- Valentin Schwind, Niklas Deierlein, Romina Poguntke, Niels Henze: Understanding the Social Acceptability of Mobile Devices using the Stereotype Content Model, Proceedings of CHI, 2019
- Sven Mayer, Valentin Schwind, Huy Viet Le, Dominik Weber, Jonas Vogelsang, Johannes Wolf, Niels Henze: Effect of Orientation on Unistroke Touch Gestures, Proceedings of CHI, 2019
- Ashley Colley, Sven Mayer, Niels Henze: Investigating the Effect of Orientation and Visual Style on Touchscreen Slider Performance, Proceedings of CHI, 2019
- Valentin Schwind, Pascal Knierim, Nico Haas, Niels Henze: Using Presence Questionnaires in Virtual Reality, Proceedings of CHI, 2019
- Alexandra Voit, Sven Mayer, Valentin Schwind, Niels Henze: Online, VR, AR, Lab, and In-Situ: Comparison of Research Methods to Evaluate Smart Artifacts (GitHub), Proceedings of CHI, 2019
Finger Identification on Capacitive Touchscreens using Deep Learning
Touchscreens enable intuitive mobile interaction. However, touch input is limited to 2D touch locations which makes it challenging to provide shortcuts and secondary actions similar to hardware keyboards and mice. Previous work presented a wide range of approaches to provide secondary actions by identifying which finger touched the display. While these approaches are based on external sensors which are inconvenient, we use capacitive images from mobile touchscreens to investigate the feasibility of finger identification. We collected a dataset of low-resolution fingerprints and trained convolutional neural networks that classify touches from eight combinations of fingers. We focused on combinations that involve the thumb and index finger as these are mainly used for interaction. As a result, we achieved an accuracy of over 92% for a position-invariant differentiation between left and right thumbs. We evaluated the model and two use cases that users find useful and intuitive. We publicly share our data set (CapFingerId) comprising 455,709 capacitive images of touches from each finger on a representative mutual capacitive touchscreen and our models to enable future work using and improving them.
Huy Viet Le, Sven Mayer, and Niels Henze. 2019. Investigating the Feasibility of Finger Identification on Capacitive Touchscreens using Deep Learning. In 24th International Conference on Intelligent User Interfaces (IUI ’19), March 17–20, 2019, Marina del Ray, CA, USA. ACM, New York, NY, USA. https://doi.org/10.1145/3301275.3302295
Papers accepted at CHI 2018
- Sven Mayer, Lars Lischke, Paweł Wozniak, Niels Henze: Evaluating the Disruptiveness of Mobile Interactions: A Mixed-Method Approach, Proceedings of CHI, 2018
- Sven Mayer, Valentin Schwind, Robin Schweigert, Niels Henze: The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments, Proceedings of CHI, 2018
- Huy Viet Le, Sven Mayer, Patrick Bader, Niels Henze: Fingers’ Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen, Proceedings of CHI, 2018
- Huy Viet Le, Thomas Kosch, Patrick Bader, Sven Mayer, Niels Henze: PalmTouch: Using the Palm as an Additional Input Modality on Commodity Smartphones, Proceedings of CHI, 2018
- Rufat Rzayev, Paweł Woźniak, Tilman Dingler, Niels Henze: Reading on Smart Glasses – The Effect of Text Position, Presentation Type and Walking, Proceedings of CHI, 2018
- Tilman Dingler, Rufat Rzayev, Ali Sahami, Niels Henze: Designing Consistent Gestures Across Device Types: Eliciting RSVP Controls for Phone, Watch, and Glasses, Proceedings of CHI, 2018
- Mohamed Khamis, Anita Baier, Niels Henze, Florian Alt, Andreas Bulling: Understanding Face and Eye Visibility in Front-Facing Cameras of Smartphones used in the Wild, Proceedings of CHI, 2018
- Sven Mayer, Lars Lischke, Jens Emil Grønbæk, Zhanna Sarsenbayeva, Jonas Vogelsang, Paweł Woźniak, Niels Henze, Giulio Jacucci: Pac-Many: Movement Behaviour when Playing Collaborative and Competitive Games on Large Displays, Proceedings of CHI, 2018
- Pascal Knierim, Valentin Schwind, Anna Feit, Florian Nieuwenhuizen, Niels Henze: Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands, Proceedings of CHI, 2018
Papers on Estimating Finger Orientation and Reducing Latency using Machine Learning
We will present two papers at the International Conference on Interactive Surfaces and Spaces. For both papers, we trained models that to improve the interaction with smartphones. PredicTouch is a system to reduce touchscreen latency using neural networks and inertial measurement units. With the second paper, we provide a ground truth data set for to estimate finger orientations using capacitive touchscreens recorded with a high-precision motion capture system. Using the data set, we show that a convolutional neural network can outperform approaches proposed in previous work.
- Huy Viet Le, Valentin Schwind, Philipp Göttlich, Niels Henze: PredicTouch: A System to Reduce Touchscreen Latency using Neural Networks and Inertial Measurement Units, Proceedings of ISS, 2017
- Sven Mayer, Huy Viet Le, Niels Henze: Estimating the Finger Orientation on Capacitive Touchscreens Using Convolutional Neural Networks, Proceedings of ISS, 2017
Tutorial on Intelligent Mobile User Interfaces @ MobileHCI
Together with Sven and Huy, I’ll give a tutorial on Machine Learning for Intelligent Mobile User Interfaces using TensorFlow. One key feature of TensorFlow includes the possibility to compile the trained model to run efficiently on mobile phones. This enables a wide range of opportunities for researchers and developers. In the tutorial, we teach attendees two basic steps to run neural networks on a mobile phone: Firstly, we will teach how to develop neural network architectures and train them in TensorFlow. Secondly, we show the process to run the trained models on a mobile phone.
“These are not my hands!”: Effect of Gender on the Perception of Avatar Hands in Virtual Reality
Video for our CHI 2017 paper These are not my hands!: Effect of Gender on the Perception of Avatar Hands in Virtual Reality, written by Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, and Nico Haas, Niels Henze.
Smart & Ambient Notification and Attention Management
We’ll run the UbiTtention workshop at UbiComp 2016. The workshop brings quite cool topics together including notifications and attention management. Deadline for submissions is June 17, 2016.
CHI 2016 Videos
The Effect of Focus Cues on Separation of Information Layers
Video for our CHI 2016 paper “The Effect of Focus Cues on Separation of Information Layers”, written by Patrick Bader, Niels Henze, Nora Broy and Katrin Wolf.
Impact of Video Summary Viewing on Episodic Memory Recall
Video for our CHI 2016 paper “Impact of Video Summary Viewing on Episodic Memory Recall”, written by Huy Viet Le, Sarah Clinch, Corina Sas, Tilman Dingler, Niels Henze, and Nigel Davies.
Attention! Smarttention?
Information overload, too many notification, too many ‘smart devices’ try to get users’ attention? Together with Benjamin Poppinga, Martin Pielot, Karen Church, and Alireza Sahami, I’ll run a MobileHCI workshop on Intelligent Attention Management on Mobile Devices.
CHI 2015 Videos
Modeling Distant Pointing for Compensating Systematic Displacements
Video for our CHI 2015 paper “Modeling Distant Pointing for Compensating Systematic Displacements”, written by Sven Mayer, Katrin Wolf, Stefan Schneegass and Niels Henze.
Subjective and Objective Effects of Tablet’s Pixel Density
Video for our CHI 2015 paper “Subjective and Objective Effects of Tablet’s Pixel Density”, written by Lars Lischke, Sven Mayer, Katrin Wolf, Alireza Sahami Shirazi and Niels Henze.
Text Entry on Tiny QWERTY Soft Keyboards
Video for our CHI 2015 paper “Text Entry on Tiny QWERTY Soft Keyboards” written by Luis A. Leiva, Alireza Sahami, Alejandro Catala, Niels Henze and Albrecht Schmidt from the Universitat Politècnica de València and the University of Stuttgart.
Investigation of Material Properties for Thermal Imaging-Based Interaction
Video for our CHI 2015 paper “Investigation of Material Properties for Thermal Imaging-Based Interaction”, written by Yomna Abdelrahman, Alireza Sahami Shirazi, Niels Henze and Albrecht Schmidt.