- Sven Mayer, Lars Lischke, Paweł Wozniak, Niels Henze: Evaluating the Disruptiveness of Mobile Interactions: A Mixed-Method Approach, Proceedings of CHI, 2018
- Sven Mayer, Valentin Schwind, Robin Schweigert, Niels Henze: The Effect of Offset Correction and Cursor on Mid-Air Pointing in Real and Virtual Environments, Proceedings of CHI, 2018
- Huy Viet Le, Sven Mayer, Patrick Bader, Niels Henze: Fingers’ Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen, Proceedings of CHI, 2018
- Huy Viet Le, Thomas Kosch, Patrick Bader, Sven Mayer, Niels Henze: PalmTouch: Using the Palm as an Additional Input Modality on Commodity Smartphones, Proceedings of CHI, 2018
- Rufat Rzayev, Paweł Woźniak, Tilman Dingler, Niels Henze: Reading on Smart Glasses – The Effect of Text Position, Presentation Type and Walking, Proceedings of CHI, 2018
- Tilman Dingler, Rufat Rzayev, Ali Sahami, Niels Henze: Designing Consistent Gestures Across Device Types: Eliciting RSVP Controls for Phone, Watch, and Glasses, Proceedings of CHI, 2018
- Mohamed Khamis, Anita Baier, Niels Henze, Florian Alt, Andreas Bulling: Understanding Face and Eye Visibility in Front-Facing Cameras of Smartphones used in the Wild, Proceedings of CHI, 2018
- Sven Mayer, Lars Lischke, Jens Emil Grønbæk, Zhanna Sarsenbayeva, Jonas Vogelsang, Paweł Woźniak, Niels Henze, Giulio Jacucci: Pac-Many: Movement Behaviour when Playing Collaborative and Competitive Games on Large Displays, Proceedings of CHI, 2018
- Pascal Knierim, Valentin Schwind, Anna Feit, Florian Nieuwenhuizen, Niels Henze: Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands, Proceedings of CHI, 2018
We will present two papers at the International Conference on Interactive Surfaces and Spaces. For both papers, we trained models that to improve the interaction with smartphones. PredicTouch is a system to reduce touchscreen latency using neural networks and inertial measurement units. With the second paper, we provide a ground truth data set for to estimate finger orientations using capacitive touchscreens recorded with a high-precision motion capture system. Using the data set, we show that a convolutional neural network can outperform approaches proposed in previous work.
- Huy Viet Le, Valentin Schwind, Philipp Göttlich, Niels Henze: PredicTouch: A System to Reduce Touchscreen Latency using Neural Networks and Inertial Measurement Units, Proceedings of ISS, 2017
- Sven Mayer, Huy Viet Le, Niels Henze: Estimating the Finger Orientation on Capacitive Touchscreens Using Convolutional Neural Networks, Proceedings of ISS, 2017
Video for our CHI 2017 paper These are not my hands!: Effect of Gender on the Perception of Avatar Hands in Virtual Reality, written by Valentin Schwind, Pascal Knierim, Cagri Tasci, Patrick Franczak, and Nico Haas, Niels Henze.
We’ll run the UbiTtention workshop at UbiComp 2016. The workshop brings quite cool topics together including notifications and attention management. Deadline for submissions is June 17, 2016.
Information overload, too many notification, too many ‘smart devices’ try to get users’ attention? Together with Benjamin Poppinga, Martin Pielot, Karen Church, and Alireza Sahami, I’ll run a MobileHCI workshop on Intelligent Attention Management on Mobile Devices.
Modeling Distant Pointing for Compensating Systematic Displacements
Video for our CHI 2015 paper “Modeling Distant Pointing for Compensating Systematic Displacements”, written by Sven Mayer, Katrin Wolf, Stefan Schneegass and Niels Henze.
Subjective and Objective Effects of Tablet’s Pixel Density
Video for our CHI 2015 paper “Subjective and Objective Effects of Tablet’s Pixel Density”, written by Lars Lischke, Sven Mayer, Katrin Wolf, Alireza Sahami Shirazi and Niels Henze.
Text Entry on Tiny QWERTY Soft Keyboards
Video for our CHI 2015 paper “Text Entry on Tiny QWERTY Soft Keyboards” written by Luis A. Leiva, Alireza Sahami, Alejandro Catala, Niels Henze and Albrecht Schmidt from the Universitat Politècnica de València and the University of Stuttgart.
Investigation of Material Properties for Thermal Imaging-Based Interaction
Video for our CHI 2015 paper “Investigation of Material Properties for Thermal Imaging-Based Interaction”, written by Yomna Abdelrahman, Alireza Sahami Shirazi, Niels Henze and Albrecht Schmidt.
Smartphones currently become the most pervasive computing devices of all times. They currently become even the best-selling consumer electronic devices of all. Obviously there is a huge amount of research that investigates how people use their phones and how we can improve their experience. If doing research using smartphones, an important practical question is which platform one should choose. Basically, there are three major platforms left and alive: iOS on the iPhone, Windows Phone, and Android.
Developing for Android is nice but developing for the other platforms isn’t worse. While Java might not be the most innovative language it easily beats iOS’s Objective C (garbage collection anyone?) and is almost on par with the .NET languages (and you could also use one of the other JVM languages). What makes Java compelling is the huge number of available examples but what really sticks out (for us) is that all our computer science students have to learn Java in the first semester. This means that every single, somewhat capable, student knows how to program Java that is even used throughout their university courses. It also comes in handy (actually this is already a real show stopper) that unlike developing for iOS you don’t need a Mac and unlike Windows Phone you don’t need Windows. Linux, Windows, MacOS – yes they can all be used to develop for Android (and those who like the pain can also use BSD).
Android is free and open. Sure, it is probably free like beer and not like free speech but you can still look into the code. Being able to look into your OS’s source code might seem like an academic detail… One of my former students had to look into the Android’s sources to understand the memory management for developing commercial apps. Having the source code enabled us to understand the Android keyboard and reuse it during our studies. We even patched Android to develop handheld Augmented Reality prototypes. All this is only possible if you have the source code available. For these examples, it might not be necessary to look in the code on other platform. Still, at one point or another you might want to dig down to the hardware level and you are screwed if it isn’t Android that you have to dig through.
While developing prototypes and conducting lab studies is nice at one point or another you might want to deploy your shiny research prototype. It might be for research, it might be for fun, or just for the money. Deploying your app in the Android market takes just seconds (if you already have those screenshots and descriptions readily available). There is no approval process. No two weeks waiting until Apple decides that your buggy prototype is – just a too buggy prototype. All you need is 25$ and a credit card (and a Google Account and a soul to sell).
Windows Phone will certainly increase its market share by some 100% soon – which isn’t difficult if you start from 0.5%. However, Android overturned all other platforms, including iOS and Blackberry. The biggest smartphone manufacturer is Samsung with their Android phones. They sell more smartphones than Nokia and they sell more smartphones than Apple. Well, and they are not the only company with an Android phone in their portfolio.
Fragmentation is horrible! I developed for Windows Mobile and for JavaME. Even simple applications need to be tested on different devices to hope that it works. Things aren’t too bad for Android (if you don’t use the camera or some sensors or recent APIs or some other unimportant things…). Fragmentation can even be great for the average mobile HCI researcher. Need a device with a big screen or with a small display? Fast processor, long battery life, TV out, or NFC? There is a device for that! There are very powerful and expensive devices (the ones you will use to test your awesome interface) but also very cheap ones for less than 80€ (that you can give to your nasty students).
Usability, UX, …
Android offers the best usability of all platforms ever – well probably not. Would I buy an Android phone for my mother? If money doesn’t count I would certainly prefer an iPhone. What would I recommend to my coolish step brother? Certainly a Windows Phone to impress the girls. But what would I recommend to my students? There is nothing but Android!
There will be one billion smartphone users in 2013 and most of them will need some sort of text entry. To help people to enter text on mobile devices we aimed at studying how people type with a large number of participants. Therefore, we developed a typing game that records how users touch on the standard Android keyboard to investigate users’ typing behaviour. We published the typing game Type It! on the Android Market. The game got installed by 72,945 players and enabled us to collect 47,770,625 keystrokes from around the world.
Using the data we identified three approaches to improve text entry on mobile phones. As we found a systematic skew in users’ touch distribution we derived a function that compensates this skew by shifting touch events. In addition, we changed the keys’ labels by shifting them upwards and visualize the position where users touch the keyboard. By updating the game we conducted an experiment that investigates the effect of the three approaches. Results based on 6,603,659 further keystrokes and 13,013 installations show that visualizing the touched positions using a simple dot decreases the error rate of the Android keyboard by 18.3% but also decreases the speed by 5.2% with no positive effect on learnability. The Android keyboard outperforms the control condition but the constructed shift function further improves the performance by 2.2% and decreases the error rate by 9.1%. We argue that the shift function can improve existing keyboards at no costs.
Our paper with the lengthy title ‘Observational and Experimental Investigation of Typing Behaviour using Virtual Keyboards on Mobile Devices‘ that describes our work has recently been accepted at CHI 2012.
Flying back from another conference I had a look at the MobileHCI 2011 proceedings. Having seen a lot of fantastic talks I don’t remember a single presentation where I thought that the paper shouldn’t have been accepted (in contrast to some talks at this year’s Interact, previous MobileHCI, and similar conferences). Anyway, just as for the MobileHCI 2010 proceedings I went through all short and long papers to derive some statistics.
18 short papers and 45 long papers (20 more papers than last year) have been accepted with a slightly increased acceptance rate of 22.8%. As I focussed on the subjects that participate in the conducted studies I excluded 6 papers from the analysis because they are systems papers (or similar) and do not contain a real study.
Number of Subjects
The average number of subjects per paper is M=1,969, SD=13,757. Removing the two outliers by Böhmer et al. (4,125 subjects) and our paper (103,932 subjects) the number of subjects is M=76.62, SD=159.84. The chart below shows the distribution of subjects per paper for the considered long and short papers.
Not all papers report the subjects’ gender. If there are multiple studies in a paper and the gender is reported for one of the studies I still use the numbers. For the paper that report participants’ gender 28.28 (SD=49.07) are male and 21.84 (SD=49.26) are female. The chart below shows the number of males and females for short and long papers (error bars show the standard error).
A paired two-tailed t-test shows that there are significantly more male participants than female participants (p<.05, d=.13). The effect is also significant if only the long papers are considered (p<.01, d=.13) but not for the short papers (p=.54). The reason why the effect is not significant for short papers is The Hybrid Shopping List. Excluding this paper the effect is also significant for short papers (p<.01, d=0.68).
Not all papers report participants’ age in a consistent and complete way. Nonetheless, I tried my best to derive the age for all papers. The chart below shows the histogram for the 41 papers where I was able to derive the average age. The average for the considered papers is 27.46 years.
It is a bit difficult for me to understand why papers fail to report participants’ age and why the age is reported in so many different ways. Of course, the age might not always be seen as relevant and sometimes you just don’t know it. However, if the data is available it is so easy to provide a basic overview. Just report the age of the youngest and oldest participant along with the average age and the sample’s standard deviation. That even fits in a single line!
Getting a complete picture of the participants’ background is just impossible based on the papers alone. To many papers either report nothing about the participants’ background or only very specific aspects (e.g. ‘all participants are right handed’). Even using the sparse information it is clear to me that the fraction of students and colleagues that participate – both with a technical background – is much higher than their fraction of the population.
From my own experience I know that getting a nice sample for your study can cost a lot of resources and/or creativity. Thus, we often rely on ‘students and guys from the lab’ for our studies. IMHO it is often perfectly fine to use such a sample (e.g. when conducting a repeated-measures Fitts’ law experiment). Still I wonder if we optimize our research for this very particular target group and if this might be an issue for the field.
I analysed the MobileHCI 2011 long and short papers to determine information about the subjects that participated in the respective studies. The number of subjects per paper is more than three times higher than 2010 even if we ignore the two outliers. One reason is that there are a few papers that contribute results from online questionnaires (or similar) that attracted some hundred participants. Even if we would also exclude these papers the sample size increased. Looking at participants’ gender we found a clear bias towards male participants. Compared to 2010, however, this bias got smaller. For 2010 we found 40.89% female participants while we found 43.57% for 2011. The age distribution shows that studies with elderlies are rare.
The data seems to support my impression that the quality is higher compared to last year. The sample size and the quality of the sample have both improved. Based on my subjective impression I also assume that the way demographics are reported improved compared to last year. Thus, I conclude that MobileHCI 2011 wasn’t only fantastic to attend but also provided a program with an outstanding quality.
If it is a “Samsung GT-i9003 Galaxy SL” the answer might be yes. At least there is something strange about its touchscreen.
Recently I analysed the data we collect using Hit It! one of our Android games that has been installed more than 350,000 times. To play the game users have to tap on circles that randomly appear on the screen. While the game is played we record the user’s behaviour and send it back to our server. In particular, we record the positions that are tapped by the player’s finger. Looking at the hit positions relative to the presented circles we did some pretty nifty things. However, one would expect that the positions that the players tap are somewhat evenly distributed across the screen when combining a serious number of taps.
Interestingly, for the GT-i9003 the distribution looks strange. For the images below we took a random sample consisting of data from 40,000 devices from our data set and extracted the data produced by the GT-i9003 and by the GT-i9000 (a regular Samsung Galaxy S). Data for the GT-i9003 has been produced by 157 devices resulting in 170,205 taps. Data for the GT-i9000 has been produced by 2,321 devices resulting in 3,689,138 taps.
The pixels’ colours show how often a particular pixel has been tapped. Green pixels are tapped often while red pixels are tapped less often. Black pixels are never tapped at all. Due to the nature of the game players have to tap on the screen’s centre more often. For the GT-i9003 we see that half the pixels are NEVER tapped at all. Considering the amount of data and the small difference between the two devices (see below) this obviously can’t be by chance.
The touch-part of the GT-i9003’s touchscreen seems to have only half the resolution of the regular Galaxy S. Furthermore, the hardware or software deals with this limitation in a strange way. The pattern is interesting. There are two rows of pixels that can be tapped followed by two rows that are never tapped. I assume that if the finger taps an “untappable” row the input is mapped to one of the adjacent rows. Considering the somewhat strange pattern and that I can get sub-pixel resolution from the Android API I assume that it is a software problem rather than a hardware issue or probably a combination of hard- and software.
The GT-i9003 is the only device that returns that pattern. If someone owns a GT-i9003 I would be very interested to hear if the effect results in practical issues. But there are also other devices that might have some issues. E.g. there are some Optimus Ones that deserve further investigation (too much variance in the distribution) and I would also like to look at the Kyocera Zio.