"With mobile devices such as the smartwatch, the interactive screens are so small that only a few control commands can be triggered by individual touches," explains Jürgen Steimle, professor of human-computer interaction at Saarland University. With his research group in the Cluster of Excellence "Multimodal Computing and Interaction," he is searching for new ways to operate small body-worn mobile devices as discreetly and quickly as possible. In an earlier research project, Steimle, together with his colleague Martin Weigel, already showed that human skin can also work as an input device. During this study, the idea for the current project came to him. "We found out that our study participants did not just use the already-known smartphone gestures on their skin, but also pushed it to the side or even pinched it with two fingers, in order to control mobile devices," reports Martin Weigel.
Further investigations led to a sensor that was actually intended to make robot hands more sensitive. "Although the sensor was developed for robotics, we found its small form factor promising for body-worn mobile devices," explains Weigel. The small form factor, in this case, describes a sensor that has a diameter of only 10 millimeters -- about the size of a pea -- and that can be deformed like a balloon. From the inside, an infrared light-emitting diode illuminates the deformable membrane. The light is reflected, and measured by four photodiodes. These measurements can then be used to calculate how the sensor is being deformed.
To test their idea, the researchers integrated this sensor into a ring, a bracelet and a charm, which was hardly larger than a 50-cent piece. The challenge was to develop gestures and use them to control mobile devices. The researchers did this for a smartwatch and for virtual reality glasses. They also defined motion sequences to control a television and to play music without having to look at the device. The Saarbrücken computer scientists then had 24 people test these, a total of 18,141 times. Their results are clear. "Despite the tiny surface, the interactions are precise and expressive, since they make use of the fine motor control of the fingertips and of the three primitives pressing, pushing, and pinching," says Weigel. Professor Jürgen Steimle is convinced that "when only a tiny sensor needs to be deformed for input, mobile devices can be worn at places on the body that enable quick and discreet input. This will help the industry bring even smaller control devices to the market."