The WaveGlove Prototype. An inertial sensor is attached to each of the five fingers, measuring acceleration and angular velocity, which are used for gesture classification. All the used components are low-cost and widely available, which further showcases the potential of a mass production of a similar device in the future. Credit: Kralik & Suppa.

Over the past few decades, computer scientists have developed a wide array of models and approaches to analyze different aspects of human behavior and communication, such as speech, emotions and gestures. Most existing techniques for hand gesture recognition rely on the use of wearable technologies with a single sensor and can only recognize a limited number of basic gestures.

Researchers at Comenius University Bratislava in Slovakia have recently developed WaveGlove, a new system for that utilizes multiple inertial , instead of a single sensor. This new system, introduced in a paper pre-published on arXiv, is essentially a glove with an inertial sensor placed on each of its fingers.

"Our recent paper showcases the use of multiple inertial sensors for gesture recognition (HGR)," Matej Kralik, one of the researchers who carried out the study, told TechXplore. "By building a custom hardware prototype and proposing a novel Transformer-based model (a network architecture, which had a lot of success on processing tasks), we demonstrate that using multiple sensors can have significant effect on the classification accuracy and allows for a richer vocabulary of gestures."

Using WaveGlove, the hand gesture recognition system they created, the researchers were able to acquire two datasets containing over 11000 hand gesture samples. Kralik and his colleagues then designed two different gesture vocabularies, one containing 8 whole-hand movements (WaveGlove-single) and another containing 10 more complex, carefully designed hand gestures, for which individual fingers move differently (WaveGlove-multi).

Gestures from the WaveGlove-single dataset are similar to those commonly classified using only one sensor. The sensor moves in the direction of the arrow. Credit: Kralik & Suppa.

"The presence of multiple sensors allows us to design and classify a richer vocabulary of gestures in comparison to single handheld sensors," Kralik said. "Gestures we classify using multiple sensors are similar to those we already use in our daily lives. This makes the use of a device like WaveGlove easier and more natural."

In addition to compiling datasets of hand gestures and confirming the effectiveness of the WaveGlove system, the researchers benchmarked over 10 classification methods for hand gesture recognition, some of which they had developed as part of their previous research. They evaluated these methods on numerous different datasets, as they hoped that this would help to standardize results in the field of hand gesture recognition.

Some of the gestures from the WaveGlove-multi dataset. The presence of multiple sensors allows for differing finger movements as well as using gestures we already know from our daily lives. Credit: Kralik & Suppa.

The findings gathered by this team of researchers could have numerous valuable implications. Firstly, their work could inform research focusing on hand recognition and help to enhance existing systems. Secondly, the WaveGlove prototype they created could ultimately be used to improve communications between humans and machines, allowing the latter to better interpret human hand gestures.

"To the best of our knowledge, we provide the first publicly available multi-sensor dataset of significant size in the area of Hand Gesture Recognition using inertial sensors," Kralik said. "We also demonstrate that the optimal amount and placement of the sensors depends on the gestures being classified. We now aim to explore the further fine-tuning of the Transformer-based model we examined, increasing the size and variability of the dataset and expanding the multi-sensor vocabulary of gestures."

More information: WaveGlove: Transformer-based hand gesture recognition using multiple inertial sensors. arXiv:2105.01753 [cs.HC]. arxiv.org/abs/2105.01753