Remotely controlling electronic devices with hand gestures
Moving things with a wave of the hand: thanks to Empa technology this dream could soon become real. A sensor made of piezo-resistive fibers integrated in a wristband measures wrist movements and converts them into electrical signals. This can be used to steer drones or other electronic devices without a remote control.
A wave to the left: the drone moves to the left. A wave to the right: the drone turns right. Clench your hand into a fist and it lands gently on the table. No, not crazy talk; reality. Empa researchers headed by Frank Clemens from the Laboratory for High-Performance Ceramics have devised a sensor made of piezo-resistive fibers and incorporated it into a wristband that registers the hand's movements. The piezo-resistive fiber is electroconductive, recognizes a change in shape and converts it into an electrical signal, which can then be read by a terminal device and interpreted accordingly. This means that robots can be moved with a simple point of the finger, for instance.
Although motion sensors are nothing new, until now movements were primarily recorded using visual sensors (such as cameras), accelerometers and gyroscopes (for rotational movements). This manner of registering movements, however, requires large, clear movements within a particular speed range that are, by and large, unnatural for humans. The new Empa sensor, on the other hand, responds to the minutest of natural movements made "off the cuff". Nonetheless, Clemens doesn't want to do away with previous technologies. "It takes a combination of different sensors to develop new concepts. Only then can we spot and use movements that weren't detectable with previous technologies." Combining acceleration, rotation and orientation sensors with the new fiber sensor would facilitate completely new "commands" to control technical devices – whether it be a drone or the garage door.
Algorithms "translate" movements
For test purposes, the researchers integrated the sensor in a conventional wristwatch strap, which can be worn unobtrusively and restricts the wearer as little as possible. Run-of-the-mill decorative bracelets are also conceivable. Nevertheless, it took quite a while to reach this stage. In the first prototypes, Frank Clemens and Mark Melnykowycz succeeded in attaching the piezo-resistive fibers to a piece of fabric. This was insufficient to use the sensor on the desired scale, however. "With the aid of additive manufacturing, we managed to integrate the sensor structure in non-textile materials," explains Clemens. The sensor could thus eventually be used in existing wristwatch straps.
In collaboration with the companies STBL Medical Research AG and Idezo, Clemens's team then programmed the sensor in such a way that it could be used to control a drone with mere hand movements. Currently, the algorithm that "translates" between sensor and drone control is being optimized as part of a Bachelor's project at Bern University of Applied Sciences supervised by Marx Stampfli so that it can respond to even simpler gestures. Not only is the sensor supposed to recognize individual movements, but also entire movement sequences. For example, clenching your fist twice in quick succession would trigger a different command to once short and once long, and so on.
Wearing the sensor in a wristband might soon be a thing of the past, too. In her term paper, a student at ETH Zurich is examining the possibility of integrating the piezo-resistive sensor in a plaster. Then all that would be needed to perform diverse interactions with technical devices and robots would be a barely conspicuous plaster on the wrist. Although the project is still very much in its infancy, everything already works perfectly. "Together with our industry partner STBL Medical Research AG, we are currently discussing a potential industrial implementation with partners from various sectors," says Clemens.
