Wearable-tech glove translates sign language into speech in real time

Wearable-tech glove translates sign language into speech in real time
The system includes gloves with thin, stretchable sensors that run the length of each of the five fingers. Credit: Jun Chen Lab/UCLA

UCLA bioengineers have designed a glove-like device that can translate American Sign Language into English speech in real time though a smartphone app. Their research is published in the journal Nature Electronics.

"Our hope is that this opens up an easy way for people who use to communicate directly with non-signers without needing someone else to translate for them," said Jun Chen, an assistant professor of bioengineering at the UCLA Samueli School of Engineering and the principal investigator on the research. "In addition, we hope it can help more people learn sign themselves."

The system includes a pair of gloves with thin, stretchable sensors that run the length of each of the five fingers. These sensors, made from electrically conducting yarns, pick up hand motions and finger placements that stand for individual letters, numbers, words and phrases.

The device then turns the finger movements into , which are sent to a dollar-coin-sized circuit board worn on the wrist. The board transmits those signals wirelessly to a smartphone that translates them into spoken words at the rate of about a one word per second.

The researchers also added adhesive sensors to testers' faces—in between their eyebrows and on one side of their mouths—to capture that are a part of American Sign Language.

Short video of a wearable glove to translate sign language. Credit: Jun Chen Lab/UCLA

Previous wearable systems that offered translation from American Sign Language were limited by bulky and heavy device designs or were uncomfortable to wear, Chen said.

The device developed by the UCLA team is made from lightweight and inexpensive but long-lasting, stretchable polymers. The electronic sensors are also very flexible and inexpensive.

Wearable-tech glove translates sign language into speech in real time
A digital rendering of the system that helps convert sign language into speech. Credit: Jun Chen Lab/UCLA

In testing the , the researchers worked with four people who are deaf and use American Sign Language. The wearers repeated each hand gesture 15 times. A custom machine-learning algorithm turned these gestures into the letters, numbers and words they represented. The system recognized 660 signs, including each letter of the alphabet and numbers 0 through 9.

UCLA has filed for a patent on the technology. A commercial model based on this technology would require added vocabulary and an even faster translation time, Chen said.


Explore further

Low-cost smart glove wirelessly translates the American Sign Language alphabet into text

More information: Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays , Nature Electronics (2020). DOI: 10.1038/s41928-020-0428-6 , www.nature.com/articles/s41928-020-0428-6
Journal information: Nature Electronics

Citation: Wearable-tech glove translates sign language into speech in real time (2020, June 29) retrieved 8 August 2020 from https://techxplore.com/news/2020-06-wearable-tech-glove-language-speech-real.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
636 shares

Feedback to editors

User comments