Recognising the signs

Credit: Paul Brennan/public domain

Machine recognition of sign languages is on the cards thanks to work by a team in India who are using a Microsoft Kinect movement-identifying controller. Writing in the International Journal of Computational Vision and Robotics, the team explains how their system uses just 11 of the 20 joints tracked by a Kinect and can extract novel features per frame, based on distances, angles, and velocity involving upper body joints. The team reports how the algorithm recognizes 35 gestures from Indian sign in real time with almost 90 percent accuracy.

Jayesh Gangrade and Jyoti Bharti of the Maunala Azad National Institute of Technology, in Bhopal, India, explain how many people with a significant hearing deficit utilize gesture-based communication, hand movements, and orientation together with facial expression are used dynamically to convey meaning in as nuanced and expressive a way as any other language dialect. The development of technology that could also be competent in would give those who rely on this form of communication a new approach to interacting with machines and computers. Camera, digital gloves, and other gadgets have been investigated previously in this context. However, the potential of an inexpensive video game controller, such as the Kinect, that can track could facilitate this rapidly.

The team points out that their approach requires no markers nor special clothing with tracking objects as was necessary with some of the earlier efforts in this area. "We have experimented with a minimal set of features to distinguish between the given signs with practical accuracies," the team writes. They are now experimenting with the Kinect v2 sensor which is more accurate and could push the research closer to its ultimate goal.

More information: Jayesh Gangrade et al. Real time sign language recognition using depth sensor, International Journal of Computational Vision and Robotics (2019). DOI: 10.1504/IJCVR.2019.101527

Provided by Inderscience
Citation: Recognising the signs (2019, September 5) retrieved 13 July 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Gaming system helps with autism diagnosis


Feedback to editors