Object identification and interaction with a smartphone knock

Object Identification and Interaction with a Smartphone Knock
Professor Lee (far right) demonstrate ‘Knocker’ with his students. Credit: KAIST

A KAIST team has featured a new technology, "Knocker," which identifies objects and executes actions just by knocking on it with the smartphone. Software powered by machine learning of sounds, vibrations, and other reactions will perform the users' directions.

What separates Knocker from existing technology is the sensor fusion of sound and motion. Previously, identification used either with cameras or hardware such as RFID (Radio Frequency Identification) tags. These solutions all have their limitations. For computer vision technology, users need to take pictures of every item. Even worse, the technology will not work well in poor lighting situations. Using hardware leads to additional costs and labor burdens.

Knocker, on the other hand, can identify objects even in dark environments only with a smartphone, without requiring any specialized hardware or using a camera. Knocker utilizes the smartphone's built-in sensors such as a microphone, an accelerometer, and a gyroscope to capture a unique set of responses generated when a smartphone is knocked against an object. Machine learning is used to analyze these responses and classify and identify objects.

The research team under Professor Sung-Ju Lee from the School of Computing confirmed the applicability of Knocker technology using 23 everyday objects such as books, , water bottles, and bicycles. In noisy environments such as a busy café or on the side of a road, it achieved 83% identification accuracy. In a quiet indoor environment, the accuracy rose to 98%.

Credit: The Korea Advanced Institute of Science and Technology (KAIST)
Object identification and interaction with a smartphone knock
An example knock on a bottle. Knocker identifies the object by analyzing a unique set of responses from the knock, and automatically launches a proper application or service.

The team believes Knocker will open a new paradigm of object interaction. For instance, by knocking on an empty water bottle, a smartphone can automatically order new from a merchant app. When integrated with IoT devices, knocking on a bed's headboard before going to sleep could turn off the lights and set an alarm. The team suggested and implemented 15 application cases in the paper, presented during the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2019) held in London last month.

Professor Sung-Ju Lee said, "This new technology does not require any specialized sensor or hardware. It simply uses the built-in sensors on smartphones and takes advantage of the power of . It's a software solution that everyday users could immediately benefit from." He continued, "This technology enables users to conveniently interact with their favorite objects."

Citation: Object identification and interaction with a smartphone knock (2019, October 2) retrieved 20 June 2024 from https://techxplore.com/news/2019-10-identification-interaction-smartphone.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

A new smartphone user authentication system based on gait analysis


Feedback to editors