Wearable wristband captures entire body in 3D

body 3d
Credit: Unsplash/CC0 Public Domain

Using a miniature camera and a customized deep neural network, Cornell researchers have developed a first-of-its-kind wristband that tracks the entire body posture in 3D.

BodyTrak is the first wearable to track the full body pose with a single . If integrated into future smartwatches, BodyTrak could be a game-changer in monitoring user body mechanics in where precision is critical, said Cheng Zhang, assistant professor of and the paper's senior author.

"Since smartwatches already have a camera, technology like BodyTrak could understand the user's pose and give real-time feedback," Zhang said. "That's handy, affordable and does not limit the user's moving area."

A corresponding paper, "BodyTrak: Inferring Full-body Poses from Body Silhouettes Using a Miniature Camera on a Wristband," was published in the Proceedings of the Association for Computing Machinery (ACM) on Interactive, Mobile, Wearable and Ubiquitous Technology, and presented in September at UbiComp 2022, the ACM international conference on pervasive and .

BodyTrak is the latest body-sensing system from the SciFiLab—based in the Cornell Ann S. Bowers College of Computing and Information Science—a group that has previously developed and leveraged similar deep learning models to track hand and finger movements, and even silent-speech recognition.

The secret to BodyTrak is not only in the dime-sized camera on the wrist, but also the customized deep neural network behind it. This deep neural network—a method of AI that trains computers to learn from mistakes—reads the camera's rudimentary images or "silhouettes" of the user's body in motion and virtually re-creates 14 body poses in 3D and in real time.

In other words, the model accurately fills out and completes the partial images captured by the camera, said Hyunchul Lim, a doctoral student in the field of information science and the paper's lead author.

"Our research shows that we don't need our body frames to be fully within camera view for body sensing," Lim said. "If we are able to capture just a part of our bodies, that is a lot of information to infer to reconstruct the full body."

Maintaining privacy for bystanders near someone wearing such a sensing device is a legitimate concern when developing these technologies, Zhang and Lim said. They said BodyTrak mitigates for bystanders since the camera is pointed toward the user's body and collects only partial body images of the user.

They also recognize that today's smartwatches don't yet have small or powerful enough cameras and adequate battery life to integrate full body sensing, but could in the future.

Along with Lim and Zhang, paper co-authors are Matthew Dressa '22, Jae Hoon Kim '23 and Ruidong Zhang, a doctoral student in the field of information science; Yaxuan Li of McGill University; and Fang Hu of Shanghai Jian Tong University.

More information: Hyunchul Lim et al, BodyTrak, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (2022). DOI: 10.1145/3552312

Provided by Cornell University
Citation: Wearable wristband captures entire body in 3D (2022, November 8) retrieved 23 July 2024 from https://techxplore.com/news/2022-11-wearable-wristband-captures-entire-body.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Wearable device uses sonar to reconstruct facial expressions


Feedback to editors