Fusion system has arms, hands to show how tasks are done

“Fusion” used as a bodily driven communication system of an operator and a surrogate. Three levels of communication are realized: (A) Direct actions using gestures and indications, (B) Enforced postures by forcing surrogate body to certain positions, and (C) Induced motions by altering the perception of body posture (red arrows represent the induced motion). Credit: ACM SIGGRAPH 2018 Emerging Technologies on - SIGGRAPH '18 (2018). DOI: 10.1145/3214907.3214912

"Body surrogacy." Technology's word masters have nailed this effort easily enough, but explaining what it is takes more than a few words. Several tech-watching sites were having a close look at a project called Fusion, which is a system that give you two extra robotic hands to get things done—thanks to a person remotely working in collaboration with you to do the task well.

It's described as a "telecollaboration system," and was demonstrated at SIGGRAPH 2018 Emerging Technologies last month in Vancouver. The event looks at happenings in computer graphics and interactive techniques.

A video released in August shows it in action, as "Fusion: Full Body Surrogacy for Collaborative Communication."

Researchers have worked up a remotely operated wearable system; they think of it as one person diving into someone else's body. Diving into another body may sound a bit too sci-fi but what they really focus on is transferring body actions from one person to another, using robotic arms and other devices.

In the mix: stereo camera, arms; attachable hands with force feedback sensor. The researchers integrated the components and battery with the backpack.

Madis Kabash in Quartz told the story: "The robot is equipped with two eye-shaped cameras that look over the shoulder of the wearer, plus two pairs of hands that can be controlled by joysticks. The joysticks vibrate using 'force feedback sensors,' designed to make it easier for a remote user who is powering the backpack to grab things. The backpack-wearer can also give the robot—i.e. the remote worker—full control of their arms by attaching wristbands to their hands, but this definitely requires a level of trust."

Rima Sabina Aouf, Dezeen, provided some more details: "Syrian designer Yamen Saraiji has designed a wearable, two-armed robot that acts as an extension to the human body. Created during Saraiji's Ph.D. studies at Keio University Graduate School of Media Design in Japan, Fusion is a robot that can be worn like a backpack, giving the wearer use of two additional, remotely operated arms. The arms are manoeuvred by a distant second user wearing an Oculus Rift virtual-reality headset and operating the platform's handheld Touch controllers."

Sabina mentioned Keio University, which has The Embodied Media Projects. The latter site goes on to describe the robot-body pair. " Designed as a backpack, the system "is operated in three different modes: direct collaboration, enforced body guidance, and induced body motion, enabling effective communication."

Yamen Saraiji and Kouta Minamizawa are from the Keio University Graduate School of Media Design. Tomoya Sasaki, Reo Matsumura and Masahiko Inami are from the University of Tokyo. Looking at their construct, it is easy to imagine an application where people with physical limitations could find assistance.

One viewer commented as such on the YouTube video page. "What you people are doing can bring freedom to cripples and people who somehow can't use their locomotive functions of body."

In the bigger picture, the researchers are joining the cadre of designers exploring robotics to enhance human abilities, whether for assisting the disabled or instructing people learning new skills. Luke Dormehl in Digital Trends quoted Saraiji: "With the proposed concept of sharing, we not only solve the collaboration problem, but also propose its potentials as a skill transfer and rehabilitation system."

The video suggests scenes where the guides a person to accomplish tasks.

Dormehl in Digital Trends said, "For example, it could be used by a therapist to assist with a patient's physical practice."

Explore further: Extra arms, manipulated by feet, designed for performing tasks

More information: MHD Yamen Saraiji et al. Fusion, ACM SIGGRAPH 2018 Emerging Technologies on - SIGGRAPH '18 (2018). DOI: 10.1145/3214907.3214912 (PDF)

62 shares