Toward tactile and proximity sensing in large soft robots

In recent years, robots have become incredibly sophisticated machines capable of performing or assisting humans in all tasks. The days of robots functioning behind a security barrier are long gone, and today we may anticipate robots working alongside people in close contact.

While working alongside robots may be very practical in some situations, they should be designed to be safe and pleasant for humans to interact with. For instance, in human-robot interactions (HRIs), robots should be able to react correctly to potential collisions with humans and also respond safely and predictably to intentional physical contact.

One of the best approaches to improve HRIs is to grant robots the ability to sense their environment in multiple ways, such as by touch, sound, and sight. Of these three, tactile sensation is particularly important for robots that are likely to come into physical contact with humans during operation. Although small-scale tactile sensors have seen tremendous progress over the past decade, the development of large-scale tactile sensors has been plagued with challenges. Moreover, most researchers have focused on systems that respond to physical touch and ignore touchless stimuli, such as when an object is in close proximity.

To address these issues, a research team led by Associate Professor Van Anh Ho from Japan Advanced Institute of Science and Technology (JAIST) recently developed ProTac—an innovative soft robotic link with tactile and proximity sensing capabilities. As explained in their paper, presented at the IEEE-RAS International Conference on Soft Robotics (ROBOSOFT), the team not only engineered ProTac itself but also pioneered a new simulation and learning framework to effectively prepare the robotic link for use.

But what does a robotic link look like, and what is ProTac good for? In general, robotic links are rigid structural components of a robot that connect two or more joints. For example, robotic links can be seen as various "segments" in a robotic limb. In this study, ProTac is designed as a soft, cylindrical segment for a robotic arm. What makes it remarkable is how the researchers incorporated the tactile and proximity sensing capabilities in a very convenient and space-efficient way.

ProTac has an outer "soft magic skin" that can be slightly deformed by touch without damage. The inside of the skin is patterned with arrays of reflective markers, and fisheye cameras are installed at both ends of the robotic link looking towards these markers. The idea is that, upon physical contact and deformation of the skin, changes in the relative positions of the markers are captured by the cameras and processed to calculate the precise location and intensity of the contact. On top of this, the outer skin is of a functional polymer that can be made entirely transparent by applying an external voltage. It allows the fisheye cameras to image the immediate surroundings of ProTac, providing footage for proximity calculations.

To more easily train ProTac to make proximity and tactile measurements, the team also developed SimTacLS, an open-source simulation and learning framework based on the SOFA and Gazebo physics engines (see the paper here). This machine learning framework is trained with simulated and experimental data considering the physics of soft contact and the realistic rendering of sensor images.

"SimTacLS enabled us to effectively implement tactile perception in robotic links without the high costs of complex experimental setups," says Prof. Ho, "Furthermore, with this framework, users can readily validate sensor designs and learning-based sensing performance before proceeding to actual fabrication and implementation."

Overall, this work will help pave the way to a world where humans can harmoniously coexist and work alongside robots. Excited by the team's contribution to this dream, Prof. Ho comments, "We expect the proposed sensing device and framework to bring in ultimate solutions for the design of robots with softness, whole-body and multimodal sensing, and safety control strategies."

It is worth noting that proposed techniques can be extended to other types of robotic systems beyond the robotic manipulator demonstrated in the study, such as mobile and flying robots. Moreover, ProTac or similar robotic links could be used to enable robotic manipulation in cluttered environments or when operating in close vicinity with humans.

The research is published in the journal IEEE Transactions on Robotics.