MIT CSAIL explores virtual reality control room

Credit: arXiv:1703.01270 [cs.RO]

(Tech Xplore)—If we clear our heads of all the anxious chatter about robots marching into a factory and humans marching out, we get our arms around the likely scenario: manufacturing will adopt systems that involve robots and people.

MIT CSAIL members working on a virtual reality system foresee the two interacting in the physical and virtual worlds, allowing humans and robots to leverage each other's capabilities.

These researchers asked, how might virtual reality (VR) headsets be used to remotely operate industrial robots in a cost-reduced way?

Their robot and user in action are seen in the "Operating Robots with Virtual Reality" video published Monday.

Their system is tele-operating a robot with Oculus Rift and the user sees through the robot's eyes. The user both sees and interacts.

The Engineer said the system would place the user "in a VR control room with multiple sensor displays, making it feel like they're inside the robot's head."

The controllers can "open and close the hand grippers to pick up, move, and retrieve items," said The Engineer.

Performing precise actions like picking up screws, the robot can even use a stapler.

The team behind the system have written a paper, which is on arXiv.

"Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing" is the title, authored by Jeffrey Lipton, Aidan Fay and Daniela Rus.

The project was funded in part by Boeing and the National Science Foundation.

Darrell Etherington in TechCrunch remarked on their title: "a 'homunculus model of mind,' or the feeling that you're a small human sitting in the brain of a large humanoid robot essentially piloting it."

The paper's abstract said the system runs on a commercial gaming engine using off the shelf VR hardware.

Bringing down the high costs of automated systems for assembly tasks could be a practical outcome of this effort. They wrote in the abstract:

"Expensive specialized systems have hampered development of telerobotic systems for manufacturing systems." They said the telerobotic system demonstrated in their work "can reduce the cost of such system by leveraging commercial (VR) technology and integrating it with existing robotics control software."

To put their system to the test, they drew up tasks, such as assembly, and pick-and-place, for a user to perform.

"For manufacturing a task, we had the user staple a wire to a board. We also tested the importance of the relative utility of the hand cameras. The user needed to be able to pick and place a wire onto a board with one arm, place a staple gun with the other, and fire the gun several times to lock the wire in place."

Results: "We compared our system with state of the art automation algorithms for assembly tasks, showing a 100% success rate for our system compared with a 66% success rate for automated systems. We demonstrate that our system can be used for pick and place, assembly, and manufacturing tasks."

The Engineer had some points about how their approach differs from other systems.

"Other systems might extract 2-D information from each camera, build out a full 3-D model of the environment, and then process and redisplay the data. In contrast, the CSAIL team's approach bypasses all of that by simply taking the 2-D images that are displayed to each eye. (The human brain does the rest by automatically inferring the 3-D information.)"

Another interesting point raised by The Engineer concerns the impact this may have on humans at work—or should we say at remote stations of work.

Would some factory works be able to cut the commute?

The team showed the system could pilot the "from hundreds of miles away, testing it on a hotel's wireless network in Washington, DC to control Baxter at MIT." Jeffrey Lipton was quoted in The Engineer: "By teleoperating robots from home, blue-collar workers would be able to tele-commute and benefit from the IT revolution just as white-collar workers do now."

The authors in their paper said, "Barriers to working such as physical health, location, or security clearance could be reduced by decoupling physicality from manufacturing tasks."

Explore further: New algorithm allows human being to communicate task to robot by performing it first in virtual reality

More information: Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing, arXiv:1703.01270 [cs.RO] arxiv.org/abs/1703.01270

Abstract
Expensive specialized systems have hampered development of telerobotic systems for manufacturing systems. In this paper we demonstrate a telerobotic system which can reduce the cost of such system by leveraging commercial virtual reality(VR) technology and integrating it with existing robotics control software. The system runs on a commercial gaming engine using off the shelf VR hardware. This system can be deployed on multiple network architectures from a wired local network to a wireless network connection over the Internet. The system is based on the homunculus model of mind wherein we embed the user in a virtual reality control room. The control room allows for multiple sensor display, dynamic mapping between the user and robot, does not require the production of duals for the robot, or its environment. The control room is mapped to a space inside the robot to provide a sense of co-location within the robot. We compared our system with state of the art automation algorithms for assembly tasks, showing a 100% success rate for our system compared with a 66% success rate for automated systems. We demonstrate that our system can be used for pick and place, assembly, and manufacturing tasks.

14 shares