The RoboHow project has told the world what it's been up to in research at the High-Tech Systems 2015 fair and conference in the Netherlands. RoboHow is a four-year European research project that started in February 2012. It works toward enabling robots to competently perform everyday human-scale manipulation activities, in work and living environments.
So what have they been up to? Quartz calls it Flipping Out. The robots, said Quartz, are using instructions to figure out how to make pancakes and pizza. The chef is PR2 (a robotics R&D platform) and it carefully goes through directions. The bigger picture is that its scientists are exploring ways of teaching robots to understand language and more. Picking up the spatula and proceeding to flip is not trivial—just making sure robots handle objects so they do not drop is an uphill challenge, leave alone following cooking instructions.
Will Knight in MIT Technology Review said that teaching robots how to turn high-level descriptions into specific actions is an important but tough task. "It is straightforward for humans because we have an understanding of all sorts of basic tasks, collected over a lifetime. A human does not need to be told the specific grasp needed to remove the top from a jar of tomato sauce." The researchers, he said, want to teach robots the general knowledge required to turn high-level instructions into specific actions.
"Common sense knowledge like moving the spatula to place the pancake on the plate – it's an implicit knowledge humans have, but it's extremely hard to make that explicit for a robot," Prof. Michael Beetz, head of RoboHow, told International Business Times UK.
Another challenge is depth, said International Business Times UK's Mary-Ann Russon. "Robots have a great difficulty in understanding depth, so if there are glass bottles on a table, the robot perceives a table and two bottle caps floating in mid-air."
The Institute for Artificial Intelligence (IAI) at the University of Bremen coordinates the project; CBC News said the project involves researchers from at least nine universities across Europe.
The team has goals not only for a cognitive robot that autonomously performs complex and routine manipulations but can extend its repertoire—-can pick up entirely new skills by using web-enabled and experience-based learning, and even through watching humans.
"Once a robot has learned how a particular set of instructions relates to a task," wrote Knight, "its knowledge is added to an online database called Open Ease, so that other robots can access that understanding. These instructions are encoded in machine-readable language similar to the one used in the Semantic Web project."
Siddhartha Srinivasa, a professor at the Robotics Institute at Carnegie Mellon University and founder of the Personal Robotics Lab, said in MIT Technology Review that connecting language with action is hugely important but also difficult. "Succeeding in this domain will require a tight integration of natural language, grounding the understanding via perception, and planning complex actions via manipulation algorithms."
We cannot live by robots making pizza and pancakes alone; their work carries wider implications. Knight said, "This could make it easier for people to communicate instructions to robots and provide a way for machines to figure out how to perform unfamiliar tasks. Instead of programming a robot to perform precise movements, the goal is for a person to simply tell a robot what to do."