Every home is different. That makes it difficult for domestic robots to translate skills learned in one household — say, fetching a soda from the fridge — into another. Training in virtual reality, where the robot has access to rich information about three-dimensional objects and spaces, can make it easier for robots to generalize skills to the real world.
What’s new: Toyota Research Institute built a household robot that users can train using a virtual reality interface. The robot learns a new behavior based on a single instance of VR guidance. Then it responds to voice commands to carry out the behavior in a variety of real-world environments.
How it works: Toyota’s robot is pieced together from off-the-shelf parts, including two cameras provide stereoscopic vision. Classical robotics software controls the machine, while convolutional neural networks learn unique embeddings.
- To teach the robot new tasks, a user wears a VR headset to see through its eyes and drive it via handheld paddles.
- During training, the system maps each pixel to a wealth of information including object class, a vector pointing to the object’s center, and other features invariant to view and lighting.
- When the robot carries out a learned action in the real world, it establishes a pixel correspondence between its training and the present scene, and adjusts its behavior accordingly.
Results: The Toyota researchers trained the bot in the virtual environment on three tasks: retrieving a bottle from a refrigerator, removing a cup from a dishwasher, and moving multiple objects to different locations. Then they had the robot perform each task 10 times in two physical homes. They ran the experiments with slight alterations, for instance asking the robot to retrieve a bottle from a higher shelf than the virtual one it was trained on, or doing so with the lights turned off. The robot achieved an 85 percent success rate — though it took an average 20 times longer than a human would.
Why it matters: Researchers have given a lot of attention lately to the use of reinforcement learning on robots that are both trained and tested in a simulated environment. Getting such systems to generalize from a simulation to the real world is an important step toward making them useful.
We’re thinking: Birth rates have been slowing for decades in Japan, China, the U.S., and much of Europe. The World Health Organization estimates that 22 percent of the world’s population will be over 60 years old by 2050. Who will care for the elderly? Robots may be part of the answer.