If You Want a Robot to Learn Better, Be a Jerk to It
Matt Simon @ wired.com
In what will go down as one of the greatest robotics experiments ever, a few years back researchers in Japan let a robot loose in a mall and watched how kids reacted. Far from the sense of wonder you might expect from children, the mood soured into a sense of concern for the next generation, as the kids proceeded to kick and punch the robot and call it names.
Call it unconstructive criticism. But maybe the kids were on to something—maybe we should be challenging the robots, albeit in more constructive ways, instead of always holding their hands as they learn to navigate our world. To that end, researchers at the University of Southern California have shown that when working in a simulation, you can give robots “tough love” by trying to knock objects out of their hands, and it’ll actually help them better learn to grasp objects.
The experiment took place entirely in simulation, as so much robot training does these days. In a digital environment, a robot undergoes a supercharged form of trial and error called reinforcement learning. The environment simulates variables like friction, and a robotic arm tries to grasp an object over and over using different grips. If it stumbles on a good grip, the system tallies that as a victory—if it does something stupid, the system counts that as a defeat. Over many attempts, the robot learns what constitutes a robust grasp.
A simulated robot before training with a human "adversary."COURTESY OF USC VITERBI SCHOOL OF ENGINEERING
But in comes a so-called adversarial human actor, a sort of additional signal. If the robot finds a good grasp, the human uses a graphical interface to click on the object it’s gripping and apply a force in a certain direction. That disturbance basically tests how good the grasp really is, and it helps the robot rule out the less effective ones.
“The robot learned to grasp objects much more robustly using this additional signal that the human was providing, but also learned to generalize to new objects much better,”
To put a number on it, when a human was giving the robot tough love, the machine had a 52 percent success rate at grasping, compared to 26.5 percent without the tough love.
After about 20 minutes of training with the human adversary, the robot succeeds more frequently in grasping objects, plus its grasps are more resilient to disturbances. Notice how it maintains a grip when the object moves.COURTESY OF USC VITERBI SCHOOL OF ENGINEERING
Now, some critical caveats here. First of all, a simulation is a necessarily imperfect model of the real world—there’s no way to fully replicate all the physics and uncertainty of meatspace (or metalspace, in this case). So porting what a robot learns in simulation into a physical robotic arm is still very difficult, a challenge known as the reality gap. And two, this wasn’t willy-nilly tough love, as the human participants were working with certain rules and constraints.
All that said, the experiment shows there’s merit in challenging robots instead of constantly coddling them. This is particularly important with a problem as complex as grasping, which has little margin for error.
“If we want robots to be out there helping people with different types of motor impairments, we don’t want them to break things 10 percent of the time”
Imagine unloading your dishwasher and dropping 10 percent of the dishes, and how mad you would be at yourself. Now imagine how mad you would be if a robot in your house did the same.
I mean, just look at what the children are capable of, and that robot was just standing there minding its business.
