November 06, 2012      

Professor Julie Shah spends time with her Interactive Robotics Group at MIT developing algorithms that will allow people and robots to cooperate in the workplace. When first asked whether her research would enable robots to eventually take over human jobs, Dr. Shah says she was stunned:

?If it was easy to make robots take people?s jobs, we would have done it twenty years ago,? Dr. Shah says. ?People are using human judgment, which is really hard to program into a robot.?

Her team focuses on increasing worker efficiency by employing robots as work assistants. Dr. Shah?s research was bolstered by an early lab test, which showed that utilizing robots in an assistive capacity (via her algorithms) increased productivity by 80%.

Co-worker robots already exist; they are being manufactured by ABB, KUKA, Kawada, Rethink Robotics and others. Money is being dumped into making the newest robots easy to program?Rethink Robotics claims their robot, Baxter, can be installed and taught a new task in less than an hour?but few are discussing how robots could easily adapt to changes in the daily routine.

Dr. Shah has challenged herself with developing these kinds of intuitive work solutions (both macro and micro) for what she calls ?high intensity domains?, or environments where timing and product availability are key variables. This involves streamlining how human and robot work is scheduled in the factory as a whole as well as how the human and robot interact when working side by side.

The great divide- Humans and robots working across the factory floor

Walk through an airplane manufacturer today and you will see what Dr. Shah observed throughout her work with Boeing: one section of the building occupied by huge, barricaded industrial robots and the other half full of human employees.

If something goes wrong on the factory floor?say, a part is delayed?a human worker can easily adapt their workflow to the disruption. Robots, on the other hand, are simply stopped until they can be assigned a new task.

This occurs because, currently, tasks are pre-assigned by rigid systems that do not account for interruptions to work flow. When workflow suddenly changes, people are good at communicating with one another to come up with a spontaneous plan of action, but, according to Shah, a robot is innately out of the loop.

?If the people are coming up with one way to move forward, how does that get communicated to the robot?? Dr. Shah asks. ?How is awareness maintained across the system so that it?s all coordinated when time and money are key factors??

The answer is in the algorithm. Dr. Shah?s algorithms for intelligent agent software are designed to eliminate idle robots by quickly implementing these re-plans. As humans determine a course of action on one side of the building, her system sets tasks in real-time with the robots on the other. The result is an intuitive interpretation of workflow on a large scale.

True interactivity- Humans and robots working at arm?s length

A worker at just about any level can ?train? Rethink?s Baxter for duty by physically moving the robot?s limbs into the positions required to perform a task and saving the settings at various points throughout the process. And, like many robots, Baxter is programmed to stop if he comes into contact with a human, but he never learns anything about the person working next to him.

If, in the future, a robot like Baxter and a human worker are assembling the same part at the same time, the robot isn?t designed to anticipate the person?s next move. He performs the same task in the same order until trained to do otherwise.

That means a human partner has to work around the robot?s movements, and there is one thing that makes that level of interactivity dangerous; it?s the same reason that robots have become an enticing alternative to human labor: people are inconsistent.

As Dr. Shah observed at Boeing, ?each person and even each team on each shift develops a different way of building their piece. You need to observe or otherwise transfer this knowledge of how a particular person likes to do a particular task to a robot task plan.?

RBR contributor, John Edwards, has previously described how Dr. Shah?s algorithms for direct human-robot interaction are allowing robots to learn specific worker behavior in order to better predict their co-worker?s needs.

Team training

So, how will this ?learning? take place in tomorrow?s workplace? You can bet that traditional training DVDs won?t be involved.

Dr. Shah?s vision starts with a team approach. She turned to the training techniques used by cockpit crews, surgeons, fire fighters and the military to support her theory that human-robot teams can learn best from what works for human teaming.

Traditionally, machines learn human preferences according to a Netflix model. The system might offer a work order preference and the worker say ?good? or ?bad? until the system narrows down the worker?s inclinations, but that strategy gathers little specific visual data about how a person performs their work. Dr. Shah?s cooperative training, on the other hand, involves recurrent virtual training sessions that resemble a video game.

In the virtual environment Dr. Shah?s team has developed, the robot can take the person?s role and vice versa, so the team essentially cross-trains. The person learns from the robot and the robot learns from the person in a visual interface, where the person clicks on the screen to choose their next action.

Similar to cockpit training, the idea is that through the repetition of tasks both robot and human can develop similar visual (and mental) representations of how they will operate together. Who is responsible for what and who reports to whom becomes clear as well as how each task will be carried out and what the possible reactions to a sudden change in activity should be. According to Shah, it?s been empirically proven that rigid, command-driven team environments are less successful than others. (Just imagine if you had to continually tell your co-worker what their next move should be.)

In cases where learning involves more art, such as when teaching a robot the best way to position its hand to pick up an object, Dr. Shah?s team inputs data for the virtual environment using a motion-capture suit. According to Dr. Shah, this is not a practical solution for the industrial infrastructure.

It?s today?s sensor manufacturers who will build those data capture capabilities into robotic hardware. For instance, concurrent research into visual tracking using 3D imaging technology is being conducted in Germany. One can imagine how a robot using an advanced version of the Kinect sensor might quickly interpret and adapt to a co-workers movements in real-time.

And what about if a person suddenly changes it up on the assembly line?

Just as data patterns converge to create a model when the robot and human are interacting successfully, there is a track-able divergence in data when the human veers from the behavior they exhibited in training. The next step, according to Dr. Shah, is folding that ?uncertainty? into safety behaviors for the robot.

Just like a human, the robot can be programmed to react (say, slow down or stop completely) if it notices its coworker behaving unpredictably.
CSAIL isn?t responsible for creating safety standards, but Dr. Shah?s team can program a robot to maintain the required distance from its coworker or slow to a certain speed, according to industry standards, until it can adapt to the human?s new task performance.

Dr. Shah says it?s hard to say exactly how long workers will have to train with robots before the robot learns their behaviors. Her team is measuring the robots? learning curve over the course of three virtual sessions in the lab.

?The worker will need to ?practice? the task with the robot in simulation a few (hopefully very few, maybe 2-4) times and then further learning would happen over time, day after day when performing the real task on the assembly line.?

As far as when we can expect to see this level of interaction in the industrial workplace, Dr. Shah favors a shorter time frame.

?My opinion is that I think we?re looking at closer to five years than ten years or 15 years.?