While many look at robots and see elegance in their movements, others see motion not so elegant. Getting robots to be “less clumsy” when they grab objects is a goal of Dr. Ken Goldberg, an inventor working at the intersection of art, robotics, and social media.

He joined the UC Berkeley faculty in 1995, where he is the UC Berkeley William S. Floyd Jr Distinguished Chair in Engineering and recently served as Chair of the Industrial Engineering and Operations Research Department. He has secondary appointments in UC Berkeley’s Electrical Engineering/Computer Science, Art Practice and the School of Information.

Goldberg also holds an appointment at the UC San Francisco Medical School’s Department of Radiation Oncology, where he pursues research in medical robotics. Goldberg is Director of the CITRIS “People and Robots” Initiative and the UC Berkeley’s Laboratory for Automation Science and Engineering (AUTOLAB), where he and his students research machine learning for robotics and automation in warehouses, homes, and operating rooms.

Joanne Pransky, associate editor for Industrial Robot Journal, recently spoke with Goldberg about his personal and business perspectives from his career-long pursuit of making robots less clumsy.

This interview is available free to Robotics Business Review readers until July 1, 2019. Here is an excerpt:

Pransky: You are like a modern-day prolific da Vinci. What has been your favorite project and why?

Goldberg: Thank you, but the one thing we have in common is a pretty broad curiosity. Regarding my projects, I can’t pick favorites. I have been very fortunate to work with terrific students and collaborators. One problem that I’ve been trying to tackle for 35 years is how to make robots less clumsy. There’s been quite a lot of research, but it still remains a very challenging problem because of the inherent uncertainty in perception, control and physics.

Pransky: Of all the specific projects you’ve worked on, has there been one that you feel has the most potential for practical applications?

Goldberg: In the past five years, we made some important progress on what we call the Dexterity Network (Dex-Net). Dex-Net is an ongoing project that makes use of large datasets of synthetic grasping examples and combines these with stochastic models of uncertainty in perception and control, and deep learning. This combination has been very exciting and we’ve made surprising progress in the last few years.

We now have a robot capable of grasping novel objects that the robot has never seen before. We throw novel objects into a bin and the robot is able to very effectively clear the bin and pick up most objects. The breakthrough is combining analytic models with deep learning and depth sensing to learn grasp affordances. Being trained on millions of examples of patterns of points and reliable graphs allows the system to process a new pattern of points and find reliable grasps. It has nothing to do with the nature of the object; it doesn’t identify the object. The object could be anything from stones to crumpled-up paper.

What’s important to also realize is that there are some things that can cause the system to fail: if the objects are transparent, it doesn’t work; if the objects are too heavy, it doesn’t work; or if the objects are too small, it doesn’t work. A great example of this is a paper clip. The robot can’t pick up a paper clip.

Another critical part of this more recent development is what we call “ambidextrous” grasping; grasping with two or more end effectors, for example, a suction cup and a gripper. Ambidextrous grasping can increase the three “Rs” – rate, reliability, and range – which are useful for e-commerce order fulfillment.

Three things have changed in the past decade: The first two are 3D sensing and Deep Learning, both of which came out around 2010. The third is a huge increase in demand for e-commerce, which is a driving application for robot grasping.

Pransky: What is the proudest moment of your long and successful career?

Goldberg: My proudest moment was when I was hired at UC Berkeley in 1995. Since I was a kid in the 1960’s I’ve always idolized Berkeley including the Free Speech Movement – and social justice movements during a time when its students questioned authority. Berkeley is a public university and has this amazing reputation in terms of innovation and rigor, not only in the sciences and engineering, but also in the arts, humanities and social sciences.

I really like thinking and talking with people from different disciplines and it’s such a thrill to work with so many fine scholars. I had a close friendship with Hubert Dreyfus, who passed away in 2017. We had lunch every month. We co-taught two courses and had an ongoing dialogue about robotics and AI. It was amazing to me that I could have a regular ongoing conversation with someone like that.

I’m also proud that I’ve been able to find and attract truly amazing students to work in the AUTOLab Currently we have about thirty students: Postdocs, graduate students, and undergrads. Everyone helps each other. They also go out together socially and they work and stay up all night on projects when we have deadlines.

Pransky: What is the biggest mistake you’ve made or the greatest lesson that you’ve learned?

Goldberg: Where do I start? One thing I feel that I’ve learned is that I really enjoy starting new initiatives and programs but the administration tasks of running them is not really my specialty. I do that for my Lab but I was recently Department Chair and I just finished my two-year term. I wouldn’t call it a mistake as I was trying to do my duty and help out, but I learned that the administrative part is not my forte.

Pransky: If you could wave a magic wand to solve one technical problem, what would it be?

Goldberg: I’d really like to solve the robot grasping problem, what I’ve been working on for decades. Humans are great at grasping. Even babies are good at grasping, but robots are still clumsy, which affects everything from manufacturing to logistics to home automation.

Read more of the interview here.