Listen to this article
Evolution of Mobile Manipulation
Since the 1960s, robotic arms have been transforming the way companies do business. Although they were big, bulky, and highly specialized, early industrial robotic arms were able to automate repetitive tasks at a rate of three times the speed of humans. Several decades later, robotic arms are not only smaller and sleeker, but also able to mimic the way humans think, react, and adapt.
Although mobile robots have existed for decades, it was the U.S. Military that widely adopted and deployed unmanned ground vehicles with manipulation capability to keep troops safe while performing(EOD) missions during the wars in Iraq and Afghanistan. Considered cutting-edge at the time, these simple robotic arms with minimal degrees of freedom would be considered crude by today’s standards, especially since modern robot arms can offer more dexterity than that of a human arm.
Five Key Abilities
Today, with ongoing advancements in robotic hardware components and intelligent software capabilities, intelligent mobile manipulation has become a reality. In this article, we will define mobile manipulation and list the five key abilities that intelligent mobile robots must possess in order to automate anywhere – namely, the ability to move, interact, see, think, and communicate.
The Ability to Move
Mobility Increases Functionality
Traditional articulated robotic arms are bolted to factory floors, where tasks can only be performed within the limits of the robot’s workspace. By integrating robotic arms with mobile platforms, tasks no longer must be performed in a fixed, static location. Rather, the robot can move to any location to perform a desired task. Mobility allows businesses to think about the function of robotics differently when compared to traditional industrial robots.
Mobility can come in different forms based on the application and the environment. Mobile robots can have wheels, tracks, and even legs. No matter the form, the platform must provide the proper level of stability and power to enable robotic arm(s) to operate seamlessly with the base. To truly automate anywhere, mobile platforms should ideally be rugged enough to go where humans can go, even during inclement weather.
These mobile platforms also give robots the ability to access environments that are hazardous to humans. Long used by the military for EOD and combat engineering missions, mobile robots are now being used for hazardous tasks across numerous industries, including aviation, energy, and oil and gas, to keep front line workers out of harm’s way.
The ability to go where humans can go is the first step in enabling intelligent mobile manipulation.
The Ability to Interact
Mobile Manipulation Enables Interaction With the World
What sets mobile robots apart from their traditional counterparts? Unlike industrial arms and collaborative robots (cobots), mobile manipulators are designed to perform tasks with human-like capabilities in a variety of environments.
Mobile manipulators must be designed from the ground up with mobility in mind. As such, mobile manipulators must feature the following attributes:
Lightweight – Mobile manipulators must be lightweight to reduce dynamic affects on the platform and lengthen system runtimes. Reducing mass also improves arm speed.
Compact – Mobile manipulators must be compact to maneuver into areas designed for humans.
Power-efficient – Mobile manipulators must be power-efficient to operate for many hours, as these systems are often battery-powered.
Rugged – Mobile manipulators must be rugged to withstand weather, shock, and vibration.
Power-dense – Mobile manipulators must be power-dense to provide strength in a lightweight package while maintaining dexterity and speed.
Portable – Mobile manipulators must run on standard DC Bus, allowing the system to operate without wall power.
What do these features mean in terms of a mobile robot’s ability to interact? In short, because of its smaller footprint, the system has the ability to complete tasks that were once only accessible by humans. As a result, businesses can truly automate anywhere.
Robotic technology can also help businesses increase throughput and overcome labor shortages by augmenting staff. In addition, robots can serve as a force multiplier. Working from a command center, one person can oversee a variety of tasks being completed by multiple robots. Furthermore, since this technology transfers physically demanding tasks to the robot, it enables older workers to stay on the job longer and leverage their expertise to optimize control of the robots.
The Ability to See
Perception Systems for Any Environment
In today’s robotics landscape, you can find a variety of mobile platforms moving throughout structured industrial environments and performing critical tasks within warehouses. These platforms play an important role in transporting goods and materials indoors. However, for tasks that require interaction in an unstructured environment, intelligent mobile manipulation is optimal.
Robots must be able to perceive and conceptualize the world around them. Unlike their predecessors, today’s mobile manipulation systems are equipped with the ability to perceive the world using multi-spectrum sensing including LiDAR, monocular vision, and stereo vision. RE2’s robotic systems, for instance, use third-party multi-modal 2D- and 3D-imaging sensors coupled with proprietary algorithms, called RE2 Detect, to perceive the world. Using RE2 Detect, systems can locate and track objects in just about any indoor or outdoor environment, whether in an operating room with bright, controlled illumination or outside on a rainy, overcast day.
Continuing with this example, RE2’s control algorithms close the control loop by continually adjusting the robotic arm’s position and orientation based on real-time vision processing and reporting. These methods are accelerated by customizing the input and processing hardware. Additionally, unlike typical industrial solutions that rely on 3D models to function, RE2 Detect leverages the geometric structure of objects, including their position, shape, or pose, to enable unique grasps for varied scenarios.
The Ability to Think
AI-Driven Robotic Intelligence
Historically, mobile manipulators were teleoperated; that is, controlled by an operator from a standoff distance. Today, thanks to enhancements in artificial intelligence (AI) and machine learning, mobile manipulators are on the path to becoming fully autonomous and enhanced with the ability to not only see, but to also think and react like humans.
When combined with AI, today’s mobile manipulators have the capacity to revolutionize business operations. AI-infused manipulators have the ability to “course correct,” allowing them to learn, over time, which methods and processes are the most effective. This capability can therefore lead to better efficiency and a higher degree of reliability than what their human counterparts can typically offer.
In the case of RE2 mobile manipulation systems, ‘intelligence’ is incorporated into systems using RE2’s proprietary autonomy algorithms, known as RE2 Intellect. Using data collected using RE2 Detect, RE2 Intellect fuses geometric computer vision with traditional machine learning and deep learning techniques to provide human-like decision processing. Unlike traditional autonomy algorithms, which are based on a single method that can only work in structured environments with controlled lighting, RE2 Intellect can handle anomalies typically experienced in unstructured, outdoor environments with variable lighting – similar to the way human beings perceive and process information. In other words, RE2’s proprietary autonomy algorithms operate in any environment.
In addition to thinking, systems must then be able to act. The complete process is often described as the OODA Loop: observe, orient, decide, act. In the case of RE2 systems, RE2 Detect is responsible for observing and orienting, while RE2 Intellect decides and acts. In humans, the brain is what closes this loop and determines the action based on perception. RE2 Intellect is the brain of RE2’s intelligent mobile manipulators.
How Do Robots Learn?
Teaching a machine to learn involves data, features and algorithms working together to build a robot’s “intelligence.” The more diverse the data, the better the results will be. Likewise, features, which are considered to be the building blocks of data, provide the algorithms with the characteristics or properties needed to perform a given calculation. Finally, choosing the correct algorithm for your data and features is critical. Algorithms can include supervised methods like classification and regression, or unsupervised methods like clustering or patterns search. The selected method directly affects the precision, performance and size of the final model.
The Ability to Communicate
Bridging the Gap Between Teleoperation and Full Autonomy
Like their human counterparts, communication is crucial for robots to successfully perform tasks. Human-robot interfaces are a critical part of this equation. Some tasks require a human to be in the loop, while others require some degree of oversight by a human operator. In some controlled, structured environments, tasks are able to completed 100% by a mobile robot.
Advancements in artificial intelligence, machine learning and computer vision are certainly driving the development of autonomous robots. Yet, despite these advancements, robots are still decades away from being able to handle the “edge cases” that exist in unpredictable and unstructured environments, such as the outdoors, where lighting conditions vary. As such, there is typically still a need to keep a human operator in the loop, whether through teleoperation or supervised autonomy (Figure 1). Human-robot interfaces become crucial to enable communication between a robot and its operator.
When teleoperation is needed, mobile manipulation systems must allow robot operators to remain completely in control while performing work from a remote location. At RE2, we have developed an Imitative Controller that essentially establishes a leader-follower teleoperation between the controller and the robot, providing the user with an intuitive system that allows fast and efficient time to proficiency.
When it comes to supervised autonomy, there is a spectrum of capabilities. At one end, for example, the operator can remain in control of decision-making, but the robot is responsible for direct action. Farther down the spectrum, the robot can make high-level decisions while keeping an operator in the loop solely for error handling and recovery. On the far end of the spectrum is what many consider to be the “Holy Grail” of autonomy, the fully autonomous system where no human supervision is required. However, for the majority of today’s robotics use cases in unstructured environments, supervised autonomy is the standard.
Mobile manipulations systems are transforming the way businesses plan, operate, and address operational challenges. In some cases, the technology is designed to serve as a force multiplier, increase throughput, improve efficiency, save lives, and automate anywhere.
As businesses across all industries navigate this 4th Industrial Revolution–one that is being driven by robotics, technology and artificial intelligence – companies like RE2 Robotics are focused on developing intelligent mobile manipulation systems that empower humans to do their work faster and safer. Mobile manipulation systems can automate tasks and enable remote work, thereby overcoming labor shortages, improving production, and saving lives.
When it comes time to implement a robotic system with manipulation capabilities within your business, be sure that it possesses the key features of a truly intelligent mobile manipulation system – the ability to move, interact, see, think, and communicate. If so, your organization will be able to unlock the ability to automate anywhere.
Editors Note: Robotics Business Review would like to thank RE2 for permission to reprint the original article series (with modifications) The series in EBook format can be found HERE.
Jorgen Pedersen, President and CEO, founded RE2 Robotics in July of 2001. Jorgen has contributed to the growth of the unmanned systems market for the past 20 years, beginning his career at Carnegie Mellon’s National Robotics Engineering Center (NREC). Jorgen has held several leadership positions within the Robotics industry. He served as the Chairman of the Robotics Division of the National Defense Industrial Association (NDIA).
Jorgen has also served in key roles such as a member of the Board of Trustees for NDIA and the Board of Directors for the National Advanced Mobility Consortium. Jorgen is the 2016 Carnegie Science Start-up Entrepreneur of the Year recipient. Jorgen was presented with the 2008 Army SBIR Achievement Award for the development of the Small Robot Toolkit for unmanned ground vehicles and the 2011 Tibbetts Award for SBIR Excellence. Jorgen holds a M.S. degree in Robotics and a B.S. degree in Electrical and Computer Engineering from Carnegie Mellon University. Connect with Jorgen on LinkedIn & Twitter.
- The Role of Supervised Autonomy in Mobile Manipulation
- How Robotics Can Benefit the Utilities Industry
- RE2 Robotics Awarded $3M to Develop Underwater Robotic Hand for U.S. Navy
- ULC Robotics, SGN to Develop All Electric Excavation Robot
- Combat Medics to Get Robotic Help From RE2 Grant
- RE2 Robotics Wins $2.9 Million U.S. Air Force Contract to Develop a Robotic Airfield Damage Recovery System
- Why Humans Will Remain at Center of an Automated Workforce