January 24, 2012      

With cruise control, automatic wipers, and proximity sensors, the average automobile has steadily gained intelligence and automation over the decades. However, in the last few years that trend has accelerated, as the miniaturization and cost reduction of computers and sensors enable cars to more quickly detect and react to their environments and aid their human drivers in safe operation. Though many drivers may be hesitant to completely yield control of their cars to computers, the fact remains that from a technical standpoint, a mass-produced, fully autonomous vehicle could be just years away.

One reason for the rapid advance is that technologies developed for the military’s unmanned ground vehicles are rapidly becoming cost effective for consumer automobiles. Robotics researchers have seized the opportunity to use a ready-made mobile platform—the car—to pilot new behaviors for self-driving cars, driver interface systems, and multicar networks for systemic automation. This research is no longer limited to academic laboratories or DARPA-sponsored teams. Major car manufacturers and governments around the world are currently working to make autonomous cars a near-future reality.

Enabling Technologies

In their efforts to create autonomous vehicles, and the sophisticated algorithms used to guide them, researchers will have plenty of data to rely upon. Cars have evolved from primarily mechanical systems to complex, computer-controlled machines with a wide array of sensors that provide everything from remaining gas mileage to backup cameras.

As these technologies become standard even on entry-level vehicles, researchers have greater access to data about a car’s environment that can inform autonomous behaviors. Cruise control, for example—or more precisely what’s known as closed-loop speed control—has been available in cars for decades. With the recent decrease in cost of cameras and laser range finders, the closed-loop speed control has become adaptive, using these sensors to detect the distance to the car in front and modifying speeds appropriately if the front car slows down or speeds up.

Proximity sensors and ranging sensors have many other applications as well. In 2004, Toyota pioneered automatic parallel parking capability on Lexus vehicles, called the Intelligent Parking Assist System, using range finders on the front and back bumpers to autonomously park a car. These sensors, sometimes combined with cameras for sensing and for visual feedback to operators, have also been used to notify drivers of impending collisions while backing up.

Meanwhile, other sensors that are either already installed in cars or can be added for a small cost, are poised to provide additional capability to an autonomy system. Accelerometers and GPS receivers offer navigation with both relative and absolute positioning capability. Cameras can detect lane lines, traffic lights, street signs, and other markers. More complex arrays of range-finding sensors will enable more advanced obstacle and collision avoidance. Each of these sensor systems has a faster response time than a human being.

But while individual cars can be outfitted in many ways, one missing part of the autonomy technology equation is multivehicle management. Indeed, if cars are autonomous, it may make the most sense for researchers to treat traffic as a multi-agent “swarm” robotics problem. Humans manage their own “swarm” with visual indicators like brake lights and turn signals, along with a general understanding of human nature, which allows them to discern what other drivers are likely to do in a given traffic situation. Drawing upon humans’ ability to function within a swarm, cars could directly transmit information to each other about position, speed, or intention to pass or merge via wireless communication protocols.

Who, What, and Why

Robot car on highway

Individual drivers may not be ready for their cars to drive themselves, but many are excited about autonomous cars for the improvements they offer in highway safety, fuel economy, and road congestion. Such interest is not limited to passenger cars. In fact, there are myriad vehicles that could benefit from autonomous capabilities, such as those used in construction and for cargo handling. Military cargo and transport vehicles from companies such as Oshkosh Defense are already incorporating autonomous capabilities to reduce personnel needed in the field and, ultimately, prevent injuries from attacks and roadside bombs.

The 2004 and 2005 DARPA Grand Challenge and subsequent 2007 DARPA Urban Challenge were the first well-publicized autonomous car competitions. These cars, outfitted with GPS, LIDAR, cameras, and other sensors, successfully navigated a desert and later a suburban course. By virtue of DARPA sponsorship, however, the challenges focused on military applications, which did not necessarily resonate with the general public. Though no production vehicles were generated out of the program, it did spur development of lower-cost and higher-capability sensors that are becoming common on unmanned ground vehicles today.

The X Prize

The X Prize Foundation, which awards $10 million prizes to the winners of multiyear, cutting-edge technology challenges, hopes to address the public engagement problem. In 2012, the foundation will formally announce a new Automotive X Prize competition that will challenge participants to develop autonomous cars that can drive a track “better than the best human driver” by increasing safety and lowering the risk for accidents.

Erika Wagner, the senior director for exploration prize development at the X Prize Foundation, hopes this premise will engage more of the public and bring autonomous cars into the spotlight. Several European organizations are also sponsoring autonomous car research programs that culminate in competitions or major demonstration events.

The EUR $28 million HAVEIt (Highly Automated Vehicles for Intelligent Transport) project yielded 17 different autonomous car technology demonstrations in June 2011. HAVEit was dedicated to improving the safety of consumer cars. Volkswagen, for example, has pioneered a temporary autopilot on a VW Passat under the HAVEit program. The VW TAP, which is a semi-autonomous control system that must be supervised by a driver, will only work at speeds up to 80 mph; but in addition to controlling speed, the system maintains safe following distances, properly handles curves in the road, notices and obeys lane markings, and can safely pass other cars.

SARTRE (Safe Road Trains for the Environment) is another EU-funded framework led by an industry-academia partnership to develop autonomous vehicle “platoons.” These platoons would use a manned vehicle in the front, with semi-autonomous vehicles following behind. They could be put to use in construction and shipping applications where multiple vehicles have to traverse long distances. Using technology like adaptive cruise control, the vehicles in the back could maintain safe following distances from the vehicles in front of them, and use the lead vehicle to navigate.

Adaptive cruise control, temporary autopilots, and other developments are designed to augments driver awareness and response; other technologies, like detection or communication systems that can indicate another car in a blind spot during a lane change, can further assist drivers in operating their vehicles more safely. With the distractions available to drivers today—mobile phones, food, makeup application, conversations with other passengers, in-car entertainment systems—awareness augmentation can drastically reduce the more than 5.5 million yearly car accidents in the United States alone. These systems can even help drivers who are visually or otherwise impaired, offering a new level of mobility and autonomy for people who might normally be unable to drive.


For example, researcher Dennis Hong at Virginia Tech University, Blacksburg, Va., developed feedback systems based on vibrating gloves and driver seats that give a visually impaired vehicle operator a tactile sense of speed and other operating parameters. Hong thinks this technology could be used elsewhere in daily life to provide visually impaired people with nonvisual feedback about other systems. In a recent interview with CNN, Hong said his Robotics and Mechanisms Laboratory (RoMeLa) views autonomous vehicle research as a valuable tool for spinning off technology for the visually impaired and other physically challenged individuals.

Robot driver

The benefits of autonomous vehicle technology are not limited to safety, however. Even if a human driver operates a normal automobile in a safe, attentive way, he or she still may not be driving in the most fuel-efficient manner. Unnecessary hard accelerations, stop-and-go traffic conditions, and high speeds can reduce the fuel economy of a vehicle, resulting in more pollution and, on a large scale, higher fuel prices. If the autonomous systems in a car are optimized around fuel-efficient driving, not only will an individual car’s fuel economy be improved—which could help meet the 2025 goal of an average 54.5 mpg fuel economy for passenger cars—but stop-and-go conditions that cause widespread inefficiencies could also be reduced.

In fact, many bad traffic conditions could be improved with car-to-car communication or smart behavioral systems that recognize the most efficient way to merge into traffic or exit a freeway. While many human drivers may feel competitive with other drivers in this situation, an autonomous car can feed into a larger system for multivehicle coordination to avoid or limit traffic jams. This will require a robust vehicle operating system that can manage not only the internal vehicle network, but permit a vehicle to integrate into a larger system.

Ford is at the forefront, even taking advantage of software originally designed for robotics development. The Ford Fiestaware system stems from a collaboration between Ford and Microsoft based on the Microsoft Robotics Developer Studio. The system enables an Internet connection for passengers as well as a framework for intervehicle communication. A more mature but less capable Ford software product, called SYNC, based on the Microsoft Windows Embedded Automotive operating system, has been deployed on consumer cars since 2007. The system helps with voice control for navigation, phone calls, and entertainment system, as well as monitoring vehicle maintenance and safety information and providing 911 assist services. SYNC is already being augmented with Wi-Fi capability in a Ford collaboration with Broadcom.

Barriers to Adoption

Although the advent of autonomous cars might seem near, legal restrictions and government regulations, the lack of infrastructure, and consumer concerns may delay widespread adoption. Since autonomous car systems are still designed to be under supervisory control—wherein a human driver can take over at any time—there will be legal questions about liability in the event of an accident. Once they feel comfortable with the safety systems, human drivers might be tempted to take advantage of the vehicle’s semi-autonomous operation to inappropriately send text messages, talk on a mobile phone, or be distracted by any number of other things, potentially creating a situation in which they are unable to override the car’s systems when needed. Not only will the legal question need to be addressed early on, but strict regulations will need to be put in place to ensure adequate testing of any autonomous automobile software prior to a public offering.

Some governments have already taken steps toward regulating autonomous cars. The Nevada state legislature recently passed Assembly Bill 511, which includes a requirement that the state develop regulations for certification, testing, and licensing of autonomous cars. Interestingly, the lobbying force behind this bill was Google. While Google is not a car manufacturer, the company has already invested its software development capabilities into an autonomous car research project, headed by Sebastian Thrun, the Stanford University AI professor whose team won the 2005 DARPA Grand Challenge. The results of Google’s project have collectively logged more than 140,000 miles driving around California, and it seems Nevada will be a likely new test site.

But even when regulations are in place and consumers become convinced of autonomous systems’ safety, some drivers may not be wiling to give up control to sensors and software. In the United States in particular, many people actually enjoy the act of driving and feeling in control, which suggests that despite the benefits, any technology that removes that control may be a difficult sell in some cultures.

Depending on the direction autonomous car research takes, there may also be significant infrastructure needed to support new systems. If a central “controller” is required to determine optimal traffic flow, there is a good deal of hardware to be installed and many logistical challenges in doing so. As a starting point, the FCC has already approved a frequency allocation for vehicle-to-vehicle wireless communication. But that is only one part of a much larger picture.

The Way Ahead

For now, cars are likely to see only incremental additions of autonomy and intelligence. This will partially be for cost reasons, but it will also prevent drivers from suddenly having to—or wanting to—surrender the wheel to a computer. As long as the systems are considered augmentations rather than autonomous capabilities, drivers are more likely to maintain the attention level required for safe driving and be able to override the autopilot when necessary.

This incremental path also allows regulatory bodies and transportation authorities to understand how autonomous automobiles will be used and deployed in the future. The regulations, laws, and infrastructure needed to support autonomous cars, smart road trains, and other autonomous vehicle technology will have to be developed and implemented over time.

But it will not be long before personal automobiles, commercial vehicles, per-hour car rental services such as ZipCar, and even manned taxis are replaced with an efficiently dispatched fleet of autonomous cars ready to drive us to our destinations.