This article was republished with permission from Comet Labs, a VC fund dedicated to accelerating intelligent machine (robotics and artificial intelligence) innovation.

Imagine standing in a dark room, and the only way you can sense the environment around you is by reaching out to objects with a stick. First, you reach straight in front of you, and the stick goes 12 feet before hitting a solid object.

Then you extend the stick to your right, 8 feet until it stops. Next you try to your left, and you get 12 feet. Behind you, the stick goes 18 feet. Now, even though you can’t see anything, and you haven’t moved, you have some information about the room.

If you repeated this hundreds, or thousands of times in different directions (and had a really good memory), you would be able to produce a rough representation of the room based on how far away objects are from you.

If you angled the stick above and below horizontal, you would even be able to “see” objects around you like chairs and doors, based on their outlines. From this information, you could produce something called a “point cloud”, which is a set of points in a 3D coordinate system. With enough points, you could make a really detailed point cloud of a room.

Lidar (a portmanteau of “light” and “radar” which also stands for Light Detection and Ranging) is a sensor designed to quickly build these point clouds. By using light to measure distance, Lidar is able to sample points extremely quickly? – ?up to 1.5 million data points per second. This sampling rate has enabled the technology to be deployed on applications such as autonomous vehicles.

How Lidar Works

Lidar measures the time of flight of a pulse of light to be able to tell the distance between the sensor and an object. Imagine starting a stopwatch when the pulse of light is emitted, and then stopping the timer when the pulse of light returns (from being reflected off the first object it encounters). By measuring the “time of flight” of the laser, and knowing the speed that the pulse travels, the distance can be solved. Light travels at 300 million meters per second (186,000 miles per second), so very high precision equipment is needed to be able to generate data about distance.

Lasers as a fancy measuring stick. (Photo Credit: UC Berkeley)

To produce complete point clouds, the sensor must be able to sample the entire environment very quickly. One way that Lidar does this is by using a very high sampling rate on the individual emitters/receivers. Each one emits tens, or hundreds of thousands of laser pulses every second. That means, within 1 second, as many as 100,000 laser pulses complete a round trip from the emitter on the Lidar unit, out to the object being measured, and back to the receiver on the Lidar unit, near the emitter. Large systems have as many as 64 of these emitter/receiver pairs, or “channels”. Multiple channels enable the system to generate more than a million data points per second.

However, 64 stationary channels aren’t enough to be able to map an entire environment?- ?it would just give very clear resolution in very focused areas. Making more of these channels is expensive due to the precision required in the optics, so increasing the number of channels above 64 just increases cost faster. Instead, many Lidar systems use rotating assemblies, or rotating mirrors to enable the channels to sweep around the environment 360 degrees. Common strategies include angling each of the emitters and receivers above or below horizontal to blanket more of the environment in the field of view of the lasers. The Velodyne 64 channel Lidar system, for example has a 26.8 vertical field of view (the rotation gives it a 360 horizontal field of view). From 50 meters away, this Lidar could see the top of an object which is 12 meters tall.

Velodyne HDL-64E Lidar System (Photo Credit: Velodyne)

Below, you can see the clear bands of points corresponding to the different channels of the Lidar unit -??bands in the point cloud? – ?as the data fidelity drops off with distance. While it isn’t perfect, the higher resolution is available for closer objects, since the angle between emitters (for example, 2 degrees) results in an increased spacing between these bands as distance to the sensor increases.

Applications of Lidar Systems

The point cloud can be used to reproduce 3D models of landscapes or environments. A few applications include:

• Geological mapping/imaging to monitor erosion or other changes
• Monitoring growth of plants and trees
• Doing surveying work for construction projects
• Making accurate volumetric estimates of landfills

Probably the most common application, and one that you may have seen, is a Lidar system integrated in an autonomous vehicle? – ?such as this episode of Top Gear in which a truck uses a Lidar system to autonomously navigate off-road.

An autonomous truck on Top Gear. (Photo Credit: BBC America)

Below, you can see the point cloud of the landscape, and additional features (green and red boxes that delineate between objects that can be driven over?- ?like plants, and objects that shouldn’t be driven over? – ?like rocks, trees and cars). There are other software elements that take in the raw point cloud, and categorize the obstacles.

An autonomous truck on Top Gear. (Photo Credit: BBC America)

Lidar systems have found their way into humanoid robots as well? – ?as can be seen in this video from Boston Dynamics. The robot uses different sensors, like optical cameras to see the QR-like code in addition to the Lidar system in the robot’s head.

Another example of a Lidar application is a sensor’s axis mounted horizontally on a drone to produce a contour map of the ground. Point cloud data from the Lidar is combined with position data on the drone itself to produce these contours.

Phoenix Aerial Systems drone mapping the ground (Credit: Phoenix Aerial Systems)

The Challenges

Since Lidar is based on measuring the time it takes for a laser pulse to return to the sensor, highly reflective surfaces pose issues. Most materials have rough surfaces on a microscopic level, and scatter light in all directions. A small portion of this scattered light makes its way back to the sensor, and is sufficient to generate the distance data. If a surface is very reflective, however, the light is reflected coherently away from the sensor, and the point cloud is incomplete for that area.