A Comparison of Lidar and Camera-Based Lane Detection Systems

By Jordan Britt, David Bevly, and Christopher Rose

Nearly half of all highway fatalities occur from unintended lane departures, which comprise approximately 20,000 deaths annually in the United States.  Studies have shown great promise in reducing unintended lane departures by alerting the driver when they are drifting out of the lane. At the core of these systems is a lane detection method typically based around the use of a vision sensor, such as a lidar (light detection and ranging) or a camera, which attempts to detect the lane markings and determine the position of the vehicle in the lane. Lidar-based lane detection attempts to detect the lane markings based on an increase in reflectivity of the lane markings when compared to the road surface reflectivity. Cameras, however, attempt to detect lane markings by detecting the edges of the lane markings in the image. This project seeks to compare two different lane detection techniques-one using a lidar and the other using a camera. Specifically, this project will analyze the two sensors’ ability to detect lane markings in varying weather scenarios, assess which sensor is best suited for lane detection, and determine scenarios where a camera or a lidar is better suited so that some optimal blending of the two sensors can improve the estimate of the position of the vehicle over a single sensor.

Lidar-based lane detection

The specific lidar-based lane detection algorithm for this project is based on fitting an ideal lane model to actual road data, where the ideal lane model is updated with each lidar scan to reflect the current road conditions. Ideally, a lane takes on a profile similar to the 100-averaged lidar reflectivity scans seen in Figure 1 with the corresponding segment.
Figure 1. Lidar reflectivity scan with corresponding lane markings.

Note that this profile has a relatively constant area bordered by peaks in the data, where the peaks represent the lane markings and the constant area represents the surface of the road.  An ideal lane model is generated with each lidar scan to mimic this averaged data, where averaging the reflectivity directly in front of the vehicle generates the constant portion and increasing the average road surface reflectivity by 75 percent mimics the lane markings.  This model is then stretched over a range of some minimum expected lane width to some maximum expected lane width, and the minimum RMSE between the ideal lane and the lidar data is assumed to be the area where the lane resides. For additional information on this method, see Britt, Rose & Levy, September 2011.

Camera-based lane detection

The camera-based method for this project was built in-house and uses line extraction techniques from the image to detect lane markings and calculate a lateral distance from a second-order polynomial model for the lane marking in image space. A threshold is chosen from the histogram of the image to compensate for differences in lighting, weather, or other non-ideal scenarios for extracting the lane markings. The thresholding operation converts the image into a binary image, which is followed by Canny edge detection. The Hough transform is then used to extract the lines from the image, fill in holes in the lane marking edges, and exclude erroneous edges. Using the slope of the lines, the lines are divided into left or right lane markings. Two criteria based on the assumption that the lane markings do not move significantly within the image from frame to frame are used to further exclude non-lane marking lines in the image. The first test checks that the slope of the line is within a threshold of the slope of the near region of the last frame’s second-order polynomial model. The second test uses boundary lines from the last frame’s second-order polynomial to exclude lines that are not near the current estimate of the polynomial. second-order polynomial interpolation is used on the selected lines’ midpoint and endpoints to determine the coefficients of the polynomial model, and a Kalman filter is used to filter the model to decrease the effect of erroneous polynomial coefficient estimates. Finally, the lateral distance is calculated using the polynomial model on the lowest measurable row of the image (for greater resolution) and a real-distance-to-pixel factor. For more information on this camera-based method, see Britt, et al.

Figure 2. Camera-based lane detection (green-detected lanes,blue-extracted lane lines, red-rejected lines).

Testing

Testing was performed at the NCAT (National Center for Asphalt Technology) in Opelika, Alabama, as seen in Figure 3.  This test track is very representative of highway driving and consists of two lanes bordered by solid lane markings and divided by dashed lane markings.  The 1.7-mile track is divided into 200-foot segments of differing types of asphalt with some areas of missing lane markings and other areas where the lanes are additionally divided by patches of different types and colors of asphalt.

Figure 3. NCAT Test Facility in Opelika, Alabama.

A precision survey of each lane marking of the test track as well as precise vehicle positions using RTK GPS were used in order to have a highly accurate measurement of the ability of the lidar and camera to determine the position of the vehicle in the lane. Testing occurred only on the straights, and the performance was analyzed on the ability of the lidar and camera to determine the position of the lane using metrics of mean absolute error (MAE), mean square error (MSE), standard deviation of error (σ­_error), and detection rate. The specific scenarios analyzed included varying speeds, varying lighting conditions (noon and dusk/ dawn), rain, and oncoming traffic. Table 1 summarizes the results for these scenarios. For additional results, please see [8].

Table 1. Lidar and camera results for various environments.

Additional testing on the effects of oncoming traffic at night was examined by parking a vehicle on the test track at a known location with the headlights on. Figure 4 shows the lateral error with respect to closing distance where a positive closing distance indicates driving at the parked vehicle, and a negative closing distance indicates driving away from the vehicle. Note that the camera does not report a solution at -200 m, which is due to track conditions and not the parked vehicle.

Figure 4. Error vs. Closing Distance.

Based on these findings it would appear that the camera provided slightly more accurate measurements than the lidar while having a decrease in detection rate. Additionally the camera performed well in the rain where the lidar experienced decreased detection rates.

References

Frank S. Barickman. Lane departure warning system research and test development. Transportation Research Center Inc., (07-0495), 2007.

J. Kibbel, W. Justus, and K. Furstenberg. using multilayer laserscanner. In Proc. Lane estimation and departure warning Proc. IEEE Intelligent Transportation Systems, pages 607 611, September 13 15, 2005.

P. Lindner, E. Richter, G. Wanielik, K. Takagi, and A. Isogai. Multi-channel lidar processing for lane detection and estimation. In Proc. 12th International IEEE Conference on Intelligent Transportation Systems ITSC ’09, pages 1 6, October 4 7, 2009.

K. Dietmayer, N. Kämpchen, K. Fürstenberg, J. Kibbel, W. Justus, and R. Schulz. Advanced Microsystems for Automotive Applications 2005. Heidelberg, 2005.

C. R. Jung and C. R. Kelber, “A lane departure warning system based on a linear-parabolic lane model,” in Proc. IEEE Intelligent Vehicles Symp, 2004, pp. 891–895.

C. Jung and C. Kelber, “A lane departure warning system using lateral offset with uncalibrated camera,” in Intelligent Transportation Systems, 2005. Proceedings. 2005 IEEE, sept. 2005, pp. 102 – 107.

A. Takahashi and Y. Ninomiya, “Model-based lane recognition,” in Proc. IEEE Intelligent Vehicles Symp., 1996, pp. 201–206.

Jordan Britt, C. Rose, & D. Bevly, “A Comparative Study of Lidar and Camera-based Lane Departure Warning Systems,” Proceedings of ION GNSS 2011, Portland, OR, September 2011.