Thermal Cameras for Safer AEB
Testing Automatic Emergency Braking with FLIR Thermal Cameras
Three test cases were conducted in both daylight and darkness, giving six datasets and 35 total test runs using an adult Euro NCAP Pedestrian Target (EPTa). Test results were promising. In all runs for all six test cases, the AEB system successfully brought the VUT to a stop before reaching the EPTa.
Fused Thermal Camera and Radar Pedestrian Detection and Automatic Emergency Braking (AEB) System
Recently, FLIR Systems Inc. contracted VSI Labs to develop a proof-of-concept AEB system that fused perception data from a FLIR ADK’s ethernet-based Boson® thermal camera and a Delphi ESR 2.5 radar. The radar provided range data and the thermal camera provided object detection and classification. VSI recently conducted initial performance tests based on Euro NCAP requirements at the American Center for Mobility (ACM). Here is the full executive summary:
Executive Summary
Current typical automatic emergency braking (AEB) or pedestrian detection systems rely on systems using radar and, in some cases, visible-light cameras. There are several common conditions in which these sensors can have difficulty detecting a pedestrian. An October 2019 study by AAA tested several production AEB systems and describes many such scenarios. In nighttime conditions, the vehicle under test (VUT) traveling at 25 mph hit the soft pedestrian target (SPT) 100% of the time. During daylight tests, the VUT driving 20 mph struck the stationary SPT 60% of the time. 1
The following report describes a proof-of-concept thermal-camera-and-radar fused AEB algorithm that stopped the VUT in 100% of daytime and nighttime testing.
Pedestrian detection and AEB systems are critical to improving road safety. According to the National Highway Traffic Safety Administration (NHTSA), on average, a pedestrian is killed every 88 minutes in traffic crashes in the United States. In 2017, nearly 6,000 pedestrians’ lives were lost in the United States, an increase of 35% since 2008. All other traffic deaths decreased by 6% during that same time period. NHSTA also reported that 75% of all pedestrian deaths occurred after sunset.2
Figure 1: VSI Lab’s 2018 test vehicle with fused thermal camera and radar pedestrian detection
A thermal camera provides complementary data to RGB cameras and radar. Thermal cameras “see” heat, not reflected light, and perform well both in daylight and in darkness. They can see through sun and headlight glare, most fog, and up to 4x farther than typical headlights illuminate in darkness. Given thermal camera’s unique capability, FLIR Systems Inc. contracted VSI Labs to develop a proof-of-concept automatic pedestrian detection system that fuses radar and FLIR thermal camera data to detect and estimate the distance of a pedestrian from the front of a test vehicle. The VUT is programmed to automatically stop if a pedestrian is in the path of the vehicle at a proximity that it determines to be an
emergency-stop distance.
Figure 2: Proof-of-concept testing based on three Euro-NCAP test cases (day and night)
Initial tests were completed in December 2019 at the American Center for Mobility (ACM) near Detroit, MI. The test design was based on the Euro NCAP, but not all testing requirements were met as the winter weather was colder than the specified testing temperature range, roadways had snowy, wet, or slick surfaces and wind interfered with the test fixtures. Three test cases were conducted in both daylight and darkness, giving six datasets and 35 total test runs using an adult Euro NCAP Pedestrian Target (EPTa).
1. EPTa stationary in the middle of the VUT’s lane in Car-to-Pedestrian Longitudinal Adult 50% (CPLA-50) tests.
2. EPTa crossed in front of the VUT from the roadside in Car-to-Pedestrian Far Side Adult 50% (CPFA-50) tests.
3. EPTa crossed in front of the VUT from an obstructed position in Car-to-Pedestrian Far Side Adult Obstructed 50% (CPFAO-50) tests.
Test results were promising. In all runs for all test cases, the AEB system successfully brought the VUT to a stop before reaching the EPTa. Additional testing is recommended and planned for spring/summer of 2020 following AEB algorithm optimization, EPTa heating improvements and when weather is in test parameters.
Figure 3: AEB test result summary
______________
1: https://www.aaa.com/AAA/common/aar/files/Research-Report-Pedestrian-Detection.pdf
2: National Highway Safety Traffic Administration, "nhtsa.gov," March 2019. [Online].
https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812681#targetText=In%202017%20there%20were%205%2C977,almost%20115%20people%20a%20week