Automatic Emergency Braking Is Improved with Thermal Vision

·7 min read
Photo credit: Mark Vaughn
Photo credit: Mark Vaughn
  • Automatic Emergency Braking (AEB) systems will be on all cars soon.

  • Almost all AEB systems use radar and visual light to detect hazards. Teledyne FLIR wants to add a thermal image to the mix for a much safer system.

  • Seeing heat instead of light is kinda cool.

The last thing you want to do in your car is hit anything, and technology can help keep you from doing it. Automatic Emergency Braking systems can help drivers avoid tens of thousands of collisions a year between other cars, deer, moose, and, God forbid, pedestrians. Consider the wildlife alone: According to a 2020 State Farm annual study, nearly two million collisions between vehicles and large animals occur every year in the United States, representing approximately five percent of all reported motor vehicle collisions. These wildlife vehicle collisions (WVCs) result in 26,000 injuries to motorists, not to mention an average of $5000 per car in repairs.

Photo credit: Teledyne FLIR
Photo credit: Teledyne FLIR

Our own National Highway Safety Administration has updated progress among 20 carmakers that have pledged to install automatic emergency braking systems in all their cars by 2022. As of the end of 2019, four of those manufacturers have met that goal: Tesla, Volvo, Mercedes, and Audi. By now there are almost certainly many more.

You may be familiar with AEB.

“Automatic emergency braking systems apply the vehicle’s brakes automatically in time to avoid or mitigate an impending forward crash with another vehicle,” said the National Highway Safety Administration. “NHTSA believes AEB systems represent the next wave of potentially significant advances in vehicle safety. Dynamic brake support and crash-imminent braking are AEB systems that potentially save lives and reduce moderate and less-severe rear-end crashes that are common on our roadways.”

The AEB systems used now function with a combination of radar and visual cameras to determine exactly when to stomp on the binders. But a California company called Teledyne FLIR wants to incorporate thermal vision to the systems for added safety. FLIR says it can add such a component for “well under” $500 a car. The idea is to add thermal vision for better AEB operation at night, in inclement weather, or when driving directly into the sun.

At the Los Angeles Auto Show, we got a tour of the city’s tree-lined Elysian Park just north of Dodger Stadium well after sunset and just about the time the park rangers were locking the gates. FLIR had a demo-Lexus equipped with a pretty much standard AEB system that had been augmented with one of its own thermal cameras. The new system could blend both visual light and thermal imagery to create a far-more-thorough vision of what was out there in front of the car. We saw joggers, dog-walkers, dogs, people milling around in the roadway outside their parked cars. Everything but moose (which are still rare in LA).

“That, we think, is how you can really get the volume of thermal cameras (in consumer vehicles) up, and how you can really save lives,” said Chris Posch, Teledyne FLIR’s director of engineering for automotive. “Automatic emergency braking is in most cars today. And with the new transportation bill is now mandatory.”

Posch said the systems not only lack a thermal imaging element, but are only tested during daylight, which is only half-helpful.

“(How) is it in low light? When it’s raining? When it’s snowing? Or when there’s fog? How well will the system work in challenging situations?”

Photo credit: Teledyne FLIR
Photo credit: Teledyne FLIR

Existing AEB with radar and a visible-light camera mounted in the top center of the windshield detects a threat— a pedestrian or a car— and stomps on the brakes. That’s helpful but not enough.

“What we’ve done in the last couple years is really focused on how to make automatic emergency braking better. If I already have a radar, and that gives me my range, and I have a visible camera that identifies the target like a vehicle or a bike or a pedestrian or an animal, how do I make it better? Do I add LIDAR? Or do I add a thermal cam?”

LIDAR stands for Light Detection and Ranging. It’s a remote-sensing method that uses light in the form of a pulsed laser to measure ranges. It’s good but it’s very expensive. Thermal cameras are relatively inexpensive and can identify a warm-bodied object like a deer or a pedestrian and distinguish between that and something like a big, empty plastic trash bag or a tumbleweed blowing across the road.

“What we’ve done with the thermal camera is we put it right next to the visible camera,” Posch said. “So they work in concert together.”

With one camera next to the other you get depth perception, just like you get with your two eyeballs.

“What we’re doing is we’re saying, ‘If you have a thermal, and a visible camera combined with artificial intelligence, perception, machine learning, and then you have radar, they all work together. And so you’re driving down the road, maybe it’s at night, the cameras should hopefully identify the object. If it’s during the day, maybe the visible is better at it, at night or in challenging weather conditions, maybe the thermal is better, but they’re working in concert, they’re both saying, ‘Hey, that’s a pedestrian.’ Maybe the thermal says, ‘Absolutely. That’s a pedestrian.’ And then the radar says, ‘And it’s 50 feet away.’ And the car says, ‘And I’m going to hit it! Slam on the brakes.’ And so we think this is the best automatic emergency braking system that’s affordable right now.”

The system offered by Teledyne FLIR just does braking. It doesn’t check the lane next to it and then steer the car around an object. And while some cars today have thermal cameras, they are not integrated into the automatic emergency braking system.

Photo credit: Teledyne FLIR
Photo credit: Teledyne FLIR

FLIR is trying to get the word out about its AEB system because it will save thousands of lives every year.

“75% of pedestrian fatalities occur at night,” Posch said. “So do you want a system that’s going to work during the day? Or do you want a system that works day and night?”

Not only has FLIR advanced its system to display the thermal and the visible together—“fused,” as Posch says—it can use one or the other. “We can vary between all-visible and all-thermal.

“So you want to make an automatic emergency braking system, you put radar and you put a visible camera. Most cars are going to have that or have that now. How do you make it better? Hey, the (existing thermal/radar-only) systems have been shown by AAA, Consumer Reports, and IIHS that these things don’t work at night. ‘Hey, engineer, I need you to make our cars safer.’ ‘Okay, what am I gonna do? Well, I can put more visible cameras, I could put big spotlights on my car, I could put more radar, I could put a LIDAR on there. How much is that going to cost? And where am I going to put that? Or I could put a thermal camera right next to the visible camera and do perception just like I do with the visible camera on the thermal camera. And I know that this thermal camera is much less than $500.’ I'm not going to tell you how much it is. But it’s much less than $500 on an OEM car.”

So it sounds like a winner. Plus, in many states, there’s no law against looking for Bigfoot with a thermal camera. Or keeping those darn kids off your lawn. There are any number of things you could do with a cool thermal component to your rig.

Teledyne FLIR has all kinds of thermal vision tools. Check them out here. I myself just ordered a Teledyne FLIR unit that attaches onto an iPhone. It only cost me $349 (I paid full retail). I’ll be out spookin’ varmit as soon as I unpack it.

Share your thoughts in the comments below on whether you think a thermal camera would be helpful for you.