Replaying real life: how the Waymo Driver avoids fatal human crashes

Road safety is a major, global public health crisis. More than 1.3 million people die on the world’s roads every year, according to WHO. That’s more people than die from HIV/AIDS, and is equivalent to a passenger plane’s worth of people crashing every single hour—or one death every 30 seconds.

I’ve spent over 20 years working in crash avoidance research, in the belief that improved driving technology is the key to reducing these needless deaths.

But while we often hear that autonomous driving technology could make a dramatic difference, before today there has not been a published scenario-based study that we’re aware of that looks into how autonomous technology performs in scenarios that led to fatal crashes by human drivers.

Today, we’re releasing the results of a study into how the Waymo Driver might perform in such tragic situations, which builds on the research that we released in October. While the October study showed that the Waymo Driver was only involved in minor collisions over more than 6 million miles driven in reality on public roads, our most recent study shows how the Waymo Driver likely would have performed in the majority of fatal crashes that occured on the same roads over a 10 year period. The results are encouraging.

NHTSA statistics show that Maricopa County, which encompasses our service area, consistently makes the list of highest annual pedestrian fatality rate each year, demonstrating the unique focus on pedestrian crashes while driving on those streets. For our analysis, we collected information on every fatal crash that took place in Chandler, Arizona between 2008-2017*. We excluded crashes that didn’t match situations that the Waymo Driver would face in the real world today, such as when crashes occurred outside of our current operating domain. Then, the data was used to carefully reconstruct each crash using best-practice methods. Once we had the reconstructions, we simulated how the Waymo Driver might have performed in each scenario.

In total, the simulated Waymo Driver completely avoided or mitigated 100% of crashes aside from the crashes in which it was struck from behind, including every instance that involved a pedestrian or cyclist (20 simulations in total). This is the first time an autonomous technology company has shared its evaluation for how the system might perform in real-world fatal crash scenarios.

Figure 1: Results of the Waymo Driver’s performance in simulated tests

Carefully replaying fatal crashes

We started by simulating 72 fatal crashes as they occurred on public roads in our operating domain, which covers thousands of miles of road in southeast Phoenix. Since many of these crashes involved two vehicles, we ran separate experiments simulating the Waymo Driver in the role of each vehicle—first replacing the vehicle that initiated the crash (the “initiator”) with the Waymo Driver, and then replacing the vehicle that responded to the other vehicle’s actions (the “responder”) with the Waymo Driver. That left us with 91 simulations in total.

When we swapped in the Waymo Driver as the simulated initiator (52 simulations), it avoided every crash by consistent, competent driving, and obeying the rules of the road—yielding appropriately to traffic, executing proper gap selection, and observing traffic signals. Here, for example, you can see on the bottom of the screen that the simulated Waymo Driver avoids a reconstructed version of a real-life fatal crash by obeying the speed limit—and not running a red light, as the initiator did in real life:

Figure 2: Waymo Driver replacing the initiator. The Waymo Driver is driving the speed limit and stops at a traffic light.

But as we know, human drivers can make mistakes. They sometimes speed excessively through red lights, fail to yield on turns, and drive while tired, distracted or impaired. Since humans will be on the road for the foreseeable future, it’s important to understand how well autonomous driving systems perform in response to mistakes made by humans.

That’s one of the reasons why we ran simulations with the Waymo Driver in the role of the responder. If the Waymo Driver performed consistently better than the human drivers in these crashes, then that helps indicate the broader safety benefits our autonomous technology can achieve. In fact, that is exactly what our simulations suggest:

• When the Waymo Driver was placed in the responder role, it completely avoided 82% of simulated crashes. In fact, in the vast majority of events, it did so with smooth, consistent driving—without the need to brake hard or make an urgent evasive response.

• In another 10% of scenarios when the simulated Waymo Driver was the responder—all at an intersection when another vehicle turned across its path— it took action that mitigated the severity of the crash.

• Only 8% of responder crash simulations were unchanged. In all of these, the human-driven vehicle struck the rear of the simulated Waymo Driver when it was either stopped or traveling at a constant speed, giving the Waymo Driver little opportunity to respond.

In other words, even when a human driver did something to initiate a crash, such as running a red light, the simulated Waymo Driver avoided or mitigated the vast majority of these fatal crashes.

Replaying the same scenario discussed above, for example, the simulated Waymo Driver is approaching from the right of the screen, and has the right of way at a green light. But as it approaches the intersection, it spots the speeding car approaching from the bottom, predicts that it isn’t going to stop at the red light, and slows considerably until the speeder passes, avoiding the crash:

Figure 3: Waymo Driver replacing the responder. The Waymo Driver perceives and accurately predicts the initiator.

You can read more about our methodologies and our full results in our academic paper.

Advancing best-practice safety techniques

While these are early results, and we are cognizant that they only represent one of many indicators that should be used for assessing our performance under real-world circumstances, we believe they demonstrate the potential of AV technology to improve safety.

Given that the version of the Waymo Driver we tested in simulation is the same version that we currently have on the road, these results are a testament not only to the capabilities of the Waymo Driver, but also to the rigor of our more than a decade of experience and our safety readiness framework. Since 94% of crashes involve human error, we believe we have an opportunity to improve road safety by replacing the human driver with the Waymo Driver. This study helps validate that belief.

By sharing this data, we’re living up to our obligation to demonstrate the trustworthiness of the technology we’re building. We encourage other companies developing autonomous driving technology to do the same. We plan to continue providing additional evidence to allow people to reach their own conclusions about our safety readiness.

This research was made possible because of the thorough crash reports the Chandler Police Department and Arizona Department of Transportation (ADOT) prepare following fatal crashes, and we’re grateful to ADOT, Chandler Police Department, and Arizona Department of Public Safety for making these reports available to the public upon request. This ultimately helps make possible the improvements to roadway and vehicle design that have improved road safety over recent decades, and collaboration with public safety bodies is important for continued improvements.

The safety of autonomous driving technology is the deliberate result of careful development and thorough and continuous evaluation and refinement. This study provides yet more useful demonstration of its potential safety benefits.

_{* We chose 2017 as the cutoff date because it takes a significant amount of time to collect the records before running the study. Crashes that happen after 2017 may not have complete records given the timing.}