Collision avoidance effectiveness of an automated driving system using a human driver behavior reference model in reconstructed fatal collisions

Avoiding and mitigating any potential collision is dependent on (1) road user ability to avoid entering into a conflict, the (conflict avoidance effect), and (2) road user response should a conflict be entered, the (collision avoidance effect). This study examined the collision avoidance effect of an automated driving system (ADS) using a human
behavior reference model. The potential benefits of ADS technology have been widely anticipated, and reliable performance benchmarking methodologies for assessing ADS performance is an essential component of determining system readiness. The reference model used in this study reflects the response time and evasive action (referred to as collision avoidance) of a human driver that is non-impaired, with eyes on the conflict (NIEON) - a consistently performing, always-attentive driver that does not exist in the human population. Notably, the NIEON model is a tool
for evaluating collision avoidance only, as it inherits the pre-conflict behaviors (includes conflict avoidance) of the ADS that is being evaluated. In this way, the testing regime isolates the analysis to focus only on the collision avoidance effect.

Counterfactual simulations of the Waymo Driver and the NIEON model were run on potentially avoidable (excludes rear-end struck) responder collision scenarios (ADS is responding to some potential collision partners sudden, unexpected actions). All of these simulated events were reconstructed fatal crashes that occurred during a 10-year period in the Operational Design Domain of the Waymo ADS in Chandler, Arizona (Scanlon et al., 2021). In this area, users of the commercial Waymo One service can hail an ADS without a human in the driver’s seat.

A conflict must be entered to examine the collision avoidance effect using the NIEON model (or else there is no potential collision to examine performance). 16 total conflicts were identified for this study from a prior analysis by Scanlon et al. (2021). Regarding the collision avoidance effect, of 16 conflicts entered, 12 (75%) were prevented by the Waymo Driver, and 10 (62.5%) were prevented by the NIEON model. The NIEON Model mitigated an additional 5 collisions and did not mitigate 1 collision. In these 16 conflicts entered, 93% of serious injury risk was reduced by the Waymo Driver, whereas 84% of serious injury risk was reduced by the NIEON model. Further, in a case-by-case evaluation, the Waymo Driver’s collision avoidance led to reduced serious injury risk when compared
to the NIEON model in every simulated event.

The results of this paper demonstrate that a reference model like NIEON can be used to benchmark ADS responder performance in response to high-risk initiating behaviors performed by the current driving population.
Outperforming established benchmarks, like the Waymo Driver did versus the NIEON model in these simulations, could be used to demonstrate that fatal and serious-injury crashes will be reduced. The analysis also helps demonstrate the safety benefits of an ideal state of human driving in Chandler, Arizona, where many of these fatal collision outcomes would have been preventable.