Cities, freeways, airports: How we've built a scalable autonomous driver

Being an experienced driver means knowing how to handle a variety of road situations that can happen anytime, anywhere. At Waymo, we’re building The World’s Most Experienced Driver™. With over 20 million autonomous public road miles under its belt, the Waymo Driver is prepared to navigate the challenges of busy city streets, high-speed multi-lane roads, chaotic parking lots and more. Right now, the Waymo Driver is simultaneously providing fully autonomous rides in two locations—Phoenix’s East Valley and San Francisco—and is ready to scale further. In the coming weeks, we will begin rider-only trips —with no human driver behind the wheel— with our employees in Downtown Phoenix. Additionally, the Waymo Driver is beginning to drive at Phoenix Sky Harbor International Airport, one of the 10 busiest airports in the world, with an autonomous specialist present.

Our ability to safely accelerate how we scale across multiple locations is the result of more than a decade of building, testing, and deploying the Waymo Driver across different use cases and geographies. For example, our years of experience providing a 24/7, fully autonomous ride-hailing service in Phoenix's East Valley enabled us to remove a human from behind the wheel in San Francisco safely just about a year after we started to ramp up our operations on the 5th-generation Waymo Driver in SF. Just as our learnings from the East Valley transferred to San Francisco, the capabilities we gained from navigating dense urban environments are transferring smoothly to other places such as Downtown Phoenix and Sky Harbor Airport.

Using Machine Learning to Build a Generalizable System

For both a human and autonomous driver, the fundamentals of driving are largely the same wherever they go. For example, any driver should be prepared to navigate around construction zones no matter if it’s a Class 8 truck carrying cargo on I-45 or a passenger car ferrying riders in San Francisco or Downtown Phoenix. Rather than just validating our technology for specific routes, we've prepared it to handle the myriad of situations that can happen on many different types of streets, by testing it in a diverse range of environments across 13 U.S. states. Our approach is to enable our software to generalize from place to place, allowing us to both make our Driver better in all locations at once and to focus our attention on new aspects of a territory—like different climates, specific road rules, or other local nuances—which leads to quicker scalability.

Autonomously navigating diverse driving environments requires building a system that can handle the long tail of rare events that can occur on and near roadways. We have seen a duck on a leash crossing the street, cyclists carrying stop signs on their backs, a person walking their dog on the freeway, a motorcycle speeding past us at 70mph with no rider on the back of it, and so much more. The Waymo Driver’s experience on public roads is equivalent to multiple human lifetimes of driving, spanning from everyday mundane situations to events that only happen once in a million miles. To teach our Driver to handle the whole gamut of on-road situations, we use advanced machine learning (ML) models across our entire software stack: Perception, Behavior Prediction, and Motion Planning. 

The Waymo Driver's powerful ML models learn from each mile from our entire fleet. Our team gets creative to train our ML systems, locating and applying the datasets that are the most interesting for ML training from the tens of millions of miles our Driver has autonomously navigated in the real world. For example, we have developed innovative techniques to automatically identify interesting interactions and augment data, enabling us to create focused datasets to learn more effectively and efficiently. When combined with our suite of simulation tools—which are now so advanced that we can generate fully synthetic data at a real-world scale with remarkable realism—it significantly speeds up our data generation rates, in turn dramatically improving our iteration speeds.

Training ML models is only part of the equation — there are many pieces that all have to work seamlessly together to solve this incredibly hard problem. One important part is evaluation, which is a critical step before we begin our operations without a human in the driver's seat. We use a comprehensive set of safety methodologies and a range of methods from simulation to structured testing to driving in the real world to evaluate the Waymo Driver's performance and determine its safety readiness. Operating in multiple domains and environments, from bustling city streets to high-speed freeways to now airport roads, gives us a nuanced understanding of which parts of evaluation are fundamental across domains and which are context-specific, helping us build a safe, generalizable stack.

Scaling with the Flywheel Effect

As the fastest growing city in the U.S. Phoenix represents a diverse population with unique transport needs. With the rapidly increasing number of residents, heavy traffic, and the sheer amount of construction taking place, vibrant Downtown Phoenix presents many similar challenges to what we’ve seen in other cities like San Francisco. Being able to apply learnings directly from one city to another unlocks a powerful flywheel effect and gives us a significant head start in every new location, without overfitting our technology to a single territory.

At Sky Harbor, the Waymo Driver will apply learnings it has gained from driving in other locations to the unique challenge of airport pick up and drop off. That includes lessons our Driver learned from navigating the bustling streets of San Francisco, as well as Phoenix's East Valley's crowded shopping centers and high speed roads. Our Jaguar I-PACEs will drive autonomously at Sky Harbor Airport at all hours of the day and night, with an autonomous specialist present at first, to help us learn from the airport’s busy roads and crowded parking lots. We’ll focus our operations on the area around the 44th Street PHX Sky Train® Station, which is near the pickup and drop-off location for other ride-hail services. We will begin with Waymo employees hailing rides between Downtown Phoenix and Sky Harbor, eventually opening up to members of the public via our Trusted Tester program.

What we learn at Sky Harbor Airport will benefit us in other locations and help accelerate our progress. As we bring the Waymo Driver to more people in more places, we will be able to deploy in every new territory even faster and focus on some inherent challenges specific to each location—like driving in snow in New York City, human-like mannequins in Los Angeles' Fashion District, or the notorious Pittsburgh left. With our robust infrastructure and state-of-the-art ML systems, the Waymo Driver is well prepared to tackle the challenges down the road.