The Waymo Driver Handbook: How our highly-detailed maps help unlock new locations for autonomous driving

For any emerging technology to be trusted, it helps to first be understood. In the past, people could see how their cars worked, looking under the hood and tinkering with them with the help of a user manual. In 2020, vehicles have so much technology that they’ve become difficult for the general public to comprehend. We want to change that. With this blog series, we’ll unpack the different parts of our technology stack to explain the fundamentals of self-driving technology. How does the Waymo Driver perceive the world? How does it learn to understand its surroundings? How can it predict the intentions of other drivers and pedestrians? And how does it keep our riders safe? We’re starting with one of the foundational questions: how does a self-driving car know where it is? 

A self-driving vehicle uses multiple sensors which work together as its eyes and ears, providing information about the world around it, from other vehicles and moving objects, to traffic lights and vegetation lining the sides of the road. In addition to this powerful sensor data, Waymo’s self-driving vehicles also have the advantage of knowing what the road ahead looks like. The Waymo Driver gets this knowledge from its highly-detailed, custom maps that provide our vehicles important context in every location we drive.

While wrestling with maps and folding them up may bring nostalgic memories for some, the maps we create for the Waymo Driver are orders of magnitude more complex. The information our vehicles care about is quite different from someone trying to find their way to a restaurant. For example, it’s far more important for the Waymo Driver to know the speed limit than the name of the road. We’ve built an incredibly detailed set of mapping technologies that help our cars navigate places even where GPS struggles, like tunnels or between skyscrapers.

_{Maps are like memory for the Waymo Driver that complements what it’s actually seeing on the road. In this video, the Waymo Driver knows it’s a one way street, but detects another vehicle driving the wrong direction which allows it to quickly adjust its speed and trajectory.}

These maps give the Waymo Driver a deep understanding of its environment, from road types and the distance and dimensions of the road itself, to other features like lane merges, stop signs, crosswalks, and so much more. This prior knowledge of what each mile ahead looks like, combined with our powerful sensing and perception capabilities, allows us to drive smoother and more predictably. For example, when the Waymo Driver approaches an intersection, not only can it sense a car that might cut across its path, but because of our custom maps, it also knows that vehicle has a stop sign. If the Waymo Driver detects the car is going too fast to stop, our robust neural nets can respond to it very quickly. This results in a safer and more comfortable experience for our riders and other road users.

Building custom maps for the Waymo Driver

To create a map for a new location, our team starts by manually driving our sensor equipped vehicles down each street, so our custom lidar can paint a 3D picture of the new environment. This data is then processed to form a map that provides meaningful context for the Waymo Driver, such as speed limits and where lane lines and traffic signals are located. Then finally, before a map gets shared with the rest of the self-driving fleet, we test and verify it so it’s ready to be deployed.

Just like a human driver who has driven the same road hundreds of times mostly needs to focus only on the parts of the environment that change, such as other vehicles or pedestrians, the Waymo Driver knows permanent features of the road from our highly-detailed maps and then uses its onboard systems to accurately perceive the world around it, focusing more on moving objects. Of course, our streets are also evolving, so if a vehicle comes across a road closure or a construction zone that is not reflected in a map, our robust perception system can recognize that and make adjustments in real-time.

Over 25 cities across the USA: What we learn through mapping new locations

While the process of creating our custom maps is similar for all geographies, every place is, in many ways, unique. Traffic laws differ from city to city, so we work closely with local officials and traffic engineers to become experts at local driving rules to convey that information to our vehicles.

_{A Safety Zone in San Francisco (Image credit: Google Maps)}

For example, in San Francisco there are special areas called Safety Zones where buses and streetcars drop off and pick up passengers. If there is a bus stopped near a Safety Zone and it is not otherwise signed, it’s illegal for a car to drive more than 10 mph past the bus. We’re encoding Safety Zones into our map as a base layer, which helps ensure we are abiding by local laws.

There are many factors we look at when mapping new cities: the width of lanes, bicycle lanes, reversible lanes (think of the Golden Gate Bridge adjustable lanes that can change their direction with the traffic flow), and more. Some nuances are even more subtle. For example, many stores have roller shutter doors and curb cuts that make them almost look like driveways. Knowing which of them are actual driveways helps the Waymo Driver understand whether other cars could be emerging from these areas.

The situations we encounter mapping a new area also help improve other parts of our self-driving system. When our mapping vehicles come across a rare or complex situation, we can use this experience to help create our map and also train our perception and behavior prediction models. For example, in Los Angeles’ Fashion District, hundreds of mannequins stand outside the stores and could be mistaken for pedestrians who want to cross the street. Capturing this early means we can better prepare the Waymo Driver to handle unusual, real-world situations. Similarly, observing different traffic behavior patterns helps us evaluate and train our behavior prediction system.

_{Mannequins in front of stores in Los Angeles}

Keeping our maps up to date

Our streets are ever-changing, especially in big cities like San Francisco and Los Angeles, where there’s always construction going on somewhere. Our system can detect when a road has changed by cross-referencing the real-time sensor data with its on-board map. If a change in the roadway is detected, our vehicle can identify it, reroute itself, and automatically share this information with our operations center and the rest of the fleet in real time.

We can also identify more permanent changes to the driving environment, such as a new crosswalk, an extra vehicle lane squeezed into a wide road, or a new travel restriction, and quickly and efficiently update our maps so that our fleet has the most accurate information about the world around it at all times.

We’ve automated most of that process to ensure it’s efficient and scalable. Every time our cars detect changes on the road, they automatically upload the data, which gets shared with the rest of the fleet after, in some cases, being additionally checked by our mapping team.

As city landscapes now face new challenges brought by the COVID-19 pandemic and continue to adapt to our new lifestyles with shared spaces and slow streets, we have also proactively gone out to map and capture these changes.

We’ve been building highly-detailed maps since we started driving more than a decade ago and, combined with our state-of-the-art onboard autonomy system, they’ve been instrumental to helping us get people and things where they are going safely. In the next blog posts we will take a closer look at the other parts of our self-driving technology stack, explain how they enable the Waymo Driver to safely cope with challenges on the road, and share interesting insights from behind the scenes. Stay tuned!