A fog blog: Understanding a challenge inherent to driving in San Francisco

If you check the weather for San Francisco, there's a one third chance you’ll see clouds or fog in the forecast, according to the National Oceanic and Atmospheric Administration (NOAA). To ensure Waymo’s Trusted Testers can safely get where they are going, whether it’s to work in the misty morning or back home to a hazy Haight at night, we’re engineering the Waymo Driver to handle the challenges of fog.

The low-hanging clouds are iconic to the city, but they also pose several challenges for drivers, both human and autonomous. According to the U.S. Department of Transportation, more than 38,700 vehicle crashes occur in fog every year, resulting in over 600 fatalities and 16,000 injuries. Fog is finicky—it comes in a range of densities, it can be patchy, and can affect a vehicle’s sensors differently. Sometimes, we see fog thick enough that it deposits tiny droplets on surfaces, like our optical sensors, while other times its microdroplets just form on our sensors, impacting how far one can see. Fog can also trap other particles, such as smoke from wildfires or pollution from gasoline, creating a more opaque fog or smog that’s hard to see through.

Whether rain or shine, The Waymo Driver uses a single, integrated system consisting of lidars, cameras, and radars to perceive the world around it. With it, the Waymo Driver can see 360 degrees in every direction, day or night, and as far as three football fields away. Like a human’s own senses, Waymo’s autonomous driving technology is more powerful as a whole than the sum of its parts; each sensor complements the others, providing our stack with a richer view of the world that makes our vehicles safer and more capable. For example, while cameras are great for detecting details like reading a sign or observing the color of a stop light, like a human’s own two eyes, camera visibility can be limited by the fog.

Our fifth-generation, state-of-the-art imaging radar complements our lidar and cameras with its unmatched resolution, unparalleled field of view, and its unique ability to instantaneously measure velocity. And since it uses microwaves instead of light, it sees through weather conditions like fog and mist. We’ve also engineered a robust sensor cleaning system that employs various mechanisms, from utilizing hydrophobic films to shaping natural air flow around the sensors and utilizing a pulsed air-puffer and wiper system to keep our sensors’ surfaces clean from fog droplets, rain, dust, and other road grime.

The Waymo Driver also adheres to the many unofficial rules of the road that the National Weather Service recommends both human and autonomous drivers should do in the fog, such as leaving plenty of space from the vehicle in front to account for sudden stops or changes, and ensuring it stays in the proper lane.

We knew we would have to teach The Waymo Driver how to navigate through the fog more than a decade ago when we first began driving in the Cool Gray City of Love. In any city we enter, solving for weather can be especially challenging because it's seasonal, very local, and patterns change over time.

To ensure the Waymo Driver is prepared for any conditions, we conduct weather testing to collect comprehensive data in order to test, train, and validate our autonomous driving system under extremely high scrutiny (as outlined in our Safety Framework). This is why you might see some cars driven by the Waymo Driver with additional sensors on it. These funky looking horns you may see around town are actually mobile weather stations that help Waymo’s Weather Team collect data on fog to help understand the density of the droplets.

Performing our own weather testing also allows us to further stretch our learnings. While it is easy for human drivers to tell the difference between when it’s sunny or foggy, it’s something we have to teach our autonomous Driver to recognize. Using our learnings we have been able to co-train our sensors to detect local weather around our vehicles and adjust its behavior accordingly like when it’s really foggy, which is especially helpful in San Francisco’s microclimates. We can also create realistic perception simulations, like in Simulation City, to augment our existing datasets and continue testing the Waymo Driver, even when the desired weather conditions aren’t present in the real world.

 The fog is as much a part of San Francisco as the hills and the Golden Gate Bridge and so we have always embraced the challenge. As we continue developing the Waymo Driver to handle a more diverse range of environments, we also continue heavily investing in learning about other types of weather—from the rainy season in Miami, Florida and Seattle Metro Area to winters on the upper peninsula of Michigan and soon, New York City. We will continue chasing the weather.