Announcing the 2024 Waymo Open Dataset Challenges

Every week we are serving tens of thousands of Waymo One rides across Phoenix, San Francisco, Los Angeles, and soon, Austin. These real-world deployments are all underpinned by our long–term commitment to cutting-edge machine learning research. 

Autonomous vehicles are the most mature manifestation of AI in the real world, and Waymo is dedicated to contributing to the latest advances in machine learning research. Our progress on the road is in many ways enabled by the same type of data we make available for research to the scientific community via the Waymo Open Dataset, one of the largest and most diverse autonomous driving datasets ever released. Today we’re excited to announce the start of our 2024 Waymo Open Dataset Challenges, which will run through May 2024. 

Since launching the Waymo Open Dataset in 2019, over 40,000 researchers have used it as a high-quality tool to explore topics in autonomous vehicles, robotics, as well as computer vision and machine learning more broadly. In the last five years, scientists have published over 2,000 academic papers either citing the Waymo Open Dataset or using it as a benchmark for their own research. 

Our 5th Challenge edition will cover a range of fundamental research topics in the autonomous driving domain and include: 

• 3D Semantic Segmentation: Given one or more lidar range images and the associated camera images, produce a semantic class label for each lidar point.

• Motion Prediction: Given the past 1 second agent history on a corresponding map and the associated lidar and camera data for this time interval, predict the positions of up to 8 agents for 8 seconds into the future. Use of lidar and camera data are optional.

• Occupancy and Flow Prediction: Predict the bird’s-eye view (BEV) roadway occupancy and motion flow of all observed and occluded vehicles given observed agent tracks for the last second.

• Sim Agents: Given the agent tracks for the past 1 second on a corresponding map, and optionally the associated lidar for this time interval, simulate 32 realistic joint futures for all the agents in the scene. It’s the second year we are running the Sim Agents Challenge, but participants can now leverage our new Waymax simulator that we made available to the research community a few months ago.

We have also improved our metrics for the Motion Prediction and Sim Agents Challenges, which will be reflected in the new leaderboards. The past leaderboards for both Challenges will maintain the scores before the change.

In addition to the Challenges, we’re expanding the Motion dataset with encoded camera data for use in behavioral models, which further encourages research on end-to-end behavior prediction and perception system development. 

First-place winners in each of the four Challenges will receive $10,000 in Google Cloud credits. Second and third place winners will get $5000 and $3000 in Google Cloud credits respectively. Additionally, teams with the best performing or noteworthy submissions may be invited to present their work at the Workshop on Autonomous Driving at CVPR in June 2024. The 2024 Waymo Open Dataset Challenges close at 11:59 PM Pacific on May 23, 2024, but the leaderboards will remain open for future submissions. The Challenges’ rules and eligibility criteria are available here. 

Good luck to all our 2024 Challenge participants! We’re looking forward to seeing the products of your creativity — and we always welcome feedback about future versions of the dataset, so please reach out if you have suggestions.