Meet the Winners of the Waymo Open Dataset Challenges

In March 2020, we launched the Waymo Open Dataset Challenges, inviting researchers to build and test their machine learning models using Waymo’s diverse self-driving dataset. We received over 100 submissions from around the world and invited the winners to present their work at our virtual Workshop on Scalability in Autonomous Driving at CVPR 2020. Today, we are excited to introduce some of the winners and their learnings.

The Waymo Open Dataset Challenges were held in five categories: 2D and 3D Detection, 2D and 3D Tracking, and Domain Adaptation. All submissions were evaluated through an automated process and participants could also publish their results to an online leaderboard. The Challenges were truly international with finalists from China, England, Singapore, the US, and many other countries.

Yu Wang, leader of the winning team that came in first in four out of five challenges, said: “We were really excited when the Waymo Open Dataset was first published last year. After years of frustrations with existing small and outdated benchmarks, we could finally move on to a large-scale multi-sensor dataset that was collected for real autonomous driving scenarios.”

Many finalists explicitly mentioned the size of Waymo’s dataset among the factors that made them participate in the Challenges. Shaoshuai Shi, a Ph.D. student in the Multimedia Lab of The Chinese University of Hong Kong, whose team developed the top-performing lidar-only solutions in the 3D Detection, 3D Tracking, and Domain Adaptation tracks, noted that “with such a large dataset, the ablation experiments are more reasonable and reliable since the model is hard to overfit.”

Some participants used this opportunity to try their skills in new areas of research. For example, Almaz Zinollayev, a researcher from Kazakhstan and one of the winners in the 3D Tracking Challenge, said he had mostly worked with 2D images before and wanted to hone his skills in building 3D algorithms. “The nature of 2D and 3D data is entirely different, though it was interesting to see that there are some common ideas that can be successfully applied to 3D object detection and tracking tasks,” remarked Zinollayev. “Learning these new concepts and then building and training the model was certainly challenging but very rewarding.”

The scientists from DAI-Labor, an AI laboratory at Berlin Institute of Technology, who set out to achieve state-of-the-art tracking quality by only using a combination of existing techniques landed third place in the 2D Tracking Challenge. The team said: “The field of machine learning and autonomous driving is changing rapidly. The Waymo Open Dataset is a great contribution to the research community, as it allows us to work on a real world problem without having an autonomous car with expensive sensors.”

While the official Waymo Open Dataset Challenges have come to an end, our leaderboards will remain open to new submissions. We also plan to continue to add to the dataset over time, in the hopes that we will aid the international research community in making advancements in machine perception and self-driving technology.

Congratulations to all the winners and thank you to everyone who submitted their work. We are excited to see what the next generation of researchers will accomplish!