3D Semantic Segmentation
Overview
Given one or more lidar range images and the associated camera images, produce a semantic class label for each lidar point.
In the 1.4.3 update of the perception dataset, we have improved the quality of 3D semantic segmentation ground truth labels for train, test and validation sets.
Leaderboard
This leaderboard only displays submissions made on or after March 18, 2024, when the 2024 Waymo Open Dataset Challenges start.
Past leaderboards and challenges are available here.
Note: the rankings displayed on this leaderboard may not accurately reflect the final rankings for this Challenge.
Submit
To submit your entry to the leaderboard, upload your file in the format specified in the SemanticSegmentationSubmission proto . You can only submit against the Test Set 3 times every 30 days. (Submissions that error out do not count against this total.)
For the 3D Semantic Segmentation challenge, your submission file should be a binary file of the SemanticSegmentationSubmission proto. See the tutorial_3d_semseg.ipynb for an example.
Metrics
Mean Intersection Over Union (mIOU)
We adopt the Intersection Over Union (IOU) metric, which is defined as follows:
IOU = true_positive / (true_positive + false_positive + false_negative)
The IOU score is computed for each class. The mIOU is calculated as the mean across all classes.
Metric breakdown
In the leaderboard, we also list IOU scores for each class for detailed comparison among methods.
The 23 classes includes: Car, Truck, Bus, Motorcyclist, Bicyclist, Pedestrian, Sign, Traffic Light, Pole, Construction Cone, Bicycle, Motorcycle, Building, Vegetation, Tree Trunk, Curb, Road, Lane Marker, Walkable, Sidewalk, Other Ground, Other Vehicle, Undefined
Rules Regarding Awards
See the Official Challenge Rules here.