2D Detection

Leaderboard

Submit

Instructions

To submit your entry to the leaderboard, upload your file in the format specified in the Submission protos. This challenge does not have any awards. You can only submit against the Test Set 3 times every 30 days. (Submissions that error out do not count against this total.)

Metrics

Leaderboard ranking for this challenge is by Mean Average Precision (mAP) / L2 among "ALL_NS" (all Object Types except signs), that is, the mean over the APs of Vehicles, Cyclists, and Pedestrians. Only cameras are allowed to be used. And we enforce a causal system, i.e, for a frame at time step t, only sensor data up to time t can be used for its prediction.

Primary metric

Average Precision (AP): ∫p(r)dr where p(r)is the PR curve

IoU Overlap Threshold

Vehicle 0.7, Pedestrian 0.5, Cyclist 0.5, Sign 0.5

Sensor Names

C: All cameras
I: Invalid

Label Difficulty Breakdown

Each ground truth label is categorized into different difficulty levels (two levels for now):

• LEVEL_1, if not marked as LEVEL_2 in the released data.

• LEVEL_2, if marked as LEVEL_2 in the released data. When evaluating, LEVEL_2 metrics are computed by considering both LEVEL_1 and LEVEL_2 ground truth.

Metric Breakdown

The following metric breakdowns are supported:

• OBJECT_TYPE: Breakdown by object type ("ALL_NS" refers to all objects except signs: Vehicle, Cyclist, and Pedestrian)

• RANGE: Breakdown by the distance between object center and vehicle frame origin. [0, 35m), [35m, 50m), [50m, +inf)