tf.Example Format

roadgraph_samples/xyz

The coordinate positions of the sampled map data points. Map polylines and polygons are sampled at 0.5 meters. The roadgraph_samples/dir feature gives the direction vector from each sample to the next sample.

[num_points, 3]

float

Any global coordinates, meters

roadgraph_samples/dir

A unit direction vector for each map feature sample point.

[num_points, 3]

float

Unit vectors

roadgraph_samples/type

A unique integer for each combination of map feature type and properties. See the table in the Value Range column.

[num_points, 1]

int64

[0, 19]. LaneCenter-Freeway = 1, LaneCenter-SurfaceStreet = 2, LaneCenter-BikeLane = 3, RoadLine-BrokenSingleWhite = 6, RoadLine-SolidSingleWhite = 7, RoadLine-SolidDoubleWhite = 8, RoadLine-BrokenSingleYellow = 9, RoadLine-BrokenDoubleYellow = 10, Roadline-SolidSingleYellow = 11, Roadline-SolidDoubleYellow=12, RoadLine-PassingDoubleYellow = 13, RoadEdgeBoundary = 15, RoadEdgeMedian = 16, StopSign = 17, Crosswalk = 18, SpeedBump = 19, other values are unknown types and should not be present.

roadgraph_samples/valid

A valid flag for each map sample point

[num_points, 1]

int64

{0, 1}

roadgraph_samples/id

A unique Integer ID for the vector map feature each sample is from.

[num_points, 1]

int64

Any 64 bit integer value

scenario/id

A unique string ID for the scenario represented by this example.

[1]

string

A hexadecimal string

state/current/x

The x coordinate of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Global coordinates, meters. For all state/{current,past,future}/xx fields, if there are less objects in the scene than num_objects, values are padded with -1. The state/{current,past,future}/valid flag indicates which values are populated with valid data for these tensors.

state/current/y

The y coordinate of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Global coordinates, meters.

state/current/z

The z coordinate of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Global coordinates, meters.

state/current/bbox_yaw

The heading angle of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Angle in radians. This is the angle to rotate a unit vector in the X direction in the X-Y plane to align with the length axis of the object bounding box.

state/current/length

The length of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Length in meters.

state/current/width

The width of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Width in meters.

state/current/height

The height of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Height in meters.

state/current/speed

The speed of each object at each time step.

[num_objects, num_steps] = [128, 1]

float

Speed in m/s.

state/current/timestamp_micros

A timestamp in microseconds for each time step.

[num_objects, num_steps] = [128, 1]

int64

Time in microseconds

state/current/vel_yaw

The yaw angle of each object's velocity vector at each time step.

[num_objects, num_steps] = [128, 1]

float

Angle in radians. This is the angle to rotate a unit vector in the X direction in the X-Y plane to align with the velocity vector of the object bounding box.

state/current/velocity_x

The x component of the object velocity at each time step.

[num_objects, num_steps] = [128, 1]

float

m/s

state/current/velocity_y

The x component of the object velocity at each time step.

[num_objects, num_steps] = [128, 1]

float

m/s

state/current/valid

A valid flag for all elements of features state/current/XX. If set to 1, the element is populated with valid data, otherwise it is populated with -1.

[num_objects, num_steps] = [128, 1]

int64

{0, 1}. Indicates if the data at each element of the state/current/XX tensors is populated with valid data (rather than being padded with -1).

state/future/x

The x coordinate of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Global coordinates, meters.

state/future/y

The y coordinate of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Global coordinates, meters.

state/future/z

The z coordinate of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Global coordinates, meters.

state/future/bbox_yaw

The heading angle of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Angle in radians. This is the angle to rotate a unit vector in the X direction in the X-Y plane to align with the length axis of the object bounding box.

state/future/length

The length of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Length in meters.

state/future/width

The width of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Width in meters.

state/future/height

The height of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Height in meters.

state/future/speed

The speed of each object at each time step.

[num_objects, num_steps] = [128, 80]

float

Speed in m/s.

state/future/timestamp_micros

A timestamp in microseconds for each time step.

[num_objects, num_steps] = [128, 80]

int64

Time in microseconds

state/future/vel_yaw

The yaw angle of each object's velocity vector at each time step.

[num_objects, num_steps] = [128, 80]

float

Angle in radians. This is the angle to rotate a unit vector in the X direction in the X-Y plane to align with the velocity vector of the object bounding box.

state/future/velocity_x

The x component of the object velocity at each time step.

[num_objects, num_steps] = [128, 80]

float

m/s

state/future/velocity_y

The x component of the object velocity at each time step.

[num_objects, num_steps] = [128, 80]

float

m/s

state/future/valid

A valid flag for all elements of features state/future/XX. If set to 1, the element is populated with valid data, otherwise it is populated with -1.

[num_objects, num_steps] = [128, 80]

int64

{0, 1}. Indicates if the data at each element of the state/future/XX tensors is populated with valid data (rather than being padded with -1).

state/past/x

The x coordinate of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Global coordinates, meters.

state/past/y

The y coordinate of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Global coordinates, meters.

state/past/z

The z coordinate of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Global coordinates, meters.

state/past/bbox_yaw

The heading angle of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Angle in radians. This is the angle to rotate a unit vector in the X direction in the X-Y plane to align with the length axis of the object bounding box.

state/past/length

The length of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Length in meters.

state/past/width

The width of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Width in meters.

state/past/height

The height of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Height in meters.

state/past/speed

The speed of each object at each time step.

[num_objects, num_steps] = [128, 10]

float

Speed in m/s.

state/past/timestamp_micros

A timestamp in microseconds for each time step.

[num_objects, num_steps] = [128, 10]

int64

Time in microseconds

state/past/vel_yaw

The yaw angle of each object's velocity vector at each time step.

[num_objects, num_steps] = [128, 10]

float

Angle in radians. This is the angle to rotate a unit vector in the X direction in the X-Y plane to align with the velocity vector of the object bounding box.

state/past/velocity_x

The x component of the object velocity at each time step.

[num_objects, num_steps] = [128, 10]

float

m/s

state/past/velocity_y

The x component of the object velocity at each time step.

[num_objects, num_steps] = [128, 10]

float

m/s

state/past/valid

A valid flag for all elements of features state/past/XX. If set to 1, the element is populated with valid data, otherwise it is populated with -1.

[num_objects, num_steps] = [128, 10]

int64

{0, 1}. Indicates if the data at each element of the state/past/XX tensors is populated with valid data (rather than being padded with -1).

state/tracks_to_predict

A vector of flags to indicate which objects to predict corresponding to the first dimension of the state/XX tensors. Up to 8 objects will be selected. For training and validation sets, these indicate which object must be present in the submission proto predictions. If exactly these objects are not predicted in the submission proto, the submission will fail. This field is also present in the training set as a reference for objects to train with, but you are free to choose other objects to train with.

[num_objects]

int64

indices into the 1st dimension of the state/XX tensors.

state/objects_of_interest

A vector of flags to indicate which objects in the scene corresponding to the first dimension of the state/XX tensors have interactive behavior. Up to 2 objects will be selected. The objects in this list form an interactive group.

[num_objects]

int64

{0, 1}. Indicates if the objects corresponding to the first dimension of the state/XX tensors have interactive behavior.

state/id

An integer ID for each object.

[num_objects]

float

An object ID.

state/is_sdc

A mask to indicate if the object is the autonomous vehicle.

[num_objects]

int64

{0, 1}

state/type

An integer type for each object (Vehicle, Pedestrian, or Cyclist)

[num_objects]

float

Unset=0, Vehicle=1, Pedestrian=2, Cyclist=3, Other=4

traffic_light_state/current/state

The state of each traffic light at each time step.

[num_steps, num_light_positions] = [1, 16]

int64

Unknown = 0, Arrow_Stop = 1, Arrow_Caution = 2, Arrow_Go = 3, Stop = 4, Caution = 5, Go = 6, Flashing_Stop = 7, Flashing_Caution = 8

traffic_light_state/current/x

The X coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [1, 16]

float

Global coordinates in meters.

traffic_light_state/current/y

The Y coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [1, 16]

float

Global coordinates in meters.

traffic_light_state/current/z

The Z coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [1, 16]

float

Global coordinates in meters.

traffic_light_state/current/id

The lane ID controlled by each traffic signal.

[num_steps, num_light_positions] = [1, 16]

int64

An integer lane ID.

traffic_light_state/current/valid

A valid flag for all elements of features traffic_light_state/current/XX. If set to 1, the element is populated with valid data, otherwise it is populated with -1.

[num_steps, num_light_positions] = [1, 16]

int64

{0, 1}

traffic_light_state/current/timestamp_micros

A timestamp in microseconds for each step corresponding to traffic_light_state/current/XX

[num_steps] = [1]

int64

Time in microseconds

traffic_light_state/future/state

The state of each traffic light at each time step.

[num_steps, num_light_positions] = [80, 16]

int64

Unknown = 0, Arrow_Stop = 1, Arrow_Caution = 2, Arrow_Go = 3, Stop = 4, Caution = 5, Go = 6, Flashing_Stop = 7, Flashing_Caution = 8

traffic_light_state/future/x

The X coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [80, 16]

float

Global coordinates in meters.

traffic_light_state/future/y

The Y coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [80, 16]

float

Global coordinates in meters.

traffic_light_state/future/z

The Z coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [80, 16]

float

Global coordinates in meters.

traffic_light_state/future/id

The lane ID controlled by each traffic signal.

[num_steps, num_light_positions] = [80 16]

int64

An integer lane ID.

traffic_light_state/future/valid

A valid flag for all elements of features traffic_light_state/future/XX. If set to 1, the element is populated with valid data, otherwise it is populated with -1.

[num_steps, num_light_positions] = [80, 16]

int64

{0, 1}

traffic_light_state/future/timestamp_micros

A timestamp in microseconds for each step corresponding to traffic_light_state/current/XX

[num_steps] = [80]

int64

Time in microseconds

traffic_light_state/past/state

The state of each traffic light at each time step.

[num_steps, num_light_positions] = [10, 16]

int64

Unknown = 0, Arrow_Stop = 1, Arrow_Caution = 2, Arrow_Go = 3, Stop = 4, Caution = 5, Go = 6, Flashing_Stop = 7, Flashing_Caution = 8

traffic_light_state/past/x

The X coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [10, 16]

float

Global coordinates in meters.

traffic_light_state/past/y

The Y coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [10, 16]

float

Global coordinates in meters.

traffic_light_state/past/z

The Z coordinate of the stop light position. This point is at the beginning of the lane segment controlled by the traffic signal.

[num_steps, num_light_positions] = [10, 16]

float

Global coordinates in meters.

traffic_light_state/past/id

The lane ID controlled by each traffic signal.

[num_steps, num_light_positions] = [10, 16]

int64

An integer lane ID.

traffic_light_state/past/valid

A valid flag for all elements of features traffic_light_state/future/XX. If set to 1, the element is populated with valid data, otherwise it is populated with -1.

[num_steps, num_light_positions] = [10, 16]

int64

{0, 1}

traffic_light_state/past/timestamp_micros

A timestamp in microseconds for each step corresponding to traffic_light_state/past/XX

[num_steps] = [10]

int64

Time in microseconds

path_samples/xyz

Path coordinates

[num_paths, num_points_per_path, 3]

tf.float32

Any global coordinates, meters

path_samples/valid

Denotes whether each point is valid or not. A point can be invalid either because a trajectory has ended, ie. some suffix of a row in valid is all False, or because the trajectory itself is invalid, ie. every entry in a row of valid is False.

[num_paths, num_points_per_path]

tf.int64

0 for false and 1 for true

path_samples/id

Integer identifiers denoting which "road part" a point belongs to. Each lane, turn, and so forth is assigned a "road part" with a unique ID.

[num_paths, num_points_per_path]

tf.int64

Any integer value, denoting IDs

path_samples/arc_length

Denotes the length of the SDC path from the starting position to a particular step in the path. For example, arc_length[i, 17] == 5.2 would indicate that on the 17'th step of the i'th SDC path, the SDC would have travelled 5.2 meters.

[num_paths, num_points_per_path]

tf.float32

Path length in meters

path_samples/on_route

Denotes whether or not each SDC path is compatible with the SDC's observed route.

[num_paths, 1]

tf.int64

0 for false and 1 for true