Skip to main content
Skip to footer
Rides Our service Phoenix San Francisco Los Angeles Rides Our service Phoenix San Francisco Los Angeles Technology Technology About Our History Waymo Leadership Latest Updates Press Resources About Our History Waymo Leadership Latest Updates Press Resources Safety Safety Community Community Careers Benefits Values People Open Roles Careers Benefits Values People Open Roles
GINA-3D: Learning to Generate Implicit Neural Assets in the Wild Bokui Shen Xinchen Yan Charles R. Qi Mahyar Najibi Boyang Deng Leonidas Guibas Yin Zhou Dragomir Anguelov Abstract Update 8/2023: The large-scale, object-centric dataset we constructed (Object Assets - Waymo Open Dataset) is now available for download at []( Modeling the 3D world from sensor data for simulation is a scalable way of developing testing and validation environments for robotic learning problems such as autonomous driving. However, manually creating or re-creating real-world-like environments is difficult, expensive, and not scalable. Recent generative model techniques have shown promising progress to address such challenges by learning 3D assets using only plentiful 2D images -- but still suffer limitations as they leverage either human-curated image datasets or renderings from manually-created synthetic 3D environments. In this paper, we introduce GINA-3D, a generative model that uses real-world driving data from camera and LiDAR sensors to create realistic 3D implicit neural assets of diverse vehicles and pedestrians. Compared to the existing image datasets, the real-world driving setting poses new challenges due to occlusions, lighting-variations and long-tail distributions. GINA-3D tackles these challenges by decoupling representation learning and generative modeling into two stages with a learned tri-plane latent structure, inspired by recent advances in generative modeling of images. To evaluate our approach, we construct a large-scale object-centric dataset containing over 520K images of vehicles and pedestrians from the Waymo Open Dataset, and a new set of 80K images of long-tail instances such as construction equipment, garbage trucks, and cable cars. We compare our model with existing approaches and demonstrate that it achieves state-of-the-art performance in quality and diversity for both generated images and geometries. Share Links Download PDF ArXiv Copy BibTeX Copied! Publication CVPR 2023 Topics 2023 Perception CVPR Simulation
FAQ Blog Privacy Policy Research Terms Legal Zero Tolerance First Responders Safety Publications Let's Talk Autonomous Driving Contact Us Notice to CA Residents Sign up for updates to get the latest on Waymo and our technology. Sign up © 2019-2023 Waymo LLC Do Not Sell or Share My Personal Information Waymo may disclose user personal information to third parties to tailor advertising and offers to your interests. Such disclosures may be considered “sales” or “sharing” of personal information under the laws described above. California residents may opt-out of these below. If you opt out, Waymo will not disclose your personal information to third parties for purposes of tailoring advertising or offers to your interests. Note that any choice you make here will only affect this website on this browser and device. To learn more about how your data is shared, view our Privacy Policy page.