How Data Is Being Used to Train Autonomous Vehicles to Navigate Roadways


Autonomous vehicles will need to traverse at least 11 billion miles of on-the-road training to match the driving abilities of human drivers, but the COVID-19 pandemic has erected a major roadblock. The pandemic has grounded a number of these training fleets, including those from GM’s Cruise, Lyft, Aurora, Uber, and Argo AI. However, some of these startups say they can use data collected by manually-piloted vehicles laden with sensors. They intend to create artificial intelligence-generated simulations to train their vehicles instead of traditional practice to navigate roadways. 

These companies have compiled thousands to millions of miles of real-world data that can be used to create these simulations. From thousands of images that can be used to teach autonomous vehicles how to recognize traffic signals, to real and synthetic audio data, to data that can create detailed 3D, high-definition maps of roads, these simulations can train vehicles how to navigate real-world roadways without ever leaving the garage.

“We have experienced very little disruption to our AI model development trajectory due to the absence of road operations,” Uber’s Advanced Technologies Group (ATG) head of systems engineering and testing Adrian Thompson told VentureBeat. “Our test track testing is meant to validate our models, so we’re able to maintain, if not accelerate, our pace of development during this time.”

Research Indicates “Scene-Compliant GANs” Help Autonomous Vehicles Predict Movement on the Road

Researchers from Uber’s Advanced Technologies Group (ATG) recently published a paper in which they claimed to have enhanced the ability of autonomous vehicles to be able to accurately predict the movement of surrounding vehicles on the road. In order for these autonomous vehicles to avoid collisions, they must be able to correctly anticipate the actions of other vehicles around them.

Generative adversarial networks (GANs) are advanced networks that allow autonomous vehicles to make complex predictions. GANs employ two adversarial models, generators and discriminators, that work to improve each other. The generator creates synthetic examples, such as images, from random backgrounds. The discriminator then has to distinguish between these synthetic examples against real-world ones from training data. As the discriminator learns to accurately identify synthetic examples, it in turn trains the generator to improve its performance.

In the study, the ATG researchers proposed “scene-compliant GANs” (also known as SC-GANs) that produce trajectories in order to track constraints within roadway scenes. The SC-GAN uses this data to create detailed frames of references of other vehicles in these scenes. All of this information is then packaged into an RGB (Red Green Blue) image, which can capture 10 meters behind and 30 meters to the side of each car near the autonomous vehicle. They experimented in Google’s TensorFlow machine learning framework using 240 hours of real-world driving data. They found the SC-GANs eliminated off-road false positives (data errors that cause vehicles to incorrectly react in certain situations) by 50% in comparison to several TensorFlow baselines. For example, in a scenario where another car entered an intersection, the SC-GAN correctly anticipated the car would continue straight even though it appeared to be turning.

“Motion prediction is one of the critical components of self-driving technology, modeling future behavior and uncertainty of the tracked actors in [the self-driving vehicles] vicinity,” the authors wrote in the study. “Extensive qualitative and quantitative analysis showed that the method outperforms the current state-of-the-art in GAN-based motion prediction of the surrounding actors, producing more accurate and realistic trajectories.”

Understand Autonomous Vehicle Technology

Prepare your organization for the latest developments in autonomous vehicle technology. Offer training in foundational and practical applications of autonomous, connected, and intelligent vehicle technologies. Developed by leading experts in the field—including Steve Vozar, CTO and co-founder of May Mobility—the IEEE Guide to Autonomous Vehicle Technology is a seven-course online training program.

Connect with an IEEE Content Specialist today to learn more about purchasing the program for your organization.

Interested in purchasing the program just for yourself? Access it through the IEEE Learning Network.


Wiggers, Kyle. (28 April 2020). The challenges of developing autonomous vehicles during a pandemic. Venture Beat.

Wiggers, Kyle. (15 April 2020). Uber claims its AI enables driverless cars to predict traffic movement with high accuracy. Venture Beat. 

Wiggers, Kyle. (26 December 2020). Generative adversarial networks: What GANs are and how they’ve evolved. Venture Beat.

, , , , , , , , ,

No comments yet.

Leave a Reply