The Virtual Reality of Testing Self-Driving Cars

There is no doubt that one-day autonomous vehicles will eliminate road accidents, reduce congestion, and make the planet a greener place to live. But before they are allowed on the thorough fares for consumer and commercial use, the technology needs to be proven safer than the flawed human experience.

The question, though, is how many miles of driving would it take to make statistically significant safety comparisons between autonomous vehicles and those with human drivers? Considering that current human-piloted vehicles have a history spanning more than a century and have covered billions of miles, autonomous vehicles would have to be test-driven hundreds of billions of miles within the next few years to demonstrate their safety confidently

For example, assuming a self-driving vehicle fleet had a 20 percent lower fatality rate than that for human drivers: Substantiating this with 95 percent confidence would necessitate driving 5 billion miles. This would be equivalent to driving every mile of road in Texas nearly 16,000 times over. It would take a fleet of 100 vehicles, driving 24/7, around 225 years to deliver these results. So test-driving to prove safety seems to be mission impossible.

Virtual testing of autonomous vehicles will speed up development

Fortunately, alternatives, such as virtual testing, mathematical analysis, and accelerated testing could shoulder some of the burdens of accelerating the introduction of self-driving cars.

As a result, Google's vehicles drive a simulated 3 million miles each day, or the equivalent of circling the equator five times every hour.

The Google self-driving car development team acknowledges that, while actual road testing is crucial, virtual simulation is equally important. Each time Google improves or refines a key feature for integration into its road vehicles, it first tests the modification in a virtual environment using thousands of variations.

To accomplish this, Google has built a "Matrix-style" digital simulation of the entire Californian road system in which it tests its self-driving cars.

This virtual environment allows Google engineers to assess new software for its robot cars in a realistic virtual environment, testing how tweaks to their behavior might play out in reality.

For example, a new emergency braking system was tested over 10,000 miles in the simulator to evaluate how often it was engaged and how it performed in emergency situations. And in another test 50 virtual self-driving cars were dropped into a digital version of Google's hometown of Mountain View to see how they interacted with each other, computer-generated pedestrians and other simulated drivers

[inlinead]

Test drivers and engineers can acquire massive amounts of data from these simulations."Each time a test driver takes over from the self-driving car, we're able to play back the exact situation and predict via simulation what could have happened if the car had been left to drive itself. If the simulator shows that the driving needs to be improved, our engineers can make refinements to the software, and run those changes in simulation to test the fixes," says Google spokeswoman Katelin Jabbari.

These successes have spurred Ron Medford, Google's safety director for the self-driving car program, to argue that computer simulation should be accepted in place of real world driving. According to the Guardian, in a letter written to California state officials, Medford claims: "Computer simulations are actually more valuable, as they allow manufacturers to test their software under far more conditions and stresses than could possibly be achieved on a test track."

OEMs traverse virtual roads across the Globe

Google is not alone on the virtual roadways of California. The Ford Research and Innovation Center in Palo Alto is one of the largest automotive manufacturer research centers in the region, housing a team of more than 100 engineers and scientists. Here Ford carries out virtual test drives to study the interaction between autonomous vehicles and pedestrians, replicating real-world situations to better understand and develop responses to the likely road and traffic conditions

In Japan, Toyota Motor Company has created its version of virtual testing. Gill Pratt, CEO of the Toyota Research Institute, which was established last year with $1 billion in funding to research automated driving, artificial intelligence, and robotics, recently explained that Toyota’s “Guardian-Angel” program raises new challenges for the company; especially regarding understanding and managing driver behavior.

The company plans to test the technology in a giant moving simulator near Mt. Fuji. The simulator places the “driver” in realistic street scenes, moving around inside a hanger roughly the size of two football pitches. This testing will make it possible to see how people react in realistic crash scenarios. “Our plan is to see how humans will respond when the car temporarily takes control because it knows better,” Pratt says.

Accurately creating a virtual world

But to accurately simulate the vehicles environment you need accurate data on various environmental influences. Developers need this information to conduct tests that simulate the experience of a real car. In terms of advancing virtual testing and validation the Ricardo Agent Drive simulation software, constructed around agent-based modeling methodology, can be used to create real-world driving scenarios to test complex driving situations for autonomous vehicles.

This form of testing places agents (vehicles, people or infrastructure) with specific driving features (such as selfish, aggressive, defensive) in a defined environment (cities, freeways or test tracks) to understand complex interactions that occur in real-world situations. Employing this approach OEMs and their suppliers are able to cut product development cycles, reduce costs related to test-vehicle damage and lower the risk of harming vehicle occupants under test conditions

In Germany, researchers at the Fraunhofer Institute have developed a quick and inexpensive system that collects data at regular driving speeds and processes the information in real time as fine-grained and coarse-grained data for 3D driving simulations

The system consists of a test vehicle, a geo-referenced database, and a motor vehicle simulator. Using two 360-degree laser scanners, the Road & Environmental Data Acquisition Rover (REDAR) captures enormous amounts of environmental data at typical driving speeds. “We call it point cloud data. That means for each 3D coordinate we have environmental data,” says Dr. Klaus Dressler of the Fraunhofer Institute for Industrial Mathematics ITWM in Kaiserslautern.

The ITWM researchers have managed to arrange the terabyte-sized dataset so that it can be used in real time during 3D interactive driving simulations. “The volume of data is so large that the information cannot easily be loaded into the memory of a computer system. We have therefore developed an out-of-core method to process only the data necessary for the running time in the simulator.”

The database simulates the world‘s road network complete with topography, regulations, weather and external geo-referenced data. “With the data collected from the data acquisition vehicle, we merge real data with the coarse-grained information from the VMC. Combining the two worlds is a major step in developing test scenarios for the engineering of road-bound vehicles,” says Dressler.

Despite the strong argument put forward by the Google self-driving car team, California's Department of Motor Vehicles has so far refused to modify its rules, confirming that physical testing on private tracks was required before autonomous vehicles could take to public roads.

In response, Google says that its intent was not to skirt safety but to add a safety layer."It's safer to test in simulation to make sure you haven't broken anything before you go out on a real road. We think that simulation and actual road driving should be used for comprehensive self-driving car testing," says Jabbari.

Meanwhile, Google continues to improve and scale up its simulation efforts. And perhaps the next state to embrace autonomous vehicles will be happier doing its test driving inside the Matrix.