We’re using AI and virtual worlds with the aim of creating safer cars

Once upon a time, we only had physical ways of testing and developing new safety features. With the arrival of the computer, we could also use virtual environments to constantly push the envelope in safety.  

And now we’re taking a new step forward, using AI-generated life-like virtual worlds to find new breakthroughs that help keep you safe. With the help of advanced AI and computational techniques, we can enhance the development of our safety software, such as our advanced driver assistance systems (ADAS), all with the aim of creating even safer cars.   

So, how does this work? Thanks to an advanced computational technique called “Gaussian splatting” (this isn’t a Volvo Cars phrase we promise), we can turn real world visuals into lifelike, natural-looking 3D scenes and subjects.   

Using real-life data 
With Gaussian splatting, incident data collected by the advanced sensors in our new cars – such as emergency braking, sharp steering or manual intervention – can now be synthesised, reconstructed and explored to help us better understand how accidents could be avoided in the future.

For example, these virtual environments can be manipulated by adding or removing road users and changing the behaviour of traffic or obstacles on the road. This allows us to expose our safety software to all types of traffic situations, at a speed and scale not possible before. As a result, we now can develop software that works well in complex, rare yet potentially dangerous ‘edge cases’ and reduce the time it takes to expose our software to edge cases, from months to days. 

The virtual environments are developed by Zenseact, our in-house AI and software company that we founded around five years ago. This project, sponsored by Wallenberg AI, Autonomous Systems and Software Program (WASP), is part of a PhD programme for leading Swedish universities to explore whether neural rendering techniques will be integrated into future safety initiatives. 

“We already have millions of data points of moments that never happened that we use to develop our software" says Alwin Bakkenes, Head of Global Software Engineering at Volvo Cars. "Thanks to Gaussian splatting we can select one of the rare corner cases and explode it into thousands of new variations of the scenario to train and validate our models against. This has the potential to unlock a scale that we’ve never previously had before and even to catch edge cases before they happen in the real world.” 

One part of the puzzle 
Today, we use virtual environments alongside real-world testing for software training, development and validation because they provide a safe, scalable, and cost-effective environment.  To ensure our safety software is safe, it must be trained to handle all types of traffic situations. By using rare or unusual scenarios, we can ensure the software works well not only in normal conditions but also in complex, rare yet potentially dangerous ‘edge case’ situations.

Needless to say, the list of possible scenarios is close to infinite. Think of animals jumping into the road, objects falling off the back of a lorry, a traffic light showing all colours at once, a car driving the wrong way down a motorway, a road that has completely flooded or a dust tornado crossing the street. Capturing all possible scenarios in real life or coding them would take hundreds of thousands of years.

Integration of NVIDIA technology
Volvo Cars can explore technologies like Gaussian splatting thanks to the  recently expanded relationship with NVIDIA. The new generation of fully electric cars, built on NVIDIA accelerated compute collects data from various sensors to understand what’s happening in and around the car better than ever before. An AI supercomputing platform, powered by NVIDIA DGX systems, contextualises this data, unlocks new insights, and trains future safety models. It will improve and accelerate the development of artificial intelligence. This supercomputing platform is part of a recent investment of Volvo Cars and Zenseact to set up one of the largest data centres in the Nordics.

Want to know more?
The exploration of Gaussian splatting and generative AI forms part of our presentation at the NVIDIA GTC conference. You can watch it live or on demand via this link.