Artificial intelligence can completely transform current driver training. This is also the aim of a collaboration between the Norwegian University of Science and Technology (NTNU), the IT company Virtur, and WAY AS in Trondheim, which is Norway’s only simulator-based driving school.

The project has developed a system and an AI driving instructor that can teach learner drivers how to drive in a full-scale car simulator.

A virtual driving instructor is fully capable of immediately assessing the performance of the learner driver behind the wheel. It can also provide completely neutral feedback in real time.

This helps eliminate or reduce any personal biases towards the learner that might influence feedback given by a real instructor in a one-on-one situation.

“AI driving instructors are on par with human driving instructors when it comes to systematically evaluating learner drivers,” says Odd Erik Gundersen, associate professor at NTNU’s Department of Computer Science.

Nearly 1.2 million people die in road traffic accidents every year, according to the World Health Organization. This is the most common cause of death for children and young people between the ages of 5 and 29. The youngest drivers are most at risk and men are three times more likely than women to be killed in road traffic accidents.

Even small improvements can save many lives. While the UN aims to halve the number of road traffic deaths by 2030, the EU wants the number to be as close as possible to zero by 2050.

In Norway, 118 people died in road traffic accidents in 2023. This trend over past two years has gone in the wrong direction, so Norwegian authorities are also intensifying their efforts to improve traffic safety.

The NTNU researchers believe that one opportunity for improvement is better driving instruction. The system they have developed is based on a total of 477 driving scenarios involving three different traffic situations – overtaking manoeuvres, city driving involving major traffic light junctions, and roundabouts and various give way situations.

The goal has been for the system, i.e., the virtual instructor, to be trained to accurately and objectively assess various driving scenarios. At the same time, it should possess skills and provide feedback that are as good and precise as real driver training experts.

To train and test the virtual instructor in relation to real-world driver training, a total of 21 learner drivers and seven instructors participated in simulator sessions.

According to the report ‘A virtual driving instructor that assesses driving performance on par with human experts’: “The results show that the assessments and feedback from the AI instructor align very well with those of the real experts in these situations. This confirms that the system can perform as precisely as human experts.”

In recent years, interest in driver training in simulators with virtual instructors has grown in line with increasingly better technology and cheaper hardware.

However, despite the many advantages, relatively little research has so far been conducted on AI-based driver training. Much of the attention in the field is instead directed towards the development of self-driving cars.

Odd Erik Gundersen hopes that their research can help advance the development of virtual driver training.

“When training in a simulator, it is easier to standardize and personalize driver learning. At the same time, the simulator can ensure that all drivers are exposed to and can handle different situations. In addition, unusual and dangerous situations which cannot be tested in the real world can be practiced in a simulator,” says Gundersen.

In addition, the virtual environment around the driver can be changed as needed. Everything from complex traffic situations in big cities to driving on country roads can be adapted to individual learner drivers.

For example, the AI driving instructor can select situations that the learner driver needs to practise more on. Since stress generally decreases our ability to learn, it is also possible to remove traffic or other stressors along the way.

The researchers have also shown that the feedback from the instructor can be tailored to a learning style that best suits the learner driver.

The researchers emphasize that many of these improvements cannot be implemented in current driving lessons where the learner is behind the steering wheel and the instructor in the passenger seat. They also believe that fully automated training can make driving lessons cheaper.

The development of these types of computer models is based on comprehensive and complex programming work. The system developed by NTNU researchers, driving instructors and IT professionals consists of an assessment system and a guidance system. The former is largely designed as a rules-based system where AI uses predetermined rules to make decisions.

This differs from pure machine learning, which requires models to be fed with vast amounts of data in order to be trained to act as desired. Machine learning is a specialization within the field of artificial intelligence, where statistical methods are used to enable computers to find patterns in large amounts of data. It is said that the machine ‘learns’ instead of being programmed.

The virtual driving instructor requires a rules-based system, where it is easy to find support and explanations for the decisions it makes.

That is because it is not only supposed to assess the learner’s driving performance but also has to explain and justify good or bad feedback, and provide information about what the learner driver needs to improve.

Traffic light junctions, roundabouts, and overtaking manoeuvres – these are the most challenging situations for learner drivers.

The assessments made by the system show a very strong correlation with the assessments made by professional driving instructors. In addition, the AI driving instructor is equally accurate as a real instructor in identifying and providing feedback on both the good and the bad decisions made by the learner drivers.

Since there is limited research assessing simulator training for learner drivers, the NTNU researchers believe they have now contributed to closing a knowledge gap.

“There are limitations. As with any rules-based system, our platform also faces some demanding challenges,” the researchers wrote in their article about the work.

Among the challenges is the complicated process of developing and managing a comprehensive and intricate set of rules that identify the many different situations a driver encounters in traffic. This means that developing and maintaining the system can be both time-consuming and costly.

The 21 learner drivers in the project have each driven three test runs. They sit in a normally equipped car and use the steering wheel and pedals as usual.

However, instead of driving on the road with other traffic, the car is mounted on a moving platform surrounded by projectors, creating a virtual traffic environment on all sides. A camera captures everything the learner drivers do, even how they move their eyes during the driving lesson.

NTNU, WAY AS and Virtur want to make the system even better by adopting the latest virtual and mixed reality (MR) technology.

MR is an emerging technology where users can wear headsets equipped with advanced eye-tracking technology, blending virtual reality (VR) and augmented reality (AR). VR allows users to navigate through a completely imagined space – an artificial environment that exists only in visual form and not in real life. It is used in everything from gaming and entertainment to medicine and military operations (adobe.com). In augmented reality, additional content is overlaid on the real world that a player or user sees in front of them.

The researchers believe this will facilitate even more personalised, efficient and tailored training for each individual learner driver. People who need to practise driving around roundabouts can complete 30 attempts in 30 minutes. They believe their work highlights the enormous potential of using AI to revolutionise driver training. They go so far as to call it a paradigm shift. A paradigm shift is a revolutionary change in the fundamental assumptions and principles within a field, such as science.

How long do you think it will be before digital driving instructors can be used in Norwegian driving school cars?

“Norwegian regulations state that driver training must be conducted in a car – meaning it cannot be done in a simulator. So, if the use of digital driving instructors changes how the training is actually done, the regulations must be changed”, says Associate Professor Odd Erik Gundersen at NTNU.