Advancing ADAS testing with machine learning and optimization techniques

ADAS (Advanced Driving Assistance Systems) and AD (Autonomous Driving) systems are the next big frontier for automotive companies. The challenge lays in finding the right balance between minimizing the number of accidents and casualties while maximizing the comfort of traveling in complex conditions.

The challenge: physical testing is not financially viable

ADAS/AD functions combine a number of components, including sensors (hardware and software processing), the algorithm fusing the data coming from multiple sensors, the algorithm deciding to act upon those inputs (braking, steering, accelerating), and finally, the actuators that will be implementing the decision.

ADAS/AD functions are also divided into a number of “levels”, each dictating who is responsible for the action, the car or the driver. From level-0 to level-2, the systems are “eyes-on and hands-on”, meaning that the function is there to support the driver by supplying more information, or by automating some parts of the driving.

From level-3 to level-5, the vehicle is in charge and can eventually (in the case of level-3 and level-4) give the controls back to the driver if the driving condition is too complex.

For ADAS systems (level-1 and level-2 type) car manufacturers have been validating their functions mainly through physical testing - driving vehicles around and ensuring that a limited number of false positives would appear.

But with the increase in number and complexity of these functions, and the need to move towards level-3 and level-4 type functions, a huge amount of testing became necessary for validation purposes.

We are talking about hundreds of millions of kilometers of testing, which made physical testing not financially viable.

Machine learning and AI supporting ADAS validation

It is now necessary to have a validation procedure that, with a reduced amount of mileage combined with the use of appropriate simulation tooling, allows for an in-depth understanding of the function performance. A number of funded projects are trying to tackle the validation aspects of ADAS/AD functions as we speak.

At ESTECO we propose a viable solution to this challenge: taking advantage of statistical tools to post-process road testing and derive test scenarios which are a combination of events that have been identified during road testing. This approach is also confirmed and put in practice by the Streetwise project which TNO is actively working on.

Machine learning libraries can automatically post-process the test data. The output from this analysis is a stochastic distribution of parameters that have been identified in the various events (i.e. change in the state of the vehicle).

These events are then combined within a vehicle dynamics simulation together with each of the identified parameters. Most critical conditions can be quickly identified within the simulation environment with the help of advanced DOE techniques

It is interesting to note that within this process a number of challenging scenarios will appear that might have not been identified in reality, but could realistically occur based on a combination of events.

With this approach, we can identify the most critical conditions independently from the sensors and functions that have been used.

In this step, it is crucial to quickly identify worst-case conditions in a smart manner, avoiding the virtual testing of all possible conditions.

Quickly identify worst-case conditions

The identified scenarios are independent of the sensors and ADAS/AD functions but are dependent on the definition of what a critical condition could be - for example, time to a collision between the vehicle under test and the surrounding vehicles.

Once critical conditions are identified, it is possible to assess the performance of the selected ADAS/AD system by selecting the specific sensor types and functions and identifying the scenarios that are most challenging to the given system.

Several automotive customers already rely on ESTECO technology in this field, and in particular on our:

• Machine learning: to identify the event parameter distribution within the road test data;
• Doe and optimization algorithms: to quickly identify worst-case conditions and assess the ADAS/AD system performance within the selected load cases;
• Enterprise-wide collaboration: to allow different teams to collaborate in sharing simulation models, workflows and having still control over the simulation model, its versioning and tracing all the results that are being generated back to the initial simulation models (and their version).