Performance Comparison of Two Search-Based Testing Strategies for ADAS System Validation

Abstract : In this paper, we compare the performance of a genetic algorithm for test parameter optimization with simulated annealing and random testing. Simulated annealing and genetic algorithm both represent search-based testing strategies. In the context of autonomous and automated driving, we apply these methods to iteratively optimize test parameters, to aim at obtaining critical scenarios that form the basis for virtual verification and validation of Advanced Driver Assistant System (ADAS). We consider a test scenario to be critical if the underlying parameter set causes a malfunction of the system equipped with the ADAS function (i.e., near-crash or crash of the vehicle). To assess the criticality of each test scenario we rely on time-to-collision (TTC), which is a well-known and often used time-based safety indicator for recognizing rear-end conflicts. For evaluating the performance of each testing strategy, we set up a simulation framework, where we automatically run simulations for each approach until a predefined minimal TTC threshold is reached or a maximal number of iterations has passed. The genetic algorithm-based approach showed the best performance by generating critical scenarios with the lowest number of required test executions, compared to random testing and simulated annealing.