Optimal Minimisation of Pairwise-covering Test Configurations Using Constraint Programming - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Information and Software Technology Année : 2016

Optimal Minimisation of Pairwise-covering Test Configurations Using Constraint Programming

Aymeric Hervieu
  • Fonction : Auteur
  • PersonId : 766227
  • IdRef : 17789508X
Dusica Marijan
  • Fonction : Auteur
  • PersonId : 945251
Arnaud Gotlieb
Benoit Baudry
  • Fonction : Auteur
  • PersonId : 838700

Résumé

Context: Testing highly-configurable software systems is challenging due to a large number of test configurations that have to be carefully selected in order to reduce the testing effort as much as possible, while maintaining high software quality. Finding the smallest set of valid test configurations that ensure sufficient coverage of the system's feature interactions is thus the objective of validation engineers, especially when the execution of test configurations is costly or time-consuming. However, this problem is NP-hard in general and approximation algorithms have often been used to address it in practice. Objective: In this paper, we explore an alternative approach based on constraint programming that will allow engineers to increase the effectiveness of configuration testing while keeping the number of configurations as low as possible. Method: Our approach consists in using a (time-aware) minimisation algorithm based on constraint programming. Given the amount of time, our solution generates a minimised set of valid test configurations that ensure coverage of all pairs of feature values (a.k.a. pairwise coverage). The approach has been implemented in a tool called PACOGEN. Results: PACOGEN was evaluated on 224 feature models in comparison with the two existing tools that are based on a greedy algorithm. For 79% of 224 feature models, PACOGEN generated up to 60% fewer test configurations than the competitor tools. We further evaluated PACOGEN in the case study of large industrial highly-configurable video conferencing software with a feature model of 169 features, and found 60% fewer configurations compared with the manual approach followed by test engineers. The set of test configurations generated by PACOGEN decreased the time required by test engineers in manual test configuration by 85%, increasing the feature-pairs coverage at the same time. Conclusion: Extensive evaluation concluded that optimal minimisation of pairwise-covering test configurations is efficiently addressed using constraint programming techniques.
Fichier principal
Vignette du fichier
IST_journal.pdf (3.37 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01352831 , version 1 (10-08-2016)

Identifiants

  • HAL Id : hal-01352831 , version 1

Citer

Aymeric Hervieu, Dusica Marijan, Arnaud Gotlieb, Benoit Baudry. Optimal Minimisation of Pairwise-covering Test Configurations Using Constraint Programming. Information and Software Technology, 2016, 71, pp.129-146. ⟨hal-01352831⟩
337 Consultations
448 Téléchargements

Partager

Gmail Facebook X LinkedIn More