Test Case Selection in Industry: an Analysis of Issues Related to Static Approaches

Vincent Blondeau 1, 2 Anne Etien 2 Nicolas Anquetil 2 Sylvain Cresson 1 Pascal Croisy 1 Stéphane Ducasse 2
2 RMOD - Analyses and Languages Constructs for Object-Oriented Application Evolution
Inria Lille - Nord Europe, CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille (CRIStAL) - UMR 9189
Abstract : Automatic testing constitutes an important part of everyday development practice. Worldline, a major IT company, is creating more and more tests to ensure the good behaviour of its applications and gain in efficiency and quality. But running all these tests may take hours. This is especially true for large systems involving, for example, the deployment of a web server or the communication with a database. For this reason tests are not launched as often as they should and are mostly run at night. The company wishes to improve its development and testing process by giving to developers rapid feedback after a change. An interesting solution is to reduce the number of tests to run by identifying only those exercising the piece of code changed. Two main approaches are proposed in the literature: static and dynamic. The static approach creates a model of the source code and explores it to find links between changed methods and tests. The dynamic approach records invocations of methods during the execution of test scenarios. Before deploying a test case selection solution, Worldline created a partnership with us to investigate the situation in its projects and to evaluate these approaches on three industrial, closed source, cases to understand the strengths and weaknesses of each solution. We propose a classification of problems that may arise when trying to identify the tests that cover a method. We give concrete examples of these problems and list some possible solutions. We also evaluate other issues such as the impact on the results of the frequency of modification of methods or considering groups of methods instead of single ones. We found that solutions must be combined to obtain better results, and, problems have different impacts on projects. Considering commits instead of individual methods tends to worsen the results, perhaps due to their large size.
Type de document :
Article dans une revue
Software Quality Journal, Springer Verlag, 2016, 〈10.1007/s11219-016-9328-4〉
Liste complète des métadonnées

Littérature citée [27 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01344842
Contributeur : Lse Lse <>
Soumis le : mardi 12 juillet 2016 - 16:40:02
Dernière modification le : mardi 3 juillet 2018 - 11:48:24

Fichier

Test_Case_Selection_in_Industr...
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Collections

Citation

Vincent Blondeau, Anne Etien, Nicolas Anquetil, Sylvain Cresson, Pascal Croisy, et al.. Test Case Selection in Industry: an Analysis of Issues Related to Static Approaches. Software Quality Journal, Springer Verlag, 2016, 〈10.1007/s11219-016-9328-4〉. 〈hal-01344842〉

Partager

Métriques

Consultations de la notice

471

Téléchargements de fichiers

144