STIPI: Using Search to Prioritize Test Cases Based on Multi-objectives Derived from Industrial Practice

Abstract : The importance of cost-effectively prioritizing test cases is undeniable in automated testing practice in industry. This paper focuses on prioritizing test cases developed to test product lines of Video Conferencing Systems (VCSs) at Cisco Systems, Norway. Each test case requires setting up configurations of a set of VCSs, invoking a set of test APIs with specific inputs, and checking statuses of the VCSs under test. Based on these characteristics and available information related with test case execution (e.g., number of faults detected), we identified that the test case prioritization problem in our particular context should focus on achieving high coverage of configurations, test APIs, statuses, and high fault detection capability as quickly as possible. To solve this problem, we propose a search-based test case prioritization approach (named STIPI) by defining a fitness function with four objectives and integrating it with a widely applied multi-objective optimization algorithm (named Non-dominated Sorting Genetic Algorithm II). We compared STIPI with random search (RS), Greedy algorithm, and three approaches adapted from literature, using three real sets of test cases from Cisco with four time budgets (25 %, 50 %, 75 % and 100 %). Results show that STIPI significantly outperformed the selected approaches and managed to achieve better performance than RS for on average 39.9 %, 18.6 %, 32.7 % and 43.9 % for the coverage of configurations, test APIs, statuses and fault detection capability, respectively.
Type de document :
Communication dans un congrès
Franz Wotawa; Mihai Nica; Natalia Kushik. 28th IFIP International Conference on Testing Software and Systems (ICTSS), Oct 2016, Graz, Austria. Springer International Publishing, Lecture Notes in Computer Science, LNCS-9976, pp.172-190, 2016, Testing Software and Systems. 〈10.1007/978-3-319-47443-4_11〉
Liste complète des métadonnées

https://hal.inria.fr/hal-01643725
Contributeur : Hal Ifip <>
Soumis le : mardi 21 novembre 2017 - 15:53:20
Dernière modification le : mardi 21 novembre 2017 - 15:55:57

Fichier

 Accès restreint
Fichier visible le : 2019-01-01

Connectez-vous pour demander l'accès au fichier

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Dipesh Pradhan, Shuai Wang, Shaukat Ali, Tao Yue, Marius Liaaen. STIPI: Using Search to Prioritize Test Cases Based on Multi-objectives Derived from Industrial Practice. Franz Wotawa; Mihai Nica; Natalia Kushik. 28th IFIP International Conference on Testing Software and Systems (ICTSS), Oct 2016, Graz, Austria. Springer International Publishing, Lecture Notes in Computer Science, LNCS-9976, pp.172-190, 2016, Testing Software and Systems. 〈10.1007/978-3-319-47443-4_11〉. 〈hal-01643725〉

Partager

Métriques

Consultations de la notice

29