Skip to Main content Skip to Navigation
Conference papers

Using Robustness Testing to Handle Incomplete Verification Results When Combining Verification and Testing Techniques

Abstract : Modular verification and dynamic testing techniques are often combined to validate complex software systems. Formal verification is used to cover all input spaces and program paths. However, due to the high complexity of modern software systems, they might not achieve complete verification results. Dynamic testing techniques can easily be applied to any type of software. Current approaches use them to handle incomplete verification results by validating unverified sections. This way of combining verification and testing ignores the fact that tests can only be used to show the presence of errors, but not their absence. Undiscovered errors pose the risk to trigger further errors in vulnerable code sections. Vulnerable sections are modularly verified, but depend on the guarantees of the tested code. We include robustness testing to analyse the influence of undiscovered errors. The generated robustness tests simulate failed guarantees within the tested code. The triggered response to those simulated errors helps the developer in adding additional error handling code. This makes the system more robust against undiscovered errors and guards it against uncontrolled crashes and unexpected behaviour in case of software failures. In the second part of this paper, we introduce a reference-architecture to generate and apply robustness tests. This architecture has been applied to multiple case studies and helped to identify potential errors yet undiscovered by generated test cases.
Complete list of metadata

Cited literature [12 references]  Display  Hide  Download
Contributor : Hal Ifip Connect in order to contact the contributor
Submitted on : Tuesday, January 9, 2018 - 3:39:56 PM
Last modification on : Tuesday, January 9, 2018 - 4:05:06 PM


Files produced by the author(s)


Distributed under a Creative Commons Attribution 4.0 International License



Stefan Huster, Jonas Ströbele, Jürgen Ruf, Thomas Kropf, Wolfgang Rosenstiel. Using Robustness Testing to Handle Incomplete Verification Results When Combining Verification and Testing Techniques. 29th IFIP International Conference on Testing Software and Systems (ICTSS), Oct 2017, St. Petersburg, Russia. pp.54-70, ⟨10.1007/978-3-319-67549-7_4⟩. ⟨hal-01678963⟩



Record views


Files downloads