Assessing Test Adequacy for Black-Box Systems without Specifications

Abstract : Testing a black-box system without recourse to a specification is difficult, because there is no basis for estimating how many tests will be required, or to assess how complete a given test set is. Several researchers have noted that there is a duality between these testing problems and the problem of inductive inference (learning a model of a hidden system from a given set of examples). It is impossible to tell how many examples will be required to infer an accurate model, and there is no basis for telling how complete a given set of examples is. These issues have been addressed in the domain of inductive inference by developing statistical techniques, where the accuracy of an inferred model is subject to a tolerable degree of error. This paper explores the application of these techniques to assess test sets of black-box systems. It shows how they can be used to reason in a statistically justified manner about the number of tests required to fully exercise a system without a specification, and how to provide a valid adequacy measure for black-box test sets in an applied context.
Complete list of metadatas

Cited literature [32 references]  Display  Hide  Download

https://hal.inria.fr/hal-01583922
Contributor : Hal Ifip <>
Submitted on : Friday, September 8, 2017 - 10:05:33 AM
Last modification on : Friday, September 8, 2017 - 10:08:39 AM

File

978-3-642-24580-0_15_Chapter.p...
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Neil Walkinshaw. Assessing Test Adequacy for Black-Box Systems without Specifications. 23th International Conference on Testing Software and Systems (ICTSS), Nov 2011, Paris, France. pp.209-224, ⟨10.1007/978-3-642-24580-0_15⟩. ⟨hal-01583922⟩

Share

Metrics

Record views

65

Files downloads

94