On the use of testability measures for dependability assessment

By Lorenzo Strigini and Antonia Bertolino; IEEE Transactions on Software Engineering, Vol. 22, No. 2, February 1996

ABSTRACT
Program "testability" is, informally, the probability that a program will fail under test, if it contains at least one fault. When a dependability assessment has to be derived from the observation of a series of failure-free test executions (a common need for software subject to "ultra-high reliability" requirements), measures of testability can - in theory - be used to draw inferences on program correctness (and hence on its probability of failure in operation). In this paper, we rigorously investigate the concept of testability and its use in dependability assessment, criticising, and improving on, previously published results.

We first give a general descriptive model of program execution and testing, on which the different measures of interest can be defined. We propose a more precise definition of program testability than that given by other authors, and discuss how to increase testing effectiveness without impairing program reliability in operation. We then study the mathematics of using testability to estimate, from test results: i) the probability of program correctness and ii) the probability of failures. To derive the probability of program correctness, we use a Bayesian inference procedure and argue that this is more useful than deriving a classical "confidence level". We also show that a high testability is not an unconditionally desirable property for a program. In particular, for programs complex enough that they are unlikely to be completely fault-free, increasing testability may produce a program which will be less trustworthy, even after successful testing.

The full text of this paper is available in .pdf and .ps format.

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

CSR Home | CSR Research Projects | CSR Publications | School of Informatics | City University