Modelling the effect of diverse fault removal techniques
A paper "Modelling the effects of combining diverse software fault
removal techniques" by Bev Littlewood, Peter Popov, Lorenzo Strigini
and Nick Shryane has been published in IEEE Transactions on Software Engineering,
Vol. SE-26, no. 12, pp. 1157-1167, 2000. The abstract is given below. The
full text of the paper is available for download in .pdf
format.
Abstract
The software engineering literature contains many studies of the efficacy
of fault finding techniques. Few of these, however, consider what happens
when several different techniques are used together. We show that the effectiveness
of such multi-technique approaches depends upon quite subtle interplay
between their individual efficacies and dependence between them. The modelling
tool we use to study this problem is closely related to earlier work on
software design diversity. The earliest of these results showed that, under
quite plausible assumptions, it would be unreasonable even to expect software
versions that were developed ‘truly independently’ to fail independently
of one another. The key idea here was a ‘difficulty function’ over the
input space. Later work extended these ideas to introduce a notion of ‘forced’
diversity, in which it became possible to obtain system failure behaviour
better even than could be expected if the versions failed independently.
In this paper we show that many of these results for design diversity have
counterparts in diverse fault removal in a single software version. We
define measures of fault finding effectiveness, and of diversity, and show
how these might be used to give guidance for the optimal application of
different fault finding procedures to a particular program. We show that
the effects upon reliability of repeated applications of a particular fault
finding procedure are not statistically independent - in fact such an incorrect
assumption of independence will always give results that are too optimistic.
For diverse fault finding procedures, on the other hand, things are different:
here it is possible for effectiveness to be even greater than it would
be under an assumption of statistical independence. We show that diversity
of fault finding procedures is, in a precisely defined way, ‘a good thing’,
and should be applied as widely as possible. The new model and its results
are illustrated using some data from an experimental investigation into
diverse fault finding on a railway signalling application.
Page maintained by: Peter Popov
Last modified 30 January 2001.