Eugenio Alberdi, Andrey Povyakalo, Lorenzo Strigini, Peter Ayton

Abstract:

We report results of a study conducted to investigate the effects of incorrect output from a Computer Aided Detection (CAD) tool on the decisions of its human users. CAD tools mark (prompt) on mammograms features which are potential indicators of cancer. Our study follows on a clinical trial that evaluated the impact of introducing CAD in breast screening. The administrators of the trial reported no significant impact on readers' specificity or sensitivity.

Our follow-up study focused on a particular type of incorrect computer output: CAD's failure to detect cancers, either by failing to prompt them or by prompting them incorrectly. We used 20 experienced mammogram readers and a data set with an unusually high proportion of cancers missed by CAD.

We found the average readers' sensitivity for these cases to be very low: 52%. Strikingly, very few of the readers' decisions for the non prompted cancers (21%) and, to a lesser extent, the incorrectly prompted cancers (53%) were correct.

One plausible explanation is that the readers were affected by "automation bias", that is, they interpreted absence of prompting as a strong indication that a case is normal, and became "complacent" with non prompted cases. We are currently exploring alternative explanations.
 

List of fields: medicine
List of topics: decision making
Interest Groups: Medical Problem Solving
 

Proceedings of SPUDM 2003 (Subjective Probability, Utility and Decision Making), Zurich, August 2003.