This article reports the results of the first large-scale black-box study to establish a discipline-wide error-rate estimate for palmar comparisons.
Critics and commentators have been calling for some time for black-box studies in the forensic science disciplines to establish the foundational validity of those fields; that is, to establish a discipline-wide, base-rate estimate of the error rates that may be expected in each field. Although the well-known FBI/Noblis black-box study has answered that call for fingerprints, no research to establish similar error rates for palmar impressions has been previously undertaken. In addressing this issue, the 226 latent print examiners who participated in the current study returned 12,279 decisions over a dataset of 526 known ground-truth pairings. There were 12 false identification decisions made, yielding a false-positive error rate of 0.7 percent. There were also 552 false exclusion decisions, yielding a false negative error rate of 9.5 percent. Given their larger number, false negative error rates were further stratified by size, comparison difficulty, and area of the palm from which the mark originated. The notion of “questionable conclusions,” in which the ground truth response may not be the most appropriate, is introduced and discussed due to the data obtained in the study. Measures of examiner consistency in analysis and comparison decisions are presented along with statistical analysis of the ability of many variables, such as demographics or image quality, to predict outcomes. Two online apps are introduced that will enable readers to fully explore the results on their own, or to explore the notions of frequentist confidence intervals and Bayesian credible intervals. (publisher abstract modified)