Replication: Criterion Validity of the Empirical Scoring System with Inexperienced Scorers

• NCJ Number: 246959
• Journal: Polygraph Volume: 43 Issue: 1 Dated: 2014 Pages: 1-19
• Author(s): Raymond Nelson; Mark Handler; Stuart Senter
• Date Published: 2014
• Length: 19 pages
• Publication Format: Article
• Publication Type: Report (Technical); Measurement/Evaluation Device
• Language: English
• Country: United States of America

Annotation

This study replicated and extended the work of earlier studies of the Empirical Scoring System (ESS) in polygraph analysis by calculating criterion accuracy profiles for the ESS and two other scoring systems: 7-position and 3-position; ESS results were also compared to an unweighted ESS model.

Abstract

The weighted ESS model produced significant increases in test sensitivity to deception as well as significantly fewer inconclusive results compared to the 7-position, 3-position, and unweighted ESS models. The difference in inconclusive results was primarily loaded on deceptive subjects. Comparison of the 7-position and 3-position systems showed that the 7-position system also produced significantly fewer inconclusive results among truthful cases, suggesting that weighted scoring models may generally outperform unweighted models. Although overall differences in decision accuracy were not significant, none of the scoring models outperformed the ESS along any measured dimension of test accuracy. One implication of this finding is that limiting test data analysis activities to the interpretation of a core set of primary scoring features does not reduce decision accuracy. A practical aspect of a simple empirically based scoring system is that reliable skill acquisition can be expected to increase, and difficulty in skill acquisition can be reduced without loss of test accuracy. Another benefit of a simplified evidence-based scoring rubric is that scoring proficiency becomes a more stable skill for those examiners whose professional responsibilities include other forms of policing. Data for this study were obtained from a cohort of inexperienced polygraph examiner trainees who evaluated a sample of confirmed single-issue three-question zone comparison test (ZCT) exams (N=60) from a confirmed case archive constructed by the Department of Defense during 2002. 7 tables, 36 references, and appended ESS and two-stage decision rules