Independent Validation Test of Microscopic Saw Mark Analysis

• NCJ Number: 241745
• Author(s): Jennifer C. Love, Ph.D.; Sharon M. Derrick, Ph.D.; Jason M. Wiersema, Ph.D.
• Date Published: February 2013
• Length: 58 pages
• Grant Number(s): 2010-DN-BX-K235
• Agencies: NIJ-Sponsored
• Publication Type: Test/Measurement; Report (Study/Research); Report (Grant Sponsored)

Annotation

This report presents the results and the methodology of an independent validation test of microscopic saw-mark analysis.

Abstract

The greatest value of this study is the development of a statistically sound approach to assessing the reliability and accuracy of a class-characteristics recognition method that should serve as a model for testing similar methods. The project developed an independent validation test of microscopic saw mark analysis. The method was replicated without deviation, and an ample sample size was produced for statistically sound analysis. Four morphologically different saws were used to make 58 partial and 58 complete saw marks in human femurs. The saw marks were examined independently by three doctoral-level anthropologists using a digital microscope. Fifteen variables were documented for each saw mark. Descriptive analysis was performed using the Microsoft Excel 2007 statistical package. The data were further analyzed with classification trees and random forest classifiers grown with the "rpart" and "randomForest" libraries of the open source data analysis package R. Four variables were used in the classification tree and random forest classifier analyses: wall shape; floor shape; minimum kerf width; and average tooth hop. Several of the variables were replicable and informative in the classification of saw type; however, other variables were rarely observed. Minimum kerf width and average tooth hop were two variables that showed little variation between the three analysts; however, average tooth hop was identified on only 71 percent of the specimens. The two random forest classifier analyses returned an out-of-bag error rate of 8.62 percent and 17.82 percent, respectively. As indicated, these analyses included average tooth hop, meaning nearly 30 percent of the data points were missing. The missing data were adaptively imputed using the random forest program, but this strategy weakens the results. Suggestions are offered for strengthening the method of testing the validity of microscopic saw-mark analysis. 21 references, 9 tables, and 8 figures