This sixth episode in the National Institute of Justice's (NIJ's) Just Science podcast series is an interview with Barbara Guttman from the National Institute of Standards and Technology (NIST), who discusses the first large-scale black-box study to test the accuracy of computer and mobile phone forensics.
Background information for the study notes that black-box studies are used to measure the reliability of methods and techniques that rely on human interpretation. Barbara Guttman and her NIST team are working to measure the overall competency of the digital forensics community by releasing an open-enrollment online test that is available to interested forensic scientists. In the interview, she discusses the parameters of the test, the expected results, and the benefit of the study. In addition to the open-enrollment online test, the black-box study involved a large-scale literature review that encompasses research related to the features of the various types of digital evidence, how it is collected, the technological features of the device or app that collects the data, and the reliability of devices used by investigators to access particular types of digital evidence. The online test solicits the voluntary involvement of forensic scientists in the specialized digital forensics tasks performed by each participant to reach an investigative result. The testing team is confident they have in hand the correct accessing method and result as a measure of whether the volunteer participants have reached the correct conclusions about digital evidence used in the case provided. The benefits of such testing are discussed in the interview.