Missing Links: How Descriptive Validity Impacts the Policy Relevance of Randomized Controlled Trials in Criminology

• NCJ Number: 237286
• Journal: Journal of Experimental Criminology Volume: 7 Issue: 3 Dated: September 2011 Pages: 201-224
• Author(s): Charlotte E. Gill
• Date Published: September 2011
• Length: 24 pages
• Publication Format: Article
• Publication Type: Report (Study/Research)
• Language: English
• Country: United States of America

Annotation

The aim of this study is to build upon previous discussion and empirical investigation of reporting quality in criminology.

Abstract

To assess quality of reporting of issues that may affect internal and external validity in randomized controlled trials (RCTs) in criminology, and explore the impact of reporting quality (descriptive validity) on the policy relevance of rigorous research. Reporting indicators based on CONSORT standards from the health sciences are constructed and applied to a sample of 38 RCTs, covering a range of criminal justice interventions, published in journals between 2002 and 2008. A Descriptive Validity Matrix is constructed to visually convey information about reporting quality across a group of studies, based on the reporting indicators, to decisionmakers. Criminological RCTs are moderately well-reported. The sample of studies show medium descriptive validity in reporting on elements relevant to internal validity, and high descriptive validity for items relevant to external validity. However, there was considerable variation in the quality of reporting on key issues, especially those related to implementation of the random assignment sequence, deviations from the planned study, and attrition of participants. This study and the Descriptive Validity Matrix provide a useful framework for assessing descriptive validity. Although the indicators developed were not specific to criminology, and the analysis was limited to a small number of studies published in academic journals, this study is an important starting point for continued research and discussion on the relationship between implementation of field experimentation, reporting quality, and policymaking. The ability to report research clearly is as important as choosing the most rigorous research design for enhancing the objectives of evidence-based crime policy. (Published Abstract)