NCJ Number
235318
Journal
Journal of Experimental Criminology Volume: 7 Issue: 2 Dated: June 2011 Pages: 183-198
Date Published
June 2011
Length
16 pages
Annotation
The authors of this paper reviews Weisburd et al. study of does the quality of research design have an influence on study outcomes in crime and justice?
Abstract
Does the quality of research design have an influence on study outcomes in crime and justice? This was the subject of an important study by Weisburd et al. (2001). They found a moderate and significant inverse relationship between research design and study outcomes: weaker designs, as indicated by internal validity, produced stronger effect sizes. Using a database of evaluations (n=136) from systematic reviews that investigated the effects of public area surveillance on crime, this paper carried out a partial replication of Weisburd et al.'s study. The authors view it as a partial replication because it included only area- or place-based studies (i.e., there were no individual-level studies) and these studies used designs at the lower end of the evaluation hierarchy (i.e., not one of the studies used a randomized experimental design). In the present study, the authors report findings that are highly concordant with the earlier study. The overall correlation between research design and study outcomes is moderate but negative and significant (Tau-b= -.175, p=.029). This suggests that stronger research designs are less likely to report desirable effects or, conversely, weaker research designs may be biased upward. The authors explore possible explanations for this finding. Implications for policy and research are discussed. (Published Abstract)