NCJ Number
213670
Date Published
September 2005
Length
18 pages
Annotation
This report presents a summary of social service grantees’ challenges, strategies, and lessons learned regarding social service program evaluation.
Abstract
The 10 demonstration projects examined reported a number of challenges in combining service delivery with a rigorous program evaluation methodology, but several of the programs were able to conduct thorough evaluations, all of which reported positive program outcomes. All 10 grantees reported challenges with evaluation design and methodology. Specifically, challenges involved the selection of measurement instruments, the establishment of comparison groups, data collection techniques, and the dilemma of how to analyze the impact of individualized services. Each of the challenges is discussed in turn, including recommendations on overcoming them. For example, all 10 grantees used standardized measurement instruments for their evaluation, yet all recommend using additional measurement techniques to supplement the standardized measures. A listing is provided of the most commonly used standardized instruments. Strategies employed by grantees for overcoming data collection challenges included using multiple data collection methods and collecting data at more than one point, such as at program intake and at program exit. Among the most valuable lessons learned about conducting social service program evaluations were the importance of beginning an evaluation with a detailed evaluation plan, allowing for sufficient evaluation budget and project duration time, having strategies on hand to address staff turnover, using qualified evaluation teams or organizations, and conducting evaluations as part of a cooperative grantee cluster. Appendixes, references