NCJ Number
94058
Journal
Evaluation Review Volume: 8 Issue: 2 Dated: (April 1984) Pages: 187-204
Date Published
1984
Length
18 pages
Annotation
An evaluation of social service programs for ex-offenders in Chicago conducted during 1977-79 illustrates the 'real world' constraints on documenting, monitoring, and measuring treatment implications. Such problems seriously impede assessments of why a project failed to produce the desired results.
Abstract
Treatment integrity refers to how well treatment practice conforms to treatment plan, and the lack of such integrity can have harmful effects on policymaking. The task of documenting and measuring treatment delivery is particularly difficult when different individuals handle the roles of evaluator ANd treatment administrator. Such problems can be overcome in a closed setting, but are exacerbated when treatment is delivered in the community by nonresearch personnel. The Chicago experiment tried to compare the differential impacts of four programs: two used volunteer counselors, one used ex-offender as paraprofessionals, and one offered only employment counseling and referrals. The evaluation was disappointing and found no significant differences among the programs. Post hoc analyses showed that the programs had no formal structure, volunteers had no prescribed schedule for meeting clients, and there were no agency controls to ensure they met. Evaluators were not allowed to observe or record counseling sessions. It utlimately was determined that only 5 percent of the experimental group subjects worked with a counselor for at least 5 hours per month, a finding which makes the evaluation results understandable. Evaluation funders could enhance the value of outcome studies if they incorporated provisions for gathering implementation data into their funding criteria. The paper includes 50 references. (Author abstract modified)