NCJ Number
89665
Date Published
1983
Length
34 pages
Annotation
This paper proposes two simple constructs to assist in the evaluation of evaluations: a decision tree approach which focuses on one or more of the proposed evaluation designs and a model dealing with the realistic issues of evaluation inputs, processes, and outcomes.
Abstract
The analogy of a prospective buyer of a used car is used to illustrate the decision tree approach to 'should the decisionmaker hire an evaluator?' This example demonstrates that the decision to evaluate is an allocation of resources which can be justified only if the expected benefits outweigh the expected costs of the evaluation. The paper notes that imperfect evaluations reduce anticipated benefits, but still provide useful information for the decisionmaker. This simple analytic framework can also address costs of evaluation error, cost of the evaluator, and biasing of the evaluation design. When reviewing past decisions to conduct evaluations, however, one must attempt to replicate the state of information available to the decisionmaker at the time. The second construct provides lists of evaluation inputs, processes, and outcomes that must be examined. Inputs include the budget, duration and timing of evaluation, attributes of the program being evaluated, methodology, audience, and programmatic purpose. Examples of process components are interactions between evaluators and staff, response to the presence of evaluators, and use of information acquired by an evaluation. Outcomes include decisions by funding agency regarding the program, staff revisions in the program, and decisions by clients to modify participation. Tables, charts, 2 footnotes, and 17 references are provided.