The design framework, which assumes that the program and its evaluation design are developed concurrently, is based on a dynamic roll-back approach consisting of three steps leading to a valid and comprehensive evaluation design. The sequence rolls back in time from (1) a projected look at the range of program characteristics to (2) a prospective consideration of the threats to the validity of the final evaluation and (3) a more immediate identification of the evaluation design elements. The five related design components are test hypotheses, selection scheme, measures framework, measurement methods, and analytic techniques. The test-hypotheses component includes the range of issues leading up to the establishment of test hypotheses, while the selection-scheme component involves the development of a scheme for the selection and identification of test groups and, if applicable, control groups, using appropriate sampling and randomization techniques. The measures-framework component involves specifying the set of evaluation measures that is to be the focus of the evaluation and the construction of a model that reflects the linkages among the measures. The list of issues and elements constituting the measurement-methods component includes measurement time frame, measurement scales, measurement instruments, measurement procedures, measurement samples, measurement quality, and measurement steps. Analytic techniques are used to conduct statistical tests of significance; combine, relate, or derive measures; assist in the evaluation; provide data adjustments for nonequivalent test and control groups; and to model test and control situations. Twelve bibliographic entries are provided.
On Developing Evaluation Designs - A Summary Report
NCJ Number
92370
Date Published
1983
Length
21 pages
Annotation
This guide to program evaluation design identifies a framework which links program characteristics to design elements, defines five related design components which contain the essential design elements, and develops a linear statistical model which highlights some of the key underlying issues in program evaluation.
Abstract