NCJ Number
50181
Journal
EVALUATION QUARTERLY Volume: 2 Issue: 3 Dated: (AUGUST 1978) Pages: 401-420
Date Published
1978
Length
20 pages
Annotation
THE EFFECTS OF DIFFERENT ASSUMPTIONS ABOUT TEST RELIABILITY ON THE FINDINGS AND CONCLUSIONS OF PROGRAM EVALUATIONS ARE EXPLORED IN AN ANALYSIS OF DATA FROM AN EXPERIMENTAL EDUCATIONAL PROGRAM.
Abstract
EVALUATIONS OF EDUCATIONAL PROGRAMS TYPICALLY ARE QUASI-EXPERIMENTAL. USUALLY SOME FORM OF ANALYSIS OF CONVARIANCE IS EMPLOYED TO EQUATE TREATMENT AND COMPARISON GROUPS ON PRETREATMENT CONDITIONS. THE APPLICATION OF ANALYSIS OF COVARIANCE TO QUASI-EXPERIMENTAL DATA HAS BEEN CRITICIZED ON THE GROUNDS THAT THE PROCEDURE LEADS TO BIAS IN THE ADJUSTMENT OF OUTCOME SCORES. VARIOUS PROCEDURES, MOST DEALING WITH THE CORRECTION OF COVARIATES FOR UNRELIABILITY, HAVE BEEN DESIGNED TO YIELD UNBIASED ADJUSTMENTS. ONE SUCH PROCEDURE WAS APPLIED TO DATA COLLECTED IN THE NATIONAL EVALUATION OF PROJECT FOLLOW THROUGH, A LARGE-SCALE QUASI-EXPERIMENTAL EDUCATIONAL PROGRAM FOR DISADVANTAGED CHILDREN. PROJECT EVALUATORS SOUGHT TO DETERMINE WHETHER THE APPLICATION OF RELIABILITY CORRECTIONS ALTERS RESULTS OBTAINED WITH UNCORRECTED COVARIATES SO AS TO CHANGE APPRECIABLY THE CONCLUSIONS DRAWN FROM THE RESULTS. THE RELIABILITY CORRECTIONS LED TO SOME CHANGES IN CONCLUSIONS ABOUT THE PERFORMANCE OF VARIOUS EDUCATIONAL MODELS FUNDED UNDER PROJECT FOLLOW THROUGH. THE RESULTS OF BOTH ANALYSES WERE PRESENTED IN THE EVALUATORS' REPORT, BUT CONCLUSIONS WERE BASED ON THE ANALYSIS EMPLOYING THE UNCORRECTED DATA DUE TO CONTROVERSY SURROUNDING METHODS OF CORRECTING FOR UNRELIABILITY IN COVARIATES. DETAILS OF THE ANALYSIS, SUPPORTING DATA, AND A LIST OF REFERENCES ARE INCLUDED. (AUTHOR ABSTRACT MODIFIED--LKM)