SOME APPROACHES TO ASSESSING CHANGE (FROM IMPROVING EVALUATIONS, 1979, BY LOIS-ELLIN DATTA AND ROBERT PERLOFF - SEE NCJ-64392)

• NCJ Number: 64395
• Author(s): B S TUCHFELD
• Date Published: 1979
• Length: 7 pages
• Corporate Author: Sage Publications, Inc
• Publication Format: Document
• Publication Type: Best Practice/State-of-the-Art Review
• Language: English
• Country: United States of America

Annotation

AN OVERVIEW OF STATISTICAL METHODS FOR MEASURING CHANGE IN PROGRAM EVALUATIONS IS PRESENTED, AND THE PROBLEMS BETWEEN DESIGN AND ANALYSIS IN DRAWING CAUSAL INFERENCE IS DISCUSSED.

Abstract

ASSESSING CHANGE HAS BEEN TROUBLSOME IN PROGRAM EVALUATIONS, SINCE EXPERIMENTATION, WHICH PROVIDES THE METHODOLOGICAL FOUNDATION FOR ASSESSING CAUSAL INFLUENCES, HAS BEEN DIFFICULT TO ACHIEVE. PROBLEMS IN ASSESSING CHANGE BECOME PRONOUNCED IN NONEXPERIMENTAL RESEARCH WHEN SUBJECTS CANNOT BE CONTROLLED THROUGH RANDOMIZATION. TECHNIQUES WHICH ATTEMPT TO TAKE SUCH DIFFERENCES INTO ACCOUNT HAVE BECOME INCREASINGLY SOPHISTICATED AS A RESULT OF EVOLUTION IN TECHNIQUE AND THINKING. THREE GENERAL STRATEGIES FOR ASSESSING CHANGE IN NONEXPERIMENTAL RESEARCH HAVE EMERGED. THESE ANALYTIC APPROACHES, USEFUL WITH VARIOUS NONEXPERIMENTAL OR QUASI-EXPERIMENTAL DESIGNS, CAN BE CLASSIFIED AS SIMPLE CHANGE SCORES, COVARIANCE STRATEGIES, AND EFFECTS MODELS. THE FIRST TWO APPROACHES PREDICT CHANGE SCORES OR ADJUSTED OUTCOMES AS A FUNCTION OF SOME INDEPENDENT VARIABLE(S), WHEREAS THE EFFECTS MODEL FOCUSES ON THE DEVELOPMENT OF STRUCTURAL EQUATION MODELS THAT MAY CONTAIN MULTIPLE INDICATORS, ESTIMATES OF UNOBSERVED VARIABLES, AND ESTIMATES OF MEASUREMENT ERROR. ALTHOUGH DEPENDENT ON THE GENERAL LINEAR MODEL, THE PERSPECTIVE UNDERLYING THE ANALYSIS OF EFFECTS VIA MULTIVARIATE STRUCTURAL MODELS SUGGESTS A SHIFT FROM PREDICTION PER SE TO THE FORMULATION OF A SERIES OF CAUSAL MODELS WHICH ARE TESTED FOR ADEQUACY AGAINST DATA COLLECTED FOR THE PROBLEM AT HAND. EFFECTS MODELS GIVE SPECIAL ATTENTION TO STATISTICAL ESTIMATION AND HYPOTHESIS TESTING AND SUGGEST A SHIFT IN FOCUS FOR THE ANALYSIS OF CHANGE SINCE THEY EMPHASIZE THE ESTIMATION OF MEASUREMENT ERROR--AN EMPHASIS CRITICAL IN EVALUATIVE RESEARCH. INCORPORATING CAUSAL MODELS IN NONEXPERIMENTAL EVALUATIONS THUS ENCOURAGES PARSIMONIOUS EXPLANATIONS, ADDS SPECIFICITY, AND REINFORCES THE CAUTIONS TO WHICH ONE MUST ADHERE WHEN ANALYZING NONEXPERIMENTAL DATA. REFERENCES ARE CITED. (MJW)