ASSESSING THE IMPACT OF PLANNED SOCIAL CHANGE

• NCJ Number: 61640
• Journal: Evaluation and Program Planning Volume: 2 Issue: 1 Dated: (1979) Pages: 67-90
• Author(s): D T CAMPBELL
• Date Published: 1979
• Length: 24 pages
• Grant Number(s): GSOC-7103704-03; BNS76-23920
• Sponsoring Agency: Russell Sage Foundation; National Science Foundation
• Corporate Author: Pergamon Press, Inc
• Publication Format: Article
• Language: English
• Country: United States of America
• Note: REPRINTED WITH MINOR REVISIONS AND ADDITIONS, FROM GENE M LYON (ED), SOCIAL RESEARCH AND PUBLIC POLICIES, HANOVER, NEW HAMPSHIRE, UNIVERSITY PRESS OF NEW ENGLAND, 1975 EARLIER VERSION PRESENTED TO THE VISEGRAD, HUNGARY, CONFERENCE ON SOCIAL PSYCHOLOGY, MAY 5-10, 1974

Annotation

AN OVERVIEW IS PROVIDED OF UNITED STATES PROBLEMS WITH ASSESSING THE IMPACT OF PLANNED SOCIAL CHANGE.

Abstract

AMERICAN PROBLEMS IN EVALUATING PLANNED SOCIAL CHANGE ARE RELATED TO ISSUES INTERNAL TO THE SCIENTIFIC COMMUNITY AND MATTERS DERIVED FROM EVALUATIVE INTERACTION WITH THE SOCIETAL CONTEXT. ISSUES INTERNAL TO THE SCIENTIFIC COMMUNITY CAN BE CLASSIFIED AS METASCIENTIFIC AND STATISTICAL. PROBLEMATIC METASCIENTIFIC ISSUES ARE (1) CHOOSING QUANTITATIVE VS. QUALITATIVE METHODOLOGY, OR CHOOSING BETWEEN EVALUATIVE MEASURES BASED ON OBJECTIVE DATA AND EVALUATIVE JUDGMENTS MADE BY TRAINED EVALUATORS; (2) SEPARATING IMPLEMENTATION AND EVALUATION, OR DEVELOPING A PROGRAM INDEPENDENT OF ANY EVALUATION DESIGN AND THEN IMPOSING A SEPARATE EVALUATION PROCEDURE AFTER A PERIOD OF OPERATION; (3) MAXIMIZING REPLICATION AND CRITICISM, OR EXPANDING THE IMPACT AND ESTABLISHING THE VALIDITY OF PLANNED SOCIAL CHANGE EVALUATIONS; AND (4) VIEWING EVALUATION RESEARCH AS NORMAL RATHER THAN EXTRAORDINARY SCIENCE, OR THE CONDUCT OF SOCIAL CHANGE EVALUATION WITHIN A FUNDAMENTALLY STABLE SOCIETY IN CONTRAST TO A CONTEXT OF REVOLUTIONARY CHANGE. STATISTICAL ISSUES IN EVALUATION ASSUME A SOLUTION TO METASCIENTIFIC ISSUES THAT FAVORS A QUANTITATIVE EXPERIMENTAL APPROACH. THE INTERRUPTED TIME-SERIES DESIGN (MEASURING IMPACTS OVER A PERIOD OF TIME WITHOUT AND WITH THE PLANNED INTERVENTION) INVOLVES PROBLEMS OF ATTRITION FROM EXPERIMENTAL AND CONTROL GROUPS AND ASSURING RANDOMIZATION. THE REGRESSION-DISCONTINUITY DESIGN IS AN ALTERNATIVE (ADMITTING PROGRAM PARTICIPANTS ON THE BASIS OF STRICT ELIGIBILITY SCORES). PROBLEMS ASSOCIATED WITH EVALUATIVE INTERACTION WITH THE SOCIETAL CONTEXT INCLUDE A NATURAL RESISTANCE TO EVALUATION BY THOSE WHOSE PROGRAMS ARE BEING EVALUATED AND THE CORRUPTING EFFECT OF EVALUATIVE QUANTITATIVE INDICATORS. THOSE BEING EVALUATED TEND TO GEAR THEIR BEHAVIOR AND PROGRAM GOALS EXCLUSIVELY TOWARD PRODUCING FAVORABLE QUANTITATIVE MEASURES OF PERFORMANCE, FREQUENTLY BY CORRUPTING THE INTEGRITY OF THE PROCESSES (FOR EXAMPLE, INDISCRIMINATE SLAUGHTER OF VILLAGERS TO INCREASE BODY COUNT AS A MEASURE OF EFFECTIVENESS IN THE VIET NAM WAR). GRAPHIC DATA AND REFERENCES ARE PROVIDED. (RCB)