NCJ Number
47883
Journal
EVALUATION QUARTERLY Volume: 2 Issue: 2 Dated: (MAY 1978) Pages: 343-358
Date Published
1978
Length
16 pages
Annotation
OVERT AND COVERT BIASES, DISTORTION, AND SELECTIVE DISCLOSURE IN DATA COLLECTION AND INTERPRETATION FOR PROGRAM EVALUATION MAY MAKE SUCH EVALUATION A MEANS FOR SELF-LEGITIMIZATION AND MAINTENANCE OF THE STATUS QUO.
Abstract
PROGRAM EVALUATION REFERS TO THE PROCESS OF OBJECTIVELY ANALYZING THE IMPACT OF SOCIAL ACTION PROGRAMS ON THE BASIS OF DATA FOR THE PURPOSE OF MAKING DECISIONS ABOUT THEIR EFFECTIVENESS. PROGRAM EVALUATION DIFFERS FROM OTHER FORMS OF APPLIED RESEARCH IN THAT THE DATA COLLECTED ARE USED TO MAKE JUDGMENTS ABOUT SPECIFIC PROGRAMS RATHER THAN TO TEST THEORETICAL PROPOSITIONS OF A MORE GENERAL NATURE. ONE IMPORTANT CHARACTERISTIC OF PROGRAM EVALUATION IS THAT THOSE WHO SUPPLY INFORMATION AND THOSE WHO EVALUATE IT MAY HAVE A PERSONAL INVESTMENT, IN TERMS OF EMPLOYMENT OR PROGRAM CREDIBILITY, IN THE OUTCOME OF THE ASSESSMENT. GIVEN THE CAPACITIES AND LIMITATIONS OF HUMAN BEINGS, THE REWARD STRUCTURES INVOLVED MAY RESULT IN CONSCIOUS OR UNCONSCIOUS DISTORTION WHICH SERVES TO LIGITIMIZE THE PROGRAM OR THE INDIVIDUAL'S INVOLVEMENT IN IT. MOREOVER, COGNITIVE PROCESSES AND SOCIAL MORES TEND TO STRONGLY BIAS PROGRAM EVALUATION RESEARCH TOWARD MAINTAINING THE STATUS QUO RATHER THAN IMPLEMENTING PROGRAM INNOVATION AND CHANGE. WHILE COMPETITION, UNOBTRUSIVE MEASURES OF EFFICACY, AND A FOCUS ON MULTIPLE SOURCES OF DATA INPUT HAVE BEEN PROPOSED AS POSSIBLE METHODS FOR OVERCOMING THESE BIASES, ONLY THE LAST METHOD IS LIKELY TO BE TRULY EFFECTIVE. A FURTHER PROBLEM IN OVERCOMING SUCH BIASES IS THAT SCIENCE AND SCIENTIFIC RESEARCH MAY ALSO SERVE TO MAINTAIN THE STATUS QUO BY VIRTUE OF GIVEN PARADIGMS, CATEGORIES, AND DEFINITIONS WHICH LIMIT INQUIRY TO HISTORICALLY ACCEPTED PERSPECTIVES AND PROCEDURES. MOREOVER, DATA MAY DISTORT UNDERSTANDING AND MAY LEAD TO ASSUMPTIONS OF CASUAL RELATIONSHIPS AND DISTINCTIONS WHERE SUCH DISTINCTIONS OR RELATIONS MAY MERELY BE ARTIFACTS OF THE HYPOTHESIS OR PRECONDITIONS RATHER THAN THEIR CAUSES. IT IS SUGGESTED THAT BY BEING AWARE OF THESE STATUS-QUO-MAINTAINING FEATURES OF PROGRAM EVALUATION AND SCIENTIFIC RESEARCH, SOCIAL SCIENCE RESEARCHERS MAY BE ABLE TO BRING A MORE ACCURATE PERSPECTIVE TO THE DESIGN AND INTERPRETATION OF POLICY RESEARCH. REFERENCES ARE INCLUDED. (JAP)