RESPONSE-SHIFT BIAS - A PROBLEM IN EVALUATING INTERVENTIONS WITH PRE/POST SELF-REPORTS

• NCJ Number: 65179
• Journal: Evaluation Review Volume: 4 Issue: 1 Dated: (FEBRUARY 1980) Pages: 93-106
• Author(s): G S HOWARD
• Date Published: 1980
• Length: 14 pages
• Corporate Author: Sage Publications, Inc
• Sale Source: Sage Publications, Inc; Institute for Scientific Information
• Language: English
• Country: United States of America
• Note: PRICE QUOTED IS FOR SINGLE ISSUE

Annotation

THIS ARTICLE REVIEWS THE LITERATURE ON RESPONSE-SHIFT EFFECTS AND RECOMMENDS THE USE OF RETROSPECTIVE PRETESTS AS ONE APPROACH TO REMOVING THIS SOURCE OF CONTAMINATION FROM EVALUATION STUDIES EMPLOYING SELF-REPORT MEASURES.

Abstract

SELF-REPORT INSTRUMENTS ARE FREQUENTLY USED TO EVALUATE THE EFFECTIVENESS OF TREATMENT OR TRAINING INTERVENTIONS. THE TYPICAL DESIGN INVOLVES COLLECTING PRETEST AND COMPARING THESE DATA WITH SIMILAR RATINGS FOR AN APPROPRIATE CONTROL GROUP. HOWEVER, A TESTING PROBLEM ARISES WHEN PSYCHOLOGICAL TREATMENT/INTERVENTION HEIGHTENS SUBJECTS' AWARENESS OF THEIR PERFORMANCE ON A PARTICULAR VARIABLE SO THAT THEIR SELF-REPORT ON THE DIMENSION FOR PRETESTING WOULD, IN THEIR OPINION, BE NO LONGER VALID. HENCE, THE DIFFERENCE BETWEEN PRETEST AND PROTTEST MEASURE WOULD BE EXAGGERATED. THIS IS KNOWN AS RESPONSE-SHIFT BIAS. IN SUCH CASES, EVEN THE TRUE EXPERIMENTAL DESIGNS CAN PROVIDE INTERNALLY INVALID RESULTS. RETROSPECTIVE PRETEST RATINGS ARE RECOMMENDED AS ONE WAY IN WHICH RESPONSE-SHIFT BIAS MIGHT BE ATTENUATED. THE APPROACH PROFERRED IN THIS STUDY IS TO UTILIZE A RETROSPECTIVE PRETEST TO OBTAIN A PRETREATMENT ESTIMATE OF SUBJECTS' LEVELS OF FUNCTIONING THAT IS MADE FROM THE SAME PERSPECTIVE AS THE POSTTEST RATING. TWO PROBLEMS WITH THIS APPROACH ARE (1) HISTORICAL OR PHILOSOPHICAL SUSPICION OF MENTALISTIC APPROACHES TO RESEARCH AND (2) THE PROBLEMS OF RESPONSE-STYLE BIAS (E.G., SUBJECT COMPLIANCE, MEMORY DISTORTION). HOWEVER, RETROSPECTIVE MEASURES WOULD NOT BE ADVISABLE IF THE DATA DEMONSTRATING THE DELETRIOUS EFFECTS OF RESPONSE-SHIFT BIAS WERE NOT SO CONVINCING OR AN ALTERNATIVE METHOD FOR REMOVING THIS SOURCE OF CONTAMINATION FROM TRADITIONAL PRETEST OR POSTTEST SELF-REPORT DESIGNS COULD BE DEVELOPED. OVERALL, THE RESEARCH SUGGESTS THAT A THEN/POST ANALYSIS IS EQUAL, OR SUPERIOR, TO A PRE/POST SELF-REPORT APPROACH. USE OF PRETEST, POSTTEST, AND RETROSPECTIVE PRETEST SELF-REPORT DATA PROVIDES A MORE SENSITIVE ASSESSMENT OF A SUBJECT'S PERSPECTIVE OF PERSONAL CHANGE. REFERENCES ARE PROVIDED. (PRG) (PRG)