NCJ Number
65370
Journal
International Journal of the Addictions Volume: 13 Issue: 8 Dated: (1978) Pages: 1263-1284
Date Published
1978
Length
22 pages
Annotation
THE EVALUATION OF MENTAL HEALTH, EDUCATION, AND WELFARE PROGRAMS HAS RECEIVED INCREASING EMPHASIS IN RECENT YEARS. HOWEVER, IN ORDER FOR THESE EVALUATIONS TO BE USEFUL, CERTAIN REQUIREMENTS MUST BE MET.
Abstract
THESE ARE (1) INTERNAL VALIDITY, (2) EXTERNAL VALIDITY, (3) SPECIFICATION OF THE POPULATION AND TREATMENT BEING IMPLEMENTED, AND (4) STANDARDIZATION OF TREATMENT IMPACT INDICATORS. TO DETERMINE THE EXTENT TO WHICH PUBLISHED PROJECT IMPACT EVALUATIONS MEET THESE CRITERIA, A STUDY WAS UNDERTAKEN TO EVALUATE THE EVALUATIONS IN HEROIN ADDICTION TREATMENT PROGRAMS. SIX HIGH-YIELD JOURNALS AND 100 RANDOM SOURCES WERE SYSTEMATICALLY SEARCHED FOR REPORTS OF EVALUATIONS WHICH PROVIDED MEASURES OF CONSUMER SUCCESS. THE STUDY DESIGN ASSUMED THAT A PRESTRUCTURED CODING SYSTEM COULD BE DEVELOPED THAT WOULD PROCESS AND IMPACT PHASES OF EVALUATIONS. ARTICLES WERE ANALYZED, BASED ON THE FOUR EVALUATION CRITERIA, FOR CROSS-PROJECT COMPARISONS REGARDING PROCESS VARIABLES, IMPACT VARIABLES, AND METHODOLOGIES. PROBLEMS WERE ENCOUNTERED, HOWEVER, THAT PREVENTED THE ATTAINMENT OF THE ORIGINAL EVALUATION OBJECTIVES. THESE WERE INADEQUATE DESCRIPTIONS OF THE TREATMENT MODALITIES EMPLOYED; FREQUENT EXCLUSION OF SOCIODEMOGRAPHIC CHARACTERISTICS OF THE PATIENT POPULATION; EXTREME VARIATION IN STANDARDS FOR REPORTING CONCEPTUALLY SIMILAR OUTCOME MEASURES; REPORTING OF OUTCOMES IN UNOPERATIONALIZED TERMS, PARTICULARLY IN MEASURES OF SOCIAL ADJUSTMENT AND PSYCHOLOGICAL STATE; AND OTHER METHODOLOGICAL WEAKNESSES SUCH AS THE PHENOMENON OF THE 'SHRINKING' SAMPLE SIZE. SUGGESTIONS FOR FUTURE RESEARCH INCLUDE AWARENESS OF THE CRUCIAL RELATIONSHIP BETWEEN METHODOLOGY AND THE INTERPRETATION OF EVALUATION RESULTS, AND THE REQUIREMENT BY FUNDING AGENCIES OF A MINIMUM SET OF STANDARDS FOR REPORTING EVALUATIVE DATA WHILE ALLOWING CONTINUATION OF PROJECT-SPECIFIC MEASURES. IN ADDITION, THE RECOGNITION OF THE NEED FOR A GOOD WORKING RELATIONSHIP BETWEEN THE PROGRAM EVALUATOR AND THE PROFESSIONAL STAFF WILL CONTINUE TO BE IMPORTANT. REFERENCES AND TABULAR DATA ARE INCLUDED. (AUTHOR ABSTRACT MODIFIED-AOP)