APPLYING TIME SERIES STRATEGIES - AN UNDERUTILIZED SOLUTION (FROM IMPROVING EVALUATIONS, 1979, BY LOIS-ELLIN DATTA AND ROBERT PERLOFF - SEE NCJ-64392)

• NCJ Number: 64396
• Author(s): M S KNAPP
• Date Published: 1979
• Length: 17 pages
• Corporate Author: Sage Publications, Inc
• Publication Format: Document
• Publication Type: Program/Project Evaluation
• Language: English
• Country: United States of America

Annotation

THIS RESEARCH PAPER DESCRIBES THE USE OF TIME-SERIES DATA TO ROLE OUT COMPETING HYPOTHESES AND ESTABLISH CAUSALITY IN PROGRAM EVALUATION.

Abstract

TIME-SERIES STRATEGIES USE QUANTITATIVE DATA, BUILD CAREFULLY CONTROLLED RESEARCH DESIGNS, AND INCREASINGLY INVOLVE SOPHISTICATED FORMS OF DATA ANALYSIS. THE FRAMEWORK FOCUSES DIRECTLY ON THE INDIVIDUAL UNIT AND ALLOWS INVESTIGATION OF EFFECT PATTERNS IN ADDITION TO EFFECTS MEASURABLE AT ANY ONE POINT IN TIME. IT IS AN EVOCATIVE MECHANISM WHEREBY HYPOTHESES CAN BE GENERATED SYSTEMATICALLY, AND IT DEMONSTRATES THE PERSISTENCE OF EFFECTS OVER TIME AND THE REPLICABILITY OF EFFECTS ACROSS INDIVIDUAL UNITS. IN ADDITION TO THESE CONCEPTUAL CONTRIBUTIONS, TIME-SERIES STRATEGIES BRING TO CERTAIN TYPES OF EVALUATION SITUATIONS ADVANTAGES OF A MORE PRACTICAL KIND. UNSTABLE TREATMENTS CAN BE ACCOMMODATED MORE FLEXIBLY BY DESIGNS LOOKING FOR CHANGING PATTERNS OVER TIME. TIME-REFERENCED EVALUATIONS CAN BE MOUNTED AFTER THE FACT, IF RELIABLE RECORDS EXISTS, OR A MORE SOLID BASIS THAN THE SELECTIVE MEMORIES OF PARTICIPANTS. DATA FROM AN EVALUATION STUDY MAY BE UNDERSTOOD MORE INTUITIVELY, AND MORE FLEXIBILITY EXISTS IN DATA REPORT TIMING. FURTHERMORE, TIME-SERIES EVALUATIONS ARE CHEAP TO IMPLEMENT. SPECIAL LIMITATIONS EXIST, HOWEVER. DESIGN COMPLEXITIES MAY BE DIFFICULT TO IMPLEMENT. VARIABLES ASSESSED BY COMPLEX TESTS OR INTERVIEWS ARE EXCLUDED FROM TIME SERIES TREATMENT. TIME SERIES STUDIES OFTEN TAKE A LONG TIME TO COMPLETE. ESTABLISHING A LONG ENOUGH BASELINE TO ALLOW SOLID INTERPRETATIONS OF CHANGES IN THE DATA SERIES IS ANOTHER FREQUENT PROBLEM, AND RESULTS, WHEN AVAILABLE ARE GENERAL TO LARGE POPULATIONS ONLY IN A SPECIAL SENSE. CHANGES IN RECORDKEEPING, SHIFTS IN THE COMPOSITION OF AN AGGREGATED UNIT OVER TIME, AND MISSING DATA CAN COMPLICATE INTERPRETATION OF A TIME SERIES. NONETHELESS, TIME-SERIES APPROACHES SEEM TO OFFER A POWERFUL POTENTIAL TO ALLOW THOSE WHO WISH TO FORM AN EVALUATIVE JUDGMENT OF AN ONGOING PROGRAMMATIC EFFORT TO COMBINE SYSTEMIC ARRAYS OF PRECISE MEASUREMENT AT REGULAR INTERVALS WITH THE UNFOLDING NARRATIVE OF EVENTS WHICH INCLUDES A KNOWLEDGE OF THE START AND DURATION OF THE PROGRAM. REFERENCES ARE CITED. (MJW)