NCJ Number
56066
Journal
Evaluation and The Health Professions Volume: 1 Issue: 5 Dated: (MARCH 1979) Pages: 100-109
Date Published
1979
Length
10 pages
Annotation
IT HAS BEEN SUGGESTED THAT THE RELIABILITY AND VALIDITY OF CONFIDENCE TESTS COULD BE IMPROVED BY CORRECTING SCORES FOR REALISM. AN EXPERIMENT USING 36 STUDENTS FINDS THIS TRUE, ALTHOUGH RESULTS WERE NOT SIGNIFICANT.
Abstract
THE CONCEPT OF REALISM IN MULTIPLE-CHOICE TESTING IS EXPLAINED IN DETAIL. THIS PAPER SUGGESTS THAT THE PREDICTIVE RELIABILITY OF SUCH TESTING IS INCREASED IF THE STUDENT IS ALLOWED TO ASSIGN PROBABILITY WEIGHTS TO THE VARIOUS ANSWERS. THIS CLAIM WAS TESTED BY ADMINISTERING A SET OF 18 QUESTIONS ON CELLULAR AND MOLECULAR BIOLOGY TO A GROUP OF 10 FEMALE AND 11 MALE MEDICAL STUDENTS AND 5 FEMALE AND 10 MALE DENTAL STUDENTS, ALL ENROLLED AT THE UNIVERSITY OF CONNECTICUT. RELIABILITY WAS MEASURED BY COMPARING SCORES ON THIS SUBSET OF 18 QUESTIONS TO SCORES OBTAINED BY THESE STUDENTS ON A FINAL EXAMINATION IN A CELLULAR AND MOLECULAR BIOLOGY COURSE AND, FOR 27 OF THE STUDENTS, ON PART I OF THE NATIONAL MEDICAL BOARDS. COMPUTER TERMINALS PROVIDING VISUAL DISPLAYS WERE USED IN THE EXPERIMENT AND THE STUDENTS MOVED A LIGHTED SPOT ON THE DISPLAY TO INDICATE DEGREE OF CONFIDENCE IN THE ANSWER. THE 18-ITEM SCALE PROVED MODERATELY PREDICTIVE OF PERFORMANCE ON THE LARGER TESTS. IT WAS FOUND THAT REALISM DECREASED AS KNOWLEDGE DECREASED. IT IS SUGGESTED THAT A QUADRATIC RELATIONSHIP EXISTS BETWEEN REALISM AND KNOWLEDGE WITH SOME INTERMEDIATE POINT OF IGNORANCE AT WHICH A LITTLE KNOWLEDGE BECOMES A DANGEROUS THING. ALTHOUGH THE DIFFERENCES IN RELIABILITY AND VALIDITY WERE NOT STATISTICALLY SIGNIFICANT, THEY WERE IN THE EXPECTED DIRECTION. FURTHER APPLICATION OF THE METHOD IS URGED. REFERENCES AREE APPENDED. (GLR)