Scribd
Upload
104 views

Student Detection of Errors in Examinations

Students were able to detect errors in exam answers with 74% sensitivity but attributed errors differently than tutors. The study aimed to establish a taxonomy of common error types, determine students' ability to detect errors marked by tutors as wrong, and encourage student reflection. Methods included a mock exam where 67 students self-marked and tutors later marked answers. Results found students detected most errors but differed from tutors in attributing specific error types. Future work could examine error detection between student years and factors affecting correlation with tutors.

Uploaded by

Chris Hebbes
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
104 views

Student Detection of Errors in Examinations

Students were able to detect errors in exam answers with 74% sensitivity but attributed errors differently than tutors. The study aimed to establish a taxonomy of common error types, determine students' ability to detect errors marked by tutors as wrong, and encourage student reflection. Methods included a mock exam where 67 students self-marked and tutors later marked answers. Results found students detected most errors but differed from tutors in attributing specific error types. Future work could examine error detection between student years and factors affecting correlation with tutors.

Uploaded by

Chris Hebbes
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 16

Student detection of errors in

examinations
Dr Christopher Hebbes ([email protected])
Dr Jenny Briggs
Dr Emmanuel Davidson
Dr Ricky Bell
Dr Heather Crick
Introduction
• 10% of the UK medical student population
struggle to pass exams
• Multiple reasons
• Implications for patient safety

Yates J, James D. Risk factors at medical school for subsequent professional misconduct: multicentre
retrospective case-control study. BMJ 2010;340:c2040
Self assessment
• Self assessment often inaccurate
•
Score = True score + Error component
•
• Why?
• Do students give the same error attribution
as their tutors?
•

I. Colthart, et al. (2008). `The effectiveness of self-assessment on the identification of learner needs, learner
activity, and impact on clinical practice: BEME Guide no. 10'. Medical Teacher 30(2):124-145.
Aims
• Establish a taxonomy of common types of
errors
• Determine ability to detect “gold standard
errors”
• Encourage reflection in learners
• Student engagement in assessment
Methods
• Pilot study SAQ Exam

• 67 students included
of 84
Self-marked Tutor-marked
– Failed one or both
semester exams
• Student consent
Errors coded Errors coded
• Mock exam
– 10 SAQ questions
Standardised
 Comparison coding sheet
Errors (overall)
False Negative

N Y
Tutor mark = max mark? Student mark = max mark?
N
Y True Positive
N
Student mark = max mark False Positive
Tutor
Y Error No Error
True Negative Student
Error
True PositiveFalse Positive

No Error
False Negative
True Negative
Errors (specific)
An error has occurred
YTrue Positive
Student code = X*

NFalse Negative
Y

N
Tutor mark = max mark? Tutor code = X*

Y N
YFalse Positive
Exclude Student code = X*
N
True Negative

*Where X is the test


Results
Results

Cronbach’s Alpha 0.81


Tutor: 29.45
Student: 44.29
p < .001
Eta2 .608
Error Detection (overall)
• 2107 Errors*
Tutor
• Students detected
Error No Error
74%
Student
Error
True Positive False Positive
• Error prevalence 0.88
1560 102 • Sensitivity 0.74
No Error
•
False Negative True Negative •
547 173

*Instances where full marks were not achieved for a part of a question
Error Type Detection
Comparison of tutor and student error attribution

Sensitivity Specificity
A 0.016 0.919
B 0.012 0.908
C 0.166 0.723
D 0.032 0.837
Summary
• Students are able to identify errors with
high sensitivity
• There is a difference in error attribution
between students and their tutors
Future work
• Is there a difference in error attribution
between first, second and third year
students?
• What factors are associated with
correlation between student and tutor
error attribution?
• Do interventions to help students attribute
errors improve correlation and improve
academic performance?
Acknowledgements
• Professor Stewart Petersen, Professor of Medical
Education, University of Leicester
• Dr Heather Crick, Lecturer, University of Leicester
• Dr Gordon French, Deputy Postgraduate Dean,
East Midlands Healthcare Workforce Deanery
• Dr Adrian Hastings, Senior Clinical Educator,
University of Leicester
• Dr David Matheson, Lecturer in Medical
Education, Medical Education unit, University of
Nottingham & East Midlands Healthcare
Workforce Deanery
http :// tinyurl . com / hebbes

?
References

You might also like