Reliability and validity of short answer questions in Pharmacology
DOI:
https://doi.org/10.18203/2319-2003.ijbcp20191594Keywords:
Atropine, Definition, Reliability, Short answer questions, ValidityAbstract
Background: Short answer question (SAQ) is an excellent tool for assessment. Objective of this study was to understand the practices adopted in answering SAQ, assessment of reliability and validity of SAQ and faculty feedback on valuing SAQs.
Methods: A cross-sectional study was done in the Department of Pharmacology of a Government Medical College in Central Kerala from December 2018-January 2019. The study participants were given 10 minutes to answer the two chosen SAQs after which the response sheets were collected and valued by 2 independent examiners. Feedback on correction of the same questions was sought from Pharmacology faculties of various Medical Colleges through online survey. The data were analysed using SPSS software.
Results: 100% participants attempted to answer the SAQ ‘Definition of a drug’ but none of them came out with the World Health Organisation definition of drug. For the second SAQ, ‘Two uses of atropine’ around 25(20.8%) participants had written two uses of the drug correctly. The reliability was 0 for ‘definition of a drug’ and 0. 50 (95% CI 0.082-0.702) for ‘Two uses of atropine’ and both the questions had no validity. The feedback on valuing definition and two use of atropine was variable.
Conclusions: None of the students wrote definition of a drug correctly and majority gave multiple responses to uses of atropine instead of two valid responses. The short answer questions we tested were found to have moderate to no reliability and no validity. The faculties had variable perceptions in valuing the two short answer questions.Metrics
References
Pham H, Trigg M, Wu S, O'Connell A, Harry C, Barnard J, et al. Choosing medical assessments: Does the multiple-choice question make the grade?. Educ Health. 2018;31:65-71.
Hift RJ. Should essays and other “open‑ended”‑type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014;14:249.
Sreedharan S, Hyma V, Palappallil DS. Concept learning through question framing in Pharmacology. Int J Basic Clin Pharmacol. 2018;7:319-23.
Palmer EJ, Duggan P, Devitt PG, Russell R. The modified essay question: Its exit from the exit examination? Med Teach. 2010;32:e300‑7.
How do you determine if a Test has Validity, Reliability, Fairness, and Legal Defensibility? Last Accessed on 20 February 2019. Available at: http://www.proftesting.com/test_topics/test_quality.php
Classroom Assessment. Last accessed on 20 February 2019. Available at: https://fcit.usf.edu/assessment/basic/basicc.html
Tripathi KD. Essentials of Medical Pharmacology. 8th Edition. Jaypee Brothers Medical Publishers; 2019:2.
Schuwirth LW, van der Vleuten CP, Donkers HH. A closer look at cueing effects in multiple‑choice questions. Med Educ. 1996;30:44‑9.
Sam AH, Hameed S, Harris J, Meeran K. Validity of very short answer versus single best answer questions for undergraduate assessment. BMC Med Educ. 2016;16:266.
Wood T. Assessment not only drives learning, it may also help learning. Med Educ. 2009;43:5-6.
Kesavan KP, Palappallil DS. Effectiveness of Formative Assessment in Motivating and Improving the Outcome of Summative Assessment in Pharmacology for Medical Undergraduates. J Clin Diagn Res. 2018;12(5):FC08-11.
Downloads
Published
How to Cite
Issue
Section
