Evaluation of novel methods of assessments in pharmacology to develop new attitudes and skills in Caribbean Medical School
Keywords:Pharmacology, Teaching, Assessment, Curriculum, Problem-solving multiple choice questions
Background: Assessment is said to drive student learning and define the curriculum. The problem-solving type of multiple choice questions (MCQs), which can be used to probe and assess medical students in pharmacology should have a clinical vignette containing presenting complaints, abstract history, physical examination and laboratory data, followed by a single or series of questions based on it. National Board of Medical Examination (NBME), USA has an extensive bank of problem-solving MCQs, and these questions are often regarded as similar in format and focus to MCQs of United States Medical Licensing Examination (USMLE). The objective assessment of teaching and curriculum in this study is done by comparison of students’ performance in pharmacology comprehensive exam of NBME, USA before and after curriculum changes. This study was designed to obtain an objective assessment of teaching and curriculum by comparison of students’ performance in terms of student mean grades, percentage of students passed, percentage of students failed, percentage of students with honors, and individual highest scores of five semesters before and five semesters after curriculum changes in pharmacology comprehensive exam of NBME, USA among the 5th semester students of American University of the Caribbean, School of Medicine, St. Maarten.
Methods: We have compared the students’ performance of pharmacology comprehensive exam of NBME using five parameters like student mean grades, percentage of students passed, percentage of students failed, percentage of students with honor, and individual highest score of five semesters May 2009, September 2009, January 2010, May 2010 and September 2010 semester batches before the introduction of curriculum changes with subsequent semesters January 2011, May 2011, September 2011, January 2012 and May 2012 semesters after the introduction of curriculum changes.
Results: The pre-curriculum student performances were compared with post-curriculum changes using the Student’s t-test. The students mean scores improved significantly from 50.76 before curriculum changes to 56.54, students passed (%) increased from 94.57% before curriculum changes to 96.93% after curriculum changes and students with honors (%) increased significantly from 64.72% before curriculum change to 75.51% after curriculum changes and also seem to have remained consistently better. The students failed (%) decreased dramatically from 5.43% to 3.07% after curriculum changes. The highest individual mean score also improved significantly from 72.4 to 80.8 after curriculum changes and have remained consistent in the following semesters.
Conclusions: There seems to be obvious improvements in student performance as reflected by a significant increase in mean scores, students pass (%), and students with honors (%) probably due to inclusion of problem-solving MCQs in formative and summative assessments in new curriculum compared with declarative MCQs in old curriculum. The student failed (%) decreased dramatically, which could be attributed to the changes in teaching content and format brought by curriculum changes in pharmacology. The teaching of pharmacology principles as pathophysiology of drug therapy also seemed to have prepared students better for NBME comprehensive exam and also USMLE Step 1. The clinical pharmacology exercises in small groups as role playing sessions seem to have really improved students’ comprehension and retention of the basic sciences knowledge for clinical application based on students’ feedback.
Gwee MC. Teaching of medical pharmacology: the need to nurture the early development of desired attitudes for safe and rational drug prescribing. Med Teach. 2009;31(9):847 54.
Vollebregt JA, van Oldenrijk J, Kox D, van Galen SR, Sturm B, Metz JC, et al. Evaluation of a pharmacotherapy context-learning programme for preclinical medical students. Br J Clin Pharmacol. 2006;62(6):666-72.
O’Shaughnessy L, Haq I, Maxwell S, Llewelyn M. Teaching of clinical pharmacology and therapeutics in UK medical schools: current status in 2009. Br J Clin Pharmacol. 2010;70(1):143-8.
Wormald BW, Schoeman S, Somasunderam A, Penn M. Assessment drives learning: an unavoidable truth? Anat Sci Educ. 2009;2(5):199-204.
Hughes I. Teaching Pharmacology in 2010 – New knowledge, new tools, new attitudes. Nihon Yakurigaku Zasshi. 2003;122(5):411-8.
Faingold CL, Dunaway GA. Teaching pharmacology within a multidisciplinary organ system-based medical curriculum. Naunyn Schmiedebergs Arch Pharmacol. 2002;366(1):18 25.
Sadaf S, Khan S, Ali SK. Tips for developing a valid and reliable bank of multiple choice questions (MCQs). Educ Health (Abingdon) 2012;25(3):195-7.
Dijksterhuis MG, Schuwirth LW, Braat DD, Teunissen PW, Scheele F. A qualitative study on trainees’ and supervisors’ perceptions of assessment for learning in postgraduate medical education. Med Teach. 2013;35(8):e1396-402.
Qadir F, Zehra T, Khan I. Use of concept mapping as a facilitative tool to promote learning in pharmacology. J Coll Physicians Surg Pak. 2011;21(8):476-81.
Palmer E, Devitt P. Constructing multiple choice questions as a method for learning. Ann Acad Med Singapore. 2006;35(9):604-8.
Schuwirth LW, Van der Vleuten CP. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach. 2011;33(6):478-85.
Al-Faris EA, Alorainy IA, Abdel-Hameed AA, Al Rukban MO. A practical discussion to avoid common pitfalls when constructing multiple choice questions items. J Family Community Med. 2010;17(2):96-102.
Al-Rukban MO. Guidelines for the construction of multiple choice questions tests. J Family Community Med. 2006;13(3):125-33.
Kazubke E, Schüttpelz-Brauns K. Review of multiple-choice-questions and group performance - A comparison of face-to-face and virtual groups with and without facilitation. GMS Z Med Ausbild. 2010;27(5):Doc68.
Abdel-Hameed AA, Al-Faris EA, Alorainy IA, Al Rukban MO. The criteria and analysis of good multiple choice questions in a health professional setting. Saudi Med J. 2005;26(10):1505-10.
Al-Rubaish AM, Al-Umran KU, Wosornu L. An audit of assessment tools in a medical school in eastern saudi arabia. J Family Community Med. 2005;12(2):101-5.
Collins J. Education techniques for lifelong learning: writing multiple-choice questions for continuing medical education activities and self-assessment modules. Radiographics. 2006;26(2):543-51.
Baig M, Ali SK, Ali S, Huda N. Evaluation of Multiple Choice and Short Essay Question items in Basic Medical Sciences. Pak J Med Sci. 2014;30(1):3-6.
Schuwirth LW, van der Vleuten CP. Different written assessment methods: what can be said about their strengths and weaknesses? Med Educ. 2004;38(9):974-9.
Gajjar S, Sharma R, Kumar P, Rana M. Item and Test Analysis to Identify Quality Multiple Choice Questions (MCQs) from an Assessment of Medical Students of Ahmedabad, Gujarat. Indian J Community Med. 2014;39(1):17-20.