Determination of Difficulty Index in End of Block Examinations of Preclinical Undergraduate Medical Students

Background: Multiple Choice Questions (MCQs) are considered a good choice for undergraduate formative assessment as they have higher reliability and are generally feasible. The objective of this study was to evaluate the difficulty index of Multiple-Choice Questions in the end of block examination of preclinical undergraduate medical students at Rawalpindi Medical University.
Methodology: This descriptive cross-sectional study was conducted at Rawalpindi Medical University. End of Block assessment data was collected from the Department of Medical Education, RMU for first-year and second-year MBBS comprising of 60 MCQs for each year. The difficulty index of the total 120 MCQs was calculated after entering data in MS Excel. Data was then entered in SPSS version 24. Means and standard deviations of Difficulty indices were calculated and compared between first-year and second-year students by independent samples students t-test and between subjects by ANOVA.
Results: Out of 120 Multiple Choice Items analyzed, in the first year MBBS block exam, 30% were easy, 65% were acceptable and 5% were difficult. In the second year MBBS block exam, 36.67% were easy, 56.67% were acceptable and 6.67% were difficult. There was no significant difference (p=0.986) between the mean difficulty index of first-year MBBS students and second-year MBBS students. However, the mean difficulty index was highest in physiology (66.53 ± 16.262) followed by biochemistry (64.36 ± 16.756) and anatomy (54.80 ± 17.665), and the mean difference between the subjects was statistically significant (p=0.005). The mean difficulty index in first-year MBBS students was highest for Biochemistry followed by Physiology and Anatomy. In second-year MBBS students, the mean difficulty index was highest in Physiology followed by Biochemistry and Anatomy.
Conclusion: The difficulty index of Anatomy MCQs was lower indicating that the students find them more difficult than the other two subjects i.e., Physiology and Biochemistry in the first two years of their undergraduate medical studies.
Keywords: Anatomy, Biochemistry, Medical education, Physiology

Assessment is an essential component of the education cycle. Formative assessment which is done during a training program is intended to provide feedback to the students regarding their knowledge base¹. It also provides feedback to the educator to evaluate the efficacy of teaching and training². The best assessment method must meet five criteria which include reliability, validity, acceptability, feasibility, and educational impacts on learning and practice³.
Multiple Choice Questions (MCQs) are considered a good choice for undergraduate formative assessment as they have higher reliability due to greater number of items and are generally feasible and acceptable to both the students and assessors^{4, 5}. However, the reliability and validity of this form of assessment can be compromised if the MCQs are overly complex, out of course, or beyond the level of the learner.6 Post hoc evaluation of MCQs involves various measures to ensure the quality of assessment⁷. One of these measures is the Difficulty index which can inform the assessor how easy or difficult the questions were for the examinees⁸.
At Rawalpindi Medical University, each undergraduate class has approximately three hundred and sixty students. They undergo end of block formative exams after every block. Each block exam has a multiple choice and short answer component which is designed by subject specialist. Each MCQ has a single stem with five options including one correct answer and four distractors and carries one mark. Maximum marks in MCQ assessment are 60 and the minimum is zero, with no negative marking.
As there is no current study on the difficulty index of MCQs in our local context especially preclinical undergraduate medical students, our objective in this study was to evaluate the degree of difficulty of MCQs at the end of the block examination by analyzing the post hoc difficulty index. This will help the medical educationists in developing better guidelines and recommendations for end of block assessments.

This descriptive cross-sectional study was conducted at Rawalpindi Medical University. A single block assessment was selected for each year. Each block assessment comprised of 60 MCQ. A total of 120MCQs (60 for first year and 60 for second year). The results of both papers were ranked in descending order, from highest to lowest marks. Then they were divided into quartiles. The Upper quartile (high score group) and lower quartile (low score group) were selected for analysis. Papers with average scores (middle quartiles) were excluded from the study. Data was entered into Microsoft Excel and the Difficulty Index (DIF) of each item was calculated using the following formula,
DIF= [(H+L)/N] × 100 Where H is the number of students who gave correct options in the high score group, L is the number of students who gave correct options in the low score group, N is the total number of students in both groups. Results were interpreted as Difficult (DIF<30%), Acceptable (DIF 30-70%), and Easy (DIF >70%). For analysis, the data was entered in SPSS version 24. Means and standard deviations of Difficulty indices were calculated and compared between first-year and second-year students by independent samples student t-test and between subjects by ANOVA.

Out of 120 Multiple Choice Items analyzed, in the first year MBBS block exam, 18 items (30%) were easy, 39 items (65%) were acceptable and 3 items (5%) were difficult. In the second year MBBS block exam, 22 items (36.67%) were easy, 34 items (56.67%) were acceptable and 4 items (6.67%) were difficult. There was no significant difference (p=0.986) between the mean difficulty index of first and second-year MBBS students. (Table-I)

Table-I: Comparison of Mean and Standard Deviation of Difficulty Index in First and Second Year MBBS Students by Independent Samples t-test

There was a significant statistical difference (p=0.005) between the mean difficulty index of questions of various subjects for both classes. The mean difficulty index was highest in Physiology followed by Biochemistry and Anatomy. (Table-II)

Table-II: Comparison of Mean and Standard Deviation of Difficulty Index of MCQs of different subjects by ANOVA

The mean difficulty index in first-year MBBS students was highest for Biochemistry followed by Physiology and Anatomy. Among second-year MBBS students, the mean difficulty index was highest in Physiology followed by Biochemistry and Anatomy. (Figure-II)

Figure-II: Bar Graph for comparison of Mean Difficulty Index in various subjects in First and Second Year MBBS students

Assessment in education has a key role in evaluating whether the goals of instruction are achieved.4 Multiple Choice Questions are a frequently used tool for formative assessment of undergraduate medical students due to their high reliability compared to other assessment tools.9 Post hoc difficulty analysis of MCQs can give us an insight into how easy or difficult the questions are for the students.10
For our undergraduate medical students, most of the items were of appropriate difficulty. The percentage of easy items was more than the difficult ones. Ideally, most MCQs should have an appropriate difficulty which is noted in our block exams both for first and second-year students^{11, 12}. In current practice for the block exams, no method of standard setting is used which could be a reason of this finding. Other studies have also reported the average difficulty of MCQs in undergraduate medical assessments¹³. According to a similar study conducted by Thompson AR and Giffin BF, there was no significant difference in the mean difficulty index on items between the first year and second-year MBBS students which indicates similar findings in both years¹⁴. Overall, the items of Anatomy were the most difficult followed by Physiology and Biochemistry. In the first year MBBS block exam, Biochemistry items were the easier and in the second year MBBS block exam Physiology questions were easier compared to the other subjects. Anatomy is generally reported to be a difficult subject for undergraduate medical students¹⁵. Areas such as Pelvis and Neuroanatomy are reported to be particularly challenging¹⁶. Due to the extensive nature of the subject comprising of various domains such as Embryology, Histology, and Gross Anatomy, it is more extensive and complex for undergraduate medical students compared to the other two subjects¹⁷.
As our curriculum progresses into integration between subjects, the threat remains that students will focus less on difficult subjects such as Anatomy and will gravitate towards spending more time on easier subjects such as Physiology and Biochemistry to pass their examinations. This may result in less retention of the subject over the course of MBBS program in the integrated modular curriculum^{18, 19}.
Multicenter studies are required to see if a similar pattern of difficulty is observed in other medical institutes of the region. Follow-up qualitative studies can further give insight into the key areas which our undergraduate medical students find difficult and appropriate academic interventions can be planned to address them.
This is the first study on item analysis of undergraduate end of block examinations at Rawalpindi Medical University. Due to lack of optical mechanical scanner, other item analysis indices i.e Reliability coefficient, Discrimination index and Distractor analysis could not be performed which would give more information about the quality of these assessments.

The MCQs in Anatomy were more difficult than the other two subjects i.e., Physiology and Biochemistry in the first two years of undergraduate medical studies in the MBBS program. In future, mixed-method studies are recommended to determine the reasons and devise strategies to improve assessment in this subject.

1. Palacios BAP. Formative assessment as tool to improve on teaching process–learning for students. Int J Soc Sci Humanit. 2019;3(3):36-49. doi:10.29332/ijssh.v3n3.354
2. McCarthy J. Enhancing feedback in higher education: Students’ attitudes towards online and in-class formative assessment feedback models. Act Learn High Educ. 2017;18(2):127-141. doi:10.1177/1469787417707615
3. Dent J, Harden RM, Hodges BD, Hunt D. A Practical Guide for Medical Teachers. Elsevier; E-Book. Elsevier health sciences; 2021 Apr 24.
4. Haris Ali S, Carr PA, Ruit KG. Validity and Reliability of Scores Obtained on Multiple-Choice Questions: Why Functioning Distractors Matter. J Scholarsh Teach Learn. 2016;16(1):1-14. doi:10.14434/josotl.v16i1.19106.
5. Holzinger A, Lettner S, Steiner-Hofbauer V, Capan Melser M. How to assess? Perceptions and preferences of undergraduate medical students concerning traditional assessment methods. BMC Med Educ. 2020;20(1):1-7. doi:10.1186/s12909-020-02239-6
6. Quaigrain K, Arhin AK. Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Educ. 2017;4(1). doi:10.1080/2331186X.2017.1301013
7. Ramzan M, Imran SS, Bibi S, Khan KW, Maqsood I. Item Analysis of Multiple-Choice Questions at the Department of Community Medicine, Wah Medical College, Pakistan. Life Sci. 2020;1(2):4. doi:10.37185/lns.1.1.53
8. Mahjabeen W, Alam S, Hassan U, Zafar T, Butt R, Konain S, et al. Difficulty index, discrimination index and distractor efficiency in multiple choice questions. Annals of PIMS. 2017;13(4):310-5.
9. Rai N. Multiple Choice Questions: As Formative Assessment. Int J Med Biomed Stud. 2019;3(3). 75-79. DOI:10.32553/ijmbs.v3i3.137
10. D'Sa JL, Visbal-Dionaldo ML. Analysis of Multiple Choice Questions: Item Difficulty, Discrimination Index and Distractor Efficiency. International Journal of Nursing Education. 2017 Jul 1;9(3):109-114. doi: 10.5958/0974-9357.2017.00080.0
11. Patel RM. Use of Item analysis to improve quality of Multiple Choice Questions in II MBBS. Journal of Education Technology in Health Sciences. 2017 Apr 29;4(1):22-9.
12. Mehta G, Mokhasi V. Item Analysis of Multiple Choice Questions-An Assessment of the Assessment Tool. Int J Heal Sci Res. 2014;4. www.ijhsr.org. Accessed November 17, 2020.
13. Islam ZU, Usmani A. Psychometric analysis of Anatomy MCQs in Modular examination. Pak J Med Sci. 2017 Sep-Oct;33(5):1138-1143. doi: 10.12669/pjms.335.12382.
14. Thompson AR, Giffin BF. Higher‐Order Assessment in Gross Anatomy: A Comparison of Performance on Higher‐ versus Lower‐Order Anatomy Questions between Undergraduate and First Year Medical Students. Anat Sci Educ. October 2020. doi:10.1002/ase.2028
15. Alvarez S, Nikendei C, Schultz J-H. Development of a didactical training concept for peer tutors in gross anatomy. Anat Sci Educ. 2017;10(5):495-502. doi:10.1002/ase.1691
16. Hall S, Stephens J, Parton W, Myers M, Harrison C, Elmansouri A, et al. Identifying Medical Student Perceptions on the Difficulty of Learning Different Topics of the Undergraduate Anatomy Curriculum. Med Sci Educ. 2018;28(3):469-472. doi:10.1007/s40670-018-0572-z
17. Zill SN. Rethinking gross anatomy in a compressed time frame: Clinical symptoms, not case studies, as the basis for introductory instruction. Clin Anat. July 2020:ca.23645. doi:10.1002/ca.23645
18. Doomernik DE, van Goor H, Kooloos JGM, ten Broek RP. Longitudinal retention of anatomical knowledge in second-year medical students. Anat Sci Educ. 2017;10(3):242-248. doi:10.1002/ase.1656
19. Hołda MK, Stefura T, Koziej M, Skomarovska O, Jasińska KA, Sałabun W, et al. Alarming decline in recognition of anatomical structures amongst medical students and physicians. Ann Anat. 2019;221:48-56. doi:10.1016/j.aanat.2018.09.004