AUCTORES
Globalize your Research
Research Article | DOI: https://doi.org/DOI:10.31579/2690-4861/154
*Corresponding Author: Santhi Silambanan, Professor, Department of Biochemistry, Sri Ramachandra Institute of Higher Education and Research, Porur, Chennai, Tamil Nadu, India.
Citation: S Silambanan, Sowmya K, Prabhu Kumar CM. (2021) Evaluating the best tool to assess the cognitive domain of first year MBBS students in the subject of Biochemistry. International Journal of Clinical Case Reports and Reviews. 7(5); DOI:10.31579/2690-4861/154
Copyright: © 2021 Santhi Silambanan, This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Received: 01 July 2021 | Accepted: 23 July 2021 | Published: 26 July 2021
Keywords: cognitive domain; assessment tool; essays; multiple choice questions; short-answer questions; case-based questions
Introduction: Assessment is an essential part of the learning process in education. Students follow a surface approach when assessment emphasis is on recall of factual knowledge. Students adopt a deep approach if assessment demands cognitive abilities. The aim of the study was to evaluate the best tool for assessing the cognitive domain of the first year MBBS students.
Materials and Methods: I MBBS students of 28 in number who had scored more than 60 percent marks in the formative assessment examinations were included. Institutional Ethics Committee approval was obtained. Ten types of assessment tools were used. Answer scripts were evaluated by two examiners. SPSS version 16.0 was used for statistical analysis. ANOVA and Tukey posthoc analysis were used. A P value less than 0.05 was considered statistically significant.
Results: All the students performed well in the close-ended questions with statistical significant P=0.000
Conclusion: Good performers did well in all types of questions. Average performers did well in closed ended questions. Poor performers did not perform in both. Essays and SAQs remain the choice to differentiate a best performer in good performing candidates.
Assessment drives learning. Assessment should instill interest and encourage the students to learn more. Present system enforces on just passing the examination. It should meet the challenges of changing attitudes of students and patients. Teaching and learning in Biochemistry by undergraduate medical students is primarily knowledge specific. There are various teaching learning evaluation methods to assess the impact of teaching and learning Biochemistry. Lot of studies have been done to assess the effectiveness of various assessment tools in assessing the knowledge gained by the students. But it has not been proven till date which could serve as the best tool to effectively assess the cognitive domain of the students. Each tool has its inherent advantages and disadvantages [1]. This study was undertaken to assess the best tool to assess the cognitive domain of first year medical students.
Study participants were 28 first year MBBS students from a tertiary care teaching hospital in Chennai, India. Students who had attended and secured more than 60 percent marks in the six formative assessment examinations were included in the study. Students who missed an examination or obtained less than 60 percent in an examination were excluded. The study was conducted after the completion of the first year university examination so that the results of this examinations did not have a bearing on the marks of the students who were participating in the study. The students who were willing to participate alone were included and the study was conducted as per Helsinki declaration. Institutional Ethics Committee approval was obtained (IEC-NI/20/SEP/75/87 dated 10-11-2020) and waiver of consent was obtained since the students have already completed the course. The student's identity was not disclosed at any time during the study as well as during the publication of this article.
Ten types of written assessment tools were used such as essays, short answer questions (SAQs), Visuals, case-based questions, fill in the blanks, true/false, MCQs, match the following, odd man out and fill in the matching words. The duration of the examination was for three hours with total score of 140 marks. The marks for the individual assessment tools were allotted depending on the complexity of the assessment tools. Answer scripts were evaluated by two senior professor; in situations where the discrepancy between the two professors was more than ten percent, third evaluation was sought.
The average of the marks obtained by the students under each assessment tool was expressed as mean and standard deviation. ANOVA and Tukey posthoc analysis were used to compare the performance of the students in the various assessment tools. SPSS version 16.0 was used for statistical analysis. A P value less than 0.05 was considered statistically significant.
Table 1: Comparison of average of the marks obtained by the students in each Assessment Tool
Values are expressed as Mean±SD; *: P Value: significant; **: P value: highly significant
Figure 1: Percentage of Students with performance > 50% in each of the assessment tools
Figure 2: Comparison of performances in Open (Essays, Short answer) - Vs Closed (others) -ended Questions
Assessment in medical curriculum is of cardinal importance. The assessment procedures have a powerful influence over the learning process [2]. It has been reported that one of the most important factors influencing students’ choice of learning approach is the way how assessment is being conducted [3]. Evaluation is a systematic process of determining the extent to which the objectives have been achieved by the student [4]. Cognitive domain focuses on intellectual skills. Bloom’s taxonomy [knowledge, comprehension, application, analysis, synthesis, and evaluation] is frequently used to describe the increasing complexity of cognitive skills as students move forward from a beginner to more advance level in their knowledge. Cognitive domain is the core of the learning domain [5].
The students who are good in skills and knowledge can handle any type of examinations with ease. The problem arises from the average students and low achievers since they lack interest in the subject due to various reasons and they require special attention. There is no uniformity in the assessment methods/tools due to various reasons such as poor understanding of the subject, examiner bias, assessment not based on defined objectives, choosing wrong assessment tool and inadequate expertise in the preparation of assessment tools [1].
Effectiveness of teaching and learning Biochemistry is by various methods such as group discussions, tutorials, problem-based approach, and seminars along with traditional didactic lectures [6, 7]. Biochemistry needs to be taught and learned effectively in the context of the disease to facilitate the transfer of this knowledge in terms of diagnosis and treatment of the patients [8, 9]. There have been many attempts at developing curricula to make biochemistry more relevant to the study of medicine and textbooks have been written with this objective [10, 11]. The undergraduate teaching curricula for medical students should be switched over to ‘Students centered’, which would focus on ‘problem-solving’ rather than on ‘information gathering’ skills. To achieve this, we should implement a combination of different teaching and learning methods [12, 13].
Formative assessment is most effective when it is nested in the ongoing teaching and learning program in order to facilitate timely, specific and actionable feedback to learners. Students can regulate their own performance with the help of good formative assessments [14]. Each assessment method has its own advantages and disadvantages, it is better to employ a variety of assessment methods the shortcomings of one can be overcome by the advantages of another [15]. Before 1950s, assessments were mostly based on the written examinations [16]. Globally it is accepted that the assessment in medical education should not be simply based on evaluation on pre-set criteria and make a judgement but also to facilitate learning through consistent involvement and timely feedback, providing an opportunity to improve [17].
In the present study, average of the marks obtained by all the students in each of the assessment tool showed that there was statistical significant difference in the marks obtained between the various assessment tools, as shown by P =0.0000. On an average almost all students had scored the highest in the true/false assessment tool. Next higher performance was noted in three assessment tools such as match the following, choosing the odd man out and fill in the matching words. Medium performance was seen in the assessment tools such as in essays, visuals, case-based questions and fill in the blanks. They had performed poorly in short answer questions (SAQs) and MCQs. The assessment tool- true/ false has left with only two options which could have been easier for the students to respond. When compared true/false tool with other assessment tools, true/ false tool showed highly statistically significant increase in scores against rest of the assessment tools except the assessment tool, odd man out (Table 1).
Formal assessment in most medical colleges is focussed on the satisfactory attainment of knowledge and skills [18]. Classroom assessments done by the process of gathering, recording, interpreting, using and communicating information about their progress and achievement during the development of knowledge, concepts, skills and attitudes [19]. An ideal assessment tool would possess features such as accountability, flexibility, comprehensiveness, feasibility, timeliness and relevance to both the examiner and examinee [20].
In the present study, the assessment tools with mean score of more than 50 indicated that these assessment tools could give some cues which facilitated them in answering correctly. Also these tools were not being routinely used, hence the students could have found them interesting and easier to answer. When compared between the groups there was statistically decreased performance against the assessment tool, true/false. But there was statistically significant increased performance against the two routinely used tools that is short answer questions (SAQs) and multiple choice questions (MCQs). The assessment tools with mean score of more than 40 indicated that in essay, the student tends to regurgitate whatever the student had learnt under the topic and thus the student were eligible to get marks. In visuals, case-based questions and fill in the blanks the students could perform probably because they were interesting to them and that could correlate clinical settings. When compared between the tools, it showed that there was statistical significant lesser performance in the items with mean scores of more than 50. But statically significant increase in performance compared to SAQs (Table 1).
MCQ is a time-tested method of assessment of knowledge in medical education for the purpose of ranking in the order of merit [2]. Students with recall ability score better in MCQs and students who had best analytical or interpretative skills score better in essays. Scoring was better by MCQs as it is exclusive of the examiners bias [1]. A well-structured MCQ is appropriate for measuring knowledge, comprehension and for analysis due to reliability and validity and better in scoring. MCQs are better to assess all domains of a student and thus a better assessment tool. The advantages with MCQs are that at a time many students can be evaluated and reliability in assessment can be ensured [1]. MCQs are being used increasingly due to their higher reliability, validity, and ease of scoring [3]. A major disadvantage of MCQs is they are often poorly written in a way that test memory recall rather than application of knowledge. However, writing MCQs that evaluate application of knowledge can be challenging, and most faculties are not formally trained; therefore, making it difficult to develop test questions that require application and critical thinking skills [21].
In essays, and short answer questions (SAQs), students need higher order cognitive skills (analytical, interpretative and application skills) [1]. SAQ involves writing short answers to questions sampled from a large part of the curriculum. SAQ carries greater objectivity and reliability and their range of subject areas tested is extended. Essay-type assessment is a sensitive test requiring students not only to recall facts but also to use higher-order cognitive skills. Essay questions though time consuming provides a unique evaluation tool particularly suited for the undergraduate settings [3]. Essays are harder than MCQs for demonstrating the ability to grasp minute details. Writing skills that includes handwriting and organizing skills affect scoring. Anxiety level goes up while answering essay type of questions [22]. In depth studies have gone on to suggest that students generally prefer MCQs but become more enthusiastic about essays when they are well prepared for the examinations [22].
High quality questions are important for medical student assessments. This is even more essential for summative assessments where medical student performance influences their progression within the program and results in a final grade for a course. Problem-solving questions involving short vignettes can be used to assess application of knowledge (procedural knowledge) rather than simple factual recall (declarative knowledge). If the vignettes are not constructed correctly, the examination question can become a pseudovignette that ends with a declarative knowledge (recall) question. It is common when it is written by a faculty member who has not received training in test development. It is time consuming and challenging to write a true clinical vignette to ask basic science questions; however, this can lead to better test questions and the assessment of student knowledge at a higher level [21].
In the present study, the performances in SAQs and MCQs were the least of all the assessment tools used. The student was expected to give the apt answers in few points; the students read the topics on the whole in general, but they could not understand what is clinically important or relevant. If the students should answer the MCQs correctly, the student must be read thorough with each and every point in that chapter. It was not just learning, the student must be able to analyze critically. The MCQs were generally tougher because of the question pattern. The question has a stem and four or five plausible distractors. The distractors are very closely connected differing in minute details. This showed that the student needed to improve thinking, analyzing and correlating aspects. The performance could have better, if the students were involved in lot of discussions within the peers, with the senior undergraduate students, the postgraduate students and the teachers. (Table 1)
Written examination questions are typically classified according to whether they are open-ended or MCQs. In addition, questions can be “context rich” or “context poor.” MCQs may create situations in which an examinee can answer a question by recognizing the correct option, but could not have answered it in the absence of options. This effect is called cueing. Extended matching items (several questions, all with the same long list of possible answers), as well as open-ended short-answer questions, can minimize cueing. Structured essays preclude cueing. In addition, they involve more complex cognitive processes and allow for more contextualized answers than do MCQs [23].
The present study showed that more than 15% of students scored more than 50% in the assessment tools, true/false and fill in the matching words. 10-14% of students scored more than 50% in the assessment tools such as case based learning, match the following and odd man out. 9% of the students scored more than in essays, SAQs, visuals, fill in the blanks and MCQs (Figure 1).
Three forms are commonly used- formative assessment, summative or end-point assessment and self-assessment. Self or ipsative assessment - measuring oneself; test against standards or specific parameters, giving an idea of the performance. This form of assessment becomes very important in teaching methods such as problem-based learning, where self-directed learning is required [24]. As the students of today are more inclined to question and challenge grades, there is a growing emphasis on ensuring that assessment methods are reliable, valid and able to sustain legal scrutiny. Assessment has three goals: to optimise the capabilities of all learners, to protect the public by identifying incompetent physicians and to provide a basis for choosing applicants for advanced training [20, 25].
In the index study, figure 2 shows the performance of the students in each of the assessment tool individually. To simplify the interpretation grouping was done as open ended and close ended questions. Essays and SAQs were grouped as open-ended questions which are being traditionally followed. The rest of the assessment tools were grouped as closed ended questions. On the whole the performance of all the candidates were better in closed ended questions compared to open ended questions. Also, it was inferred that the student who scored the second highest in close ended questions, scored the highest in the open-ended questions (student 22). Whereas the student who scored the highest in the open-ended questions did not score well in the open-ended questions (student 13). Generally, students who were good performers in close ended questions were not that good in open ended questions. Probably they might have difficulty in comprehending their answers. These students need moral boosting with clarity of the subject which could remarkably enhance their performance. There was another category of students whose performances were alike in both the groups of questions. These students require close monitoring and training since they had more chances of scoring less than required to pass the examination (Figure 2).
The summative purpose of assessment was often justified by the social accountability of the schools to ensure that the graduates are competent [26]. The framework for single assessments identifies construct validity, reproducibility, equivalence, acceptability, feasibility, educational benefit and timely feedback as key elements. Single methods of assessment alone are unable to assess all the attributes required to become a competent health professional. [27, 28] In planning and designing assessments, it is essential to recognize the stakes involved in it. The higher the stake, the greater the implications of the outcome of the assessment. Measuring progress in acquiring core knowledge and competencies may be a problem if the examinations are designed to measure multiple integrated abilities, such as factual knowledge, problem-solving, analysis and synthesis of information [29].
Limitations: Use of small sample size; feedback from faculty and students; incorporation into formative assessments
Good performers did well in both open- and close-ended questions which require more complex cognitive processes, structuring and logical reasoning. Average performers did well in closed ended questions. Poor performers did not perform in both. Essays and SAQs remain the choice to differentiate a best performer in good performing candidates. The use of multiple formats is recommended in assessment of medical students. However, assessment tools should be valid and reliable and be able to measure the different aspects of professional competencies. Essays, SAQs and MCQs in definite proportion are a better method for assessing the cognitive domain of undergraduate medical students instead of using only one or two assessment tools. When clear grading guidelines are in place, structured essays can be psychometrically robust. We highly recommend that medical schools take the time and effort to evaluate their current assessments to ensure that students are being asked the application questions necessary to measure student learning.
The authors wish to thank the management for providing financial support for doing oral presentation in the 11th Asia Pacific Medical Education Conference held at National University of Singapore, Singapore. The authors wish to thank the management for providing necessary infrastructure for conducting the study. The authors also wish to express their thanks to the students who participated in the study.
Dr Santhi Silambanan: concept, design, statistical analysis, manuscript preparation, manuscript editing and manuscript review; Dr K Sowmya: literature search, manuscript preparation; Dr CM Prabhu Kumar: design, literature search, statistical analysis.
There is no conflicts of interest during the conductance of the study or publication of the article
Funding details: The study was not supported by any funds from any national or international funding agencies
Ethics statement: ethics approval was obtained from institutional ethics committee.
Clearly Auctoresonline and particularly Psychology and Mental Health Care Journal is dedicated to improving health care services for individuals and populations. The editorial boards' ability to efficiently recognize and share the global importance of health literacy with a variety of stakeholders. Auctoresonline publishing platform can be used to facilitate of optimal client-based services and should be added to health care professionals' repertoire of evidence-based health care resources.
