RELATIONSHIP OF AWARDS IN MULTIPLE CHOICE QUESTIONS AND STRUCTURED ANSWER QUESTIONS IN THE UNDERGRADUATE YEARS AND THEIR EFFECTIVENESS IN EVALUATION
Abstract
Background: A number of evaluation tools for assessing the cognitive and affective domains inaccordance with Bloom’s taxonomy are available for summative assessment. At the University ofHealth Sciences, Lahore, Multiple Choice Questions (MCQs) and Structured Answer Questions(SAQs) are used for the evaluation of the cognitive domain at all six hierarch levels of taxonomyusing the tables of specifications to ensure content validity. The rationale of having two evaluationtools seemingly similar in their evaluative competency yet differing in feasibility of construction,administration and marking is being challenged in this study. Methods: The MCQ and SAQawards of the ten percent sample population amounting to 985 students in fifteen Medical andDental Colleges across Punjab were entered into SPSS-15 and correlated according to thecognitive and affective level of assessment in relation to the Bloom’s taxonomy and their groupingin the Tables of Specifications, using parametric tests. 3494 anonymously administeredquestionnaires were analyzed using ethnograph. Results: No statistically significant differencewas found in the mean marks obtained by the students when MCQs and SAQs were comparedaccording to their groupings in the Tables of Specifications at all levels of cognitive hierarchicaltesting. End-of-year cognitive level testing targets set were not met and more questions were set atthe lower cognitive testing levels. Expenses incurred in setting MCQs and SAQs were comparablebut conduct and assessment costs for MCQs and SAQs were 6% and 94% of the total respectively.In both MCQs and SAQs students performed better at higher cognitive testing levels whereas theSAQs and MCQs were able to marginally test the lower levels of affective domain only. Student’sfeedback showed that attempting MCQs required critical thinking, experience and practice.Conclusion: MCQs are more cost effective means at levels of cognitive domain assessment.Keywords: Cognitive Domain, Affective Domain, Bloom’s Taxonomy, Tables of Specifications,EvaluationReferences
Schultheis NM. Writing cognitive educational objectives and
multiple-choice test questions. Am J Health Syst Pharm
;55:2397–401
Writing Multiple-Choice Questions that Demand Critical
Thinking. Available at: http://cit.necc.mass.edu/atlt/
TestCritThink.htm#anchor1046303 [Accessed January 2nd 2008].
Dressel, PL, and Associates. Evaluation in higher education.
Boston: Houghton Mifflin; 1961.
Airasian, PW. Classroom Assessment: Concepts and
Applications (4th Ed.). New York: McGraw-Hill; 2001.
McMillan JH. (Ed.). New directions in teaching and learning:
Assessing students’ learning. No. 34. San Francisco: JosseyBass; 1988.
Gronlund, NE. How to write and use instructional objectives (6th
Ed.) Upper Saddle River, NJ: Merril/ Prentice Hall; 2000.
Footlick JK. Truth and Consequences: How Colleges and
Universities Meet Public Crises. Phoenix Ariz: Ammerica
Council on Education; 1997.
Mayer D. Calamen DL. Gunderson A. Barach P. Telluride
Interdisciplinary Roundtable. Designing a Patient Safety
Undergraduate Medical Curriculum: The Telluride
Interdisciplinary Roundtable Experience. Teach Learn Med
;21(1):52–8.
Tarrant M, Knierim A, Hayes SK, Ware J. The frequency of item
writing flaws in multiple-choice questions used in high stakes
nursing assessments. Nurse Educ Today 2006;26:662–71.
Shepard, LA. Evaluating test validity. In L. Darling-Hammond
(Ed.),. Review of research in education (Vol. 19) Washington,
DC: American Educational Research Association; 1993.
p.405–50.
Palmer EJ, Devitt PG. Assessment of higher order cognitive
skills in undergraduate education: modified essay or multiple
choice questions?: research paper. BMC Med Edu 2007;7(1):49.
Louis C, Lawrence M, Keith M. Research Methods in Education
(6th Ed.) New York: Routledge; 2007.
Bloom Benjamin S, David R. Krathwohl. Taxonomy of
Educational Objectives: The Classification of Educational Goals,
by a committee of college and university examiners. Handbook I:
Cognitive Domain. New York: Longmans, Green; 1956.
Bloom B, Englehart M, Furst E, Hill W, Krathwohl D.
Taxonomy of educational objectives: The classification of
educational goals. Handbook I: Cognitive domain. New York,
Toronto: Longmans, Green; 1956.
Mukhopadhyay M, Bhowmick K, Chakraborty S, Roy D, Sen
PK, Chakraborty I. Evaluation of MCQs for Judgment of higher
levels of Cognitive learning. Gomal J Med Sci 2010; 8(2):112–6.
Moeen-Uz-zafar, Aljarallah B. Evaluation of MEQ and MCQ as
a tool for assessing the cognitive skills of undergraduate students
at department of medicine, Qassim University. J Fam
Community Med 2010;17(1):50–67.
Palmer EJ, Duggan P, Devitt PG, Russell R.. The modified essay
question: its exit from the exit examination?
Marshall J. Assessment of problem-solving ability. Med Educ
;11:329–34.
Rabinowitz HK, Hojat MD. A comparison of the modified essay
question and multiple choice question formats: Their
relationships to clinical performance. Fam Med 1989;21:364–7.
Palmer EJ, Devitt P. Constructing multiple choice questions as a
method for learning. Ann Acad Med Singapore 2006;35:604–8.
Collins J. Education techniques for lifelong learning: writing
multiple-choice questions for continuing medical education
activities and self-assessment modules. Radiographics
;26:543–51.
Rabinowitz HK. The modified essay question: an evaluation of
its use in a family medicine clerkship. Med Educ 1987;21:114–8.
Webber RH. Structured short answer questions: an alternative
examination method. Med Educ1992;26(1):58–62.
Pai MRSM, Sanji N, Pai PG, Kotian S. Comparative Assessment in
Pharmacology Multiple Choice Questions Versus Essay with focus
on Gender difference. J Clin Diagnostic Res 2010;4:2515–20.
Epstein RM. Assessment in medical education. N Engl J Med
;356:387–96.
DFES (Department for education and skills) Towards a Unified
e-Learning Strategy: consultation document, Nottingham: DFES
Publications; 2003.
Rust C. The impact of assessment on student learning: how can
the research literature practically help to inform the development
of departmental assessment strategies and learner-centered
assessment practices? Active Learning in Higher Education
;3(2):145–58.
Moosa MYH, Jeenah FY. The assessment of undergraduate
psychiatry training: a paradigm shift. A S Psychiatry Re Rev
;10:88–91.
Swanson DB. A measurement framework for performance based
tests. In: IR Hart and RM Harden, Editors, Further developments
in assessing clinical competence. Montreal: Can-Heal;
p.13–45.
Newble DI, Swanson DB. Psychometric characteristics of the
objective structured clinical examination. Med Educ
;22:325–34
Dauphinee, D. Determining the content of certification
examinations. In: D Newble, B Jolly and R Wakeford, Editors,
The certification and recertification of doctors: issues in the
assessment of clinical competence, Cambridge: Cambridge
University Press: 1994p. 92–104.
Hettiaratchi ES. A comparison of student performance in two
parallel physiology tests in multiple choice and short answer
forms. Med Educ 1978;12:290–6.
Leamnson R. Thinking about teaching and learning. Sterling,
VA: Stylus Publishing; 1999.
Published
Issue
Section
License
Journal of Ayub Medical College, Abbottabad is an OPEN ACCESS JOURNAL which means that all content is FREELY available without charge to all users whether registered with the journal or not. The work published by J Ayub Med Coll Abbottabad is licensed and distributed under the creative commons License CC BY ND Attribution-NoDerivs. Material printed in this journal is OPEN to access, and are FREE for use in academic and research work with proper citation. J Ayub Med Coll Abbottabad accepts only original material for publication with the understanding that except for abstracts, no part of the data has been published or will be submitted for publication elsewhere before appearing in J Ayub Med Coll Abbottabad. The Editorial Board of J Ayub Med Coll Abbottabad makes every effort to ensure the accuracy and authenticity of material printed in J Ayub Med Coll Abbottabad. However, conclusions and statements expressed are views of the authors and do not reflect the opinion/policy of J Ayub Med Coll Abbottabad or the Editorial Board.
USERS are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open access.
AUTHORS retain the rights of free downloading/unlimited e-print of full text and sharing/disseminating the article without any restriction, by any means including twitter, scholarly collaboration networks such as ResearchGate, Academia.eu, and social media sites such as Twitter, LinkedIn, Google Scholar and any other professional or academic networking site.
