INTER-RATER RELIABILITY OF OBJECTIVE STRUCTURED LONG EXAMINATION RECORD

Authors

  • Syed Inamullah Shah Foundation University Medical College, Islamabad
  • Mehreen Baig Foundation University Medical College, Islamabad
  • Sajida Shah Shifa International Hospital, Islamabad
  • Eitezaz Ahmad Bashir Foundation University Medical College, Islamabad
  • Hajira Sarwar Foundation University Medical College, Islamabad
  • Jamil Ahmad Shah Fauji Foundation Hospital, Rawalpindi

Abstract

Background: Objective Structured Long Examination Record (OSLER) scale was introduced in 1997 by Gleeson to improve the long case examination. There is no psychometric evidence to support reliability of OSLER. This study was done to analyse inter-rater reliability of OSLER. Methods: Two groups of examiners assessed 105 students in long case examination of their final professional examination, using OSLER scale. Group 1 was composed of actual examiners while Group 2 was mock examiners. Kappa statistic and intraclass correlation coefficient (ICC) were used on SPSS 23 to calculate reliability. Results: Mean score awarded by actual examiners was 55.36 (SD=11.2) whereas mean score by mock examiners was 57.74 (SD=14.1). Cronbach’s alpha was 0.586, Kappa was 0.019 whereas inter-rater reliability on ICC was 0.413. Conclusion: Although OSLER is a practical modification of long case examination with good validity, the scale needs to be more structured to improve its reliability.Keywords: Long case; OSLER; Reliability

Author Biographies

Syed Inamullah Shah, Foundation University Medical College, Islamabad

Associate Professor Surgery

Mehreen Baig, Foundation University Medical College, Islamabad

Assistant Professor Surgery

Sajida Shah, Shifa International Hospital, Islamabad

Assistant Consultant Radiologist

Eitezaz Ahmad Bashir, Foundation University Medical College, Islamabad

Professor and HOD Surgery FUMC, Islamabad

Hajira Sarwar, Foundation University Medical College, Islamabad

Senior Registrar Surgery

Jamil Ahmad Shah, Fauji Foundation Hospital, Rawalpindi

Registrar Surgery Deptt.

References

Dare AJ, Cardinal A, Kolbe J, Bagg W. What can history tell us? An argument for observed history-taking in the trainee intern long case assessment. N Z Med J 2008;121(1282):51–7.

Wilkinson TJ, Campbell PJ, Judd SJ. Reliability of the long case. Med Educ 2008;42(9):887–93.

Smee S. ABC of learning and teaching in medicine. Skill based assessment. Br Med J 2003;326(7391):703–6.

Troncon EA, Fernando ROD, Figueiredo C, Ferriolli E, Moriguti Lio C, Martinelli Ana LC, et al. A standardized, structured long-case examination of clinical competence of senior medical students. Med Teach 2000;22(4):380–5.

Wass V, Van der Vleuten C. The long case. Med Educ 2004;38(11):1176–80.

Teoh NC, Bowden FJ. The case for resurrecting the long case. BMJ 2008;336(7655):1250.

Thornton S. A literature review of the long case and its variants as a method of assessment. Educ Med J 2012;4(1):e5–14.

Ponnamperuma GG, Karunathilake IM, McAleer S, Davis MH. The long case and its modifications: a literature review. Med Educ. 2009;43(10):936–41.

Sood R. Long case examination - Can it be improved? J Indian Acad Clin Med 2001;2(4):252–5.

Norcini JJ, Lipner RS, Kimball HR. Certifying examination performance and patient outcomes following acute myocardial infarction. Med Educ 2002;36(9):853–9.

Newble DI. The observed long case in clinical assessment. Med Educ 1991;25(5):369–73.

Abouna GM. The integrated direct observation clinical encounter examination (IDOCEE) - an objective assessment of students’ clinical competence in a problem-based learning curriculum. Med Teach 1999;21(1):67–72.

Gleeson F. AMEEN medical education guide no 9: Assessment of clinical competence using the Objective Structured Long Examination Record (OSLER). Med Teach 1997;19(1):7–14.

Bannerji M, Capozzoli M, McSweeney L, Sinha D. Beyond Kappa: A review of inter-rater agreement measures. Can J Stat 1999;27(1):3–23.

Weir JP. Quantifying test-retest reliability using intraclass correlation coefficient and SEM. J Strength Cond Res 2005;19(1):231–40.

Viera JA, Garrett JM. Understanding inter-observer agreement: The Kappa Statistic. Fam Med 2005;37(5):360–3.

Barzansky B, Etzel SI. Medical schools in the United States, 2009-2010. JAMA 2010;304(11):1247–54.

Bentley BS, Hill RV. Objective and subjective assessment of reciprocal peer teaching in medical gross anatomy laboratory. Anat Sci Educ 2009;2(4):143–9.

Kamarudin MA, Mohamad N, Halizah MN, Yaman MN. The Relationship between Modified Long Case and Objective Structured Clinical Examination (OSCE) in final professional examination 2011 held in UKM Medical Centre. Procedia-Soc Behav Sci 2012;60:241–8.

Malik A, Bhugra D. Workplace based assessment methods: literature overview. In: Malik A, Bhugra D, Brittlebank A, editor. Workplace-based assessments in psychiatry. 2nd ed. London: RCPsych Publications, 2011; p.14–27.

Kroboth FJ, Hansusa BH, Parker S, coulehan JL, Kapoor WN, Brown FH, et al. The inter-rater reliability and internal consistency of a clinical evaluation exercise. J Gen Intern Med 1992;7(2):174–9.

Wass V, Jolly B. Does observation add to the validity of the long case? Med Educ 2001;35(8):729–34.

Nithyanandam S, Joseph M, Vasu U. Can conventional long case examination be improved? Indian J Ophthalmol 2012;60(4):333.

Published

2018-04-17

Issue

Vol. 30 No. 2 (2018): JOURNAL OF AYUB MEDICAL COLLEGE, ABBOTTABAD

Section

ORIGINAL ARTICLE

License

Journal of Ayub Medical College, Abbottabad is an OPEN ACCESS JOURNAL which means that all content is FREELY available without charge to all users whether registered with the journal or not. The work published by J Ayub Med Coll Abbottabad is licensed and distributed under the creative commons License  CC BY ND Attribution-NoDerivs. Material printed in this journal is OPEN to access, and are FREE for use in academic and research work with proper citation. J Ayub Med Coll Abbottabad accepts only original material for publication with the understanding that except for abstracts, no part of the data has been published or will be submitted for publication elsewhere before appearing in J Ayub Med Coll Abbottabad. The Editorial Board of J Ayub Med Coll Abbottabad makes every effort to ensure the accuracy and authenticity of material printed in J Ayub Med Coll Abbottabad. However, conclusions and statements expressed are views of the authors and do not reflect the opinion/policy of J Ayub Med Coll Abbottabad or the Editorial Board.

USERS are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, without asking prior permission from the publisher or the author. This is in accordance with the BOAI definition of open access.

AUTHORS retain the rights of free downloading/unlimited e-print of full text and sharing/disseminating the article without any restriction, by any means including twitter, scholarly collaboration networks such as ResearchGate, Academia.eu, and social media sites such as Twitter, LinkedIn, Google Scholar and any other professional or academic networking site.