Abstract and Introduction
Abstract
Purpose Despite standardized curricula and mandated accreditation, concern exists regarding the variability and imprecision of medical student evaluation. The authors set out to perform a complete review of clerkship evaluation in U.S. medical schools.
Method Clerkship evaluation data were obtained from all Association of American Medical Colleges–affiliated medical schools reporting enrollment during 2009–2010. Deidentified reports were analyzed to define the grading system and the percentage of each class within each grading tier. Inter- and intraschool grading variation was assessed in part by comparing the proportion of students receiving the top grade.
Results Data were analyzed from 119 of 123 accredited medical schools. Dramatic variation was detected. Specifically, the authors documented eight different grading systems using 27 unique sets of descriptive terminology. Imprecision of grading was apparent. Institutions frequently used the same wording (e.g., "honors") to imply different meanings. The percentage of students awarded the top grade in any clerkship exhibited extreme variability (range 2%–93%) from school to school, as well as from clerkship to clerkship within the same school (range 18%–81%). Ninety-seven percent of all U.S. clerkship students were awarded one of the top three grades regardless of the number of grading tiers. Nationally, less than 1% of students failed any required clerkship.
Conclusions There exists great heterogeneity of grading systems and imprecision of grade meaning throughout the U.S. medical education system. Systematic changes seeking to increase consistency, transparency, and reliability of grade meaning are needed to improve the student evaluation process at the national level.
Introduction
The training and education of a physician is a complex endeavor that focuses on the development of competency within three principle domains: the mastery of medical knowledge, proficiency in the practice of medical skills, and the ability to doctor within a culturally competent and professional environment. Though societies have defined healers within their communities for thousands of years, the central tenets of modern-day medical education have their origins in Abraham Flexner's work during the early 20th century. Flexner's 1910 report on medical education in the United States and Canada argued that variation in curriculum standards and training was unacceptable. Shortly after this report was published, nearly all medical school curricula in the two countries were modified to provide a foundation of core basic science material followed by clinical training in the major medical disciplines. Today, medical education in the United States is primarily structured within a four-year developmental curriculum. Student evaluation is based primarily on individual demonstration of competency in the knowledge, skill, and behavioral domains described above. Mastering these competencies is required for promotion toward and attainment of a medical degree.
Although numerous means of student assessment are well described, the process of incorporating such assessment into a final evaluation (or grade) has proven challenging. This is particularly true within the third-year clinical clerkships, when students are taught by numerous faculty and housestaff physicians in an apprenticeship model. The Liaison Committee on Medical Education (LCME) mandates that every accredited U.S. medical school have a system in place for the assessment of medical student achievement which employs a variety of measures to assess students' knowledge, skills, and behaviors. Furthermore, the LCME directs each school to ensure that their faculty understand the uses and limitations of various test formats, the benefits of criterion-referenced versus norm-referenced grading, and the reliability and validity of each modality. Indeed, obtaining LCME accreditation implies that both internal and external reviews of clinical curricula meet these standards.
Despite the progress of the last century, however, concern remains that medical school clerkship evaluations are imprecise, highly variable, and difficult to interpret outside of the microenvironment of a clerkship itself. Recognizing these issues, the Association of American Medical Colleges (AAMC) mandated modifications to the medical student performance evaluation (MSPE), formerly referred to as the "dean's letter." Following an extensive effort from 2000 to 2002, the AAMC defined a novel and standard set of measurable professional attributes expected of all medical students. Furthermore, the AAMC requested that all medical schools provide corresponding policies and consistent procedures for a systematic, performance-based assessment of all students across their core clinical clerkships. This has led to increased transparency of medical schools' clerkship evaluations.
The transparency of the evaluation process is widely considered a positive step forward. However, the inter- and intraschool variability of clerkship grading raises questions about the ultimate meaning of each grade. This is of great importance because an MD degree granted in the United States, as viewed by patients and colleagues alike, implies completion of a medical program that met standardized, consistent, and reliable standards. Furthermore, all LCME-accredited schools should seek to assess similar developmental competencies from their students. To date, no complete review of clerkship grading within U.S. medical schools has been performed. Therefore, with this background, we sought to broadly and systematically review clerkship grading across all LCME-accredited AAMC member schools in the United States. Our goal was to catalogue the types and distributions of grading systems and investigate how these systems affect grade distribution. We believe these data will serve as a launching point for an important national discussion on this topic and for movement toward more reliable and competency-based student assessment.