TY - JOUR
T1 - The emergency medicine group standardized letter of evaluation as a workplace-based assessment
T2 - The validity is in the detail
AU - Love, Jeffrey N.
AU - Doty, Christopher I.
AU - Smith, Jessica L.
AU - Deiorio, Nicole M.
AU - Jordan, Jaime
AU - van Meter, Michael W.
AU - Edens, Mary Ann
AU - Hegarty, Cullen B.
N1 - Publisher Copyright:
Copyright: © 2020 Love et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/licenses/by/4.0/
PY - 2020
Y1 - 2020
N2 - Introduction: Interest is growing in specialty-specific assessments of student candidates based on clinical clerkship performance to assist in the selection process for postgraduate training. The most established and extensively used is the emergency medicine (EM) Standardized Letter of Evaluation (SLOE), serving as a substitute for the letter of recommendation. Typically developed by a program's leadership, the group SLOE strives to provide a unified institutional perspective on performance. The group SLOE lacks guidelines to direct its development raising questions regarding the assessments, processes, and standardization programs employ. This study surveys EM programs to gather validity evidence regarding the inputs and processes involved in developing group SLOEs. Methods: A structured telephone interview was administered to assess the input data and processes employed by United States EM programs when generating group SLOEs. Results: With 156/178 (87.6%) of Accreditation Council of Graduate Medical Education-approved programs responding, 146 (93.6%) reported developing group SLOEs. Issues identified in development include the following: (1) 84.9% (124/146) of programs limit the consensus process by not employing rigorous methodology; (2) several stakeholder groups (nurses, patients) do not participate in candidate assessment placing final decisions at risk for construct under-representation; and (3) clinical shift assessments don't reflect the task-specific expertise of each stakeholder group nor has the validity of each been assessed. Conclusion: Success of the group SLOE in its role as a summative workplace-based assessment is dependent upon valid input data and appropriate processes. This study of current program practices provides specific recommendations that would strengthen the validity arguments for the group SLOE.
AB - Introduction: Interest is growing in specialty-specific assessments of student candidates based on clinical clerkship performance to assist in the selection process for postgraduate training. The most established and extensively used is the emergency medicine (EM) Standardized Letter of Evaluation (SLOE), serving as a substitute for the letter of recommendation. Typically developed by a program's leadership, the group SLOE strives to provide a unified institutional perspective on performance. The group SLOE lacks guidelines to direct its development raising questions regarding the assessments, processes, and standardization programs employ. This study surveys EM programs to gather validity evidence regarding the inputs and processes involved in developing group SLOEs. Methods: A structured telephone interview was administered to assess the input data and processes employed by United States EM programs when generating group SLOEs. Results: With 156/178 (87.6%) of Accreditation Council of Graduate Medical Education-approved programs responding, 146 (93.6%) reported developing group SLOEs. Issues identified in development include the following: (1) 84.9% (124/146) of programs limit the consensus process by not employing rigorous methodology; (2) several stakeholder groups (nurses, patients) do not participate in candidate assessment placing final decisions at risk for construct under-representation; and (3) clinical shift assessments don't reflect the task-specific expertise of each stakeholder group nor has the validity of each been assessed. Conclusion: Success of the group SLOE in its role as a summative workplace-based assessment is dependent upon valid input data and appropriate processes. This study of current program practices provides specific recommendations that would strengthen the validity arguments for the group SLOE.
UR - http://www.scopus.com/inward/record.url?scp=85084881360&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85084881360&partnerID=8YFLogxK
U2 - 10.5811/westjem.2020.3.45077
DO - 10.5811/westjem.2020.3.45077
M3 - Article
C2 - 32421507
AN - SCOPUS:85084881360
SN - 1936-900X
VL - 21
SP - 600
EP - 609
JO - Western Journal of Emergency Medicine
JF - Western Journal of Emergency Medicine
IS - 3
ER -