TY - JOUR
T1 - When the first try fails
T2 - re-implementation of SIMPL in a general surgery residency
AU - Hsu, Phillip J.
AU - Wnuk, Gregory
AU - Leininger, Lisa
AU - Peterson, Samantha
AU - Hughes, David T.
AU - Sandhu, Gurjit
AU - Zwischenberger, Jay B.
AU - George, Brian C.
AU - Aubry, Staci
N1 - Publisher Copyright:
© The Author(s) 2024.
PY - 2024/12
Y1 - 2024/12
N2 - Background: Workplace-based assessment (WBA) can facilitate evaluation of operative performance; however, implementation of WBA is sometimes unsuccessful. The American Board of Surgery Entrustable Professional Activities WBA project was launched in July 2023. Some programs will face the challenge of re-implementation of a WBA following previous failures. It is unknown what interventions are most effective for WBA re-implementation. Our goal is to identify barriers and facilitators to re-implementing SIMPL, an operative performance WBA. Methods: The System for Improving and Measuring Procedural Learning (SIMPL) was implemented at our residency in 2018, but usage rates were low. We interviewed residents and faculty to identify barriers to usage and opportunities for improvement. Residents reported that SIMPL usage declined because of several factors, including a low faculty response rate, while some faculty reported not responding because they were unable to login to the app and because usage was not mandated. We then re-implemented SIMPL using a plan based on Kotter’s Model of Change. To evaluate impact, we analyzed rates of SIMPL usage when it was first implemented, as well as before and after the date of re-implementation. Results: In September 2022, we re-implemented SIMPL at our program with measures addressing the identified barriers. We found that, in the six months after re-implementation, an average of 145.8 evaluations were submitted by residents per month, compared with 47 evaluations per month at the start of the original implementation and 5.8 evaluations per month just prior to re-implementation. Faculty completed 60.6% of evaluations and dictated feedback for 59.1% of these evaluations, compared with 69.1% at implementation (44% dictated) and 43% prior to re-implementation (53% dictated). Conclusions: After identifying barriers to implementation of a WBA, we re-implemented it with significantly higher usage by faculty and residents. Future opportunities exist to implement or re-implement assessment tools within general surgery programs. These opportunities may have a significant impact in the setting of national standardization of workplace-based assessment among general surgery residencies.
AB - Background: Workplace-based assessment (WBA) can facilitate evaluation of operative performance; however, implementation of WBA is sometimes unsuccessful. The American Board of Surgery Entrustable Professional Activities WBA project was launched in July 2023. Some programs will face the challenge of re-implementation of a WBA following previous failures. It is unknown what interventions are most effective for WBA re-implementation. Our goal is to identify barriers and facilitators to re-implementing SIMPL, an operative performance WBA. Methods: The System for Improving and Measuring Procedural Learning (SIMPL) was implemented at our residency in 2018, but usage rates were low. We interviewed residents and faculty to identify barriers to usage and opportunities for improvement. Residents reported that SIMPL usage declined because of several factors, including a low faculty response rate, while some faculty reported not responding because they were unable to login to the app and because usage was not mandated. We then re-implemented SIMPL using a plan based on Kotter’s Model of Change. To evaluate impact, we analyzed rates of SIMPL usage when it was first implemented, as well as before and after the date of re-implementation. Results: In September 2022, we re-implemented SIMPL at our program with measures addressing the identified barriers. We found that, in the six months after re-implementation, an average of 145.8 evaluations were submitted by residents per month, compared with 47 evaluations per month at the start of the original implementation and 5.8 evaluations per month just prior to re-implementation. Faculty completed 60.6% of evaluations and dictated feedback for 59.1% of these evaluations, compared with 69.1% at implementation (44% dictated) and 43% prior to re-implementation (53% dictated). Conclusions: After identifying barriers to implementation of a WBA, we re-implemented it with significantly higher usage by faculty and residents. Future opportunities exist to implement or re-implement assessment tools within general surgery programs. These opportunities may have a significant impact in the setting of national standardization of workplace-based assessment among general surgery residencies.
KW - Barriers to usage
KW - Change in culture
KW - Evaluation
KW - Re-implementation
KW - Workplace-based assessment
UR - http://www.scopus.com/inward/record.url?scp=85203512231&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85203512231&partnerID=8YFLogxK
U2 - 10.1186/s12893-024-02557-2
DO - 10.1186/s12893-024-02557-2
M3 - Article
C2 - 39261888
AN - SCOPUS:85203512231
SN - 1471-2482
VL - 24
JO - BMC Surgery
JF - BMC Surgery
IS - 1
M1 - 257
ER -