Catherine Apaloo, MD
Athens Regional Medical Center
Program Size: 31-100 residents
Academic Setting: Community-Based
Clinical Setting: Patient Safety/ QI
Overview
This evaluation tool is designed to assess resident presentations at the Patient Safety and Quality Improvement Morbidity and Mortality Conference. A resident will present an error that was made by someone at the hospital, and develop a strategy to ensure the error will not be made again. The goal of the evaluation is for the resident to systematically learn more about the error while addressing the six core competencies. The evaluation is divided into two sections: Content and Presentation.
The “Content” section of the evaluation uses a framework that assesses the content of the information in the presentation including: identification and classification of the error, the S.T.E.E.E.P. patient care model, the competencies addressed, and how S.M.A.R.T. the Action Plan goals were. Part of the learning process is that the resident must use the S.T.E.E.E.P. framework for quality and identify which of those areas were violated in the error. The Institute of Medicine developed the S.T.E.E.E.P. acronym as a framework for quality improvement. S.T.E.E.E.P. stands for Safe, Timely, Effective, Efficient, Equitable, and Patient-Centered Care. The resident must develop an action plan in order to learn from the error, and the goals of the action plan must be S.M.A.R.T. The S.M.A.R.T. acronym stands for: Specific, Measurable, Attainable, Relevant, and Timely. The “Presentation” section of the evaluation examines the manner in which the resident presented the information. This tool is used by faculty members, residents, medical students, and other health professionals to evaluate the overall effectiveness of resident presentations at the Patient Safety and Quality Improvement Morbidity and Mortality Conferences.
Download the Tool
Development
The key step in developing this evaluation tool was determining how to best to assess the resident’s identification and then mitigation of the adverse event. To identify the error, we used the STEEEP model to identify patient care that is safe, timely, effective, efficient, equitable, and patient centered. Once the error is identified, it must then be mitigated. We used the SMART goals to allow the systems error to be mitigated. The next step was to ensure that the evaluation accurately mapped back to the ACGME milestones.
Lessons Learned
One concern during the development and implementation is how best to ensure that the resident learns from the adverse events. We believe that using the Institute of Medicine’s STEEEP criteria and SMART Action Plan objectives as part of the goals and objectives for the M&M conference as well as the evaluation tool, enables the resident to use this as a learning tool.
Faculty Development and Training
The evaluation was designed to be user friendly and fairly self-explanatory. A 30 minute faculty development session is required to use this tool; this could be incorporated into an on-line module. During the faculty development session, frame of reference rater training using the STEEEP and SMART standards is used to teach raters to share a common conceptualization of performance to improve the accuracy of their ratings in the content section. Using behavioral observation training sample standardized M& M presentations are used to train raters on important characteristics to look for in the performance section to provide more objective assessments of the resident’s performance.
How Used to Inform Decisions about a Learner's Milestone
The M&M Conference Evaluation questions can be mapped back to milestones that the CCC can then use to make decisions about the competency of the resident. In addition some of the questions map back to several milestones.
For more information, please contact Catherine.Apaloo@athenshealth.org.