A Milestone-Based Assessment Tool for a General Medicine Ward Rotation Incorporating Positive Narratives to Describe Performance
Murat O. Arcasoy, MD, C. William Hargett, MD, Aimee K. Zaas, MD
Duke University School of Medicine
Program Size: More than 101 residents
Academic Setting: University-Based
Clinical Setting: Inpatient wards
Overview
The General Medicine ward rotation is offered in each year of post-graduate training (Duke Internal Medicine Residency Program), providing an ideal vantage point for graduated performance assessment. Our approach focused on integrating ABIM/ACGME milestones into both our curriculum goals and objectives for this rotation and a new milestone-based evaluation tool. Each curricular and reporting milestone was discussed by our core educator panel and integrated into our curriculum in accordance with our programmatic and institutional priorities. Using our revised curriculum goals and objectives for each year of training, a comprehensive, summative evaluation form was generated based on graduated performance assessment during training, encompassing a total of 21 sub-competencies categorized according to one of six ACGME competency domains. Behavioral anchors (narratives) were articulated for each sub-competency describing the observable skills for each performance level and for each year of training. The focus was on the use of positive language in the narratives rather than emphasize the deficient aspects of performance. A five-tier evaluation scale was utilized, with the steepest earning curve anticipated during intern year. Four intermediate levels were incorporated into the scale for use in circumstances where the learner displayed features of two different levels of performance. Each skill was subsequently mapped to reporting milestones and 84 of the 142 curricular milestones. To generate a practical evaluation form, the views of our educators were assessed using Qsort methodology to select the most important elements of clinical competence. A video was created for faculty education on how to utilize the evaluation tool.
Download the Tool
Development
- A core group of educators in our Program articulated the unique aspects and strengths of our program to define graduated levels of responsibility
- General Medicine rotation goals and objectives were revised to integrate reporting and curricular milestones
- Most important elements of clinical competence were identified by assessing the opinions and perceptions of our educators using Qsort methodology after watching a tutorial on the survey technique
- A practical evaluation tool was created using Qsort results and by articulating behavioral anchors for each sub-competency
- A video illustrating the effective use of the new assessment tool was created and circulated to teaching faculty
Lessons Learned
- Convening multiple meetings of our educator panel and integrating milestones both into our curriculum and the evaluation tool were challenging
- The description of each graduated performance level on the evaluation tool was not included on the final electronic form. Possibly as a result of this, very few interns were “graded” as “requires complete supervision”. Consideration is being given to include supervision level required as headings Periodic refreshers of faculty education are needed to promote appropriate use of the tool We have >100 faculty who teach on our ward rotations across three hospitals. Providing “in-person” faculty development is challenging in scope
Faculty Development and Training
We developed a video describing the milestones and the evaluation process that was embedded into our evaluation software for use as ‘just-in-time’ faculty development. We were able to track which faculty members viewed or chose not to view the video.
How Used to Inform Decisions about a Learner's Milestones
The milestones achieved can be viewed for an individual rotation as well as in aggregate for discussion during competency committee meeting. The progress over time as milestones are achieved can be displayed and reviewed to determine whether interns, junior or senior residents are on track.
For more information, please contact murat.arcasoy@dm.duke.edu.