Society of General Internal Medicine (SGIM) Professionalism Milestones Assessment Tools for Competency-Based Progression in the Ambulatory Setting
Download Evaluation Tool
Overview
The SGIM Milestones working group developed these tools to assess the Professionalism competency in the ambulatory setting for learner progression. To determine the highest priority ACGME/ABIM milestones for assessment in the ambulatory setting, an anonymous electronic survey was sent to AAIM and SGIM members in 2013. The survey results highlighted assessment of Professionalism as a priority with three specific milestones selected by the majority of the respondents.
- Professionalism 1: Has professional and respectful interactions with patients, caregivers and members of the interprofessional team (e.g. peers, consultants, nursing, ancillary professionals and support personnel). (PROF1)
- Professionalism 2: Accepts responsibility and follows through on tasks. (PROF2)
- Professionalism 4: Exhibits integrity and ethical behavior in professional conduct. (PROF4)
In addition to the question regarding importance of achieving each milestone in ambulatory, survey responses were collected to ascertain who, where, what types of assessment tool, and what challenges for implementation should be considered.
The SGIM tools are designed to facilitate direct observation and multi-source feedback to help inform the Professionalism core competency for the ambulatory setting. These tools clearly define expectations for successful resident (Learner) advancement and may be utilized as a primary component of diverse assessment systems with the goal of enhancing learner development and progression.
Development
Members of the SGIM Education Committee Milestones working group with extensive experience in medical education and competency-based progression collaborated to design an electronic survey to prioritize ambulatory milestones as determined by AAIM and SGIM members. The final survey underwent a modified seven-step iterative process prior to dissemination to the SGIM Open Forum and AAIM list serve.
The survey questions asked the following questions of each of the 22 ACGME Reporting milestones:
- How important was it for residents to achieve that particular milestones in their continuity clinic (using a 5 point likert scale of strongly disagree to strongly agree)
- Who is best suited to evaluate the resident’s achievement of this milestone in continuity clinic (i.e. peers, nurses, interpreters, faculty, etc.)
- Where in the continuity clinic is best suited to observe the resident’s achievement of this milestone (i.e. precepting room, bedside, etc.)
- What types of assessment tools would be most applicable to assess this milestone in the continuity clinic (i.e. direct observation, chart-simulated recall, etc)
- What difficulties were expected during implementation of the assessment tool (i.e. time and faculty buy-in)
Survey response rate was 54.5% over a two month period (255 total response; 139 full survey completion and 116 partial survey completion). Based on the results, the Milestones workgroup decided to focus on the Professionalism competencies for direct observation tool development. Three specific professionalism milestones (PROF1, PROF2, PROF4) were chosen based on their high rankings (five on the 5-point likert scale) of greater than 75% of respondents selecting as very important to assess in the ambulatory environment.
Survey results further suggested that the creation of a standardized, multi-source tool for assessment of these professionalism milestones would be of most benefit to learners and residency programs. The Milestones subgroup performed an extensive literature review and ultimately selected Professionalism tools from nine institutions and organizations across the USA and Canada as prospective building blocks for the creation the toolkit.
- ABIM Peer Assessment Instrument
- Baystate/Tufts University School of Medicine
- Indiana University Nursing MSF Evaluation
- McGill University P-MEX Instrument
- NBME Assessment of Professional Behaviors
- Partners HealthCare System Evaluations
- University of Colorado MSF Evaluations
- University of Massachusetts Evaluations
- University of Washington Clinic Evaluations
Through conference calls and electronic reviews, the Milestones workgroup and SGIM education committee created and expanded the professionalism milestones tools to include complementary assessments by patients, peers, nurses and Teaching Faculty. The tools’ design was individualized to emphasize common observable behaviors that could be directly assessed by each of these stakeholders in the ambulatory setting.
Lessons Learned
Developing and implementing a survey as well as designing milestones-based tools with educators from nine institutions across the country are time-consuming and challenging endeavors. The process required a considerable amount of time and dedication from these educators. As we move forward, piloting the tools during an era where the educational and clinical expectations for learners and faculty have grown exponentially will be a challenge. Taking advantage of direct observation and multi-source feedback opportunities that occur during the normal flow of care will be essential for success since using these tools will require direct observation and therefore a re-organization of faculty effort.
Faculty Development and Training
Minimal faculty development is anticipated to implement these tools as similar models have been successfully incorporated across multiple institutions and ambulatory centers. A concise explanation of the tools is usually adequate to orient new faculty and learners and can be reinforced during Faculty: Learner meetings. However, new junior faculty or new clinician educators may need some faculty skills development in direct observation assessments.
How Used to Inform Decisions about a Learner's Milestone
The SGIM tools were designed to assist Learners and Faculty to understand and incorporate specific milestone-based Professionalism goals for education and development. The tools create a guidebook for success and clearly define expectations for the learner and those doing the assessments.
The SGIM Professionalism Milestones tools are designed to be used as longitudinal assessment tools that facilitate direct observation and inform milestones in the ambulatory setting. We believe these tools will allow identification of problem behaviors, as well as areas for learner growth and success through the delineation of specific steps for progression in a competency-based system.