Timeliness Assessment Methods & Process Quality Assessment National Resources
Program leaders are encouraged to use institutional and national tools, as well as the implementation ideas collated by the leadership of the University of Arizona College of Medicine – Phoenix GME programs and listed on these resource pages to support their improvement efforts.
Feedback & Assessment 1: Timeliness
| Level 1 |
Level 2 |
Level 3 |
Level 4 |
Level 5 |
| The program assigns evaluations to faculty and sends out reminders for completion of assessments within two weeks of the completion of the rotation. |
When asked, faculty, residents/fellows know that the program has a requirement for all faculty to complete evaluations in a timely fashion. |
Residents/fellows are satisfied with the timeliness of assessments. |
Program regularly uses data about timely completion of assessments to maintain or improve the process. |
Program is sharing its best practices for achieving timely completion of assessments with other programs. |
💡Idea 1: There is a standard in the program that all written evals are completed within 2 weeks of assignment (level 2)
Implementation:
- Add the standard to your program manual
- PD makes announcements at faculty and trainee meetings on the requirement to complete evaluations in a timely fashion
- Coordinator emails faculty and trainees two weeks prior to start of rotation that evaluations have been assigned
- Coordinator sends out emails after conclusion of the rotation to remind faculty and trainees to complete the evaluations
|
💡Idea 2: Regularly create reports on the timeliness of completion of evals to improve compliance (Level 4)
Tools:
Implementation:
- Program pulls data every month to assess the completeness rate evaluations
|
💡Idea 3: Includes data about faculty compliance as part of faculty feedback (level 2-4)
Tools:
Implementation:
- Create an award to recognize trainees and faculty who meet/exceed the program’s expectations for timely feedback
- PD sends a personal email to faculty to complete evals if they are more than one month late. The email reminds them of the 2-week standard
- Report of evaluation timeliness included in annual faculty feedback or other regular communication
|
Literature search
Feedback & Assessment 2: Assessment Methods & Process
| Level 1 |
Level 2 |
Level 3 |
Level 4 |
Level 5 |
| The evaluations of the trainees are linked to specialty specific milestones. |
Residents/fellows agree that they receive semi-annual feedback on their progress towards the milestones and the evaluation is documented in new innovations. |
Annually, all trainees have at least one documented assessment focused on direct observation of a specific clinical skill relevant to the specialty/sub-specialty. |
The program annually makes adjustments to enhance the value of assigned assessments. |
The program’s process is a role model for competency-based assessment utilizing emerging tools and making regular updates. They regularly share best practices with others. |
💡Idea 1: Enhance assessment tools and the anchors (Level 1-5)
Tools:
Implementation:
- The STFM CBME task force recommends the following entrustment scale:
- Participated only as an observer or required faculty to do.
- Required significant faculty assistance.
- Required minimal faculty assistance.
- Completed independently, without need for assistance.
- Demonstrated the ability to role model or teach others.
|
💡Idea 2: Semi annual meetings and documentation (level 2)
Tools:
Implementation:
- Each trainee receives a copy of their milestones at semi-annual evaluation with PD/APD/advisor
- Trainees can review their evaluations and milestones on new innovations.
|
💡Idea 3: Direct observation of specialty specific skills/entrustable professional activities (Level 3)
Tools:
Implementation:
- Create a QR code linking to a Microsoft form so that it is easy to access for all
- Create a New innovations “on demand eval” for the EPA
|
💡Idea 4: Annual review of assessment forms (level 4)
Tools:
Implementation:
Add to the CCC agenda &/OR use faculty survey- Question to consider are: Which forms/questions are helping? What gaps do we have in our assessments. Could they be shortened? Are people using the NI app to help complete in more real time, etc.
|
Literature search
|
Feedback & Assessment 3: Quality
| Level 1 |
Level 2 |
Level 3 |
Level 4 |
Level 5 |
| Faculty members directly observe, evaluate, and frequently provide feedback on resident performance during each rotation or similar educational assignment. |
Most narrative feedback comments from faculty are clear, specific, frequent, focused on actions. |
The CCC and or PD provide at least annual program (and individual when appropriate) level faculty development on methods to improve the quality and reliability of completed assessments. |
Members of the CCC agree that the quality of assessment feedback is sufficient to make decisions regarding progression, disciplinary action, and independent learning plans. In general, Residents/Fellows are satisfied with the quantity and quality of feedback they receive. |
Program is sharing its best practices for achieving high quality feedback with other programs. |
💡Idea 1: Narrative feedback enhancements
Tools:
Implementation:
- Overall brief and general feedback terms are avoided by faculty like: “good job,” “perfect,” “well done,” “read more,” “at level of PGY year”
- PD review evaluations and identify faculty members that should be coached on how to provide descriptive, non-judgmental language on the narrative box.
|
💡Idea 2: Faculty Development Related to Assessment (level 2-4)
Tools:
Implementation:
- PD or CCC chair has individual conversations with faculty who seem to be overly harsh, lenient or consistent (all trainees get the same rating) in the use of the scale on assessment forms.
- Add a faculty development assessment pearl to division/program meetings.
- Invite an institutional or national leader to present grand rounds for your faculty.
|
💡Idea 3: Regularly create reports on the timeliness of completion of evals to improve compliance (Level 4)
Tools:
Implementation:
- Program pulls data every month to assess the completeness rate evaluations.
|
Literature search
Do you have a tool to share or need more help? Contact Yasmin.Alishahi@va.gov
Assessment National Resources (update quarterly)