The 360-degree evaluation tool incorporates feedback from multiple sources including faculty, peers, clinical support staff, and patients to evaluate resident's performance. A total of thirty-two dermatology residents over seven years were evaluated with one of two versions of the 360-degree evaluation tool as well as with an overall standard evaluation tool, the Visual Analog Scale (VAS). The evaluations of residents with the second version of the 360-degree evaluation tool correlated strongly (r = 0.92, p < 0.0001) with their evaluations via the VAS. While the implementation of this tool has been reported in other residency programs, this represents the first reported validation of the 360-degree evaluation tool in a dermatology residency program. As a multisource, competency-based metric, the 360-degree evaluation tool is a recommended tool for resident evaluation given the requirements for milestone-based tracking and incorporation of multiple evaluators in the Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System.

The Accreditation Council for Graduate Medical Education as part of its Common Program Requirements for accreditation mandates that resident evaluations utilize data from multiple evaluators including faculty, peers, patients, self, and other professional staff [1]. The 360-degree evaluation tool, which incorporates multisource feedback on resident performance, is a recommended method of evaluating residents on the six ACGME-defined core competencies [2]. Although this tool has been widely implemented across residency programs [3-5], there are no data on its validity in dermatology residency programs. We sought to determine the validity of two versions of the 360-degree evaluation tool in evaluating dermatology residents by comparison to global resident assessments performed via a visual analog scale (VAS) tool.

Exemption was obtained from the Institutional Review Board (IRB) and a total of thirty-two dermatology residents from the University of North Carolina at Chapel Hill were included in this study over seven years. The initial 360-degree evaluation tool (360-1) addressed 15 specific skills to cover all six ACGME competencies. The 360-degree evaluation was completed by faculty members, peer residents, one departmental administrator, nurses, clinical support staff (e.g. front desk), and patients. Due to evaluation fatigue, the assessment was later shortened to one Likert scale response for each of the six competencies (360-2). Twenty-four residents were evaluated by the 360-1 tool and thirteen residents were evaluated by the 360-2 tool; all thirty-two residents were evaluated by a consistent panel of seven senior faculty members (Chairperson, Program Director, Director of Dermatology Surgery, Director of Dermatopathology, Basic Science Researcher/Clinician, Former Medical School Dean and Department Chairperson, and full-time Clinician) using the program gold standard evaluation tool, the VAS. A numeric total score between 0-100 for each evaluation was calculated based upon percentage of perfect score for rated items, and scores were then averaged across raters for each resident. Using SAS version 9.2, convergent validity for both versions of the 360-degree evaluation was estimated by calculating the correlation of resident scores on the 360-degree evaluation tool with scores on the gold standard VAS using Pearson correlation coefficients. The mean score from the VAS was 72.81 ± 21.09 out of a possible 100 points (n = 32). The mean score for the 360-1 assessments was 96.23 ± 2.90 (n = 24) out of a possible 100 points, which was higher than the mean score for the 360-2 assessment of 72.81 ± 21.09 (n = 13). Resident scores on 360-1 were not well correlated with scores on the VAS (r = 0.23, p < 0.30). Scores on the revised version of the 360-degree evaluation (360-2) were highly correlated with VAS scores (r = 0.92, p < 0.0001).

This is the first study of the validity of a 360-degree evaluation tool in a dermatology residency program. The study was limited by a small sample size drawn from a single institution. Our results combined with the growing literature demonstrating the potential effectiveness of 360-degree feedback for resident education across a variety of specialties [6,7], support the use of this multisource feedback tool. In contrast to the longer, behavior-based tool, the shortened format 360-degree evaluation tool (360-2) offers an efficient and valid method for resident assessment. As graduate medical education continues to change with the transition to competency-based milestone tracking required by the ACGME Next Accreditation System [8].