Based on this feedback, the baseline education expectations were revised to include completion of a percentage of resident post-shift assessments over the academic year, inversely proportional to a faculty member’s clinical load. During the first year , the requirements included only conference and module participation.Table 1 lists the final baseline education expectations required of faculty members. Before employing these education requirements, all faculty members were notified of the consequences of not fulfilling expectations, which included ineligibility for any academic incentive and an inability to participate in the voluntary ARVU system.In May 2018, stage two began, which involved the creation of an ARVU system to encompass all other academic activities. It was decided that the ARVU system would be voluntary, but to participate the baseline education expectations outlined in stage one had to be fulfilled. For the first step of this stage, the vice chair for education created a list of preliminary activities to be included in the ARVU system, such as teaching, lecturing, publications, grants, committee memberships, and leadership positions. These additional activities were ones in which faculty were already participating that aligned with the academic mission of the department, but had not been captured within the baseline education expectations, did not earn clinical hours reduction from the department or institution, or were not an implicit part of a faculty member’s role based on his or her leadership position. The thought was that activities that earned a clinical reduction in hours were already being financially rewarded, and this system was designed to recognize activities not yet distinguished. An example includes fellowship activities, which were not included because fellowship directors have a reduction in clinical hours to support their leadership role. After the initial list was assembled, it was shared with a select group of 11 leaders within the department,ebb and flow flood table including residency leadership, undergraduate medical education leadership, fellowship directors, the research division, and the pediatric emergency medicine division.
The participants were selected due to their various leadership roles in the department, their dedication to scholarly achievement in their own careers, and the high priority they placed on these activities within their respective divisions. These qualifications placed these faculty members in a prime position to help generate a comprehensive list of activities relevant to each division. After multiple discussions and written communications using a modified Delphi method, the group reached consensus on the activities that were to be included. As expected, some subjectivity was involved in the voting for various reasons, such as the activity being one in which the responsive faculty member participated in himself or herself, or differing opinions regarding how much preparation time might be needed for such things as a lecture. To help reduce this bias, the survey was sent to many faculty members with different roles and responsibilities to obtain a consensus and to diluteidiosyncratic points of view. Furthermore, the knowledge of and dedication to each activity that the chosen faculty members had and the descriptions provided helped to further reduce bias in the points system. The survey also included free-text fields where faculty could input additional activities that they felt should be added to the list. Of the 60 faculty members surveyed, 49 responded and completed the survey in its entirety. The activities, ranked from highest to lowest based on the mean score including standard deviations, are presented in Table 2.Activities with higher means earned more points. Any activities that were similar in description and mean score were assigned the same number of final points. We introduced the final list and point system at a faculty meeting prior to implementing, and after this final feedback round, we launched the system in December 2018. The free-text responses were also reviewed, and these activities were added to the list and also voted on by the faculty group to create the final list with points. The next steps for the project included creating a database where faculty could log their completed activities. We created a Google form that listed all activities in the ARVU system where faculty members could select the activity in which they participated.
Each activity had an associated drop down menu that asked for additional information, such as title, date, location, description, proof of activity, and an ability to upload documents. We then created a dashboard in the analytics platform Tableau , containing all activities. Statistics for the baseline educational expectations automatically loaded into the dashboard and could not be edited by faculty members.The full dashboard displayed each faculty member’s baseline education expectations, whether they had met requirements, the activities that they had entered into the ARVU point system, and total points earned to date . Final points were earned after academic leadership reviewed, approved, and signed off on each submitted activity. Each month, the system automatically e-mailed a link to each individual’s dashboard notifying faculty how many points they had earned to date and of any participation deficiencies. The medical school requires a teaching portfolio for faculty seeking promotion on the scholar track. This portfolio requires faculty to document their achievements in the following categories: teaching effort, mentoring and advising, administration and leadership, committees, and teaching awards. All ARVU activities were reviewed and categorized based on the elements of the teaching portfolio. These activities not only show up as itemized items with points, but they are also grouped into the appropriate portfolio category and are displayed on each individual faculty member’s dashboard. This allowed each faculty member to see how much scholarship they had completed within each of the teaching portfolio categories and in which areas they were lacking that deserved more attention. This provided faculty with a readily accessible repository of activities that could be transferred directly into the correct category of their teaching portfolio,hydroponic drain table facilitating tracking of activities upon which one needed to focus for promotion. This project has resulted in preliminary positive effects on both education and documentation of scholarly work within our department. The first stage resulted in an overall increase in conference attendance and participation even prior to implementing the ARVU system.
It is possible that these positive findings were a result of the academic incentive being dependent on meeting education expectations. However, in offline discussions with multiple faculty members, it appears that there was a shame factor that also contributed to improved attendance. Multiple faculty expressed their relief that many were being called out on their low attendance and participation and that faculty who had historically carried much of the teaching responsibility were now being recognized. In the same vein, resident assessments increased in the second year by a considerable amount, without any other changes being made to the system, and therefore were likely a result of the new expectations. The increase in assessments does not necessarily mean better quality, and this will need to be evaluated going forward to determine full impact. The improved participation in educational activities as a result of financial incentives or other measures is consistent with reports from other institutions and existing literature. There is a clear correlation between faculty documentation of scholarly output and the ARVU system, as there was no system in place prior that allowed tracking of activities. The increase in activities and documentation will need to be followed from year to year to draw conclusions on overall scholarly activity among individual faculty members and throughout the department. Unlike previous literature describing ARVU systems, our project has emphasized the ability to house activities in one place that can be transferred into a faculty member’s teaching portfolio, thereby further incentivizing the use of this system outside of financial rewards. We will continue to track baseline education expectations and the ARVU system across the department as well as continuously seek feedback from faculty and make changes as needed. This process will continue to be refined over time based on faculty feedback and departmental and institutional priorities. The majority of faculty who did not qualify for the academic bonus last year worked more than 28 clinical hours per week, and thus time issues may have affected compliance. To further probe this finding and facilitate educational commitments, we will solicit additional feedback from this group of faculty members to explore participation barriers that may be addressed in the future. We hope to follow the scholarly output of the department over time using the ARVU system as an estimate of faculty productivity. Our longer-term goals will be to see the effects of this system on the promotion process within the department with an expectation that more junior faculty will become eligible for advancement. These effects will be evaluated by tracking the progress and content of junior faculty teaching portfolios compared to previous years and time to successful promotion. With a bottom-heavy young faculty group, our expectation is that this system will better prepare people for promotion as they can track their activities and determine where they need to place more effort to enhance their portfolio. Finally, this system will be used to improve the mentorship infrastructure within the department. Assigned faculty mentors will use the ARVU dashboard to mentor junior faculty on their progress for promotion.
This dashboard will provide another data point for mentors to advise junior faculty where they need to focus their efforts in order to progress professionally. There was likely subjectivity and bias in faculty assigning points to activities based on effort. Faculty may have ranked certain activities higher than others due to their own participation in the activity in question. In addition, faculty have different opinions on what type of effort may go into an activity; for example, a lecture may be easily prepared by some and take a lot of effort for others. We attempted to remove some of this subjectivity and bias by including faculty in this process who are the most committed to academics in our department. Many of these faculty participate in these activities on a regular basis and, therefore, we believed they were most committed to creating a fair transparent system to reward achievements. Furthermore, the standard deviation for each activity was not large enough to have created significant discrepancies in where a particular activity was ranked. This was a project initiated at a single site, which may limit its generalizability to other institutions. However, similar methods could be used to create site-specific prioritized activities that may enhance its use at other institutions. Finally, it is possible that the increase in conference attendance and resident assessments was confounded by other factors. The changes could have been simply due to faculty feeling the need to attend more conferences or better evaluate our learners, but the effects coinciding with the implementation of new expectations is unlikely to be coincidental. Gender disparities exist in academic medicine.Previous studies in other professional fields have shown that there are differences in language used in describing men and women in letters of recommendation.Additional studies have shown that evaluations of women medical students are more likely to describe women as “caring,” “compassionate,” and “empathetic,” in addition to “bright” and “organized,” than male medical students.In addition, women are more often portrayed as teachers and students, and less often portrayed as researchers or professionals compared to men.Within emergency medicine the letter of recommendation, including both standardized letters and traditional letters, has been cited as one of the top four most important factors in selecting applicants to residency, along with EM rotation grade, interview, and clinical grades.10 More specifically, the letter of recommendation has been cited as the most important factor in selecting applicants to interview.Historically, in EM, letters of recommendation were written without guidelines or restrictions. In 1996, the Council of Residency Directors in Emergency Medicine implemented the standardized letter of recommendation , which was renamed the standardized letter of evaluation in 2013. The SLOE contains both a quantitative evaluation of an applicant and a narrative portion of 250 words or less.The SLOE narrative provides a focused assessment of the noncognitive attributes of potential residency candidates.The standardized format and universal instructions make the SLOE a good text sample to study for variation in language by gender.