A Team-based Approach Evaluating Surgical Residents’ Performance in the Operating Room

Shreyash Pradhan DO¹, Graal Diaz PhD², Anthony McCloud DO¹, Javier Romero MD¹, Shawn Steen MD²

1. Community Memorial Hospital – Graduate Medical Education 147 N. Brent St., Ventura, CA, 93003
2. Ventura County Medical Center 300 Hillmont St., Ventura, CA, 93003

OPEN ACCESS

PUBLISHED: 31 May 2025

CITATION: Pradhan, S., et al., 2025. A Team-based Approach to Evaluating Surgical Residents’ Performance in the Operating Room: A Prospective Survey Study. Medical Research Archives, [online] 13(5).
https://doi.org/10.18103/mra.v13i5.6537

COPYRIGHT: © 2025 European Society of Medicine. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

DOI https://doi.org/10.18103/mra.v13i5.6537

ISSN 2375-1924

---

ABSTRACT

Introduction:
The operating room serves as an indispensable arena for the training and assessment of surgical residents, playing a pivotal role in their professional development. Within this high-stakes environment, the surgical team collectively assumes a significant duty towards ensuring patient safety and well-being. Every member of the team, regardless of their role, is in a unique position to offer critical insights that can aid in evaluating and enhancing the competencies of their peers. This collaborative approach not only fosters a culture of continuous learning and improvement but also directly contributes to the advancement of patient care standards. Through such dynamic inter-professional interactions, the operating room becomes a cornerstone for both educational excellence and clinical proficiency, ensuring that surgical residents are well prepared to meet the challenges of their demanding profession.

Methods:
This prospective survey study aimed to determine if operating room (OR) nurses evaluate surgical residents differently than attending surgeons. 18 OR nurses and 10 attending surgeons in two hospitals evaluated 15 general surgery residents using seven questions from the ACGME Surgical Milestones (CCC) tool.

Results:
A total of 195 evaluations were completed for 15 surgical residents (3 residents from each PGY) by 18 OR nurses and 10 attending surgeons. Significant differences were found in mean scores between attending surgeons and OR nurses in all seven domains:

• Question 1, patient evaluation and decision making (4.01 vs. 2.77, P < 0.004)

• Question 2, intraoperative care and performance of procedures (3.83 vs. 2.63, P < 0.005)

• Question 3, operative patient care and technical skills (3.86 vs. 2.63, P < 0.003)

• Question 4, medical knowledge/anatomy (3.87 vs. 2.70, P < 0.01)

• Question 5, professional behavior and accountability (2.63 vs. 1.59, P < 0.002)

• Question 6, interprofessional and team communication (3.97 vs. 2.70, P < 0.002)

• Question 7, communication within the healthcare system (4.03 vs. 2.67, P < 0.001)

Conclusion:
This study reveals significant differences in how OR nurses and attending surgeons evaluate surgical residents across all seven domains of the ACGME Surgical Milestones. Attending surgeons consistently rated residents higher than OR nurses. These findings suggest that incorporating evaluations from a broader range of team members, including OR nurses, could provide a more comprehensive assessment of surgical residents’ performance and identify areas for improvement. Enhancing interprofessional feedback mechanisms leads to improved training outcomes and better patient care.

Keywords: surgical resident evaluation, operating room, multidisciplinary evaluation, ACGME Surgical Milestones

Introduction

The operating room (OR) is a crucial setting for the training and assessment of surgical residents. However, assessing specific aspects of resident performance in the OR should be more routinely measured and documented.¹ Currently, the residents in our five-year accredited general surgery residency program are evaluated using several methods, including the Accreditation Council for Graduate Medical Education (ACGME) case logs, monthly evaluations, bi-annual Clinical Core Committee (CCC) scores by attending surgeons, and 360-degree evaluations from non-physician staff.

The ACGME case log system tracks the surgical experience of each resident, with residents documenting each procedure performed and indicating their level of participation.² While this system addresses the extent of resident participation in various operations, it does not adequately assess a resident’s overall performance in the OR.³ Monthly rotation evaluations may cover some aspects of OR performance. However, they typically provide a broad overview of a resident’s time in service, requiring faculty to recall multiple individual performances over an extended period.

In addition to these tools, our program utilizes quarterly 360-degree evaluations from non-physician staff, which include feedback from nursing and ancillary staff in hospital wards and outpatient clinical settings. The 360-degree evaluation is the only instrument mediated by non-physicians among the evaluation tools. However, it does not include assessments of residents in the OR, even though surgical residents spend the majority of their training time with OR nurses, sharing direct patient care responsibilities.³

Despite this close interaction, OR nurses are not included in the routine evaluations of surgical residents. Most existing OR assessment instruments focus on generic evaluations of resident operative skills rather than specific operations, procedural knowledge, anatomy, or interactions with the surgical team. According to a study by Guerlain et al., there is a need for better integration between attending surgeons and the rest of the OR team.¹ The study highlighted the importance of procedural-focused discussions and clear team hierarchies, noting that attending surgeons are vital in optimizing OR communication and interactions. Integration is necessary for the comprehensive evaluation of resident performance in the OR.⁴ Further research by S. Bakhtari et al. found that OR nurses viewed an “effective learning and teaching process” as vital for successful teamwork, linking effective collaboration with positive patient care outcomes. They also noted the importance of overseeing the performance of higher-ranking team members, such as surgical residents, as part of successful teamwork. This insight suggests that including OR nurses in the evaluation process could enhance the assessment of surgical residents and improve educational outcomes.⁵

The primary objective of this study is to determine if there is a discrepancy in how OR nurses and attending surgeons evaluate surgical residents’ performance in the OR. Based on our review, no relevant literature addresses this proposed study, making it a novel contribution to the field of surgical education.

---

Method

This prospective survey targeted attending surgeons and OR nurses in the elective and emergency operating rooms of hospitals affiliated with the surgical residency program at Community Memorial Healthcare System (CMHS) and Ventura County Medical Center (VCMC). A specific strategy known as purposive sampling was employed to select individuals for participation. This method involves deliberately choosing participants based on specific criteria that are expected to generate the most valuable insights or data for the research objectives. This approach allows the researchers to focus on particular characteristics or experiences among the participant pool, ensuring that the study’s outcomes are highly relevant and informative in relation to the research questions being investigated.

The inclusion criteria required OR nurses to have a minimum of six months of work experience with the evaluated residents and a willingness to participate. This study aimed to compare evaluations from OR nurses and attending surgeons, focusing on specific aspects of resident performance within the OR. We selected seven domains (questions) from the ACGME Surgical Milestones (CCC) tool, tailored to the operating room setting. Each participating OR staff member completed the survey for up to ten residents with whom they had worked for at least six months.

The survey questions were rated on a scale from 1 to 5 (Figure 1). The selected domains addressed critical areas of resident competency, ensuring a comprehensive assessment. This study’s structured and validated tool provided a reliable measure of resident performance, capturing detailed feedback from OR nurses and attending surgeons. An Internal Review Board (IRB) approved this study. Descriptive and bivariate analyses were done using SAS version 9.4.

Results

In total, 195 evaluations were completed for 15 surgical residents (3 residents from each PGY) by 18 OR nurses and attending surgeons.

Results indicated that OR nurses rated residents significantly higher than attending surgeons overall in all domains. For question 1, patient evaluation and decision making (3.83 vs. 2.63, P<0.0005); question 3, operative patient care and technical skills (3.63 vs. 2.63, P<0.0005); question 4, medical knowledge/anatomy (3.87 vs. 2.63, P<0.0005); question 5, professional behavior and accountability (2.63 vs. 1.59, P<0.0005); question 6, interpersonal and team communication (3.97 vs. 2.70, P<0.0005); and question 7, patient care in the healthcare system (3.03 vs. 2.67, P<0.001) (Table 1).

Table 1. Comparative analysis of ACGME Surgical Milestones (CCC) evaluation scores between operating room nurses and attending surgeons.

Domain	OR Nurses	Attending Surgeons
Patient evaluation and decision making	3.83	2.63
Operative patient care and technical skills	3.63	2.63
Medical knowledge/anatomy	3.87	2.63
Professional behavior and accountability	2.63	1.59
Interpersonal and team communication	3.97	2.70
Patient care in the healthcare system	3.03	2.67

Discussion

The surgical field is inherently team-based, especially within the operating room (OR). The ultimate goal of this teamwork is to provide the best patient care possible. To achieve this, continually evaluating and improving our practices is crucial. Our study found that, on average, OR staff ratings of surgical residents were significantly higher than those of attending surgeons across all seven domains evaluated. Specifically, OR nurses rated residents an average of 3.7/5, whereas attending surgeons rated them an average of 2.52/5, with p < 0.01 in all domains.

HARD SKILLS VS. SOFT SKILLS

When breaking down the evaluations into procedural hard skills (patient evaluation and decision making, intraoperative care and performance of procedures, operative patient care and technical skills, medical knowledge/anatomy), OR nurses rated residents an average of 3.89/5. In contrast, surgical attendings rated them 3.54/5, with p < 0.006. This higher score from nurses may be due to a need for more understanding of the procedural skills and knowledge base taught at higher training levels (i.e., medical school and surgical residency). According to Schlitzkus et al., surgical nurses often need a greater understanding of surgical residents’ capabilities, particularly regarding procedural skills. This lack of understanding may lead OR nurses to perceive residents as more competent than they are, resulting in higher evaluations compared to those given by attending surgeons.⁶⁷

In contrast, when evaluating non-procedural soft skills (professional behavior and accountability, interprofessional and team communication, and communication within the healthcare system), OR nurses rated residents an average of 2.68/5, while attending surgeons rated them 2.32/5, all with p < 0.002. Both groups rated residents lower on soft skills. Surgery is a hands-on, technical field where training focuses primarily on surgical skills, potentially leading to lower scores in soft skills areas. Moreover, OR nurses spend more perioperative time with residents compared to attending surgeons, allowing them more opportunities to observe and evaluate interprofessional interactions, which may contribute to higher ratings.⁸¹⁰

IMPACT OF HIERARCHAL DYNAMICS

The difference in ratings may also be attributed to the diminished hierarchical dynamics between OR staff and residents compared to those between attending surgeons and residents. Residents and OR staff often work closely together more equally, leading to more collegial and friendship-based relationships. Viewing residents as friends may influence OR nurses to rate them higher. Conversely, attending surgeons have more hierarchical and strictly educational relationships with residents, focusing on rigorous evaluation standards, resulting in lower scores. Attendings grade residents higher once they have demonstrated sufficient progress in their training.¹¹¹³

INCORPORATING MULTISOURCE FEEDBACK

Although attending surgeons are currently the primary evaluators, they may only directly oversee residents occasionally throughout their interactions with the surgical team.¹⁴¹⁶ This presents an opportunity to involve OR nurses in the evaluation process, especially in non-procedural areas where they can witness residents’ teamwork, communication, and leadership skills. Incorporating multisource feedback, including evaluations from OR nurses, can provide a more comprehensive assessment of surgical residents and highlight areas for improvement.¹⁷ This approach aligns with the shift in medicine from a paternalistic model to a more patient-centered care model, emphasizing the importance of well-rounded care providers.¹⁸¹⁹

STUDY LIMITATIONS AND FUTURE DIRECTIONS

One limitation of our research was the time discrepancy of about six months between the CCC attending evaluations and the OR nurse evaluations. Given the potential for rapid growth in resident education, this time gap contributed to higher ratings from OR nurses. Another limitation is the educational background of the nurses concerning the evaluation criteria. Alternatively, nurses may need more formal

training in detailed anatomy, surgical procedures, or technical skills required to be a proficient surgeon, leading to potential misperceptions of residents’ abilities. We recommend creating a simultaneous evaluation tool for OR nurses and surgeons to address these limitations in future studies. Additionally, educating nurses on the evaluation parameters of attending surgeons can help ensure a more equitable assessment basis. By refining the evaluation process, we can better identify areas for improvement and enhance surgical residents’ overall education and performance.

Conclusion

On average, the OR nurses’ ratings were significantly higher than those of the attending surgeons in all seven domains. Alternatively, nurses evaluated residents ~3.7 out of 5, whereas attending physicians evaluated ~2.5 out of 5 for residents, with P<0.1 for all domains. Based on this data, OR nurses evaluate surgical residents’ clinical and communication skills differently and more leniently than attending surgeons. The significant difference in scoring may be due to less hierarchical dynamics between OR nurses and residents than between attending surgeons and residents, the attending surgeon’s perception that residents need to “earn” a higher score, and possible lack of complete understanding of OR nurses on the requirements for a higher score in practical surgical skills/knowledge. In the future, a novel questionnaire given to attending surgeons and OR nurses simultaneously may result in more equitable and proportional evaluations of residents. Additionally, facilitating the education of OR nurses on what faculty look for in residents may improve the ability of ancillary staff to contribute more accurately to complete evaluations of surgical residents. Residency is a time of growth, and this is better facilitated when we use all of our available resources to reflect and improve in all aspects.

 

References

1. Guerlain, Stephanie, et al. “Assessing team performance in the operating room: development and use of a “black-box” recorder and other tools for the intraoperative environment.” Journal of the American College of Surgeons 200.1 2005: 29–37.

2. Holmboe, Lobst, William.Eric. Accreditation Council for Graduate Medical Education: Assessment Guidebook. Accreditation Council for Graduate Medical Education. Accessed from
https://www.acgme.org/globalassets/pdfs/milestones/guidebooks/assessmentguidebook.pdf

3. Doyle, Jeffrey D., Eric M. Webber, and Ravi S. Sidhu. “A universal global rating scale for the evaluation of technical skills in the operating room.” The American journal of surgery 193.5 2007: 551–555.

4. Wohaibi, Eyad M., et al. “A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance.” Journal of Surgical Education 64.6 2007: 333–341.

5. Bakhtiari, Soheila, et al. “Perspective and experience of hospital operating room nurses with the concept of excellence: A qualitative study.” Risk Management and Healthcare Policy 13 2020: 125.

6. Schlitzkus, Lisa L., et al. “What do surgical nurses know about surgical residents?.” Journal of Surgical Education 66.6 2009: 383–391.

7. Gumbs, Andrew A., Nancy J. Hogle, and Dennis L. Fowler. “Evaluation of resident laparoscopic performance using global operative assessment of laparoscopic skills.” Journal of the American College of Surgeons 204.2 2007: 308–313.

8. Aggarwal, Rajesh, et al. “Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room.” Annals of surgery 247.2 2008: 372–379.

9. Grabski, David F., et al. “Compliance with the Accreditation Council for Graduate Medical Education duty hours in a general surgery residency program: challenges and solutions in a teaching hospital.” Surgery 167.2 2020: 302–307.

10. Chang, Lily, et al. “Reliable assessment of laparoscopic performance in the operating room using videotape analysis.” Surgical innovation 14.2 2007: 122–126.

11. Deptula, Peter, and Maria BJ Chun. “A literature review of professionalism in surgical education: suggested components for curriculum development.” Journal of Surgical Education 70.3 (2013): 408-422.

12. Igel, Lee H., et al. “Moving Past Individual and “Pure” Autonomy: The Rise of Family-Centered Patient Care.” AMA Journal of Ethics January 2016.

13. Imani, B. and Jalal, S. (2021). Explaining the impact of surgical team communication skills on patient safety in the operating room: a qualitative study. https://doi.org/10.21203/rs.3.rs-1082850/v1

14. Mishra, A., Catchpole, K., Dale, T., & McCulloch, P. (2007). The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy. Surgical Endoscopy, 22(1), 68-73. https://doi.org/10.1007/s00464-007-9346-1

15. Parnikh, H., Kalantari, R., Alaei, E., Khajoogh, Z., Nourani, S., & Movahednia, Z. (2022). Assessment of surgical teams’ teamwork skills in pediatric surgery: a cross-sectional study. Medical – Surgical Nursing Journal, 11(2). https://doi.org/10.5812/msnj-134001

16. Pasquer, A., Ducarroz, S., Lifante, J., Skinner, S., Poncet, G., & Duclos, A. (2024). Operating room organization and surgical performance: a systematic review. Patient Safety in Surgery, 18(1).
https://doi.org/10.1186/s13037-023-00388-3

17. Thomaschewski, M., Kist, M., Zimmermann, M., Benecke, C., Kalff, J., Krüger, C., … & Hummel, R. (2024). Conception and prospective multicentric validation of a robotic surgery training curriculum (rostrac) for surgical residents: from simulation via laboratory training to integration into the operation room. Journal of Robotic Surgery, 18(1). https://doi.org/10.1007/s11701-023-01813-6

18. Yule, S., Gupta, A., Gazarian, D., Geraghty, A., Smink, D., Beard, J., … & Paterson‐Brown, S. (2018). Construct and criterion validity testing of the non-technical skills for surgeons (notss) behaviour assessment tool using videos of simulated operations. British Journal of Surgery, 105(6), 719-727. https://doi.org/10.1002/bjs.10779

19. Fann, J., Calhoon, J., Carpenter, A., Merrill, W., Brown, J., Poston, R., … & Feins, R. (2010). Simulation in coronary artery anastomosis early in cardiothoracic surgical residency training: the boot camp experience. Journal of Thoracic and Cardiovascular Surgery, 139(5), 1275-1281. https://doi.org/10.1016/j.jtcvs.2009.08.04