Article Test

Introduction
Entrustable Professional Activities (EPAs) are increasingly being introduced  in postgraduate medical education (PGME) programmes. Recommendations and guidelines for the design of EPA-based training programmes have been widely published. In addition, numerous studies have addressed the technical and procedural aspects of entrusting residents with an EPA without extensive evaluation and testing * 3 However, studies that systematically evaluate the experiences of professionals (residents and supervisors) working with an EPA-based training programme in daily practice are scarce.

The concept of EPAs was first described in 2005 and further developed in an AMEE guideline in 2015 4. EPAs are components of professional practice that can be entrusted to residents once they have demonstrated the ability to successfully integrate multiple competencies^. EPA-based training programmes have been  introduced worldwide to operationalise and translate competency-based medical education into daily practice ^ 7 This translation was necessary because existing competency frameworks (e.g., ACGME, CanMED) only describe the personal, individual roles of professionals and do not describe the integration of these roles in daily clinical work.

The EPA-based programme provides supervisors with guidance and supports them in assessing the progress of residents in clinical workplaces^. The summative entrustment decision of an EPA is made by a Clinical Competence Committee (CCC), a group of well-informed supervisors. The acquirement of an EPA also involves patient safety, since it is the starting point for residents to practice unsupervised . Since EPAs play an essential role in the professional development of residents, a valid, reliable and reproducible process is required 2 Trust plays a key role in the entrustment decision-making    process. However, it is personal and challenging to substantiate. Trust in the context of traineeships has been described as ‘the reliance of a supervisor or medical team in a trainee to execute a given professional task correctly and in their willingness to ask for help when needed” Many entrusting factors (trainee-, supervisor-, task- and system factors) are involved in gaining trust 0.

Despite the growing understanding of how an entrustment decision is made, the design and implementation of an EPA-based residency training programme is challenging and can be optimised by the sharing of knowledge and experience by professionals in daily practice. Therefore, the current study was conducted to assess the operationalisation and to gain a more comprehensive understanding of the experiences of professionals working with a national EPA-based paediatric residency training programme in the Netherlands.

Materials and methods

Setting
To achieve the aim of this study, we focused on the first EPA-based residency training programme in the Netherlands: Training Optimisation for Postgraduate Paediatrics in 2020 (TOP2020) 1 The TOP2020 was implemented nationally in 2017 and is now operational in all eight medical specialist teaching regions. Each region includes several general hospitals and a university medical centre (UMC). A paediatric resident will receive workplace-based training in at least one UMC and one general teaching hospital in a medical specialist teaching region. Because EPA-based programmes intend to enable flexible and time-variable individualised training, the length of rotations is not the same for each resident.

Entrustment Decision-Making in TOP2020
The    TOP2020    programme    includes    nine generic EPAs that have been developed according to existing guidelines 2 7. During the rotations in the different hospitals, residents will be exposed to all EPAs. A standardised procedure has been introduced to decide whether a resident can acquire an EPA2. In brief, the procedure is as follows: it starts with the collection of evidence of performance as described in the training plan (e.g., progress and    feedback reports and test results). Subsequently, a group of well-informed staff members will discuss the resident during the CCC meeting, concluding with a decision on entrustment . The level of supervision required during rotation depends on a residents level of training and competency. Each level of competence - novice, advanced and competent - requires a different level of supervision 3. For example, a resident with minimal clinical experience requires close supervision. As the rotation progresses, residents will gain more and more (clinical) experience; in most cases this will lead to a gradual reduction in supervision. When the resident is competent and the EPA is acquired, the resident is ready for unsupervised practice. Unsupervised practice means that supervision is not mandatory but available at the resident’s request. In this way, residents experience considerable independence and autonomy during their training programme.

Study design and participants
We conducted a qualitative study to gain a deeper understanding of the experiences of the entrustment decision-making process in an EPA-based residency training programme 4. We set up two independent face-to-face focus groups in the teaching regions, one for residents and one for supervisors. This approach ensured that participants could speak freely, without the constraints of possible hierarchical relationships. All eligible participants were contacted by an independent office to avoid any potential conflict of interest. After at least two weeks of reflection, the focus group session took place, and informed consent was signed beforehand.
The focus groups consisted of 6-10 participants and lasted 60-90 minutes. Supervisor focus groups included supervisors from each participating centre in the teaching region. Since more than enough paediatric residents were interested in participating, we used purposive sampling to create representative focus groups. We ensured that the groups were as diverse as possible by taking into account gender, age, years of training and current hospital of placement (general or academic). The requirement for inclusion was that participants were trained according to TOP2020. There were no exclusion criteria.

Constructivist approach
The constructivist theory describes the coexistence of multiple realities and how individuals construct new knowledge or understanding based on past experiences and knowledge 5. This approach is suitable for addressing our research aim of constructing an understanding of a new training programme in a complex healthcare setting with many interacting aspects and individuals.
We followed the constructivist approach to data collection by scheduling focus group sessions during which participants could interact, influence and add to each others responses^. In addition, the researchers’ beliefs and experiences influenced the focus group because they designed the semi- structured interview guide used during the group discussions. The constructivist approach was also used to analyse and interpret the transcripts 5. We enriched the coding and identification of the themes with our perspectives by taking into account what was said by the participants, both literally and implicitly. Reflexivity was practised by regularly discussing the analytical process, perspectives and findings of the research team members 4.

Research team
The research team consisted of six researchers with diverse backgrounds and areas of expertise. The principal investigator (MS) is a PhD-student and paediatric resident. MdV and KvL are medical educationalists, experienced educators and senior researchers. MdH, RG and JvdV are paediatricians, (former) directors of the pediatric residency training programme and professors of medical education. RG and MdH were involved in developing TOP2020, endorsed by the Dutch Association of Paediatrics, and JvdV was responsible for the implementation and evaluation of TOP2020. To avoid potential conflicts of interest, MdH, RG and JvdV neither conducted nor participated in the focus proup sessions.

Data collection
We developed a semi-structured interview guide to help explore the phases of the entrustment process. The focus proup sessions took place between March and June 2019. MdV was the main moderator, ensuring that all participants had the opportunity to share their experiences and opinions. MS had an observational role, payinp particular attention to non-verbal nuances and askinp for clarification when necessary. Data saturation was reached after ten focus proup sessions in five different teaching regions. These ten focus proups included data from five supervisors and five residential focus groups.

Data analysis
All focus proup sessions were audio recorded and transcribed verbatim. The tapes were transcribed confidentially by a professional apency. MS read and reread all transcripts in order to anonymise them and prevent traceability to participants. The anonymized transcripts were entered into qualitative data analysis software (ATLAS-ti@). The transcripts were analysed usinp inductive thematic analysis following the guidelines of Braun and Clarke 20. Transcripts were coded in 3 phases: descriptive codinp, interpretive codinp and identifying overarching themes.
In the first phase, MS and MdV independently coded a transcript without a predefined codebook. They coded usinp a constructivist approach but with a focus on the aim of the study. After codinp one transcript, they conducted a comprehensive analysis and reached a consensus on a shared codebook. Durinp the interpretive codinp process MS and MdV identified connections between codes and interpreted the underlying meaning of what the participants had said. Disagreements tended to revolve around codes that were closely related in meaning. Disagreements were resolved through discussion or, where necessary, by expanding the definition of the code. Final adjustments were made to the codebook. After five transcripts (2 supervisor and 3 resident proups) consensus on the codebook was reached. MS used the final version to code all the transcripts. Finally, the overarching themes were identified following a discussion about the coded phrases and the relationships between them within the research group.

Results
Our data show that residents and supervisors distinguished two phases in the EPA-based residency training programme. 1) The training phase before the entrustment of an EPA; and
2) The phase following the entrustment of an EPA. The process of entrustment decision making with the CCC meeting takes place between these two phases. Further analysis revealed three central themes in the phases: individualised training, role of supervisors, and the practical process.

Training phase before the entrustment of an EPA

Individualised training
A main finding was that both residents and supervisors felt that the EPA-based residency training programme facilitated individualisation. It also emerged that individualisation had several prerequisites, including personal leadership, self-reliance, and independence. Personal leadership enabled residents to explore opportunities for individualisation. They learn to take matters into their own hands, set goals and take responsibility for achieving those goals (Quotations 1 and 2, see table 1). In an EPA-based curriculum, residents may seize opportunities to organise and personalise their training programme in consultation with the programme director and accounting for constraints of the clinical setting (Quotation 3, see table 1).

Role of supervisors
Many residents need support and guidance from their supervisors and programme director throughout the process of EPA entrustment (Quotation 4, see table 1). Guidance at the start of the training programme is crucial, as the programme director determines the entry level and takes into account previous experience, resulting in an individualised training programme.

Supervisors were very content to have a supportive role, to provide tailored supervision, and to allow residents to grow towards the EPA acquirement. However, some supervisors felt that residents did not keep sufficient track of their progress towards the acquirement of an EPA (Quotation 5, see table 1). They noted the risk of shifting responsibility for successful completion of a rotation and personal goals from resident to supervisor.

Many residents do not recognise themselves in this shift of responsibility. They feel minimal supportive guidance from supervisors or programme director in particular when it comes to practical matters and the possibility of individualisinp their rotation (Quotation 6, see table 1). Nevertheless, most residents feel supported in discussing their development, recopnisinp professional limits and askinp for supervision when needed. Residents reported almost unanimously that they discuss an important consideration with their supervisors and the programme director: Am I reaching the required level of competence?

Practical process
According to supervisors, residents are responsible for obtaining the mandatory requirements and orpanisinp the portfolio as defined in the TOP2020 training plan (Quotation 7, see table 1). Residents’ opinions on this issue were mixed. Some of them found it worthwhile to collect evidence of performance since it contributed to their learning curve. Others considered it a time- consuming burden and did not see the added value (Quotation 8, see table1), believing that feedback based on, for example, a brief observation does not represent their overall performance.

Supervisors are expected to reflect on a residents performance, which many find difficult for a number of reasons. First, providing comprehensive feedback in a set format is difficult, which is why it often remains practical because trust issues or gut feelings are difficult to put into words. Second, some fear the possible consequences of their feedback reports (Quotation 9, see table 1). Many programme directors recognised this reluctance on the part of their colleagues (Quotation 9, see table 1).

Table 1. Training phase before the acquirement of an EPA Overview of themes, corresponding codes, quotation examples.

The entrustment decision-making process of an EPA
Residents and supervisors identify the entrustment decision as a key moment in the growth of resident competence that takes place between the two phases. Several themes emerged from the analysis that are relevant to this key moment.

Portfolio
Residents and supervisors submit their evaluation reports independently, which encourages reflection before the CCC meeting. Residents submit their portfolio, which they both felt was primarily the resident’s responsibility.

Many residents felt that supervisors did not really need the portfolio to assess their level of performance; they reported that they already knew what level the resident was working at through daily contact. When asked which role the portfolio should have in the decision-making process, residents and supervisors unanimously said that, the portfolio should not be decisive’. They made it clear that trust of supervisors in residents is more important than mandatory feedback forms and reports.

According to supervisors, the requirements for the portfolio are minimal, but still helpful for making residents level of performance more explicit. From the summarised portfolio and the preparation for the CCC meeting, supervisors have the  impression that the whole group is well informed, up to date and prepared.

CCC meeting
Most residents reported that they had no insight into or understanding of the role of trust during the CCC meeting. They felt that supervisors already knew from daily contact whether or not they could trust a resident to work without direct supervision. Furthermore, they considered the CCC meeting a necessary Normalisation, since their independence was already gradually increasing during the rotation (Quotation 10, see table 2).

Supervisors recognised the natural and gradual progression towards independence, but did not see the CCC meeting as simply formalising what was already known or done. Supervisors said that trust in a resident is determined by multiple trust factors (supervisor, trainee, task and system factors) and can therefore be an elusive concept (Quotations 11 and 12, see table 2). They found it helpful and essential to discuss residents’ performance with other supervisors. Examples of relevant discussion topics: Do the promising qualities become apparent in practice? is the resident aware oI his/her limitations? Will the resident ask for supervision when needed? Supervisors reported that the standardised entrustment procedure, with the CCC meeting as the final step, helps decide whether a resident is competent and capable of working without direct supervision. This mainly concerned residents with previous experiences in different hospitals. Supervisors experienced that the final decision-making in the CCC meeting was almost always unanimous. They also reported that they felt relieved that the entrustment decision was a group responsibility.

Feedback
After the CCC meeting, residents receive substantive feedback and conclusion from the programme director, sometimes accompanied by a mentor. Residents sometimes felt that the programme directors found it difficult to translate the group- supported feedback into a more nuanced and practical feedback, since they often watered down critical remarks. Many residents speculated that programme directors were afraid to jeopardise their friendly working relationship by giving critical feedback. Most residents considered it a missed opportunity to develop themselves for professional development and said that they preferred to receive specific and constructive feedback. Many programme directors recognised themselves in the opinions of residents, and found it challenging to make the feedback specific and practical for residents without being too critical (Quotation 13, see table 2).

Table 2. The entrustment decision-making process of an EPA Overview of themes, corresponding codes, quotation examples

The phase following the acquirement of an EPA

Individualised training
At the start of each focus proup session, residents and supervisors indicated that the EPA-based programme did not contribute to a gradual increase in autonomy and responsibility. It was argued that the progression to eventually unsupervised practice is autonomous and not linearly related to the desipn of the training programme (Quotation 14, see table 3). Later, as the focus proup session progressed, more and different insights were presented, and the initial scepticism gradually made way for a more nuanced view.
Many residents found that EPAs helped them to take more control during their training and to communicate their learning goals to their supervisors. They found it easier to explain their level of competence and whether or not they need supervision during shifts (Quotation 15, table 3). Residents had different opinions about the feasibility of designing their training programme. Some see opportunities to broaden their personal educational goals in addition to general clinical requirements. For example, by taking time to develop their own science and management skills, or by attending specialist outpatient clinics. Contributing factors to arrange these possibilities included assertiveness, self- confidence and discussing special wishes and interest with the programme director at an early stage. Some residents felt discouraged from individualising their programme because the programme director did not provide practical resources or time. They felt personally disadvantaged when, for example, other residents were allowed to take an extra course or rotation. Other residents reported that clinical healthcare tasks consumed all their time and energy, and that they experienced no opportunities for individualisation following the acquirement of an EPA. These residents felt underprivileged because other colleagues seemed to have more possibilities for individualisation (Quotation 16, table 3).

Supervisors indicated that EPAs are helpful for individualisation, since EPAs are requested and acquired at different times, allowing for individual trajectories of the training programme for each resident. The inter-individual differences between the possibilities for  realizing  individualisation were acknowledged by supervisors and programme directors. Supervisors almost unanimously agreed that the design of the training programme was not essential for creating opportunities for individualisation. There will always be residents who see opportunities to fulfil  their  individual learning goals and aspirations, with or without EPAs. Supervisors and programme directors agreed that they were more likely to support self-directed residents in fulfilling their wishes.

Role of supervisors
Supervisors found it reassuring that they are provided with information about the level of expertise of each individual resident. At the start of a shift, they only need to ask a resident what level of supervision for which EPA is desired. However, many supervisors still found it challenging to reduce control and allow residents to work unsupervised, even those with an acquired EPA. The almost unanimous explanation was that they had a strong sense of responsibility for their vulnerable patients.

Practical Process
Both residents and supervisors found it difficult to tailor rotations because of their continuing clinical care responsibilities and the need to fill shifts. In practice, adjustments to the schedule could only be made prior to the start of the rotation, makinp it difficult to tailor rotations based on a resident’s development durinp rotations.

Table 3. The phase following the acquirement of an EPA Overview of: theme, corresponding codes, and quotation examples.

Table 4. Summary of main results.
Experiences of residents and supervisors working and learning in an EPA-based residency training programme

Discussion
This study provides a more comprehensive understanding of how medical residents and supervisors value an EPA-based residency training programme. Three main themes emerged when learning and working in an EPA-based programme in daily practice: self- regulated learning, entrustment and autonomy.

Self-regulated learning
EPAs aim to provide  opportunities  for residents to follow their learning path and individualise their training in a flexible and time-variable way™  EPAs  encourage residents to reflect on their personal competencies and needs, which  is  an essential step towards developing self- regulated learning (SRL) skills required for individualised training21. SRL can  be  defined as ‘the degree to which students are metacognitively, motivationally, and behaviourally active participants in their own learning process’ 2*. Further development of SRL will be beneficial, not only in terms of individualisation in the EPA-based training but also because high levels of SRL IS associated with more effective learning and easier lifelong learnin 23 24.

Our study revealed that an essential prerequisite for individualising the training programme is the ability of residents to show leadership. For instance, combining personal goals with the organisational rotation schedule was considered a major challenge by some residents in realising individualised training programmes. Potential inequalities arise since less self-directed and self- confident residents effectuate fewer opportunities to individualise their training programme. Some residents were unaware of the importance of personal leadership in enabling individualisation. Our findings raise the question whether sufficient attention and guidance is paid to SRL during residency training2*. Guiding residents and faculties to improve SRL is likely to be beneficial, as improving implementation and outcomes of SRL will only be possible if it is seen as a team effort.

A lifelong learning mindset is essential for residents and medical specialists to keep up with the constant changes and innovations in healthcare. In such a mindset, entrustment of an EPA is not the end point of learning, but a milestone on a lifelong learning journey. If, in daily practice, residents experience entrustment as an end point, working and learning with EPAs may encourage assessment-driven learning, which is potentially detrimental to a lifelong learning mindset 25. Therefore, the focus of SRL guidance should not be on how to tick off EPAs, but how to use EPAs during a lifelong learning journey.

Entrustment
Summative entrustment involves a complex decision-making process, for which the CCC meeting was highly valued by supervisors. For supervisors, it was important to have a group discussion about gut feelings and trust. The entrustment decision of a professional activity was no longer seen as an individual responsibility, but as a group-supported decision. The individual assessment of residents by CCC members prior to the CCC meeting ensured that the views of all supervisors were represented and collected independently. Supervisors reported that CCC meeting outcomes were almost always unanimous. When opinions were divided at first, short discussions were sufficient to reach consensus.

Well-prepared meetings and detailed guidance on the aim and conduct of CCC meetings in this national EPA-based residency programme might have contributed to the positive experience of CCC meetings in our study. Previous literature shows the importance of a ‘shared mental model’, where group members have a common understanding of the purpose of the meeting, know how to interpret information, and how to make decisions2^ 2 . Group assessments have proven to be valuable. Furthermore, a group decision-making reduces the halo effect compared to individual judgement 3 2’ 30.

The portfolio was appreciated by supervisors, and they often felt more comfortable and guided by semi-quantitative Likert scales and quantitative questions such as ‘how many critically ill patients did the resident see in the emergency department?’ Although checking boxes in the portfolio may seem like a convenient tool for supervisors, this usually does not reflect actual exposure, level of performance or professional growth during rotations31. Narrative feedback and group discussion  on trust  during  the CCC meeting provides more insight into the resident’s level of performance. Especially as trust is a subjective element, also called gut feeling, which becomes reproducible when professionals share their experiences’.
Despite its merits, many residents felt that preparing their portfolio was a burden with no added value. They argued that supervisors already knew their level of professional performance through daily contact, and emphasised that acquisition of competences is a continuous, iterative process that cannot be exhaustively captured in quantitative measures.

Recent literature shows that the use of portfolios does not support SRL in workplace- based settings, especially when there is frequent or close supervision 3. However, portfolios can add value by ensuring that all CCC members are well informed and prepared for the meeting, since multiple sources of information are necessary for the validity of entrustment decisions’. Furthermore, the portfolio can capture conditional and indispensable information that is necessary before an EPA can be entrusted. Since portfolios play an increasingly important role in the recertification of medical specialists, we hypothesise that working with portfolios during residency may help prepare for future obligations.

Following the  entrustment  decision, residents receive aggregated group-based feedback. Data analysis revealed that residents and supervisors felt that the feedback could be improved, particularly by  making  it  more  specific  and learning-oriented.  Both  groups  acknowledged that the fear that critical feedback could affect the continuation of a constructive working relationship, sometimes hampered the manner of feedback. Therefore, in professional assessment and high stakes decision-making it is essential that critical feedback is formulated  in  a  constructive way that  facilitates  a  resident’s  subsequent I earn ing curve3 33 . To establish a supportive and safe feedback culture, four domains need to be considered: feedback providers, feedback recipients, feedback relationships, and institutional context  33  Several suggestions were made during the focus group meetings to improve the feedback culture. It was suggested to include  a mentor, as an independent  supervisor,  in the CCC meeting and have the mentor give feedback on behalf of other supervisors  and programme director  in  order  to  create a safe feedback relationship. Another suggestion was to invite residents to join the CCC meeting for the final minutes, which would allow CCC members to clarify their feedback to the resident face to face. However, residents can feel intimidated when they meet their supervisors after they have just discussed their performance, which could put the safe context required for receiving feedback at risk34.

Autonomy
The EPA-concept aims to guide residents and supervisors when estab lishin g a graded increase in autonomy and responsibility towards readiness for independent practice of key tasks of the profession. Many residents experienced that acquiring EPAs empowered them to express  their supervision needs. Neverth eless, supervisors recognised challenges in working with entrusted residents. Reducing control sometimes conflicted with their high sense of responsibility for their patients.
Many factors play a role in helping supervisors provide autonomy supportive supervision, e.g., medical  legislation, national  and  local  protocols  and policies35 3^. This chal len ge n eeds to be addressed, since it  hinders  the development of autonomy in an EPA-based residency training programme. Furthermore, it may be helpful for  supervisors if residents are more explicit about knowing and monitoring their own limits during the entrustment process.

Strengths and limitations
We studied a fully operational national EPA-based residency training programme in a country ranked top five in the world for healthcare quality and efficiency 37. This study provided a comprehensive insight into and understanding of how professionals experience working and learning in an EPA-based residency training programme. Since the implementation of our EPA-based residency training programme in 2017, the programme has been used as a template for other residency training programmes in the Netherlands.
Limitations of the study were that only paediatric residents and supervisors in one health care system were included, thereby limiting extrapolation to other settings. In addition, there was no interim analysis of the data. We analysed the data after the focus group sessions had taken place.

Future implications
This study was conducted two years after the national implementation of the EPA- based residency training programme. In line with our findings, we believe that further research could focus on faculty development, with a particular focus on how to optimise SRL, feedback, and autonomy following the entrustment. In addition, longitudinal research would be interesting. After all, since there is more familiarity with working, acquiring and entrusting EPAs, experiences of residents and supervisors may chanpe over time. We are also interested in comparing differences, experiences and well-beinp affected by EPAs between various EPA- based residency training programmes at national and international level.

Conclusion
We found that residents and supervisors see added value in working and learning in an EPA-based residency training programme. The central themes were: self-regulated learning, entrustment and autonomy. Awareness and encouragement of self- regulated learning skills could potentially help create a balance between programme requirements and individualisation. When discussing a portfolio, trust and gut-feelings during CCC meetings is fundamental and helpful for supervisors to get a comprehensive view of the resident’s performance. Following the entrustment decision, residents are open to concrete feedback since it contributes to their professional development. However, providing critical feedback can be challenging for supervisors. In order to encourage residents’ autonomy and practice of working without direct supervision, we need to identify what is missing at a national, regional and local level that could contribute to autonomy- supportive supervision.

Corresponding author:    
Prof. J. van der Velden
Radboudumc Amalia Children’s Hospital, 
Geert Grooteplein Zuid 32, 6525 GA,
Nijmepen, The Netherlands.
Tel: +31 243614415
Email: [email protected]

Acknowledgements:
The authors would like to thank residents and supervisors for their participation in the focus proups and their willingness to share their experiences.

Conflict of interest statement:
The authors declare that they have no conflicts of interest.

Funding statement:
No external funding for this study.
 
Ethics:
This study received approval from the Ethical Review Board of the Netherlands Association for Medical Education (NERB number 2019.1.7)

References:
1. El‐Haddad C, Damodaran A, McNeil HP, Hu W. The ABCs of entrustable professional activities: an overview of ‘entrustable professional activities’ in medical education. Internal medicine journal. 2016;46(9):1006- 1010.

2. Smit MP, de Hoog M, Bracket HJ, Ten Cate O, Gemke RJ. A national process to enhance the validity of entrustment decisions for Dutch pediatric residents. Journal of graduate medical education. 2019;11(4s):158- 164.

3. Duitsman ME, Fluit CR, van Allen-van der Velden J, et al. Desipn and evaluation of a clinical competency committee. Perspectives on medical education. 2019;8(1):1-8.

4. Ten Cate O. Entrustability of professional activities and competency-bases training. Medical education. 2005;39:1176-1177.

5. ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Medical teacher. 2015;37(11):983-1002.

6. ten Cate O, Scheele F. Competency- based postgraduate training: can we bridge the gap between theory and clinical practice? Academic Medicine. 2007;82(6):542-547.

7. ten Cate O. Nuts and bolts of entrustable professional activities. Journal of graduate medical education. 2013;5(1):157-158. 

8. Peters H, Holzhausen Y, Boscardin C, ten Cate O, Chen HC. Twelve tips for the implementation of EPAs for assessment and entrustment decisions. Medical teacher. 2017;39(8):802-807.

9. ten Cate O, Hart D, Ankel F, et al. Entrustment Decision Making in Clinical Training. Academic Medicine. 2016;91 (2):191- 198. doi:10.1097/acm.0000000000001044

10. Choo KJ, Arora VM, Barach P, Johnson JK, Farnan JM. How do supervising physicians decide to entrust residents with unsupervised tasks? A qualitative analysis. Journal of hospital medicine. 2014;9(3):169-175.

11. TOP2020 part 1. Dutch Association for Pediatricians. Accessed May 2022, https://assets.nvk.nl/p/491522//files/TOP2020
%20dee %20 %20de initie aa %20201 df

12. van Rossum TR, Scheele F,  Sluiter HE, Bosman PJ, Rijksen L, Heylipers IC. Flexible competency based medical education: more time efficient, hipher costs. Medical teacher. 2018;40(3):3 15-317.

13. Dreyfus SE, Dreyfus HL. A five-stage model of the mental activities involved in directed skill acquisition. 1980.

14. Tavakol M, Sandars J. Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part I. Medical Teacher. 2014;36(9):746-756.

15. Mann K, MacLeod A. Constructivism: learning theories and approaches to research. In: Durning SJ, Cleland J, eds. Researching
medical education. Wiley Blackwell; 2015:49-66.

16. Stalmeijer RE, McNaughton N, Van Mook 24. van Houten-Schat MA, Berkhout JJ, van WN. Usinp focus proups in medical education Dijk N, Endedijk MD, Jaarsma ADC, Diemers research: AMEE Guide No. 91. Medical AD. medical education in review. Medical teacher. 2014;36(11):923-939.    Education. 2018;52:1008-1015.

17. Frambach JM, van der Vleuten CP, Durning SJ. AM last page: Quality criteria in qualitative and quantitative research. Academic Medicine. 2013;88(4):552.

18. Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qualitative health research. 1999;9(1):26-44.

19. Tavakol M, Sandars J. Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part II. Medical teacher. 2014;36(10):838-848.

20. Braun V, Clarke V. Usinp thematic analysis in psychology. Oualitative research in psychology. 2006;3(2):77-101.

21. Williams GC, Saizow RB, Ryan RM. The importance of self-determination theory for medical education. Academic Medicine. 1999;74(9):992-5.

22. Zimmerman BJ. Investipatinp Self- Regulation and Motivation: Historical Background, Methodological Developments, and Future Prospects. American Educational Research Journal. 2008;45(1):166-183. doi:10.3102/0002831 2073 1 2909

23. Siddaiah-Subramanya M, Nyandowe M, Zubair O. Self-regulated learning: why is it important compared to traditional learning in medical education? Advances in medical education and practice. 2017:243-246.

25. Pacifico JL, Villanueva JAS, Heeneman S, van der Vleuten C. How perceptions of residents toward assessment influence learning: a qualitative study. Asia Pac Scholar. 2020;5(1):46-53.

26. Lu L, Yuan YC, McLeod PL. Twenty-five years of hidden profiles in group decision making: A meta-analysis. Personality and Social Psychology Review. 2012;16(1):54-75.

27. Janis IL. Groupthink. IEEE Engineering Management Review. 2008;36(1):36.

28. Hauer KE, Cate Ot, Boscardin CK, et al. Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees. Journal of graduate medical education. 2016;8(2):156-164.

29. Stasser G, Titus W. Pooling of unshared information in group decision making: Biased information sampling during discussion. Journal of personality and social psychology. 1985;48(6):1467.

30. Stasser G, Titus W. Effects of information load and percentage of shared information on the dissemination of unshared information during group discussion. Journal of personality and social psychology. 1987;5J(1):81.

31. Van Der Gulden R, Heeneman S, Kramer A, Laan R, Scherpbier-de Haan N, Thoonen B. How is self-regulated learning documented in e-portfolios of trainees? A content analysis. BMC medical education. 2020;20(1):1-13.

32. Van der Gulden R, Timmerman AA, Sagasser MH, et al. How does portfolio use support self-regulated learning during general practitioner specialty training? A qualitative focus group study. BMJ open. 2023;13(2):e066879. 

33. Ramani S, Konings KD, Ginsburg S, van der Vleuten CP. Twelve tips to promote a feedback culture with a growth mind-set: Swinging the feedback pendulum from recipes to relationships. Medical teacher. 2019;41(6):625-631.

34. Sargeant JM, Mann KV, Van der Vleuten CP, Metsemakers JF. Reflection: a link between receiving and using assessment feedback. Advances in health sciences education. 2009;14(3):399-410. 

35. Biondi EA, Varade WS, Garfunkel LC, et al. Discordance between resident and faculty perceptions of resident autonomy: can self-determination theory help interpret differences and guide strategies for bridging the divide? Academic Medicine. 2015; 90(4):462-471. 

36. Baldwin CD, Craig MS, Garfunkel LC, et al. Autonomy-Supportive Medical Education: Let the Force Be: Within: You! Academic Medicine. 2012;87(11):1468-1469. 

37. Schneider EC, Shah A, Doty MM, Tikkanen R, Fields K, Williams II RD. Reflecting Poorly: Health Care in the US Compared to Other High-Income Countries. 2021.