“Validation and reliability of the self-efficacy scale to assess the competence of pediatricians in ECHO-Pediatrics course”
Main Article Content
Le Hong Nhung, MD, PhD, M.Sc.
http://orcid.org/0000-0002-6048-6570
Nguyen Phuong Lan, M.Sc.
Piter Martinez Benitez, MD
Pham Van Hoc, MD
Pham Duc Han, Pharmacist, MSc.
Matthew F Bouchonville, MD, CDCES
http://orcid.org/0000-0003-2852-0510
http://orcid.org/0000-0002-6048-6570
Nguyen Phuong Lan, M.Sc.
Piter Martinez Benitez, MD
Pham Van Hoc, MD
Pham Duc Han, Pharmacist, MSc.
Matthew F Bouchonville, MD, CDCES
http://orcid.org/0000-0003-2852-0510
Abstract
Background: Self-assessment of competency in CME/CPD has been fully approved by the American Medical Association (AMA) in its program of retention of practice certificates. To be accepted and applicable in the CME/CPD course in Vietnam, the scale of self-efficacy or self-assessment of competency requires modification. It is helpful for the future studies related to the ECHO courses that the scale of self-efficacy or self-assessment of competency are applied with its respect to validity and reliability.
Aims: In the current study, we aimed to introduce the theoretical framework for developing a set of scales to facilitate self-assessment of professional capacity among PHWs participating in project ECHO. Additionally, we proposed a viable method for creating a self-efficacy scale applicable to the continuing medical training programs. We hypothesized that the scale of efficacy would integrate into the research and evaluation procedure at VNCH.
Methods: We applied a cross-sectional study design, implementing mixed methods including quantitative and qualitative in order to adjust the scale of self-efficacy to assess the professional capacity improvement of the healthcare workers to ensure the validity and reliability of the scale. The study proceeds in the following steps: 1) Reviewing the sets of scales for self-assessment of competency (self-efficacy) according to the ECHO model in the world; 2) Collecting the decisions from the panel of experts to determine the content of primary scale to improve professional capacity through continuous medical training; 3) Test the scale on a sample of learners participating in a continuing medical training program using the ECHO model; 4) Evaluate the surface validity, the content of validity including the convergent value and discriminant value and the structural reliability with Cronbach's Alpha internal consistency index.
Conclusion: The 22-item self-assessment of competency scale was developed based on the results of qualitative and quantitative research, and is considered to be a reliable scale applicable to further studies on the ECHO model in Vietnam.
Keywords:
self-efficacy, CME, Delphi method, reliability, validity
Article Details
How to Cite
NHUNG, Le Hong et al.
“Validation and reliability of the self-efficacy scale to assess the competence of pediatricians in ECHO-Pediatrics course”.
Medical Research Archives, [S.l.], v. 12, n. 10, oct. 2024.
ISSN 2375-1924.
Available at: <https://esmed.org/MRA/mra/article/view/5771>. Date accessed: 22 dec. 2024.
doi: https://doi.org/10.18103/mra.v12i10.5771.
Section
Research Articles
The Medical Research Archives grants authors the right to publish and reproduce the unrevised contribution in whole or in part at any time and in any form for any scholarly non-commercial purpose with the condition that all publications of the contribution include a full citation to the journal as published by the Medical Research Archives.
References
1. Faherty LJ, Rose AJ, Chappel A, Taplin C, Martineau M, Fischer SH. Assessing and Expanding the Evidence Base for Project ECHO and ECHO-Like Models: Findings of a Technical Expert Panel. Journal of general internal medicine. Mar 2020;3 5(3):899-902. doi:10.1007/s11606-019-05599-y
2. Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. The New England journal of medicine. Jun 9 2011;364(23):2199-207. doi:10 .1056/NEJMoa1009370
3. Salvador JG, Bhatt SR, Jacobsohn VC, et al. Feasibility and acceptability of an online ECHO intervention to expand access to medications for treatment of opioid use disorder, psychosocial treatments and supports. Substance Abuse. 2020:1-8.
4. Pollack TM, Nhung VTT, Vinh DTN, et al. Building HIV healthcare worker capacity through telehealth in Vietnam. BMJ global health. 2020;5(4) :e002166.
5. Eaton LH, Godfrey DS, Langford DJ, Rue T, Tauben DJ, Doorenbos AZ. Telementoring for improving primary care provider knowledge and competence in managing chronic pain: A randomised controlled trial. Journal of telemedicine and telecare. Jan-Feb 2020;26(1-2):21-27. doi:10.1 177/1357633x18802978
6. Holmes CM, Keyser-Marcus, L., Dave, B., & Mishra, V Project ECHO and opioid education: a systematic review. Current Treatment Options in Psychiatry. 2020:14.
7. Carlin L, Zhao J, Dubin R, Taenzer P, Sidrak H, Furlan A. Project ECHO telementoring intervention for managing chronic pain in primary care: Insights from a qualitative study. Pain Medicine. 2018;19 (6):1140-1146.
8. Johnson KL, Hertz D, Stobbe G, et al. Project Extension for Community Healthcare Outcomes (ECHO) in multiple sclerosis: increasing clinician capacity. International journal of MS care. 2017;19 (6):283-289.
9. Fisher E, Hasselberg M, Conwell Y, et al. Telementoring primary care clinicians to improve geriatric mental health care. Population health management. 2017;20(5):342-347.
10. Gordon MJ. Self-assessment programs and their implications for health professions training. Academic medicine : journal of the Association of American Medical Colleges. Oct 1992;67(10):672-9. doi:10.1097/00001888- 199210000-00012
11. Approved initiatives for Maintenance of Certification for the ABMS (2006).
12. Arora S, Kalishman SG, Thornton KA, et al. Project ECHO: A Telementoring Network Model for Continuing Professional Development. The Journal of continuing education in the health professions. Fall 2017;37(4):239-244. doi:10.1097/ ceh.0000000000000172
13. Komaromy M, Duhigg D, Metcalf A, et al. Project ECHO (Extension for Community Healthcare Outcomes): A new model for educating primary care providers about treatment of substance use disorders. Substance abuse. 2016;3 7(1):20-4. doi:10.1080/08897077.2015.1129388
14. Shelley BM, Katzman JG, Comerci GD, Jr., et al. ECHO Pain Curriculum: Balancing Mandated Continuing Education With the Needs of Rural Health Care Practitioners. The Journal of continuing education in the health professions. Summer 2017;37(3):190-194. doi:10.1097/ceh.000 0000000000165
15. Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The Impact of Project ECHO on Participant and Patient Outcomes: A Systematic Review. Academic medicine : journal of the Association of American Medical Colleges. Oct 2016;91(10) :1439-1461. doi:10.1097/acm.0000000000001328
16. Arora S, Geppert CM, Kalishman S, et al. Academic health center management of chronic diseases through knowledge networks: Project ECHO. Academic medicine : journal of the Association of American Medical Colleges. Feb 2007;82 (2):154-60. doi:10.1097/ACM.0b013e31802d8f68
17. A. B. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: PrenticeHall; 1986.
18. Mind in Society: The Development of Higher Psychological Processes. MA: Harvard University Press; 1978.
19. Situated Learning: Legitimate Peripheral Participation. . Cambridge university press; 1991.
20. Control. BASETEo. W.H. Freeman and Co; 1997.
21. Coach DSaDBM. Increasing Student Mathematics Self-Efficacy Through Teacher Training. Journal of Advanced Academics. 2007;18(2):44.
22. Mazurek MO, Brown R, Curran A, Sohl K. ECHO Autism. Clinical pediatrics. Mar 2017;56(3):247-256. doi:10.1177/0009922816648288
23. Arora S, Kalishman S, Thornton K, et al. Expanding access to hepatitis C virus treatment--Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology (Baltimore, Md). Sep 2010;52(3) :1124-33. doi:10.1002/hep.23802
24. Masi C, Hamlish T, Davis A, et al. Using an established telehealth model to train urban primary care providers on hypertension management. Journal of clinical hypertension (Greenwich, Conn). Jan 2012;14(1):45-50. doi:10.1111/j.1751-7176.20 11.00559.x
25. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of advanced nursing. 2006;53(2):205-212.
26. Mengual-Andrés S, Roig-Vila R, Mira JB. Delphi study for the design and validation of a questionnaire about digital competences in higher education. International Journal of Educational Technology in Higher Education. 2016;13:1-11.
27. Blasco Mira JE, López Padrón A, Mengual Andrés S. Validación mediante el metodo Delphi de un cuestionario para conocer las experiencias e interés hacia las actividades acuáticas con especial atención al Winsurf. 2010;
28. Campbell DT, & Fiske, D. W. Convergent and discriminant validation by the multitrait-multimethod matrix.Psychological Bulletin. 1959;5 6(2):25. doi:https://doi.org/10.1037/h0046016
29. Landeta O, Calvete E. Adaptación y validación de la escala multidimensional de apoyo social percibido. Ansiedad estrés. 2002:173-182.
30. Lawshe CH. A QUANTITATIVE APPROACH TO CONTENT VALIDITY. Personnel Psychology. 1975; 28:563-575.
31. Arora S, Thornton K, Bradford A. Access to care: Management of hepatitis C viral infection in remote locations†. Clinical Liver Disease. 2012; 1(3):87-90. doi:10.1002/cld.46
32. Arora S, Brakey HR, Jones JL, Hood N, Fuentes JE, Cirolia L. Project ECHO for Cancer Care: a Scoping Review of Provider Outcome Evaluations. Journal of Cancer Education. 2023/10/01 2023;38 (5):1509-1521. doi:10.1007/s13187-023-02292-x
33. Tian J, Atkinson NL, Portnoy B, Lowitt NR. The Development of a Theory-Based Instrument to Evaluate the Effectiveness of Continuing Medical Education. Academic Medicine. 2010;85(9):1518-1525. doi:10.1097/ACM.0b013e3181eac3fb
34. Ramos MM, Sebastian RA, Murphy M, Oreskovich K, Condon TP. Adolescent substance use: Assessing the knowledge, attitudes, and practices of a school-based health center workforce. Substance abuse. Apr-Jun 2017;38 (2):230-236. doi:10.1080/08897077.2017.1287149
35. Löfmark A, Mårtensson G. Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts. Nurse Education Today. 2017/03/01/ 2017;50:82-86. doi:https://doi.org/10.1016/j.nedt.2016.12.009
36. Hong QN, Pluye P, Fàbregues S, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. Journal of Clinical Epidemiology. 2019/07/01/ 2019;111:49-59.e1. doi:https://doi.org/10.1016/j.jclinepi.2019.03.008
37. Gandek B, Ware J, Aaronson N, et al. Tests of data quality, scaling assumptions, and reliability of the SF-36 in eleven countries: results from the IQOLA Project. International Quality of Life Assessment. Journal of clinical epidemiology. 1998;51 11:1149-58.
38. Ware J, Gandek B. Methods for testing data quality, scaling assumptions, and reliability: the IQOLA Project approach. International Quality of Life Assessment. Journal of clinical epidemiology. 1998;51 11:945-52.
39. McHorney CA, Ware JE, Jr., Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical care. Mar 1993;31(3):247-63. doi:10.1097 /00005650-199303000-00006
40. Sockalingam S, Rajaratnam T, Gambin A, Soklaridis S, Serhal E, Crawford A. Interprofessional continuing professional development programs can foster lifelong learning in healthcare professionals: experiences from the Project ECHO model. BMC Medical Education. 2022/06/06 2022; 22(1):432. doi:10.1186/s12909-022-03500-w
41. Hojat M, Veloski JJ, Gonnella JS. Measurement and Correlates of Physicians’ Lifelong Learning. Academic Medicine. 2009;84(8):1066-1074. doi:10 .1097/ACM.0b013e3181acf25f
42. Ware J, Brook R, Davies A, et al. Conceptualization and Measurement of Health for Adults in the Health Insurance Study: Vol. I, Model of Health and Methodology. 1979:
43. Humphrey-Murto S, Varpio L, Wood TJ, et al. The Use of the Delphi and Other Consensus Group Methods in Medical Education Research: A Review. Academic Medicine. 2017;92(10):1491-1498. doi:1 0.1097/acm.0000000000001812
44. Nguyen Thanh Huong LNK. Designing and validation of a living standards questionnaire in Vietnam's elderly community. Medical Practice Journal. 2010;
2. Arora S, Thornton K, Murata G, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. The New England journal of medicine. Jun 9 2011;364(23):2199-207. doi:10 .1056/NEJMoa1009370
3. Salvador JG, Bhatt SR, Jacobsohn VC, et al. Feasibility and acceptability of an online ECHO intervention to expand access to medications for treatment of opioid use disorder, psychosocial treatments and supports. Substance Abuse. 2020:1-8.
4. Pollack TM, Nhung VTT, Vinh DTN, et al. Building HIV healthcare worker capacity through telehealth in Vietnam. BMJ global health. 2020;5(4) :e002166.
5. Eaton LH, Godfrey DS, Langford DJ, Rue T, Tauben DJ, Doorenbos AZ. Telementoring for improving primary care provider knowledge and competence in managing chronic pain: A randomised controlled trial. Journal of telemedicine and telecare. Jan-Feb 2020;26(1-2):21-27. doi:10.1 177/1357633x18802978
6. Holmes CM, Keyser-Marcus, L., Dave, B., & Mishra, V Project ECHO and opioid education: a systematic review. Current Treatment Options in Psychiatry. 2020:14.
7. Carlin L, Zhao J, Dubin R, Taenzer P, Sidrak H, Furlan A. Project ECHO telementoring intervention for managing chronic pain in primary care: Insights from a qualitative study. Pain Medicine. 2018;19 (6):1140-1146.
8. Johnson KL, Hertz D, Stobbe G, et al. Project Extension for Community Healthcare Outcomes (ECHO) in multiple sclerosis: increasing clinician capacity. International journal of MS care. 2017;19 (6):283-289.
9. Fisher E, Hasselberg M, Conwell Y, et al. Telementoring primary care clinicians to improve geriatric mental health care. Population health management. 2017;20(5):342-347.
10. Gordon MJ. Self-assessment programs and their implications for health professions training. Academic medicine : journal of the Association of American Medical Colleges. Oct 1992;67(10):672-9. doi:10.1097/00001888- 199210000-00012
11. Approved initiatives for Maintenance of Certification for the ABMS (2006).
12. Arora S, Kalishman SG, Thornton KA, et al. Project ECHO: A Telementoring Network Model for Continuing Professional Development. The Journal of continuing education in the health professions. Fall 2017;37(4):239-244. doi:10.1097/ ceh.0000000000000172
13. Komaromy M, Duhigg D, Metcalf A, et al. Project ECHO (Extension for Community Healthcare Outcomes): A new model for educating primary care providers about treatment of substance use disorders. Substance abuse. 2016;3 7(1):20-4. doi:10.1080/08897077.2015.1129388
14. Shelley BM, Katzman JG, Comerci GD, Jr., et al. ECHO Pain Curriculum: Balancing Mandated Continuing Education With the Needs of Rural Health Care Practitioners. The Journal of continuing education in the health professions. Summer 2017;37(3):190-194. doi:10.1097/ceh.000 0000000000165
15. Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The Impact of Project ECHO on Participant and Patient Outcomes: A Systematic Review. Academic medicine : journal of the Association of American Medical Colleges. Oct 2016;91(10) :1439-1461. doi:10.1097/acm.0000000000001328
16. Arora S, Geppert CM, Kalishman S, et al. Academic health center management of chronic diseases through knowledge networks: Project ECHO. Academic medicine : journal of the Association of American Medical Colleges. Feb 2007;82 (2):154-60. doi:10.1097/ACM.0b013e31802d8f68
17. A. B. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: PrenticeHall; 1986.
18. Mind in Society: The Development of Higher Psychological Processes. MA: Harvard University Press; 1978.
19. Situated Learning: Legitimate Peripheral Participation. . Cambridge university press; 1991.
20. Control. BASETEo. W.H. Freeman and Co; 1997.
21. Coach DSaDBM. Increasing Student Mathematics Self-Efficacy Through Teacher Training. Journal of Advanced Academics. 2007;18(2):44.
22. Mazurek MO, Brown R, Curran A, Sohl K. ECHO Autism. Clinical pediatrics. Mar 2017;56(3):247-256. doi:10.1177/0009922816648288
23. Arora S, Kalishman S, Thornton K, et al. Expanding access to hepatitis C virus treatment--Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology (Baltimore, Md). Sep 2010;52(3) :1124-33. doi:10.1002/hep.23802
24. Masi C, Hamlish T, Davis A, et al. Using an established telehealth model to train urban primary care providers on hypertension management. Journal of clinical hypertension (Greenwich, Conn). Jan 2012;14(1):45-50. doi:10.1111/j.1751-7176.20 11.00559.x
25. Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. Journal of advanced nursing. 2006;53(2):205-212.
26. Mengual-Andrés S, Roig-Vila R, Mira JB. Delphi study for the design and validation of a questionnaire about digital competences in higher education. International Journal of Educational Technology in Higher Education. 2016;13:1-11.
27. Blasco Mira JE, López Padrón A, Mengual Andrés S. Validación mediante el metodo Delphi de un cuestionario para conocer las experiencias e interés hacia las actividades acuáticas con especial atención al Winsurf. 2010;
28. Campbell DT, & Fiske, D. W. Convergent and discriminant validation by the multitrait-multimethod matrix.Psychological Bulletin. 1959;5 6(2):25. doi:https://doi.org/10.1037/h0046016
29. Landeta O, Calvete E. Adaptación y validación de la escala multidimensional de apoyo social percibido. Ansiedad estrés. 2002:173-182.
30. Lawshe CH. A QUANTITATIVE APPROACH TO CONTENT VALIDITY. Personnel Psychology. 1975; 28:563-575.
31. Arora S, Thornton K, Bradford A. Access to care: Management of hepatitis C viral infection in remote locations†. Clinical Liver Disease. 2012; 1(3):87-90. doi:10.1002/cld.46
32. Arora S, Brakey HR, Jones JL, Hood N, Fuentes JE, Cirolia L. Project ECHO for Cancer Care: a Scoping Review of Provider Outcome Evaluations. Journal of Cancer Education. 2023/10/01 2023;38 (5):1509-1521. doi:10.1007/s13187-023-02292-x
33. Tian J, Atkinson NL, Portnoy B, Lowitt NR. The Development of a Theory-Based Instrument to Evaluate the Effectiveness of Continuing Medical Education. Academic Medicine. 2010;85(9):1518-1525. doi:10.1097/ACM.0b013e3181eac3fb
34. Ramos MM, Sebastian RA, Murphy M, Oreskovich K, Condon TP. Adolescent substance use: Assessing the knowledge, attitudes, and practices of a school-based health center workforce. Substance abuse. Apr-Jun 2017;38 (2):230-236. doi:10.1080/08897077.2017.1287149
35. Löfmark A, Mårtensson G. Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts. Nurse Education Today. 2017/03/01/ 2017;50:82-86. doi:https://doi.org/10.1016/j.nedt.2016.12.009
36. Hong QN, Pluye P, Fàbregues S, et al. Improving the content validity of the mixed methods appraisal tool: a modified e-Delphi study. Journal of Clinical Epidemiology. 2019/07/01/ 2019;111:49-59.e1. doi:https://doi.org/10.1016/j.jclinepi.2019.03.008
37. Gandek B, Ware J, Aaronson N, et al. Tests of data quality, scaling assumptions, and reliability of the SF-36 in eleven countries: results from the IQOLA Project. International Quality of Life Assessment. Journal of clinical epidemiology. 1998;51 11:1149-58.
38. Ware J, Gandek B. Methods for testing data quality, scaling assumptions, and reliability: the IQOLA Project approach. International Quality of Life Assessment. Journal of clinical epidemiology. 1998;51 11:945-52.
39. McHorney CA, Ware JE, Jr., Raczek AE. The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical care. Mar 1993;31(3):247-63. doi:10.1097 /00005650-199303000-00006
40. Sockalingam S, Rajaratnam T, Gambin A, Soklaridis S, Serhal E, Crawford A. Interprofessional continuing professional development programs can foster lifelong learning in healthcare professionals: experiences from the Project ECHO model. BMC Medical Education. 2022/06/06 2022; 22(1):432. doi:10.1186/s12909-022-03500-w
41. Hojat M, Veloski JJ, Gonnella JS. Measurement and Correlates of Physicians’ Lifelong Learning. Academic Medicine. 2009;84(8):1066-1074. doi:10 .1097/ACM.0b013e3181acf25f
42. Ware J, Brook R, Davies A, et al. Conceptualization and Measurement of Health for Adults in the Health Insurance Study: Vol. I, Model of Health and Methodology. 1979:
43. Humphrey-Murto S, Varpio L, Wood TJ, et al. The Use of the Delphi and Other Consensus Group Methods in Medical Education Research: A Review. Academic Medicine. 2017;92(10):1491-1498. doi:1 0.1097/acm.0000000000001812
44. Nguyen Thanh Huong LNK. Designing and validation of a living standards questionnaire in Vietnam's elderly community. Medical Practice Journal. 2010;