Sustainability of a Child Mental Health Intervention in Child Welfare Services: Case study
Main Article Content
Abstract
From an implementation science lens, sustaining interventions in large, heterogeneous organizations such as child welfare requires attunement to the inner and outer contexts in which the innovative practice is delivered. This case study describes how one child welfare agency sustained implementation of a modified child mental health intervention since Spring 2019 after formal supports were removed and provides a retrospective look at their experiences. Using qualitative data from interviews with a key informant, this case study employed a priori codes from an existing sustainability framework to organize and understand factors of innovation, adopters, leadership and management, inner context and process, outer context, and outcomes. Findings offer insights for child welfare agencies to consider when sustaining an intervention without research supports, including the intervention’s alignment with the agency’s culture and mission, the agency’s ability to adapt and embed the intervention, and child welfare leadership and staff buy-in to the intervention. Implications for policy, practice, and research are discussed.
Keywords:
child welfare, child mental health, evidence-based interventions, sustainability
Article Details
How to Cite
PISCIOTTA, Caterina et al.
Sustainability of a Child Mental Health Intervention in Child Welfare Services: Case study.
Medical Research Archives, [S.l.], v. 11, n. 10, oct. 2023.
ISSN 2375-1924.
Available at: <https://esmed.org/MRA/mra/article/view/4524>. Date accessed: 15 may 2024.
doi: https://doi.org/10.18103/mra.v11i10.4524.
Section
Case Reports
The Medical Research Archives grants authors the right to publish and reproduce the unrevised contribution in whole or in part at any time and in any form for any scholarly non-commercial purpose with the condition that all publications of the contribution include a full citation to the journal as published by the Medical Research Archives.
References
Dopp AR, Hanson RF, Saunders BE, Dismuke CE, Moreland AD. Community-based implementation of trauma-focused interventions for youth: Economic impact of the learning collaborative model. Psychol Serv. 2017;14(1):57-65. doi:10.1037/ser0000131
2. Saldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J. The cost of implementing new strategies (COINS): A method for mapping implementation resources using the stages of implementation completion. Child Youth Serv Rev. 2014;39:177-182. doi:10.1016/j.childyouth.2013.10.006
3. Hall A, Shoesmith A, Doherty E, et al. Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review. Implement Sci. 2022;17(1):1-28. doi:10.1186/s13012-022-01252-1
4. Lengnick-Hall R, Gerke DR, Proctor EK, et al. Six practical recommendations for improved implementation outcomes reporting. Implement Sci. 2022;17(1):16. doi:10.1186/s13012-021-01183-3
5. Mundey P, Slemaker A, Dopp AR, Beasley LO, Silovsky JF. Sustaining treatment for youth with problematic sexual behavior: Administrator and stakeholder perspectives following implementation. J Behav Health Serv Res. 2021;48(3):410-426. doi:10.1007/s11414-020-09726-0
6. Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88. doi:10.1186/s13012-015-0274-5
7. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39(1):55-76. doi:10.1146/annurev-publhealth-040617-014731
8. Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):110. doi:10.1186/s13012-017-0637-1
9. Rabin BA, Brownson RC. Developing the terminology for dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press; 2018:0. doi:10.1093/acprof:oso/9780199751877.003.0002
10. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):4-23. doi:10.1007/s10488-010-0327-7
11. Chambers D. Building a lasting impact: Implementation science and sustainability. Published 2013. Accessed August 30, 2023. https://cancercontrol.cancer.gov/is/training-events/webinars/building-a-lasting-impact-implementation-science-and-sustainability
12. Moullin JC, Sklar M, Ehrhart MG, Green A, Aarons GA. Provider report of sustainment scale (PRESS): Development and validation of a brief measure of inner context sustainment. Implement Sci. 2021;16(1):86. doi:10.1186/s13012-021-01152-w
13. Moullin JC, Sklar M, Green A, et al. Advancing the pragmatic measurement of sustainment: a narrative review of measures. Implement Sci Commun. 2020;1(1):76. doi:10.1186/s43058-020-00068-8
14. Urquhart R, Kendell C, Cornelissen E, et al. Defining sustainability in practice: views from implementing real-world innovations in health care. BMC Health Serv Res. 2020;20(1):87. doi:10.1186/s12913-020-4933-0
15. Nadalin Penno L, Davies B, Graham ID, et al. Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks. Implement Sci. 2019;14(1):108. doi:10.1186/s13012-019-0952-9
16. McKay MM, Gopalan G, Franco L, et al. A collaboratively designed child mental health service model: Multiple family groups for urban children with conduct difficulties. Res Soc Work Pract. 2011;21(6):664-674. doi:10.1177/1049731511406740
17. Gopalan G, Fuss A, Wisdom JP. Multiple family groups for child behavior difficulties: Retention among child welfare–involved caregivers. Res Soc Work Pract. 2015;25(5):564-577. doi:10.1177/1049731514543526
18. Gopalan G, Lee KA, Pisciotta C, Hooley C, Stephens T, Acri M. Implementing a child mental health intervention in child welfare services: Stakeholder perspectives on feasibility. J Emot Behav Disord. 2023;31(3):204-218. doi:10.1177/10634266221120532
19. Yin RK. Case Study Research: Design and Methods. SAGE; 2009.
20. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. DSM Library. Published 2013. Accessed August 30, 2023. https://dsm.psychiatryonline.org/doi/book/10.1176/appi.books.9780890425596
21. U.S. Department of Health and Human Services. The family first prevention services act. Published 2018. Accessed August 30, 2023. https://www.acf.hhs.gov/cb/policy-guidance/pi-18-07
22. McKay MM, Gonzales J, Quintana E, Kim L, Abdul-Adil J. Multiple family groups: An alternative for reducing disruptive behavioral difficulties of urban children. Res Soc Work Pract. 1999;9(5):593-607. doi:10.1177/104973159900900505
23. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320-347. doi:10.1177/1098214005278752
24. Beidas RS, Stewart RE, Adams DR, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Adm Policy Ment Health Ment Health Serv Res. 2016;43(6):893-908. doi:10.1007/s10488-015-0705-2
25. Fuller T, Braun M, Chiu Y ling. Increasing worker buy-in for child welfare reform: Examining the influence of individual, organizational, and implementation factors. Child Youth Serv Rev. 2018;93:301-306. doi:10.1016/j.childyouth.2018.08.003
26. Beidas R, Kendell P. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clin Psychol Publ Div Clin Psychol Am Psychol Assoc. 2010;17(1). doi:10.1111/j.1468-2850.2009.01187.x
27. Jones AS. Effective implementation strategies of differential response in child welfare: A comparative case analysis. J Public Child Welf. Published online December 8, 2015. Accessed August 30, 2023. https://www.tandfonline.com/doi/abs/10.1080/15548732.2015.1090365
28. Mildon R, Shlonsky A. Bridge over troubled water: using implementation science to facilitate effective services in child welfare. Child Abuse Negl. 2011;35(9):753-756. doi:10.1016/j.chiabu.2011.07.001
29. Aarons GA, Green AE, Trott E, et al. The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Adm Policy Ment Health Ment Health Serv Res. 2016;43(6):991-1008. doi:10.1007/s10488-016-0751-4
30. Winters AM, Antle BF, Collins-Camargo C. Implementing trauma-responsive screening and assessment: Lessons learned from a statewide demonstration study in child welfare. Prof Dev Int J Contin Soc Work Educ. 2021;24(1).
31. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24-34. doi:10.1007/s10488-008-0197-4
32. Hooley C, Winters AM, Pisciotta C, Gopalan G. Caregiver-relevant perspectives from a multi-stakeholder collaborative advisory board on adapting a child mental health intervention to be delivered in child-welfare settings. J Public Child Welf. 2021;15(3):318-340. doi:10.1080/15548732.2020.1724238
33. Kemp SP, Marcenko MO, Hoagwood K, Vesneski W. Engaging parents in child welfare services: bridging family needs and child welfare mandates. Child Welfare. 2009;88(1):101-126.
34. Kerkorian D, McKay M, Bannon WM. Seeking help a second time: parents’/caregivers’ characterizations of previous experiences with mental health services for their children and perceptions of barriers to future use. Am J Orthopsychiatry. 2006;76(2):161-166. doi:10.1037/0002-9432.76.2.161
35. Anderson CM, Robins CS, Greeno CG, Cahalane H, Copeland VC, Andrews RM. Why lower income mothers do not engage with the formal mental health care system: Perceived barriers to care. Qual Health Res. Published online September 1, 2006. doi:10.1177/1049732306289224
36. Scholte EM, Colton M, Casas F, Drakeford M, Roberts S, Williams M. Perceptions of stigma and user involvement in child welfare services. Br J Soc Work. 1999;29(3):373-391. doi:10.1093/oxfordjournals.bjsw.a011463
37. Palinkas LA, Chavarin CV, Rafful CM, et al. Sustainability of evidence-based practices for HIV prevention among female sex workers in Mexico. PLOS ONE. 2015;10(10):e0141508. doi:10.1371/journal.pone.0141508
38. Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child mental health programs. J Child Fam Stud. 2010;19(5):629-645. doi:10.1007/s10826-009-9350-2
39. Wong JJ, Roubinov DS, Gonzales NA, Dumka LE, Millsap RE. Father enrollment and participation in a parenting intervention: Personal and contextual predictors. Fam Process. 2013;52(3):440-454. doi:10.1111/famp.12024
40. Crampton DS, Crea TM, Abramson-Madden A, Usher CL. Challenges of street-level child welfare reform and technology transfer: The case of team decisionmaking. Fam Soc. Published online May 3, 2018. Accessed August 30, 2023. https://journals.sagepub.com/doi/10.1606/1044-3894.3823
41. Willging CE, Gunderson L, Green AE, et al. Perspectives from community-based organizational managers on implementing and sustaining evidence-based interventions in child welfare. Hum Serv Organ Manag Leadersh Gov. 2018;42(4):359-379. doi:10.1080/23303131.2018.1495673
42. Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Adm Policy Ment Health. 2014;41(2):228-236. doi:10.1007/s10488-012-0461-5
43. Gopalan G, Hooley C, Winters A, Stephens T. Perceptions among child welfare staff when modifying a child mental health intervention to be implemented in child welfare services. Am J Community Psychol. 2019;63(3-4):366-377. doi:10.1002/ajcp.12309
44. Stephens T, Gopalan G, Acri MC, Bowman M, McKay MM. Culturally relevant, trauma-informed engagement strategies for child welfare workers: Moving beyond compliance to engagement with families experiencing high levels of exposure to trauma and stress. In: Trauma Responsive Child Welfare Systems. Springer International Publishing/Springer Nature; 2018:67-86. doi:10.1007/978-3-319-64602-2_5
45. Children’s Defense Fund, American Academy of Pediatrics, Child Focus, et al. Implementing the Family First Prevention Services Act: A technical guide for agencies, policymakers, and other stakeholders. Published online n.d. https://www.childrensdefense.org/wp-content/uploads/2020/07/FFPSA-Guide.pdf
46. World Health Organization, PEPFAR, UNAIDS. Task Shifting: Rational Redistribution of Tasks among Health Workforce Teams: Global Recommendations and Guidelines. World Health Organization; 2007. Accessed August 30, 2023. https://apps.who.int/iris/handle/10665/43821
47. Sanders MR, Kirby JN. Consumer engagement and the development, evaluation, and dissemination of evidence-based parenting programs. Behav Ther. 2012;43(2):236-250. doi:10.1016/j.beth.2011.01.005
48. Johnson AM, Moore JE, Chambers DA, Rup J, Dinyarian C, Straus SE. How do researchers conceptualize and plan for the sustainability of their NIH R01 implementation projects? Implement Sci. 2019;14(1):1-9. doi:10.1186/s13012-019-0895-1
49. Casaneuva C, Ringeisen H, Wilson E, Smith K, Dolan M. NSCAW II baseline report: Child well-Being, final report. Published April 29, 2019. Accessed August 30, 2023. https://www.acf.hhs.gov/opre/report/nscaw-ii-baseline-report-child-well-being-final-report
2. Saldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J. The cost of implementing new strategies (COINS): A method for mapping implementation resources using the stages of implementation completion. Child Youth Serv Rev. 2014;39:177-182. doi:10.1016/j.childyouth.2013.10.006
3. Hall A, Shoesmith A, Doherty E, et al. Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review. Implement Sci. 2022;17(1):1-28. doi:10.1186/s13012-022-01252-1
4. Lengnick-Hall R, Gerke DR, Proctor EK, et al. Six practical recommendations for improved implementation outcomes reporting. Implement Sci. 2022;17(1):16. doi:10.1186/s13012-021-01183-3
5. Mundey P, Slemaker A, Dopp AR, Beasley LO, Silovsky JF. Sustaining treatment for youth with problematic sexual behavior: Administrator and stakeholder perspectives following implementation. J Behav Health Serv Res. 2021;48(3):410-426. doi:10.1007/s11414-020-09726-0
6. Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88. doi:10.1186/s13012-015-0274-5
7. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. 2018;39(1):55-76. doi:10.1146/annurev-publhealth-040617-014731
8. Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017;12(1):110. doi:10.1186/s13012-017-0637-1
9. Rabin BA, Brownson RC. Developing the terminology for dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press; 2018:0. doi:10.1093/acprof:oso/9780199751877.003.0002
10. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):4-23. doi:10.1007/s10488-010-0327-7
11. Chambers D. Building a lasting impact: Implementation science and sustainability. Published 2013. Accessed August 30, 2023. https://cancercontrol.cancer.gov/is/training-events/webinars/building-a-lasting-impact-implementation-science-and-sustainability
12. Moullin JC, Sklar M, Ehrhart MG, Green A, Aarons GA. Provider report of sustainment scale (PRESS): Development and validation of a brief measure of inner context sustainment. Implement Sci. 2021;16(1):86. doi:10.1186/s13012-021-01152-w
13. Moullin JC, Sklar M, Green A, et al. Advancing the pragmatic measurement of sustainment: a narrative review of measures. Implement Sci Commun. 2020;1(1):76. doi:10.1186/s43058-020-00068-8
14. Urquhart R, Kendell C, Cornelissen E, et al. Defining sustainability in practice: views from implementing real-world innovations in health care. BMC Health Serv Res. 2020;20(1):87. doi:10.1186/s12913-020-4933-0
15. Nadalin Penno L, Davies B, Graham ID, et al. Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks. Implement Sci. 2019;14(1):108. doi:10.1186/s13012-019-0952-9
16. McKay MM, Gopalan G, Franco L, et al. A collaboratively designed child mental health service model: Multiple family groups for urban children with conduct difficulties. Res Soc Work Pract. 2011;21(6):664-674. doi:10.1177/1049731511406740
17. Gopalan G, Fuss A, Wisdom JP. Multiple family groups for child behavior difficulties: Retention among child welfare–involved caregivers. Res Soc Work Pract. 2015;25(5):564-577. doi:10.1177/1049731514543526
18. Gopalan G, Lee KA, Pisciotta C, Hooley C, Stephens T, Acri M. Implementing a child mental health intervention in child welfare services: Stakeholder perspectives on feasibility. J Emot Behav Disord. 2023;31(3):204-218. doi:10.1177/10634266221120532
19. Yin RK. Case Study Research: Design and Methods. SAGE; 2009.
20. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. DSM Library. Published 2013. Accessed August 30, 2023. https://dsm.psychiatryonline.org/doi/book/10.1176/appi.books.9780890425596
21. U.S. Department of Health and Human Services. The family first prevention services act. Published 2018. Accessed August 30, 2023. https://www.acf.hhs.gov/cb/policy-guidance/pi-18-07
22. McKay MM, Gonzales J, Quintana E, Kim L, Abdul-Adil J. Multiple family groups: An alternative for reducing disruptive behavioral difficulties of urban children. Res Soc Work Pract. 1999;9(5):593-607. doi:10.1177/104973159900900505
23. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320-347. doi:10.1177/1098214005278752
24. Beidas RS, Stewart RE, Adams DR, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Adm Policy Ment Health Ment Health Serv Res. 2016;43(6):893-908. doi:10.1007/s10488-015-0705-2
25. Fuller T, Braun M, Chiu Y ling. Increasing worker buy-in for child welfare reform: Examining the influence of individual, organizational, and implementation factors. Child Youth Serv Rev. 2018;93:301-306. doi:10.1016/j.childyouth.2018.08.003
26. Beidas R, Kendell P. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clin Psychol Publ Div Clin Psychol Am Psychol Assoc. 2010;17(1). doi:10.1111/j.1468-2850.2009.01187.x
27. Jones AS. Effective implementation strategies of differential response in child welfare: A comparative case analysis. J Public Child Welf. Published online December 8, 2015. Accessed August 30, 2023. https://www.tandfonline.com/doi/abs/10.1080/15548732.2015.1090365
28. Mildon R, Shlonsky A. Bridge over troubled water: using implementation science to facilitate effective services in child welfare. Child Abuse Negl. 2011;35(9):753-756. doi:10.1016/j.chiabu.2011.07.001
29. Aarons GA, Green AE, Trott E, et al. The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Adm Policy Ment Health Ment Health Serv Res. 2016;43(6):991-1008. doi:10.1007/s10488-016-0751-4
30. Winters AM, Antle BF, Collins-Camargo C. Implementing trauma-responsive screening and assessment: Lessons learned from a statewide demonstration study in child welfare. Prof Dev Int J Contin Soc Work Educ. 2021;24(1).
31. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24-34. doi:10.1007/s10488-008-0197-4
32. Hooley C, Winters AM, Pisciotta C, Gopalan G. Caregiver-relevant perspectives from a multi-stakeholder collaborative advisory board on adapting a child mental health intervention to be delivered in child-welfare settings. J Public Child Welf. 2021;15(3):318-340. doi:10.1080/15548732.2020.1724238
33. Kemp SP, Marcenko MO, Hoagwood K, Vesneski W. Engaging parents in child welfare services: bridging family needs and child welfare mandates. Child Welfare. 2009;88(1):101-126.
34. Kerkorian D, McKay M, Bannon WM. Seeking help a second time: parents’/caregivers’ characterizations of previous experiences with mental health services for their children and perceptions of barriers to future use. Am J Orthopsychiatry. 2006;76(2):161-166. doi:10.1037/0002-9432.76.2.161
35. Anderson CM, Robins CS, Greeno CG, Cahalane H, Copeland VC, Andrews RM. Why lower income mothers do not engage with the formal mental health care system: Perceived barriers to care. Qual Health Res. Published online September 1, 2006. doi:10.1177/1049732306289224
36. Scholte EM, Colton M, Casas F, Drakeford M, Roberts S, Williams M. Perceptions of stigma and user involvement in child welfare services. Br J Soc Work. 1999;29(3):373-391. doi:10.1093/oxfordjournals.bjsw.a011463
37. Palinkas LA, Chavarin CV, Rafful CM, et al. Sustainability of evidence-based practices for HIV prevention among female sex workers in Mexico. PLOS ONE. 2015;10(10):e0141508. doi:10.1371/journal.pone.0141508
38. Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child mental health programs. J Child Fam Stud. 2010;19(5):629-645. doi:10.1007/s10826-009-9350-2
39. Wong JJ, Roubinov DS, Gonzales NA, Dumka LE, Millsap RE. Father enrollment and participation in a parenting intervention: Personal and contextual predictors. Fam Process. 2013;52(3):440-454. doi:10.1111/famp.12024
40. Crampton DS, Crea TM, Abramson-Madden A, Usher CL. Challenges of street-level child welfare reform and technology transfer: The case of team decisionmaking. Fam Soc. Published online May 3, 2018. Accessed August 30, 2023. https://journals.sagepub.com/doi/10.1606/1044-3894.3823
41. Willging CE, Gunderson L, Green AE, et al. Perspectives from community-based organizational managers on implementing and sustaining evidence-based interventions in child welfare. Hum Serv Organ Manag Leadersh Gov. 2018;42(4):359-379. doi:10.1080/23303131.2018.1495673
42. Bond GR, Drake RE, McHugo GJ, Peterson AE, Jones AM, Williams J. Long-term sustainability of evidence-based practices in community mental health agencies. Adm Policy Ment Health. 2014;41(2):228-236. doi:10.1007/s10488-012-0461-5
43. Gopalan G, Hooley C, Winters A, Stephens T. Perceptions among child welfare staff when modifying a child mental health intervention to be implemented in child welfare services. Am J Community Psychol. 2019;63(3-4):366-377. doi:10.1002/ajcp.12309
44. Stephens T, Gopalan G, Acri MC, Bowman M, McKay MM. Culturally relevant, trauma-informed engagement strategies for child welfare workers: Moving beyond compliance to engagement with families experiencing high levels of exposure to trauma and stress. In: Trauma Responsive Child Welfare Systems. Springer International Publishing/Springer Nature; 2018:67-86. doi:10.1007/978-3-319-64602-2_5
45. Children’s Defense Fund, American Academy of Pediatrics, Child Focus, et al. Implementing the Family First Prevention Services Act: A technical guide for agencies, policymakers, and other stakeholders. Published online n.d. https://www.childrensdefense.org/wp-content/uploads/2020/07/FFPSA-Guide.pdf
46. World Health Organization, PEPFAR, UNAIDS. Task Shifting: Rational Redistribution of Tasks among Health Workforce Teams: Global Recommendations and Guidelines. World Health Organization; 2007. Accessed August 30, 2023. https://apps.who.int/iris/handle/10665/43821
47. Sanders MR, Kirby JN. Consumer engagement and the development, evaluation, and dissemination of evidence-based parenting programs. Behav Ther. 2012;43(2):236-250. doi:10.1016/j.beth.2011.01.005
48. Johnson AM, Moore JE, Chambers DA, Rup J, Dinyarian C, Straus SE. How do researchers conceptualize and plan for the sustainability of their NIH R01 implementation projects? Implement Sci. 2019;14(1):1-9. doi:10.1186/s13012-019-0895-1
49. Casaneuva C, Ringeisen H, Wilson E, Smith K, Dolan M. NSCAW II baseline report: Child well-Being, final report. Published April 29, 2019. Accessed August 30, 2023. https://www.acf.hhs.gov/opre/report/nscaw-ii-baseline-report-child-well-being-final-report