A Measure for Evaluating Basic Psychological Interviewing Skills

Ralph E. Cash, Ph.D., ABPP ^1; Sarah A. Valley-Gray, Psy.D., ABPP ^1; Shannon Worton, Psy.D. ^1; Alyssa Newman, Psy.D. ²

1. Faculty at the College of Psychology at Nova Southeastern University in Ft. Lauderdale, Florida.
2.  University of Nevada in Las Vegas, Nevada.

OPEN ACCESS

PUBLISHED: 30 November 2024

CITATION: Cash, RE., Valley-Gray, SA., et al., 2024. A Measure for Evaluating Basic Psychological Interviewing Skills. Medical Research Archives, [online] 12(11). https://doi.org/10.18103/mra.v12i11.5943

COPYRIGHT: © 2025 European Society of Medicine. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

DOI: https://doi.org/10.18103/mra.v12i11.5943

ISSN 2375-1924

ABSTRACT

Traditionally, graduate programs in psychology have relied upon applied practicum experiences and supervisor evaluations to establish a student’s clinical competence. However, guidelines established by the American Psychological Association (APA) Standards of Accreditation (SoA) for Health Service Psychology require that programs accredited by the APA assess every graduate student in vivo or via recording each time an evaluation is conducted (usually each grading period). There are currently no published, standardized measures in applied psychology to gather most of these data in a reliable and valid fashion.

The purpose of the current study was to develop and to take steps toward standardization of an empirically validated measure for use within graduate training in psychology. The Skills in Psychological Interviewing: Clinical Evaluation Scales (SPICES) was developed in response to the need for standardized methods of evaluating psychology students’ skills across the nine profession-wide competency areas, as identified by the SoA.

Evaluation of the SPICES measure provided empirical support for its reliability, validity, and usefulness in graduate training, lending support for use of the measure as one method to evaluate student performance across the APA Competency Benchmarks in Professional Psychology. Importantly, the measure is congruent with the SoA’s profession-wide competency areas and provides programs with a framework for helping to evaluate each student’s readiness for practicum at various levels as well as internship.

Keywords

psychological interviewing, clinical competence, SPICES, graduate training, competency evaluation

Introduction

Traditionally, graduate programs in psychology have relied upon practicum experiences and supervisor evaluations to establish whether a student has attained clinical competency. However, there is an increasing emphasis in the American Psychological Association’s Standards of Accreditation for Health Service Psychology for graduate programs accredited by the APA to assess every graduate student in vivo or via recording each time an evaluation is conducted. Moreover, there are very few opportunities to observe a student’s developing competencies in a standardized way within formal coursework, particularly among first-year students, who are typically not yet seeing clients.

COMPETENCIES IN PROFESSIONAL PSYCHOLOGY

The Competency Benchmarks in Professional Psychology were developed by the APA as a framework for assessing the competencies required to practice health service psychology as broadly defined. This framework focuses on readiness for each transition which demands increasing autonomy, progressing from practicum to internship and entry into practice. The framework covers 16 domains: 1) professional values and attitudes, 2) individual and cultural diversity, 3) ethical legal standards and policy, 4) reflective practice/self-assessment/self-care, 5) relationships, 6) scientific knowledge and methods, 7) research/evaluation, 8) evidence-based practice, 9) assessment, 10) intervention, 11) consultation, 12) teaching, 13) supervision, 14) interdisciplinary systems, 15) management/administration, and 16) advocacy. The framework has a corresponding rating system, with acknowledgements by APA that it may be adapted depending on the specific needs of the program.

According to the SoAs, which went into effect in 2017, all doctoral students being trained as health service providers in clinical, counseling, and/or school psychology are expected to develop competence in nine Profession-Wide Competency (PWC) areas: 1) research, 2) ethical and legal standards, 3) individual and cultural diversity, 4) professional values, attitudes, and behaviors, 5) communication and interpersonal skills, 6) assessment, 7) intervention, 8) supervision, and 9) consultation and interprofessional/interdisciplinary skills. As a result, training programs are tasked with developing mechanisms by which all students are evaluated within each of these domains and are provided with regular feedback regarding their level of attainment of the competencies.

EVALUATION OF PSYCHOLOGY TRAINEES

A “toolkit” was established to assist in the evaluation of students’ competencies, corresponding with each of the 16 Competency Benchmarks for Professional Psychology (Kaslow et al., 2009). Two such tools which have proven useful in the evaluation of student competencies include Competency Evaluation Rating Forms (CERFs) and Live or Recorded Performance Ratings. Competency Evaluation Rating Forms are written documents that utilize behavioral indicators for selected foundational and functional competencies. They involve evaluating individuals on each behavioral indicator according to a numerical system that corresponds with levels of attained competence. These rating forms can be either formative or summative and were deemed either a very useful method or a useful method of evaluating clinician competence for all elements under each of the competency benchmarks with the exception of two, i.e., participation in enhancement of interdisciplinary outcomes under the Interdisciplinary Systems domain and skills under the Intervention domain. The CERFs demonstrate high face, content, construct, and discriminant validity and have moderate to good reliability with trained raters. Moreover, CERFs can be used within the context of live/recorded evaluations of student competence and have demonstrated good interrater reliability and content validity in medical settings. In addition to routine and systematic evaluation of student competence, the assessment process provides an opportunity to give specific feedback to trainees, which can contribute to strengthening their skills within each of the competency benchmark areas.

There is currently no standard assessment practice utilized to evaluate basic interviewing skills among graduate students. While there is a requirement to assess graduate students developmentally and sequentially, programs have the freedom to develop and to utilize informal assessment measures as they deem appropriate. There is a need for reliable and valid measures to evaluate the readiness of students to begin practicum, as well as to assess their skill-level development throughout their graduate training. These measures should, insofar as possible, take diversity issues into account.

CURRENT STUDY

The purpose of the current study was to develop an empirically validated measure to evaluate and to provide helpful feedback to graduate students in psychology regarding their basic interviewing skills. This instrument was designed to recognize areas of strength as well as to point out needed improvement which could be targeted for intervention/remediation within the context of supervision. The measure was developed to be related to the nine profession-wide competency areas, as identified by the SoA, to help provide graduate education programs with a method of evaluating students’ readiness for practicum, internship, and independent practice. The measure targets the specific skills required for interviewing and rapport building, as these lay the foundation for further clinical work.

Method

DEVELOPMENT OF THE MEASURE

The development of the Skills in Psychological Interviewing: Clinical Evaluation Scales (SPICES) occurred in four distinct phases.

Phase 1: Development of the instrument

The SPICES instrument was developed within Nova Southeastern University’s (NSU) College of Psychology. Using the APA Competency Benchmarks in Professional Psychology (2012), faculty, post-doctoral residents, and graduate students generated a list of essential skills required for effective interviewing. These skills included areas such as the development of rapport, the effective use of time within a session, basic counseling functions, discussion/exploration of critical issues, and overall professionalism. After the specific skills were identified, items were clustered into categories based on the overarching competency areas. Behavioral anchors were then created to ensure that each item was measurable through direct observation of relatively brief (15 minute or more) interviews.

Phase 2: Evaluation of the measure

Each item was reviewed for relevance and necessity by experienced faculty members within the school psychology and clinical psychology programs at NSU, all of whom had expertise in teaching/supervising clinical interviewing skills. Faculty reviewers were deemed experts in the area of clinical interviewing based on 10 or more years of experience within the field and whether they had taught at least one course in basic clinical interviewing and/or supervised practicum students who were interviewing clients. Faculty were asked to provide feedback regarding a) the effectiveness of each item (i.e., did the item successfully assess the targeted skill), b) the relevance of each item (i.e., was the item necessary/unnecessary), and c) any additional comments/feedback.

Phase 3: Revision of the measure

Following the feedback phase, each item within the scale was reviewed by the research team for relevance and validity. Revisions based on the experts’ feedback occurred on several levels: a) items deemed unnecessary, irrelevant, or redundant by faculty experts were removed; b) items that did not correspond with observable, measurable skills were removed; c) wording of items was revised to achieve greater clarity and specificity, including the wording of behavioral anchors; and d) select items were renamed/relocated to reflect more accurately the domain/competency being measured.

Phase 4: Piloting the measure

Following the development of SPICES, the measure was piloted for use within the Doctor of Philosophy and the Doctor of Psychology (Ph.D. and Psy.D.) in Clinical Psychology programs at NSU. All students within the program (approximately 80-100 admitted annually) enroll in a pre-practicum course during the winter semester of their first year of study. The pre-practicum course provides the students with the opportunity to learn and to practice a variety of interviewing and clinical skills necessary for effective functioning in an assessment/therapeutic context. Students are enrolled concurrently in a course which covers the theoretical and practical aspects of clinical interviewing. Emphasis in the interviewing course is placed on developing communication skills, building rapport, utilizing evaluation strategies, collecting diagnostic data, addressing diversity issues, and examining mental status. The goal of pre-practicum is to provide students with the opportunity to learn and to practice a variety of interviewing and clinical skills necessary for effective functioning as beginning clinicians. During the semester-long pre-practicum experience, students engage with simulated patients (SPs), i.e., actors who are trained to portray symptoms of a psychological/psychiatric diagnosis accurately and consistently. Testing with SPs occurs at the onset and at the end of the course to assess the development of students’ competencies in basic interviewing. During the pre- and the post-test, each student engages in a 15-minute SP encounter, during which time they are tasked with conducting an abbreviated clinical interview with the SP. Each SP is provided with training in the portrayal of a specific diagnostic entity, including the specifics of the case as well as the diagnostic criteria. Students are unaware of the presenting concern(s), as this experience is meant to simulate an actual case that students may encounter during their first practicum placement.

Student interactions with the SPs are recorded to allow for evaluation by multiple raters at a later date. Each recording, both pre- and post-test (i.e., at the beginning and end of the pre-practicum course), was reviewed by two separate raters who were assigned at random from among 10 pre-doctoral interns and post-doctoral residents in psychology who were trained for approximately two hours in a group on the use of SPICES. Using the SPICES measure, raters assessed each student conducting the SP encounter. Raters also provided the researchers with qualitative feedback regarding their experience with SPICES (e.g., ease of use, clarity of items). A minimum level of achievement (MLA) of 80% on the SPICES measure was established to be consistent with other MLAs in both programs. Students who earned scores of less than 80% at post-test were identified and provided with remediation, after which they were re-evaluated to ensure that they evidenced the requisite skills to begin practicum.

Item-total correlation analyses utilizing Cronbach’s alpha were conducted to examine the internal consistency of the SPICES measure. Pearson product-moment correlations were calculated to determine reliability between the scores of the two ratings of each video. Finally, the means and standard deviations of the pretest and post-test data were compared using correlated t-tests to help determine validity.

Results

Using the raters’ assessments of each student, SPICES was evaluated for its validity and reliability. The reader is referred to Appendix A for a complete version of the SPICES measure. Appendix B contains an abbreviated rating form used to gather data.

RELIABILITY

Item-total alpha levels were high (Cronbach’s alpha=.843), suggesting strong internal consistency. Moreover, each of the domains reflected some degree of internal consistency (highest: Cronbach’s alpha=.648, Nonverbal Communication; lowest: Cronbach’s alpha=.119, Diversity). Variability in internal consistency for the domains reflects not only differences in the number of items within each domain but also the possibility that some reorganization of the domain structure may be warranted. See Table 1 for an overview of the item-total correlation data.

Among the items that contributed least to the measure’s internal consistency were questions regarding appearance (Items 7 and 8) and an item regarding diversity (Item 14). While appearance (i.e., mode of dress, hygiene, and grooming) is not the best indicator of other aspects of interviewing effectiveness, it does matter. However, the main reason for the limited contribution of these items as well as the diversity item to internal consistency was their restricted range; these ratings did not vary much in this setting. As a result, the scoring of both items was changed to a maximum of two points each, a change which, conveniently, resulted in a maximum possible score of 100 points.

Among the items that were most impactful on internal consistency were Item 2 (Definition and Limits of Confidentiality), Item 9 (Non-Judgmental Attitude), Item 16 (Response to Client’s Expressions of Concerns), and Item 19 (Management of Ambiguity and Uncertainty). The item that had the highest overall impact was Item 23 (Nonverbal Communication). These items were also judged to be critical to the effectiveness of interviewing.

Additionally, Cronbach’s alpha coefficients were calculated for the scale with each item deleted. It is clear that each item contributes positively to the total score and that removal of any of the items would not significantly improve the overall internal consistency.

Table 1. Item-total statistics.

Scale Mean if Item Deleted	Scale Variance if Item Deleted	Corrected Item-Total Correlation	Cronbach’s Alpha if Item Deleted
Informed Consent	79.7500	63.358	.375	.841
Definition and Limits of Confidentiality	79.3889	63.632	.519	.832
Suicide Assessment	79.7840	65.328	.252	.848
Threat Assessment	81.0833	65.470	.302	.843
Abuse Assessment	81.2377	67.104	.267	.842
Personal Boundaries	78.6420	69.630	.244	.842
Personal Hygiene	78.6451	69.809	.261	.842
Attire	78.6667	70.037	.169	.843
Non-Judgmental Attitude	78.9568	65.769	.594	.833
Appreciation for Client’s Life Circumstances	79.0123	65.480	.591	.832
Compassion for the Client	78.9321	66.224	.510	.835
Structure of the Interview	79.3951	64.946	.562	.832
Time Management	79.1420	67.881	.267	.842
Diversity	80.2500	69.872	.119	.845
Response to Client’s Feelings	79.1080	65.589	.547	.833
Responses to Client’s Expression of Concerns	78.9877	65.684	.508	.834
Indirect Messages/Communication	79.8056	64.355	.335	.842
Management of Interpersonal Conflict	78.7222	68.294	.343	.839
Management of Ambiguity and Uncertainty	79.3395	63.352	.617	.829
Language in Professional Communication	78.7562	68.414	.354	.839
Tone of Speech	78.9444	67.050	.458	.836
Communication of Ideas and Information	78.9074	67.440	.399	.838
Nonverbal Communication	79.1728	63.716	.648	.829
Open-Ended Questioning	79.5000	65.080	.522	.833
Paraphrasing or Summarizing	79.2654	64.920	.564	.832
Closure of the Session	79.5679	67.980	.167	.848

INTERRATER RELIABILITY

Pre-test interrater reliability was moderate (r=0.448; p<0.0001). Post-test interrater reliability, however, was relatively weak (r=0.3214; p<0.004). It is noteworthy that 10 raters were used, with each rating 20 interviews in a short period of time in order to submit grades for the course, which was a daunting task. In addition, cases were assigned at random to the raters, so they were not consistently paired, and they had varying levels of experience with SPICES. Moreover, training on the scoring of SPICES occurred only when the raters were beginning their participation in the project, and they did not receive additional training prior to the post-test. Interrater reliability results indicate that additional training should be provided for each rater, before both the pre-test and the post-test. Because of the concerns regarding interrater reliability, the anchors for a number of items were modified slightly based on rater feedback, and the averages of the two raters’ scores were used as pre-test and post-test scores.

VALIDITY

As previously implied, because of the way in which it was constructed, SPICES has both face and content validity. In addition, Table 1 represents Rater 1’s and Rater 2’s evaluations of students using the SPICES assessment tool within the pre-practicum course at both pre- and post-test. Students scored an average of 12-points higher on SPICES (based on averages of the two ratings for each student) at post-test when compared to pre-test, indicating that the group of students being measured obtained higher scores on SPICES, presumably as a result of their coursework and training in the area of clinical interviewing. This difference was significant at the p<.0001 level. This finding lends support to SPICES validity, as this result is what is expected if SPICES is a valid measure (i.e., the measure demonstrates that students improve significantly, when, in fact, they should as a result of their course work and practice in basic interviewing). Additionally, as expected, the standard deviations for both ratings decreased at post-test. The decrease in standard deviations is likely reflective of the students’ being more consistent in their skill development following completion of their course in interviewing and practice within the pre-practicum class.

Table 2. Pre and post-test evaluation scores using the SPICES assessment tool.

Rater	Pre/Post-test	N	Mean	SD
Rater 1	Pretest	84	77.05	6.75
Rater 1	Posttest	84	89.49	5.09
Rater 2	Pretest	81	76.78	6.50
Rater 2	Posttest	84	88.55	4.81

Table 3. Two-tailed test of significance.

Rater	Pre/Posttest	df	P<
Rater 1	Pretest	83	.0001
Rater 1	Posttest	83	.0001
Rater 2	Pretest	80	.0001
Rater 2	Posttest	83	.0001

Discussion

The Skills in Psychological Interviewing: Clinical Evaluation Scales (SPICES) was developed in response to the call for standardized methods of evaluating psychology students’ competence across the nine APA profession-wide competency areas. An evaluation of the measure provided empirical support for its reliability, validity, and usefulness across graduate training, lending credence to the use of the measure to evaluate student performance in clinical interviewing, which includes aspects of several PWCs.

PSYCHOMETRIC PROPERTIES OF SPICES

Evaluation of SPICES yielded support for its validity and reliability. However, there are notable limitations with regard to interrater reliability. Given the moderate to weak interrater reliability, especially at post-test, anchors for several of the items were modified based on rater feedback. Moreover, it is recommended that with future use, raters should have multiple sessions of prior training and experience with the measure. This should include receiving extensive, specialized training on scoring SPICES. Specifically, raters should observe videos of psychotherapy sessions/interviews and independently rate the clinicians using the SPICES measure. Next, they should discuss the nature of their ratings with a trained rater to understand the nuances of the measure. Additionally, behavioral anchors for each item should be reevaluated with the objective of enhancing clarity and ease of use. Finally, raters may benefit from a booster session of training if there is time between multiple evaluations.

Analyses demonstrated high overall internal consistency of the SPICES measure, though variability existed regarding the impact of each item on the measure’s overall reliability. Specifically, regarding appearance, interactions with SPs were scheduled in advance, allowing adequate time for students to present themselves in a professional manner with respect to attire and grooming. As a result, there was low variability among students’ scores on items reflecting appearance (Items 7 and 8), thereby helping to explain the impact these items had on the overall internal consistency of the measure. Additionally, SP encounters typically lasted 15 minutes, not always allowing for adequate time to address all essential elements of an initial biopsychosocial interview. Specifically, Item 14 (evaluating students’ explorations of relevant diversity variables) evidenced a low correlation with total scores. Raters typically scored students highly on this item, likely in response to the time limitation and the presumption that the student would address this variable if given more time. Moreover, while the SPs were diverse with respect to race, ethnicity, religion, and background, the cases they portrayed did not make it obvious that these characteristics should be addressed fully, as doing so could have created significant variability in cases, thus complicating standardization of the presentations. As a result, there was limited opportunity for students to explore diversity variables based on observed characteristics or factors which might have emerged during the interviews.

When examining the items most strongly correlated with the measure’s internal consistency, all, with the exception of Item 2 (Definition and Limits of Confidentiality), reflected students’ demeanor and approach to the SPs. Given the importance of developing a therapeutic relationship, which is one of the strongest predictors of a client’s change and growth, these findings support SPICES as a critical measure in the evaluation of students’ interpersonal skills when approaching work with actual clients.

INTEGRATION OF APA SoA

A primary benefit of SPICES is its use in evaluating graduate students’ clinical skills. Given the requirement of the SoA to evaluate each psychology trainee in vivo or via video- or audiotaping as a source of information to be used for each evaluation period, SPICES affords the unique opportunity to establish observed competence with regard to six of the nine SoA profession wide competencies, i.e., ethical and legal standards; individual and cultural diversity; professional values, attitudes, and behaviors; communication and interpersonal skills; assessment; and intervention (See Table 4 for SPICES/SoA crosswalk). SPICES may be used as an evaluation tool prior to entry into an initial practicum placement and may continue to be utilized throughout graduate training as a measure of clinical growth and preparedness.

Although three SoA profession-wide competencies are not directly assessed in SPICES, the measure has utility within each. A discussion of these three areas, i.e., research, supervision, and consultation and interprofessional/interdisciplinary skills is provided below.

Profession-wide competency: research

SPICES is related to the domain of research, as one of the main uses of SPICES is within the context of program evaluation. Training programs should continuously evaluate the efficacy of their academic and clinical training approaches. This measure may be utilized as a program evaluation tool each semester or annually to monitor student progress and to demonstrate the efficacy of each student in meeting the competencies within the curriculum. It may also be used following specific training efforts to demonstrate the acquisition and application of knowledge.

Profession-wide competency: supervision

Consensus within the field has pushed for a framework in approaching supervision to ensure supervisors’ competence, thereby supporting supervisors in the adequate and accurate evaluation of their trainees. With the implementation of the SoAs, graduate programs in professional psychology are likely to include a course that incorporates the models of supervision within the curriculum. In training students on approaches to supervision, SPICES may be used as a foundational tool to enhance the assessment of supervisees’ interviewing skills. Training within this area may involve role-play, during which students take the role of supervisor and develop their skills within this domain. By providing structure in the form of a measure like SPICES, students may begin to familiarize themselves with the clinical skills necessary to evaluate supervisees. SPICES can provide them with a framework for doing so. Moreover, the areas of strength as well as the areas for growth identified using the SPICES measure can be discussed in the context of supervision sessions.

Profession-wide competency: Consultation and interprofessional/interdisciplinary skills

As a significant source of collaboration occurs within the context of medical settings, psychologists and graduate students are frequently tasked with providing consulting services to medical students, residents, and fellows, particularly with respect to bedside manner. Given the usefulness of the SPICES measure in evaluating students’ demeanors and interpersonal skills when relating to clients, this tool may prove particularly useful when evaluating and providing feedback to medical trainees, as it involves a structure to follow and items which can serve as targeted, specific feedback.

While the current research has focused on the use of the SPICES measure within the context of initial evaluation sessions, future research should examine the usefulness of the tool within a consultation dyad or within an interprofessional/interdisciplinary arena.

Table 4. SPICES/SoA Profession-Wide Competencies crosswalk

Profession-Wide Competency Domain	SPICES Item
Research	N/A
Ethical and Legal Standards	1. Informed Consent	2. Definition and Limits of Confidentiality
Individual and Cultural Diversity	14. Diversity (e.g., age, gender, race, religion, culture, ethnicity, sexual orientation, language)
Professional Values, Attitudes, and Behaviors	6. Personal Boundaries	7. Personal Hygiene	8. Attire
Communication and Interpersonal Skills	9. Non-Judgmental Attitude	10. Appreciation for Client’s Life Circumstances	11. Compassion for the Client	15. Response to Client’s Feelings	16. Response to Client’s Expressions of Concerns	17. Indirect Messages/Communications	18. Management of Interpersonal Conflict	19. Management of Ambiguity and Uncertainty	20. Language in Professional Communication	21. Tone of Speech	22. Communication of Ideas and Information (i.e., taking into account client’s educational and developmental level)	23. Nonverbal Communication (e.g., eye contact, posture, attention to client)
Assessment	3. Suicide Assessment	4. Threat Assessment	5. Abuse Assessment (i.e., physical, verbal, emotional, and sexual)	12. Structure of the Interview	13. Time Management	24. Open-Ended Questioning (when appropriate to question)	25. Paraphrasing or Summarizing	26. Closure of the Session
Intervention	*Items subsumed under Communication and Interpersonal Skills and Assessment can also be included in the Intervention domain.
Supervision	N/A
Consultation and Interprofessional/Interdisciplinary Skills	N/A

Future Directions for Research

Future research efforts should focus on developing additional versions of SPICES, including a measure specific to working with children and adolescents, and a version focused on crisis assessment and intervention. Additionally, the psychometric properties of the existing measure will be refined, with items loading less to be reduced in weight for scoring purposes. Finally, further research is needed to determine the effects of additional rater experience and training on interrater reliability.

Conclusion

The Skills in Psychological Interviewing: Clinical Evaluation Scales, an instrument which was designed to assess and to monitor competency in basic interviewing skills among first year graduate students in health service psychology, has evidence of reliability; validity; and, as a result, utility. In addition, it shows promise in helping to assess at least six of the nine profession-wide competencies required by the American Psychological Association’s Standards on Accreditation. Although additional work and further research is required to improve interrater reliability, current results suggest that the results may be worth the effort.

References

1. American Psychological Association Commission on Accreditation (APA CoA) (2015). Standards of accreditation for health service psychology. Accessed 11.01.2024. http://www.apa.org/ed/accreditation/about/policies/standards-of-accreditation
2. Kaslow NJ, Grus CL, Campbell LF, Fouad NA, Hatcher RL, Rodolfa ER. Competency Assessment Toolkit for professional psychology. Training and Education in Professional Psychology. 2009;3(4, Suppl):S27-S45. doi:10.1037/a0015833
3. American Psychological Association. Compentency benchmarks in professional psychology. (2009). Accessed 11.01.24. https://www.apa.org/ed/graduate/revised-competency-benchmarks.doc
4. American Psychological Association. A practical guidebook for the competency Benchmarks. Updated July, 2012. Accessed October 31,2024, https://www.apa.org/ed/graduate/benchmarks-evaluation-system
5. Fouad NA, Grus CL, Hatcher RL, et al. Competency benchmarks: A model for understanding and measuring competence in professional psychology across training levels. Training and Education in Professional Psychology. 2009;3(4, Suppl):S5-S26. doi:10.1037/a0015832
6. Andrews LB, Burruss JW. Core competencies for psychiatric education: Defining, teaching, and assessing resident competence. Core competencies for psychiatric education: Defining, teaching, and assessing resident competence. American Psychiatric Publishing, Inc.; 2004:145-145.
7. Lievens F, Sanchez JI. Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment. Journal of Applied Psychology. 2007;92(3):812-819. doi:10.1037/0021-9010.92.3.812
8. Roberts MC, Borden KA, Christiansen MD, Lopez SJ. Fostering a Culture Shift: Assessment of Competence in the Education and Careers of Professional Psychologists. Professional Psychology: Research and Practice. 2005;36(4):355-361. doi:10.1037/0735-7028.36.4.355
9. Rivas-Vazquez RA, Lengnick T, Rivas‑Vazquez AA. The diagnostic interview: Conceptual and practical considerations. Journal of Health Service Psychology: An Official Journal of the National Register of Health Service Psychologists. 2024;50(2):61-75. doi:10.1007/s42843-024-00104-4
10. Kaslow NJ, Rubin NJ, Bebeau MJ, et al. Guiding principles and recommendations for the assessment of competence. Professional Psychology: Research and Practice. 2007;38(5):441-451. doi:10.1037/0735-7028.38.5.441
11. Falender CA, Cornish JA, Goodyear R, et al. Defining competencies in psychology supervision: a consensus statement. J Clin Psychol. Jul 2004;60(7):771-85. doi:10.1002/jclp.20013
12. Fischer CT. Individualizing psychological assessment: A collaborative and therapeutic approach. Routledge; 2012.
13. Multicultural issues in counseling: New approaches to diversity, 4th ed. Multicultural issues in counseling: New approaches to diversity, 4th ed. American Counseling Association; 2013:xxiv, 327-xxiv, 327.
14. Hays PA. Connecting across cultures: the helper’s toolkit Sage Publications;2013. http://public.eblib.com/choice/publicfullrecord.aspx?p=1207753
15. Ratts MJ, Singh AA, Nassar-McMillan S, Butler SK, McCullough JR. Multicultural and Social Justice Counseling Competencies: Guidelines for the Counseling Profession. Journal of Multicultural Counseling and Development. 2016;44(1):28-48. doi:https://doi.org/10.1002/jmcd.12035
16. Sue S. Cultural competency: From philosophy to research and practice. Journal of Community Psychology. 2006;34(2):237-245. doi:10.1002/jcop.20095
17. Collaborative/therapeutic assessment: A casebook and guide. Collaborative/therapeutic assessment: A casebook and guide. John Wiley & Sons Inc; 2012:xxiv, 440-xxiv, 440.
18. Kottler JA. On Being a Therapist. Wiley; 2010.
19. Yalom ID. The Gift of Therapy: An Open Letter to a New Generation of Therapists and Their Patients.
20. Zalewski B, Guziak M, Walkiewicz M. Developing Simulated and Virtual Patients in Psychological Assessment – Method, Insights and Recommendations. Perspectives on Medical Education. 2023. doi:10.5334/pme.493
21. Cormier LS, Nurius, P. S., Osborn, C. J. Interviewing and change strategies for helpers: Fundamental skills and cognitive-behavioral interventions (8th ed.) 2017.Boston, MA: Cengage.
22. Connelly BS, McAbee ST, Oh I-S, Jung Y, Jung C-W. A multirater perspective on personality and performance: An empirical examination of the trait–reputation–identity model. Journal of Applied Psychology. 2022;107(8):1352-1368. doi:10.1037/apl0000732
23. Allen DN, Becker ML. Clinical interviewing. Handbook of psychological assessment, 4th ed. Elsevier Academic Press; 2019:307-336.
24. Stubbe DE. The Therapeutic Alliance: The Fundamental Element of Psychotherapy. Focus (Am Psychiatr Publ). Oct 2018;16(4):402-403. doi:10.1176/appi.focus.20180022

Appendix

Appendix A

Skills in Psychological Interviewing: Clinical Evaluation Scales (SPICES) Readiness for Practicum-Level, Intake Interview Rating Form

Instructions: When rating an interview, raters should not penalize students for the same construct multiple times for the same error.

Clinical Competency: An individual’s demonstrated ability to understand and to engage in specific tasks in a manner consistent with the expectations for training in a specific profession (Kaslow et al., 2009)

FOUNDATIONAL COMPETENCIES

PROFESSIONALISM

A. Ethical and Legal Standards and Policy: Application of ethical concepts and awareness of legal issues regarding professional activities with individuals, groups, and organizations

Knowledge of Ethical, Legal, and Professional Standards and Guidelines: Demonstrates basic knowledge of the principals of the APA Ethical Principles and Code of Conduct (ethical practice and basic skills in ethical decision making); demonstrates beginning level knowledge of legal and regulatory issues in the practice of psychology that apply to practice while placed at practicum setting

1. Informed Consent* (as demonstrated by referring to an agency-approved consent form, thoroughly explaining to the client [or parent/guardian, if working with a minor], attempting to ensure that client understands, indicating the interviewer’s training status, and obtaining or referring to obtaining the client’s [or the parent’s/guardian’s] signature.)
   1. Does not address verbal or written consent
   2. Addresses informed consent issues (either accurately or inaccurately) but does not obtain (or refer to obtaining) signature
   3. Obtains (or refers to obtaining) signature but explains informed consent issues inaccurately or incompletely
   4. Obtains (or refers to obtaining) signature and explains informed consent issues accurately and completely
2. Limits of Confidentiality**
   1. Fails to address limits of confidentiality
   2. Addresses limits to confidentiality inaccurately or incompletely
   3. Indicates limits to confidentiality but inadequately explains one or both
   4. Indicates limits of confidentiality fully
3. Suicide Assessment
   1. Fails to ask about suicidality or to follow-up if client communicates suicidal ideation
   2. Asks about suicidality inappropriately (e.g., “You’re not suicidal, are you?”)
   3. Asks about suicidality appropriately but fails to evaluate for ideation, intent, plans, and means when necessary
   4. Asks about suicidality appropriately and evaluates for ideation, intent, plans, and means (or client indicates none)
4. Threat Assessment
   1. Fails to ask about homicidality or to follow-up if client communicates homicidal ideation
   2. Asks about homicidality inappropriately (e.g., “You’re not homicidal, are you?”)
   3. Asks about homicidality appropriately but fails to evaluate for ideation, intent, and plans
   4. Asks about homicidality appropriately and evaluates for ideation, intent, and plans (or client indicates none).
5. Abuse Assessment (physical, verbal, emotional, and sexual)
   1. Fails to ask about possible abuse or to follow-up if client communicates abuse issues
   2. Asks about possible abuse inappropriately (e.g., “You don’t abuse anybody, do you?”)
   3. Asks about possible abuse appropriately but fails to evaluate including history, signs of abuse, and current threat level
   4. Asks about possible abuse appropriately and evaluates including history, signs of abuse, and current threat level
6. Personal Boundaries
   1. Conveys non-constructive or non-constructive, excessively intimate information
   2. Conveys constructive but excessively intimate information
   3. Self-discloses constructively in an attempt to normalize or to build rapport, but focus is largely shifted from the client
   4. Self-discloses constructively to normalize client concerns and to build rapport or interview does not require self-disclosure

B. Professional Values and Attitudes: as evidenced in behavior and comportment that reflect the values and attitudes of psychology

Appearance: Understands how to present oneself in a professional manner

1. Personal Hygiene
   1. Disheveled and/or unclean (e.g., unkempt hair or too much makeup/cologne/perfume, etc.) OR mostly clean cut, but may have loud nail polish, shaggy beard, etc.
   2. Clean cut, neat hair OR meticulously groomed and coiffed
2. Attire
   1. Dressed in a provocative or unkempt manner (e.g., low cut, tight, and/or short clothing for females; low slung and/or tight for males) OR dressed too casually or inappropriately (e.g. jeans or shorts and/or t-shirt and/or sandals and/or dirty/stained clothes)
   2. Dressed in appropriate but casual attire OR dressed in professional attire
3. Non-Judgmental Attitude
   1. Is consistently critical of client either verbally, non-verbally, or both
   2. Expresses criticism toward client verbally, non-verbally, or both at times
   3. Rarely displays criticism toward client
   4. Demonstrates consistent acceptance of client
4. Appreciation for Client’s Life Circumstances
   1. Fails to acknowledge or is dismissive of client’s stressors
   2. Minimizes the significance of client’s stressors
   3. Acknowledges client’s stressors but without conveying the significance of the impact
   4. Acknowledges and conveys the importance of client’s life circumstances
5. Compassion for the Client
   1. Fails to demonstrate compassion for the client
   2. Rarely demonstrates compassion for the client
   3. Sometimes demonstrates compassion for the client
   4. Often/always demonstrates compassion for the client
6. Structure of the Interview
   1. Fails to provide any identifiable structure (e.g., allows conversation to wander, no discernible goals for session, minimal conversation, or entirely client-dominated)
   2. Provides some structure, but allows frequent digressions (many questions are followed by tangential client comments without redirection)
   3. Provides structure allowing only occasional digressions (a few questions are followed by tangential client comments without redirection)
   4. Provides consistent, responsive structure so client is redirected to salient issue(s)
7. Time Management (Note: Key points consist of presenting problem(s), support system, family history, substance use/abuse, job/school history, mental health treatment history, medical history, and legal history).
   1. Uses time inefficiently; obtains fewer than one key points per 15-minute interval and/or fails to address critical issue(s)
   2. Obtains one key point per 15-minute interval
   3. Obtains two key points per 15-minute interval
   4. Obtains three or more key points per 15-minute interval

C. Individual and Cultural Diversity: Awareness, sensitivity, and skills in working professionally with diverse individuals, groups and communities who represent various cultural and personal backgrounds and characteristics defined broadly and consistent with APA policy.

Interaction of Self and Others as Shaped by Individual and Cultural Diversity and Context:

Demonstrates knowledge, awareness, and understanding of interactions between self and diverse others.

1. Diversity (e.g., age, gender, race, religion, culture, ethnicity, sexual orientation, language)
   1. Conveys intolerance, either explicitly or implicitly, for salient diversity issues (e.g. makes insensitive/disrespectful comments, nonverbal communication conveys insensitivity/disrespect)
   2. Does not acknowledge salient diversity issues
   3. Promotes some discussion of client’s diversity but does not explore its impact upon presenting problem(s)
   4. Explores client’s diversity and its impact upon presenting problem(s)

Concern for the Welfare of Others: Demonstrates awareness of the need to uphold and to protect the welfare of others and to facilitate client disclosure

Efficiency: Demonstrates organization and effectiveness within the session

RELATIONAL

D. Relationships: Relates effectively and meaningfully with individuals, groups, and/or communities

Interpersonal Relationships: Displays interpersonal skills (e.g., develops rapport through posture, facial expression, and voice tone)

1. Response to Client’s Feelings
   1. Ignores or does not reflect client’s feelings
   2. Reflects client’s feelings inaccurately and responds ineffectively
   3. Reflects client’s feelings accurately but responds ineffectively
   4. Reflects client’s feelings accurately and responds effectively
2. Response to Client’s Expressions of Concerns
   1. Consistently interrupts client while he/she is trying to share information and fails to acknowledge client’s concerns
   2. Does not interrupt client constructively and/or fails to acknowledge client’s concerns
   3. Acknowledges client’s concerns, but interrupts the client frequently
   4. Only interrupts client constructively and acknowledges client’s concerns
3. Indirect Messages/Communications
   1. Only responds to the direct messages communicated and ignores or does not acknowledge incongruent tone, non-verbals, etc.
   2. Acknowledges client’s indirect messages inaccurately and responds ineffectively
   3. Acknowledges client’s indirect messages accurately but treats them as of secondary importance
   4. Acknowledges and responds effectively to both the direct and the indirect communication of the client
4. Management of Interpersonal Conflict
   1. Actively argues and is inappropriately emotional with the client
   2. Does not overtly argue, but is inappropriately emotional (e.g., withdraws or appears hostile) with the client
   3. Does not overtly respond to conflict, but is noticeably negatively affected (e.g, appears anxious or upset)
   4. Manages interpersonal conflict in a mature and professional manner (e.g., using a calm tone and a reflective statement) or no interpersonal conflict observed
5. Management of Ambiguity and Uncertainty
   1. Does not tolerate ambiguity and uncertainty and rushes to problem definition and resolution
   2. Demonstrates difficulty tolerating ambiguity and uncertainty and rushes to problem definition without sufficient data
   3. Tolerates ambiguity and uncertainty but does not clarify problem definition
   4. Tolerates ambiguity and uncertainty and facilitates clear problem description(s)
6. Language in Professional Communication
   1. Uses profanity or inappropriate language (e.g., slang) during session
   2. Uses language more typical of informal social interactions
   3. Uses occasional professional jargon (e.g., use of “technical” terms/acronyms without clarification)
   4. Uses language that is clear, coherent, socially appropriate, and consistent with the client’s cognitive and emotional level
7. Tone of Speech
   1. Uses a tone that is harsh and impairs the development of rapport
   2. Uses a tone that is difficult to understand and may interfere with rapport
   3. Uses a tone that is intelligible but reflects anxiety (e.g., pressured speech) or is inadequately comforting
   4. Uses a tone that sounds comforting and relaxed
8. Communication of Ideas and Information (taking into account client’s educational and developmental level)
   1. Fails to communicate information and ideas
   2. Communicates ideas and information in a confusing or difficult to interpret manner
   3. Communicates information and ideas with some lack of clarity and at times fails to be congruent with client’s educational and developmental level
   4. Communicates information and ideas clearly and consistent with the client’s educational and developmental level
9. Nonverbal Communication (e.g., eye contact, posture, attention to client)
   1. Poor nonverbal communication
   2. Fair nonverbal communication
   3. Good nonverbal communication
   4. Excellent nonverbal communication

FUNCTIONAL COMPETENCIES APPLICATION

E. Assessment: Assessment and diagnosis of problems, capabilities and issues associated with individuals, groups and/or organizations

Skills: Displays basic helping skills

1. Open-Ended Questioning (when appropriate to question)
   1. Utilizes only closed-ended questions
   2. Utilizes primarily closed-ended questions
   3. Utilizes open-ended questions at least half of the time
   4. Primarily utilizes open-ended questions
2. Paraphrasing or Summarizing
   1. Fails to utilize paraphrasing and/or summarizing
   2. Inappropriately or excessively utilizes paraphrasing and/or summarizing
   3. Appropriately but rarely utilizes paraphrasing and/or summarizing
   4. Consistently and appropriately utilizes paraphrasing and/or summarizing
3. Closure of the Session
   1. Ends the session abruptly
   2. Does not end abruptly but fails to summarize or to suggest a plan
   3. Does not end abruptly and either summarizes the session or suggests a plan but not both
   4. Does not end abruptly, summarizes the session, and suggests a plan

Appendix B

Skills in Psychological Interviewing: Clinical Evaluation Scales (SPICES) Feedback Form

Student’s Class ID Number: Rater ID: Time:

Professionalism:

1. Informed Consent
2. Definition and Limits of Confidentiality
3. Suicide Assessment
4. Threat Assessment
5. Abuse Assessment
6. Personal Boundaries
7. Personal Hygiene
8. Attire
9. Non-Judgmental Attitude
10. Appreciation for Client’s Life Circumstances
11. Compassion for the Client
12. Structure of the Interview
13. Time Management
14. Diversity

Relational:

1. Response to Client’s Feelings
2. Response to Client’s Expressions of Concerns
3. Indirect Messages/Communications
4. Management of Interpersonal Conflict
5. Management of Ambiguity and Uncertainty
6. Language in Professional Communication
7. Tone of Speech
8. Communication of Ideas and Information
9. Nonverbal Communication

Application:

1. Open-Ended Questioning
2. Paraphrasing or Summarizing
3. Closure of the Session

Professionalism Score

Relational Score

Application Score

Total Score

Comments: