Southern Adventist University Counselor Education Program ...€¦ · Counseling Program...

Southern Adventist University

Counselor Education Program

Counseling Assessment Outcomes Report

School of Education and Psychology

Assessment System Office

September 2019

Counseling Assessment Outcomes Report 2019 SAU Counselor Education

of 29

[Page intentionally blank]

SAU Counselor Education Counseling Assessment Outcomes Report 2019

of 29

Table of Contents

Characteristics of Program Candidates ..................................................................................................... 5

Table 1. Counseling Program Demographics, Both Emphases Combined, Unduplicated at Fall 2019

.............................................................................................................................................................. 5

Key Performance Indicators Assessment .................................................................................................. 5

Table 2. Key Performance Indicator One: Ethical Practice ................................................................. 5

Table 3. Key Performance Indicator Two: Social and Cultural Diversity ............................................ 7

Table 4. Key Performance Indicator Three: Growth and Personality Development ......................... 8

Table 5. Key Performance Indicator Four: Career Development ........................................................ 8

Table 6. Key Performance Indicator Five: Counseling Skills ............................................................... 9

Table 7. Key Performance Indicator Six: Group Work ...................................................................... 10

Table 8. Key Performance Indicator Seven: Assessment Processes ................................................. 11

Table 9. Key Performance Indicator Eight: Research ........................................................................ 12

Table 10. Key Performance Indicator Nine (CMHC): Diagnostic Process ......................................... 12

Table 11. Key Performance Indicator Ten (SC): School Counseling Program ................................... 13

Follow-up Studies of Program Graduates and Employers ...................................................................... 13

Table 12. Alumni Survey Responses Common Core Areas (CCA) 1 – 4 ............................................. 14

Chart 1. Alumni Survey Responses Common Core Areas 1 – 4 (cf. Table 12) .................................. 14

Table 13. Alumni Survey Responses Common Core Areas 5 – 8 ....................................................... 15

Chart 2. Alumni Responses Common Core Areas 5 – 8 (cf. Table 13) .............................................. 15

Table 14. Alumni Responses, Entry Level Specialty Areas Clinical Mental Health Counseling and

School Counseling............................................................................................................................... 16

Chart 3. Alumni Responses, Entry Level Specialty Areas Clinical Mental Health Counseling (cf. Table

14) ....................................................................................................................................................... 16

Chart 4. Alumni Responses, Entry-Level Specialty Area School Counseling (cf. Table 14) .............. 17

Table 15. Alumni Responses, Satisfaction with Program.................................................................. 17

Chart 5. Alumni Satisfaction with Program (cf. Table 15) ................................................................. 18

Studies of Site Supervisors’ Perceptions and Evaluations of the Program ............................................. 19

Table 16. Site Supervisor Survey 2018-19, Common Core Areas 1 – 4 ............................................. 19

Chart 6. Site Supervisor Survey, Common Core Areas 1 – 4 (cf. Table 16) ....................................... 20

Table 17. Site Supervisor Survey 2018-19, Common Core Areas 5 – 8 ............................................. 20

Chart 7. Site Supervisor Survey, Common Core Areas 5 – 8 (cf. Table 17) ....................................... 21

Table 18. Site Supervisor Survey 2018-19, Clinical Mental Health Counseling Indicators............... 21

Chart 8. Site Supervisor Survey, CMHC Indicators (cf. Table 18) ...................................................... 22


of 29

Table 19. Site Supervisor Survey 2018-19, School Counseling Indicators ........................................ 22

Chart 9. Site Supervisor Survey, School Counseling Indicators (cf. Table 19) .................................. 23

Table 20. Site Supervisor Survey 2018-19, Satisfaction with Collaboration and Preparation ......... 23

Chart 10. Site Supervisor Survey, Satisfaction Indicators (cf. Table 20) ........................................... 24

Assessment of Student Learning and Performance on Professional Identity ........................................ 24

Table 21. Counseling Dispositions Assessment of 2018-19 Completers .......................................... 24

Assessment of Student Learning and Performance on Professional Practice ........................................ 25

Summary of Observations, Use of Findings ............................................................................................ 26

Chart 11. Evaluation of Candidate’s Clinical Practice, Clinical Mental Health Counseling ............. 27

Chart 12. Evaluation of Candidate’s Clinical Practice, School Counseling ....................................... 28


of 29

Counseling Assessment Outcomes Report September 2019

Characteristics of Program Candidates

Current students are from eleven states (Alabama, Florida, Georgia, Illinois, North Carolina, North Dakota, New

Jersey, New York, Tennessee, Texas, and Virginia) plus Puerto Rico. Two are international students.

As shown in Table 1, the demographics at fall term 2019 show a relatively high degree of diversity within the

student population.

Table 1. Counseling Program Demographics, Both Emphases Combined, Unduplicated at Fall 2019

Asian Black Hispanic White Mixed Total NR Alien Disability

M 1 _ 1 3 -- 5 2 --

F 3 8 4 10 3 28

Total 4 8 5 13 3 33

Key Performance Indicators Assessment

Key performance indicators are established for each of the eight common core areas that represent foundational

knowledge for all entry-level counselor education graduates.

Student learning in each of these indicators is assessed via multiple measures and over multiple points in time. The

assessment outcomes on the KPIs are reviewed and analyzed by the Assessment System Manager and the

Counseling Assessment Committee annually, with recommendations to the Counseling Programs Council as may

be indicated in the analyses.

These indicators, along with assessment data to date, are shown in Tables 2 through 11.

Table 2. Key Performance Indicator One: Ethical Practice

KPI 1 : Students will understand ethical standards of professional counseling organizations and credentialing bodies. They will demonstrate competency in

application of ethical and legal considerations in professional counseling.

Points of Assessment for KPI 1

Point 1: COUN 537 Exam and Case Presentation COUN 537 F18

Possible range: 0 - 3 Mean: 2.22

Expectation met: 2 N: 20

Lo/Hi: 1/3

SD: 0.620

Table continued next page


of 29

Table 2. Key Performance Indicator One: Ethical Practice (continued)

Points of Assessment for KPI 1 (continued)

Point 2: CMHC E-5, SC E-5 Dispositions Assessment Item 19, Practicum II

'17-18 Completers

'18-19 Completers

Possible range: 0 - 3 Mean Mean

2.00 = Proficient 2.30 2.47

N=10 N=17

Lo/Hi Lo/Hi

2 /3 2/3

SD SD

0.483 0.514

Point 2: CMHC E-4 Item 21; SC E-4: Item SC 34 at Practicum II

'17-18 Completers

'18-19 Completers

Possible range: Levels 1 - 5 Mean Mean

Minimum required at conclusion of Practicum II: Level 3

4.58 4.26

N=10 N=17

Lo/Hi Lo/Hi

3 / 5 4 / 5

SD SD

0.79 0.65

Point 3: CMHC E-5, SC E-5 Dispositions Assessment Item 19, Internship

'17-18 Completers

'18-19 Completers


2.00 = Proficient 2.80 3.00

N=10 N=17

Lo/Hi Lo/Hi

2/ 3 3 /3

SD SD

0.422 0.000

Point 3: CMHC E-4 Item 21; SC E-4: Item SC 34 at Internship

'17-18 Completers

'18-19 Completers


Minimum required at conclusion of Internship: Level 4

4.86 5.00

N=10 N=17

Lo/Hi Lo/Hi

3/ 5 5 / 5

SD SD

0.467 0.000


of 29

Table 3. Key Performance Indicator Two: Social and Cultural Diversity

KPI 2 :Students will demonstrate the knowledge, skills, and self-awareness to effectively counsel clients of various cultures and ethnicities.


Point 1: COUN 561: Identity Paper and Case Study Report

COUN 561, W19


2.00 = Expectation Met N: 11

Lo/Hi 2/3

SD: 0.45

Point 2: Dispositions Assessment, Practicum II: Item 4 in revised (CW1) E-5; item 4 in original E-5

'17-18 Completers

'18-19 Completers


2.00 = Proficient 2.80 2.82

N=10 N=17

Lo/Hi Lo/Hi

2 /3 2/3

SD SD

0.422 0.393

Point 2: Diversity Item (Added) CMHC E-4: # 9 in CW; SC E-4 #27 in CW. COUN 580 & 585

New assessment indicator, to be

collected F19 & W20

Point 3: Dispositions Assessment, Internship : Item 4 in revised E-5; item 4 in original E-5

'17-18 Completers

'18-19 Completers

Mean Mean

Possible range: 0 - 3 3.00 2.94

2.00 = Proficient N=10 N=17

Lo/Hi Lo/Hi

3/ 3 2 /3

SD SD

0.00 0.243

Point 3: E-4 Diversity Item (Added) CMHC E-4: # 9 in CW; SC E-4 #27 in CW.

COUN 581 & 586

New assessment indicator, to be

collected F19 & W20

1 CW is code for Chalk & Wire, assessment platform containing revised E-4 and E-5. The revised E-4 contains several indicators added for KPI assessments.


of 29

Table 4. Key Performance Indicator Three: Growth and Personality Development

KPI 3: Students will demonstrate a thorough knowledge of the theories of

normal and abnormal development, including personality, social and physical development, learning, and the effects trauma has on

development.


Point 1: COUN 510: Online quizzes COUN 510, S18

COUN 510, S19


2.00 = Expectation Met 2.75 2.65

N=10 N=17

Lo/Hi Lo/Hi

2/3 1/3

SD SD

0.46 0.60

Point 2: Practicum II, CMHC E-4 #10 (CW), SC E-4 # 9 (CW)

COUN 580, 579

New assessment indicator, to be collected F19

Point 3: Internship, CMHC E-4 #10 (CW), SC E-4 # 9 (CW)

COUN 581, 586

New assessment indicator, to be collected F19

Table 5. Key Performance Indicator Four: Career Development

KPI 4: Students (candidates) will demonstrate ability to: • Assess client abilities, interests, values, personality, and other factors that

contribute to career development and • facilitate client skill development for career, educational, and lifework

planning and management


Point 1: COUN 516, Final Exam COUN 516, W19



Lo/Hi: 2/3

SD: 0.5



of 29

Table 5. Key Performance Indicator Four: Career Development (continued)


Point 2: CPCE2 Comprehensive Final, Career Development

Sep 2018 Testing

Feb 2019 Testing

Southern Adventist University counseling completers

Mean Mean

10.2 10.7

N=5 N=7

Benchmark: All tests administered within the time frame, as reported by Pearson.

Overall Overall

9.4 10.5

N=289 N=977

Table 6. Key Performance Indicator Five: Counseling Skills

KPI 5 : Students will demonstrate competent application of essential interviewing, counseling and case conceptualization skills.


Point 1a, Case Conceptualization: COUN 556, Team Case Conceptualization

COUN 556, W19



Lo/Hi: 2/3

SD: 0.37

Point 1b, Interviewing and Counseling Skills: Practicum II, Microskills Analysis Part 1 and Part 2

COUN 579 S19



Lo/Hi: 1/3

SD: 0.52

Point 2: Practicum II, CMHC E-4 Items 1 - 5 and SC Items 2 - 6

'17-18 Completers

'18-19 Completers



4.38 4.11

N=10 N=17

Lo/Hi Lo/Hi

3/5 3/5

SD SD

0.74 0.81


2 Counselor Preparation Comprehensive Examination, Center for Credentialing and Education


of 29

Table 6. Key Performance Indicator Five: Counseling Skills (continued)


Point 3: Internship, CMHC E-4 Items 1 - 5 and SC items 2 - 6

'17-18 Completers

'18-19 Completers



4.58 4.79

N=10 N=17

Lo/Hi Lo/Hi

3/ 5 3 /5

SD SD

0.56 0.46

Table 7. Key Performance Indicator Six: Group Work

KPI 6: The student will demonstrate thorough understanding of theoretical foundations of group counseling and group work.


Point 1: COUN 553, Group Proposal, Reflective paper COUN 553 W19



Lo/Hi: 2/3

SD: 0.463

Point 2: Practicum II, CMHC E-4 (#18, CW) and SC E-4 (#21 , CW) CMHC #15, SC #20

'17-18 Completers

'18-19 Completers

Mean Mean

Possible range: Levels 1 - 5 4.27 3.53

Minimum required at conclusion of Practicum II :Level 3

N=10 N=17

Lo/Hi Lo/Hi

2/5 3 / 5

SD SD

1.10 0.83

Point 3: Internship, CMHC E-4 (#18, CW) and SC E-4 (#21 , CW) CMHC #15, SC #20

'17-18 Completers

'18-19 Completers

Mean Mean

Possible range: Levels 1 - 5 4.73 4.53


N=10 N=17

Lo/Hi Lo/Hi

3 / 5 3 / 5

SD SD

0.65 0.74


of 29

Table 8. Key Performance Indicator Seven: Assessment Processes

KPI 7 : The students will understand assessment processes and will be able to evaluate and provide intervention based on IQ and achievement, personality,

career, behavioral, and diagnostic assessments.


Point 1: COUN 530, Exams, Midterm and Final COUN 530, W19



Lo/Hi: 1/3

SD: 0.60

Point 2: Practicum II (CW CMHC E-4 Item 11 and SC E-4 Item 10) CMHC #8, SC #9 (v3, v4)

'17-18 Completers

'18-19 Completers

Mean Mean

4.42 3.74

Possible range: Levels 1 - 5 N=10 N=17


Lo/Hi Lo/Hi

3 / 5 2 / 5

SD SD

0.79 0.81

Point 3: Internship, (CW CMHC E-4 Item 11 and SC E-4 Item 10) CMHC #8, SC #9 (v3, v4)

'17-18 Completers

'18-19 Completers

Mean Mean

4.45 4.61

Possible range: Levels 1 - 5 N=10 N=17


Lo/Hi Lo/Hi

3 / 5 4 / 5

SD SD

0.82 0.50


of 29

Table 9. Key Performance Indicator Eight: Research

KPI 8 : The students will demonstrate knowledge and ability to conduct research, including the use of needs assessments and development of

outcome measures, to evaluate counseling interventions and programs.


Point 1: COUN 598, Research proposal COUN 598 F18



Lo/Hi: 2/3

SD: 0.53

Point 2: Internship, Case Study COUN 581 & 586 F19

Data to be collected F19

Table 10. Key Performance Indicator Nine (CMHC): Diagnostic Process

KPI 9 : Clinical Mental Health Counseling emphasis students will demonstrate competence in the diagnostic process, including differential

diagnosis and the use of current diagnostic classification systems.


Point 1: COUN 521, 2nd Case Report COUN 521 W19



Lo/Hi: 2/3

SD: 0.50

Point 2: Practicum II, (CW CMHC E-4 Item 5) COUN 580 & 585

New assessment indicator, data to be

collected 2019-20

Point 3: Internship, (CW CMHC E-4 Item 5) COUN 581 & 586

New assessment indicator, data to be

collected 2019-20


of 29

Table 11. Key Performance Indicator Ten (SC): School Counseling Program

KPI 10 : School Counseling emphasis students will demonstrate competence in the design and evaluation of a school counseling program


Point 1: COUN 577, Handbook Assessment COUN 577, W19



Lo/Hi: 2/3

SD: 0.49

Point 2: COUN 598, Program Evaluation Assessment COUN 598, F19

Data to be collected F19

The KPIs and supporting assessments were identified during the 2017-18 year and the initial data collection and

compilation in alignment with these indicators took place during 2018-19. The data as presented is to be reviewed

by the Counseling Assessment Committee at the Fall 2019 meeting. Recommendations will be made to the

Counseling Programs Council if curricular or program modifications are suggested by the review.

Follow-up Studies of Program Graduates and Employers

The revised alumni and employer surveys have been structured around competencies of the 2016 CACREP

standards. The surveys mirror each other in content and structure, each containing 14 items.

Alumni Survey

Results of the 2018-19 administration of the Alumni Survey are presented in Tables 12 through 15, with

corresponding charts 1 – 5.


of 29

Table 12. Alumni Survey Responses Common Core Areas (CCA) 1 – 4

Alumni Survey Responses

Common Core Areas 1 - 4: Ethics, Diversity, Growth, Career

Prompt: How well did the program prepare you to apply knowledge and skills in these competencies? Range and choices: 0 (Not at all); 1 (Somewhat); 2 (Well); 3 (Exceptionally Well)

CCA 1

Ethical standards of professional counseling organizations, and applications of ethical and legal considerations in professional counseling

2.44

Counselors' roles and responsibilities as members of interdisciplinary community outreach and emergency management response teams

1.89

CCA 2

The impact of heritage, attitudes, beliefs, understandings, and acculturative experiences on an individual's views of others

2.22

Multicultural counseling competencies 2.44

CCA 3

Theories of normal and abnormal personality development 1.89

Theories of individual and family development across the lifespan 2.00

CCA 4

Methods of identifying and using assessment tools and techniques relevant to career planning and decision making

1.89

Strategies for facilitating client skill development for career, educational, and life-work planning and management

2.22

Chart 1. Alumni Survey Responses Common Core Areas 1 – 4 (cf. Table 12)

2.22

1.89

2.00

1.89

2.44

2.22

1.89

2.44

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Strategies for facilitating client skill…

Methods of identifying and using…

Theories of individual and family…

Theories of normal and abnormal…

Multicultural counseling competencies

The impact of heritage, attitudes, beliefs,…

Counselors' roles and responsibilities as…

Ethical standards of professional counseling…

Counselor Education Alumni Survey (CCA 1 - 4)2018-19

N=12


of 29

Table 13. Alumni Survey Responses Common Core Areas 5 – 8


Common Core Areas 5 - 8: Counseling, Group, Assessment, Research

Prompt: How well did the program prepare you to apply knowledge and skills in these competencies?

Possible Range and Choices: 0 (Not at all); 1 (Somewhat); 2 (Well); 3 (Exceptionally Well) Actual range: 1 - 3

CCA 5

Evidence-based counseling strategies and techniques for prevention and intervention 1.78

Essential interviewing, counseling, and case conceptualization skills 2.11

CCA 6

Approaches to group formation, including recruiting, screening, and selecting members 1.78

Theoretical foundations of group counseling and group work 2.22

CCA 7

Use of assessments for diagnostic and intervention planning purposes 1.78

Procedures for assessing risk of aggression or danger to others, self-inflicted harm, or suicide

2.33

CCA 8

Evaluation of counseling interventions and programs 1.33

Making use of research to advance the counseling profession, including critiquing research to inform counseling practice

2.44

Chart 2. Alumni Responses Common Core Areas 5 – 8 (cf. Table 13)

2.44

1.33

2.33

1.78

2.22

1.78

2.11

1.78

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Making use of research to advance the…

Evaluation of counseling interventions…

Procedures for assessing risk of…

Use of assessments for diagnostic and…

Theoretical foundations of group…

Approaches to group formation,…

Essential interviewing, counseling, and…

Evidence-based counseling strategies…

Counselor Education Alumni Survey (CCA 5 - 8)2018-19

N=12


of 29

Table 14. Alumni Responses, Entry Level Specialty Areas Clinical Mental Health Counseling and School Counseling


Entry-level Specialty Areas Clinical Mental Health Counseling and School Counseling Prompt: How well did the program prepare you to apply knowledge and skills in these competencies?

Possible Range and Choices: 0 (Not at all); 1 (Somewhat); 2 (Well); 3 (Exceptionally Well) Actual range: 1 - 3

CMHC Specialty

Area

Diagnostic process, including differential diagnosis and the use of current diagnostic classification systems (DSM and ICD)

1.60

Intake interview, mental status evaluation, biopsychosocial history, mental health history, and psychological assessment for treatment planning and caseload management

2.50

SC Specialty

Area

Design and evaluation of school counseling programs 1.57

Characteristics, risk factors, and warning signs of students at risk for mental health and behavioral disorders

2.00

Chart 3. Alumni Responses, Entry Level Specialty Areas Clinical Mental Health Counseling (cf. Table 14)

2.50

1.60

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Intake interview, mental status evaluation,biopsychosocial history, mental health

history, and psychological assessment fortreatment planning and caseload

management

Diagnostic process, including differentialdiagnosis and the use of current diagnostic

classification systems (DSM and ICD)

Counselor Education Alumni Survey (CMHC)2018-19

N=5


of 29

Chart 4. Alumni Responses, Entry-Level Specialty Area School Counseling (cf. Table 14)

Table 15. Alumni Responses, Satisfaction with Program


Satisfaction with Program Prompt: How well did the program prepare you to apply knowledge

and skills in these competencies? Possible Range and Choices: 0 (Strongly Disagree); 1 (Disagree); 2 (Agree);

3 (Strongly Agree) Actual range: 1 - 3

In retrospect, the program prepared me well for the challenges of my work.

2.11

I would recommend this program of study to a friend who had career aspirations similar to mine.

1.78

2.00

1.57

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Characteristics, risk factors, and warningsigns of students at risk for mental health

and behavioral disorders

Design and evaluation of school counselingprograms

Counselor Education Alumni Survey (SC) 2018-19

N=7


of 29

Chart 5. Alumni Satisfaction with Program (cf. Table 15)

Review of the most recent alumni survey yielded the following observations:

Table 13 (Chart 2): The low of 1.33 (somewhere between “somewhat” and “well”) in Evaluation of

counseling interventions and programs is being addressed in the supporting curriculum.

Table 14 (Charts 3 and 4): Where the averages are lower (Diagnostic process at 1.60 and Design and

evaluation of school counseling programs at 1.57), changes in curriculum presentation have already been

made.

Subsequent administrations of the survey will monitor these areas.

Employer Survey

The employer survey mirrored the questions of the alumni survey. Employers responses indicate perception of

“Exceptionally-Well-Prepared” in the following indicators:

Applications of ethical and legal considerations,

Addressing normal and abnormal personality development,

Facilitation of client skill development for career, educational, and life-work planning and management,

Theoretical foundations (and/or practice) of group counseling and group work;

Procedures for assessing risk of aggression or danger to others, self- inflicted harm, or suicide.

Responses averaged between “Well-Prepared” and “Exceptionally-Well-Prepared” for these competencies:

Multicultural counseling competencies,

Essential interviewing, counseling and case conceptualization skills, and

Use of research to advance the counseling profession, and/or critiquing research to inform counseling

practice.

1.78

2.11

0 0.5 1 1.5 2 2.5 3

I would recommend this program of studyto a friend who had career aspirations

similar to mine.

In retrospect, the program prepared mewell for the challenges of my work.

Counselor Education Alumni Survey2018-19

Response N this Q = 9


of 29

A concluding question asked the level of satisfaction with the overall preparation of the practicing alum counselor:

What is your level of satisfaction with the overall preparation of this counselor?

Responses indicated “Consistently-pleased” to “Very-Pleased” levels of satisfaction.

Studies of Site Supervisors’ Perceptions and Evaluations of the Program

Following a pilot study, the Site Supervisor Survey was administered following every placement during the 2018-19

year. Like the alumni and employer surveys, questions highlighted each of the eight common core areas and

included questions specific to the two entry-level specialty areas. The additional two questions sought feedback on

collaboration between faculty and site supervisors and the overall preparation of the candidates. Results are

presented in Tables 16 through 20 (corresponding charts 6 through 10).

Table 16. Site Supervisor Survey 2018-19, Common Core Areas 1 – 4

Counselor Education Site Supervisor Survey, 2018-19

Common Core Areas (CCA) 1 - 4

Prompt: How well was your candidate prepared in the following areas?

Possible range and responses: 0 (Not at all);1 (Somewhat); 2 (Well); 3 (Exceptionally well)

Actual range: 1 - 3; N = 16

CCA 1 Applications of ethical and legal considerations in professional counseling

2.60

CCA 2 Multicultural counseling competencies 2.43

CCA 3 Addressing normal and abnormal personality development 2.40

CCA 4 Facilitation of client skill development for career, educational, and life-work planning and management

2.44


of 29

Chart 6. Site Supervisor Survey, Common Core Areas 1 – 4 (cf. Table 16)

Table 17. Site Supervisor Survey 2018-19, Common Core Areas 5 – 8


Common Core Areas (CCA) 5 - 8




CCA 5 Essential interviewing, counseling, and case conceptualization skills 2.53

CCA 6 Theoretical foundations of group counseling and group work 2.23

CCA 7 Procedures for assessing risk of aggression or danger to others, self-inflicted harm, or suicide

2.50

CCA 8 Use of research to advance the counseling profession, and/or critiquing research to inform counseling practice

2.27

2.44

2.40

2.43

2.60

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Facilitation of client skill development forcareer, educational, and life-work planning

and management

Addressing normal and abnormalpersonality development

Multicultural counseling competencies

Applications of ethical and legalconsiderations in professional counseling

Site Supervisor Survey: Counselor Education, 2018-19 (1 of 5)


of 29

Chart 7. Site Supervisor Survey, Common Core Areas 5 – 8 (cf. Table 17)

Table 18. Site Supervisor Survey 2018-19, Clinical Mental Health Counseling Indicators


Entry-level Specialty Area Clinical Mental Health Counseling




CMHC Diagnostic process, including differential diagnosis and the use of current diagnostic classification systems (DSM and ICD)

2.00

CMHC Intake, interview, mental status evaluation, biopsychosocial history, mental health history, and psychological assessment for treatment planning and caseload management

2.33

2.27

2.50

2.23

2.53

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Use of research to advance the counselingprofession, and/or critiquing research to

inform counseling practice

Procedures for assessing risk of aggressionor danger to others, self-inflicted harm, or

suicide

Theoretical foundations of groupcounseling and group work

Essential interviewing, counseling, andcase conceptualization skills

Site Supervisor Survey: Counselor Education, 2018-19(2 of 5)


of 29

Chart 8. Site Supervisor Survey, CMHC Indicators (cf. Table 18)

Table 19. Site Supervisor Survey 2018-19, School Counseling Indicators


Entry-level Specialty Area School Counseling



Actual range: 1 – 3; N = 9

SC Design and evaluation of school counseling programs 1.82

SC Characteristics, risk factors, and warning signs of students at risk for mental health and behavioral disorders

2.09

2.33

2.00

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Intake interview, mental status evaluation,biopsychosocial history, mental health

history, and psychological assessment fortreatment planning and caseload

management

Diagnostic process, including differentialdiagnosis and the use of current diagnostic

classification systems (DSM and ICD)



of 29

Chart 9. Site Supervisor Survey, School Counseling Indicators (cf. Table 19)

Table 20. Site Supervisor Survey 2018-19, Satisfaction with Collaboration and Preparation


Entry-level Specialty Area School Counseling

Possible range and responses: 0 (Not pleased);1 (Satisfied with some aspects); 2 (Well satisfied); 3 (Very pleased)


Overall Preparation

What is your level of satisfaction with the overall preparation of the Southern Adventist University candidate(s) you have supervised?

2.40

Collaboration What is your level of satisfaction with the collaboration between you and the faculty supervisor?

1.93

2.09

1.82

0.00 0.50 1.00 1.50 2.00 2.50 3.00

Characteristics, risk factors, and warningsigns of students at risk for mental health

and behavioral disorders

Design and evaluation of schoolcounseling programs



of 29

Chart 10. Site Supervisor Survey, Satisfaction Indicators (cf. Table 20)

Assessment of Student Learning and Performance on Professional Identity

Counseling dispositions are assessed every term for every student. The internship level disposition assessment of

the 2018-19 completers are presented in Table 20.

Table 21. Counseling Dispositions Assessment of 2018-19 Completers

Counseling Dispositions of 2018-19 Completers as Rated at Program Exit

Rating Levels: 1-Unacceptable; 2- Proficient; 3-Exceptional

N=17

Performance Indicator Min Max Mean

1. Demonstrates commitment to the welfare of others.

2 3 2.94

2. Shows sensitivity in interaction with others. 2 3 2.94

3. Demonstrates ability to empathize 3 3 3.00

4. Respects individuality, diversity, and freedom of choice

2 3 2.94

5. Believes in the positive potential of people. 3 3 3.00

6. Facilitates personal development in others 3 3 3.00

7. Participates actively and appropriately in class 2 3 2.92

8. Demonstrates ability to translate academic information into practice

2 3 2.82

9. Conceptualizes human behavior and the process of change

2 3 2.71


1.93

2.40

0.00 0.50 1.00 1.50 2.00 2.50 3.00

What is your level of satisfaction withthe collaboration between you and the

faculty supervisor?

What is your level of satisfaction withthe overall preparation of the SouthernAdventist University candidate(s) you

have supervised?



of 29

Table 21. Counseling Dispositions Assessment of 2018-19 Completers (continued)

Counseling Dispositions of 2018-19 Completers as Rated at Program Exit (continued)

Performance Indicator Min Max Mean

10. Maintains objectivity 2 3 2.82

11. Shows discretion and good judgment 2 3 2.94

12. Utilizes professional resources 2 3 2.88

13. Demonstrates appropriate levels of insight 2 3 2.88

14. Demonstrates appropriate levels of emotional stability 3 3 3.00

15. Demonstrates appropriate levels of tolerance for stress and frustration

3 3 3.00

16. Demonstrates appropriate levels of dependability 3 3 3.00

17. Demonstrates appropriate levels of self-confidence 2 3 2.94

18. Maintains a professional identity as a counselor 2 3 2.94

19. Models professional ethics and confidentiality 3 3 3.00

20. Commits to professional growth 3 3 3.00

21. Pays attention to personal physical health 2 3 2.76

22. Pays attention to personal mental health 2 3 2.94

23. Maintains a professional appearance 2 3 2.82

24. Expresses self well orally and in writing 2 3 2.94

25. Maintains professional boundaries and competence 2 3 2.94

24. Expresses self well orally and in writing 2 3 2.94

23. Maintains a professional appearance 2 3 2.82

22. Pays attention to personal mental health 2 3 2.94

Assessment of Student Learning and Performance on Professional Practice

The E-4 Evaluation of Candidate’s Clinical Practice (CMHC and SC, separate forms) is the key indicator for student

learning and performance in professional practice.

This rubric is organized using Bloom’s Taxonomy,3 which outlines the shifts in thinking that learners experience as

they progress form novice to more expert thinking. The five levels are described as follows:

Level 1: Is not able to remember or chooses not to remember relevant knowledge

Level 2: Remembers and understands knowledge

3 (Revised, Anderson & Krathwohl, 2001)


of 29

Level 3: Applies knowledge

Level 4: Analyzes and evaluates a situation

Level 5: Creates meaningful responses

The E-4 is administered by site supervisors at the conclusion of each semester of practicum or internship

experience. There are typically multiple placements during internship, including the multiple settings for school

counseling emphasis (elementary, middle, and high school). The five performance levels for each indicator on the

instrument allow students to track their progress toward proficiency; ratings at the level of 3 (“applies”) are

required for transitioning from practicum to internship, and ratings at the levels of 4 and 5 (“analyzes” and

“creates”) are expected by the end of internship. This summary highlights the aggregated results at exit of

candidates who completed the program within 2018-19 year.

Evaluation of Candidate’s Clinical Practice, Clinical Mental Health Counseling

Expectation: At the end of COUN 584 Clinical Internship, the onsite internship supervisor will evaluate the intern in

terms of his/her demonstrated counseling skills (Evaluation of Candidate's Clinical Practice). The combined mean

score of these evaluations will be at least 4.00. (Scale: 1-5, 4.00 required at completion of internship). Aggregated

mean for 2018-19 completers at internship is 4.80. Expectation is met. See Chart 11, page 27.

Evaluation of Candidate’s Clinical Practice, School Counseling

Expectation: At the end of COUN 584 Clinical Internship, the onsite internship supervisor will evaluate the intern in

terms of his/her demonstrated counseling skills (Evaluation of Candidate's Clinical Practice). The combined mean

score of these evaluations will be at least 4.00. (Scale: 1-5, 4.00 required at completion of internship). Aggregated

mean for 2018-19 completers is 4.68. Expectation is met. See Chart 12, page 28.

Summary of Observations, Use of Findings

Characteristics of Program Candidates

Over half of the current student population is non-white; gender representation is present. Recruiting will

continue to target all demographics.

Key Performance Indicators

The establishment of the key performance indicators and the embedding of supporting assessments at

multiple points, as well as processes for collection of assessment results and initial data collection, has

been a major achievement of the past eighteen months. The process and review will be ongoing, with

adjustments as different curricular areas are highlighted.

This first compilation or key performance indicator data will be reviewed at the fall meeting of the

Counseling Assessment Committee in November. Any recommendations referenced on the data will be

directed to the Counseling Programs Council., which meets monthly.

Follow-up Studies

The revised, piloted, and validated alumni and employer surveys have been implemented. The surveys

allow the data collected to correlate with the key performance indicators. Results of the alumni survey

were reviewed by the Counseling Assessment Committee on September 11, 2019. Efforts will focus on

more completely tracking alumni and securing adequate response rates from both alumni and employers.


of 29

Chart 11. Evaluation of Candidate’s Clinical Practice, Clinical Mental Health Counseling

4.6

5.0

4.7

5.0

5.0

4.9

4.8

4.8

4.9

4.8

4.4

4.5

4.7

5.0

4.6

4.8

4.5

4.6

4.9

4.9

4.8

4.6

4.8

5.0

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Facilitation Opt Dev

Use of Feedback

Growth Activ

Ethical Behav

Attendance

Follow-Through

Teaming

Collaboration

Consultation

Small Groups

Use of Referrals

Readiness Crisis Interv

Termination

Self-Sprv Counseling

Theoretical Congruence

Goal Setting

Assessment

Mgm of Emotions

Intro and Summary

Therapeutic Skills

Delivery of Content

Mgm Therapeutic Dialog

Management of Process

Rapport with Clients

Clinical Observation, CMHC2018-19 Completers

N=10Note: Ratings reference Levels 1 - 5.

Level 4 is required at completion.


of 29

Chart 12. Evaluation of Candidate’s Clinical Practice, School Counseling

5.0

4.7

5.0

4.8

5.0

4.8

4.8

4.8

4.0

4.5

4.9

4.7

4.8

4.4

4.6

4.1

4.7

4.7

4.5

4.0

4.6

4.7

4.7

4.9

4.7

4.7

4.8

4.8

4.9

4.7

4.8

4.7

4.8

4.8

4.8

4.8

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Use of Feedback

Growth Activ

Ethical Behav

Attendance

Follow-Through

Teaming

Collaboration

Consultation

Budgeting

Data Analysis

Diversity Competency

Exceptional Student Needs

Personal, Educational, Career Advisement

Assessment Instruments

Use of Technology

Parent Workshops

Small Groups

Guidance Curriculum Dev

Classroom Mgm

Guidance Lessons

Peer Facilitation

Use of Referrals

Readiness Crisis Interv

Termination

Self-Sprv Counseling

Theoretical Congruence

Goal Setting

Assessment

Mgm of Emotions

Intro and Summary

Therapeutic Skills

Delivery of Content

Mgm Therapeutic Dialog

Management of Process

Rapport with Students

Consultation Effectiveness

Clinical Observation, SC2018-19 Completers

N=9Note: Ratings reference Levels 1 - 5.

Level 4 is required at completion.


of 29

Studies of Site Supervisors’ Perceptions and Evaluations of the Program

The site supervisor survey has been implemented and is correlated to the key performance indicators.

Results of the site supervisor survey for 2018-19 were reviewed by the Counseling Assessment Committee

on September 11, 2019.

Assessment of Student Learning and Performance on Professional Identity

All results of the counseling dispositions assessment show satisfactory attainment of the desired

counselor dispositions.

Student demonstration of expected counseling dispositions is monitored through collaborative faculty

review every term.

Procedures are in place to address student dispositional characteristics that are found to be inconsistent

with counseling practice.

No students had to be redirected due to dispositional issues during the 2018-19 year.

Assessment of Student Learning and Performance on Professional Practice

The E-4 Evaluation of Candidate’s Clinical Practice (CMHC/SC, two forms) as completed by the site

supervisors provides assessment of student learning and performance on professional practice. Several

new indicators have been incorporated in order to monitor KPIs at multiple points.

All indicators indicate satisfactory results on these measures for the 2018-19 year. The recently

incorporated indicators will provide assessment results beginning fall 2019. i

i All data and charts in this report were compiled by the School of Education and Psychology Assessment System Office. Questions may be directed to Elaine Hayden, Assessment System Manager, at [email protected].

mailto:[email protected]

Southern Adventist University Counselor Education Program ...€¦ · Counseling Program...

Documents

Transcript of Southern Adventist University Counselor Education Program ...€¦ · Counseling Program...