Mentoring As a Judicial Sanction:

Assessing Sense of Belonging

Wendy YoungAssistant Director, Judicial Affairs

Sara Finney, Ph.D.Assistant Professor of Graduate Psychology;

Assistant Assessment Specialist

Freedom

Didn’t realize it was a policy

Rebellion

Attention-Seeking behavior

This program focuses on attention-seeking

What are some REASONS that students violate policies?

Theory to Guide Program

Academic success and personal growth is

positively correlated with: INVOLVEMENT IN WORK INVOLVEMENT WITH FACULTY Academic InvolvementAcademic Involvement Involvement with Student PeersInvolvement with Student Peers Other involvementOther involvement

Astin, Alexander W. (1993). What Matters in College. San Francisco, CA: Jossey-Bass, Inc.

Astin’s Work Applied to This Program “Student-faculty interaction has significant

positive correlations with every academic attainment outcome: college GPA, degree attainment, graduating with honors and enrollment in graduate or professional school.”

“Student-faculty interaction also has positive correlations…with a variety of personality and attitudinal outcomes: scholarship, social activism, leadership, artistic inclination, and commitment to each of three life goals: promoting racial understanding, participating in programs to clean up the environment and making theoretical contributions to science.”

Astin, Alexander W. (1993). What Matters in College. San Francisco, CA: Jossey-Bass, Inc.

Astin’s Work Applied to This Program “Holding a part-time job on campus is positively

associated with attainment of a bachelor’s degree and with virtually all areas of self-reported cognitive and affective growth.”

“Working at a part-time job on campus increases the student’s chances of being elected to a student office, tutoring other students, and attending recitals or concerts. It has positive effects on Liberalism, Leadership, and a commitment to the goals of promoting racial understanding and participating in programs to clean up the environment.”

Astin, Alexander W. (1993). What Matters in College. San Francisco, CA: Jossey-Bass, Inc.

Civic Learning—A History James Madison University’s Mission

We are a community committed to preparing students to be educated and enlightened citizens who lead productive and meaningful lives.

Judicial Affairs MissionWe are committed to promoting student

learning, civic responsibility and, through partnerships, developing the community necessary for the university to achieve its mission.

Civic Learning—A History Prior to 2005:

“Service learning”---student was assigned hours to work in an office on campus; reflection was done with one facilitator & 2 other student participants

No formal mentoring No formal assessment No consistent “treatment”; no formal training of

facilitators.

“Civic Learning” did not exist

Civic Learning—Definitions for Program Civic Learning: “the growth and development for

citizenship that results for students from civic engagement of all kinds”

Civic Engagement: “wide range of learning activities within or part of the institution…that engages the institution in partnership with its civic contexts”

Service Learning: “educational activity carried out in partnership with a public or non-profit agency, organization, or project in the community”

Mentoring/Coaching: “Unlocking a person’s potential to maximize their own performance. It is helping them to learn rather than teaching them”

References: “Coaching a Diverse Workforce; (Leading with questions)”; Coaching for Performance, Whitmore, John http://www.usm.maine.edu/cft/service/ce_sl_key_definitions.htm

Civic Learning—Year One (2005-2006)Program Description:Sanctioned students are assigned to an on campus site for 25-75 hours ANDare given a mentor for 15 sessions. The program is designed to assist students in becoming more involved on campus. (Note: 9 students were given mentoring only due to not having an available site & the students’ desire to have this type of assistance.)

Goals and Objectives:1. Students will increase their level of connection/belonging to the university. Upon completion of the civic learning sanctioned program students will:

Significantly increase their Sense of Belongingness Be able to do at least one of the following:

Identify at least two academic/student assistance resources at JMU that they were not aware of prior to participating in the program.

Report using at least one academic/student assistance resource that they had not used prior to the program.

Identify at least two new clubs/organizations that they would consider participating in. Report attending at least one university sponsored event that they learned about through the

civic learning program.2. Student will identify their civic responsibilities within the university community.

Identify at least three civic responsibilities they have within the university community.

Civic Learning—Year One (2005-2006)Program Procedures: Hearing officer sanctions student to a specific amount of hours Hearing officer fills out the overview sheet (which contains general

information and any points that must be notes) Hearing officer walks student to the Resource Center. Office Assistant or PA gives student the brochure and explains that

it is an overview of the program. PA schedules an intake interview with the Coordinator of Civic

Learning. Prior to the intake interview, the student completes the

Belongingness assessment. During the intake interview, the Coordinator asks questions and

generates discussion to discover the student’s interests and needs. The Coordinator assigns a site and mentor. The Coordinator will explain all of the program requirements (acts as a contract) and then the student will sign contract.

Assessment Procedures: Only assessed Sense of Belongingness (SB)

Pre-post design Pilot study of program

Civic Learning—Year One (2005-2006)Assessment Results N = 29 participated

76% Male 46% 1st year, 29%

Soph, 21% Junior Sense of

Belongingness increased 6 point gain on scale

with 18-90 range (8.3% gain)

Largest increase for those with only mentoring hours (~9pt)

Use of Results Added mentoring only

as a sanctioning option (15 sessions if mentoring only)

Set up a plan to investigate if differential gains in SB would replicate or if its just a function of sampling error

Received internal grant to support continued study of program

Note: aTest of mean difference was statistically (t = 4.36, p <.001) and practically significant (d = .84).Row in WHITE represents total sample of students.Row in YELLOW indicate results from students that completed ONLY mentoring hours. Row in GREEN indicate results from students that completed 20-34 community service hours. Row in GRAY indicate results from students that completed more than 34 community service hours.

α Pre-Test

Mean (SD) Pre-Test

SampleSize

α Post-Test

Mean (SD) Post-Test

Difference in Mean Scores

.81 67.89 (8.80) 27 .88 73.85 (9.51) 5.96a

.56 67.89 (5.94) 9 .92 77.33 (9.53) 9.44

.81 65.86 (9.91) 7 .85 69.86 (10.76) 4.00

.90 69.18 (8.80) 11 .87 73.54 (9.51) 4.36

Total Scale Means by Type & Amount of Activity (2005-2006)

Civic Learning—Year One (2005-2006)Assessment Results N = 29 participated

76% Male 46% 1st year, 29%

Soph, 21% Junior Sense of

Belongingness increased 6 point gain on scale

with 18-90 range (8.3% gain)

Largest increase for those with only mentoring hours (~9pt)

Use of Results Added mentoring only

as a sanctioning option (15 sessions if mentoring only)

Set up a plan to investigate if differential gains in SB would replicate or if its just a function of sampling error

Received internal grant to support continued study of program

Civic Learning—Year Two (2006-2007)Goals/Objectives Re-assessed change in SB

given change in the program.

Spent the year re-visiting remaining goals Looking at match of

goals with “new” program

Issues surrounding future assessment implementation and needs

Program Procedures Same general procedures of

implementation & goals and objectives as 2005-2006 used in assigning students to program

Implementation of program changed based on assessment results: More intentional training of

mentors

Number of mentoring hours (8, 10, 12, or 15 sessions OR mentoring only—15 sessions)

Assignment of students to sites and mentors

Civic Learning—Year Two (2006-2007): Assignment to Program

Low Challenge/Low Mentoring Very busy schedule Belongs to at least 1 JMU org. Good understanding of personal mission and goals Site Hours: 25-50Mentoring Sessions: 8 or 10

Moderate Challenge/Low Mentoring

Not involved/lots of free time Good understanding of personal mission and goals May be a repeat offenderSite Hours: 50-75Mentoring Sessions: 8 or 10

Moderate Challenge/High MentoringVery busy schedule Belongs to at least 1 JMU org.Poor understanding of personal mission and goalsNot well-adjusted/homesickSite Hours: 25-75Mentoring Sessions: 12 or 15

High Challenge/High MentoringNot involved/lots of free timePoor understanding of personal mission and goalsMay be a repeat offenderNot well-adjusted/homesickSite Hours: 0 or 25-75Mentoring Sessions: 12 or 15 or Mentoring Only

Civic Learning—Year Two (2006-2007)Goals/Objectives Re-assessed change in SB

given change in the program.

Spent the year re-visiting remaining goals Looking at match of

goals with “new” program

Issues surrounding future assessment implementation and needs

Program Procedures Same general procedures of

implementation & goals and objectives as 2005-2006 used in assigning students to program

Implementation of program changed based on assessment results: More intentional training of

mentors

Number of mentoring hours (8, 10, 12, or 15 sessions OR mentoring only—15 sessions)

Assignment of students to sites and mentors

Civic Learning—Year Two (2006-2007)Assessment Results N = 35 participated

66% Male 60% 1st year, 26% Soph, 6%

Junior SB increased

4.66 point gain (6.5% gain) Differential increase given

different “treatment” “Mentoring only” had

largest effect, but not as extreme as previous year

Examined both years together to get better idea of program impact (more stable estimates)

Use of Results SB changing

Most change related to “mentoring only”

Differential growth could be a function of the “treatment” OR type of student

Turn focus on to assessing other objectives

New Civic Learning Coordinator position provided to office

Year α Pre-Test

Sample Size

Mean (SD) Pre-Test

α Post-Test

Mean (SD) Post-Test t

EffectSize

2006 – 2007 .91 3272.31 (10.61) .84

76.97(7.54) 2.74* .48

2005 – 2006 .81 2767.89 (8.80) .88

73.85(9.51) 4.36** .84

2005 - 2007 .88 5970.28 (10.00) .87

75.54(8.57) 4.74** .62

Reliabilities & Total Scale Means Across Two Years

* p < .01; **p < .001; effect size is Cohen’s d for repeated measures

Rows in YELLOW indicate results from students that completed ONLY mentoring hours. Rows in GREEN indicate results from students that completed 20-34 community service hours. Rows in GRAY indicate results from students that completed more than 34 community service hours.

YEAR

α Pre-Test

Mean (SD) Pre-Test

SampleSize

α Post-Test

Mean (SD) Post-Test

Difference inMean Scores

2006-2007

.92 72.31 (10.92) 19 .83 77.52 (7.20) 5.21

.82 86.00 (5.66) 2 .91 83.00 (7.07) -3.00

.84 69.81 (9.31) 11 .84 74.91 (8.09) 5.10

2005-2006

.56 67.89 (5.94) 9 .92 77.33 (9.53) 9.44

.81 65.86 (9.91) 7 .85 69.86 (10.76) 4.00

.90 69.18 (8.80) 11 .87 73.54 (9.51) 4.36

2005-2007

.88 70.89 (9.72) 28 .86 77.46 (7.84) 6.57

.90 70.33 (12.51) 9 .89 72.77 (11.26) 2.44

.87 69.50 (9.67) 22 .85 74.23 (8.08) 4.73

Total Scale Means Across Two Years by Type & Amount of Activity

Civic Learning—Year Two (2006-2007)Assessment Results N = 35 participated

66% Male 60% 1st year, 26% Soph,

6% Junior SB increased

4.66 point gain (6.5% gain) Differential increase given

different “treatment” “Mentoring only” had

largest effect, but not as extreme as previous year

Examined both years together to get better idea of program impact (more stable estimates)

Use of Results SB increasing

Most change related to “mentoring only”

Differential growth could be a function of the “treatment” OR type of student

New Civic Learning Coordinator position provided to office

Turn focus on to assessing other objectives

Civic Learning,Year Three(2007-2008): Assessment ActivitiesGoal: Students will increase their level of connection and/or belonging to the university.

Objectives Activities AssessmentAs a function of the Civic Learning program, students will report a significant increase in their sense of belonging at the university. We expect a higher increase for those working with a mentor (5 point increase) than those only working with a site (2 point increase).

Mentoring: “Why College?”, free conversation, attend reflection meetings, site hours, attend mentoring meetingsSites:

Sense of Belonging instrument (pre/post)

As a function of the Civic Learning program, students will be able to list 3 ways to find information about JMU-sponsored programs/events and describe at least 3 benefits associated with attending these types of activities.

Mentoring: “Things to Do at JMU”, field trip, site hours, Motivation and Goal-setting worksheetSites:

Reflection Paper; GA qualitative exit interview (both scored using rubrics to be created)

As a function of the Civic Learning program, students will be able to list and apply university resources to potential student needs.

Mentoring: “Things to Do at JMU”, field trip, site hours, Motivation and Goal-Setting WorksheetSites:

Reflection Paper (scenarios); GA qualitative exit interview (both scored using rubrics to be created)

As a function of the Civic Learning program, students will show an increase in positive regard for administrators, faculty, and other JMU students.

Function of total program Reflection Paper (to be scored using rubric to be created)

As a function of the Civic Learning program, students will be less likely to leave JMU.

Function of total Program Analysis of transfer information

As a function of the Civic Learning program, students will be more likely to increase help-seeking behavior.

Function of total Program Help-seeking instrument (pre/post)

Civic Learning, Year Three (2007-2008)

Projects to Implement or Develop:

Continue to re-visit our goals/objectives

Continue to improve program Focus on sites

Training & recruitment of supervisorsIncorporating reflection for site work in

order to create more intentional purpose for the site.

Questions?

Wendy Young Assistant Director

James Madison UniversityJudicial Affairs540-568-6218

young2wm@jmu.edu

Sara Finney Assistant Professor of Graduate Psychology;

Assistant Assessment SpecialistJames Madison University

Center for Assessment & Research Studies540-568-6757

finneysj@jmu.edu

Contact Information