Slide 1

Getting the data off the shelf Jimmy Davis, Ph.D. Vice Provost Kimberly Thornbury, Ph.D. Dean of Students Union University

Slide 2

Where We Were What Got Us Started in Assessment Institutional Strategic Planning Long Range Planning (began 1982) Five-Year Planning & Institutional Research Office (began 1994) Regional Accreditation (SACS) Institutional Effectiveness ongoing, integrated, and institutional-wide planning and evaluation processes provides evidence of improvement based on analysis of assessment results

Slide 3

The deciding factor in assessing the effectiveness of any institutional effectiveness is evidence of the extent to which it achieves its goals and objectives. The necessity of seeking such evidence continually is inescapable; one of the primary hallmarks of faculty, administration, and governing boards is the skill with which they raise questions about institutional effectiveness, seek answers, and significantly improve procedures in light of their findings. (Commission on Higher Educations Standards for Accreditation, 1982, pp 17-18.)

Slide 4

Without assessment, student affairs is left only to logic, intuition, moral imperatives, goodwill, or seredipity in justifying its existence. (Upcraft and Schuh, Assessment in Student Affairs, pg 12.)

Slide 5

We in student affairs can no longer presume that our jobs are important and effective simply because we think they are or because weve always done them or even because were doing the right thing. There must be evidence gathered from a comprehensive assessment program to answer these questions. (Upcraft and Schuh, Assessment in Student Affairs, pg. 16.)

Slide 6

Survey Instruments How to choose How to focus How to report

Slide 7

Why We Changed CAP = FOCUS YearFallSpring 2000-2001CIRP 2001-2002SSI/IRPCSS 2002-2003CIRP 2003-2004SSI/IRPCSS 2004-2005CIRPFaculty Survey 2005-2006SSI/IRPCSS, STI 2006-2007CIRP 2007-2008SSI/IRPCSS

Slide 8

Astins I-E-O Model I refers to Inputs The personal qualities that students bring to the educational program E refers to Environment The students actual experience during college O refers to Outputs Talents that are developed in the educational program Astin, A. W. (1993). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. Phoenix: The Oryx Press.

Slide 9

What Surveys does Union University Employ? Outputs Alumni Survey General Education College BASE Major Areas Major Field Tests Inputs Cooperative Institutional Research Program ( CIRP) Freshmen Survey Environment and Outputs College Student Survey (CSS) Student Satisfaction Index (SSI) Adult Student Priorities Survey (ASPS) National Survey of Student Engagement (NSSE) HERI Faculty Survey

Slide 10

Group Discussion: What Survey Instruments Do You Employ?

Slide 11

How we collected the data CIRP: part of orientation schedule SSI: (random group selected, come at 10:00 or 3:00 hour.) NSSE: our first experience with online surveys CSS: the draw of food Faculty survey: January workshop Adult Noel Levitz: Program directors Alumni:

Slide 12

Group Discussion: How do you collect your data? (e.g. service days, in class distribution?)

Slide 13

How we distributed the data Most people wont read raw data. The importance of interpretation and presentation. Most people need trends, themes, highs and lows. Most people need categories (we used the SSI categories for SSI, NSSE, CSS).

Slide 14

Slide 15

Slide 16

Slide 17

Assessment Plunge: What we looked for CSS: Looks are the differences between men, women, and our students versus other religious 4 year and all private four year schools. (6% difference gets our attention.) NSSE: Looks at first year students and seniors versus other CCCU institutions and masters schools, and also compares data with all NSSE participants (5% difference gets our attention. SSI: Looks at our students compared to private four year schools. We look at items with a Performance Gap > 1.0, and if our scores were less than NGM. We also look at the items that are most important (already listed) as well as the items ranked low in satisfaction.

Slide 18

Distribution Provost council January Faculty Workshop In the Academy session SGA open student senate Student Life divisional workshop/retreat Open for other divisional retreats President received hard data

Slide 19

Feedback categories Academic Advising Effectiveness Campus Climate Campus Life Campus Support Services Concern for the Individual Instructional Effectiveness Recruitment and Financial Aid Effectiveness Registration Effectiveness Responsiveness to Diverse Populations Safety and Security Service Excellence Student Centeredness Spiritual Growth Decisions Factors Affecting Decision to Enroll Extent of Student Involvement in the Community Overall/General

Slide 20

Academic Advising Effectiveness

Slide 21

Academic Advising Effectiveness assesses the comprehensiveness of your academic advising program. Academic advisors are evaluated on the basis of their knowledge, competence and personal concern for student success, as well as on their approachability.

Slide 22

* CSS

Slide 23

Slide 24

Slide 25

* SSI

Slide 26

Slide 27

Campus Life

Slide 28

Campus Life assesses the effectiveness of student life programs offered by your institution, covering issues ranging from athletics to residence life. This scale also assesses campus policies and procedures to determine students perceptions of their rights and responsibilities.

Slide 29

Strengths

Slide 30

* CSS

Slide 31

Slide 32

Slide 33

Slide 34

Slide 35

SSI Seniors *** better than ngm

Slide 36

* SSI *** better than ngm

Slide 37

SSI Seniors *** better than ngm

Slide 38

SSI Seniors *** better than ngm

Slide 39

Areas for Improvement

Slide 40

There was a 13% drop in this physical health rating from freshman to senior year. (same students) * CSS

Slide 41

SSI Seniors *** better than ngm

Slide 42

SSI Seniors Same as ngm

Slide 43

SSI Seniors Same as ngm

Slide 44

SSI Seniors Same as ngm

Slide 45

Importance of context Helpful to compare your strengths and weaknesses with the overall CCCU data

Slide 46

What is most important to your students? Item No.Imp. ScoreDescription 86.63The content of the courses within my major is valuable. 166.61The instruction in my major is excellent. 686.58Nearly all of the faculty are knowledgeable in their field. 586.54The quality of instruction I receive in most of my classes is excellent 666.52Tuition paid is a worthwhile investment. 396.52I am able to experience intellectual growth here. 336.50My academic advisor is knowledgeable about requirements in my major. 296.50It is an enjoyable experience to be a student on this campus. 346.49I am able to register for classes I need with few conflicts. 746.46Being on this campus is contributing to my spiritual growth. 176.46Adequate financial aid is available for most students. 756.44My understanding of God is being strengthened by classroom and/or campus experiences. 696.43There is a good variety of courses provided on this campus. 76.43The campus is safe and secure for all students. 256.41Faculty are fair and unbiased in their treatment of individual students. CCCU SSI Averages (15 Highest Importance Scores)

Slide 47

Where are your students most satisfied? Item No.Sat. ScoreDescription 685.91Nearly all of the faculty are knowledgeable in their field. 515.81This institution has a good reputation within the community. 785.79This campus provides adequate opportunities for involvement in ministry. 775.79Males and females are treated with equal respect on this campus. 725.78On the whole, the campus is well-maintained. 395.77I am able to experience intellectual growth here. 25.75The campus staff are caring and helpful. 65.73My academic advisor is approachable. 455.70Students are made to feel welcome on this campus. 35.69Faculty care about me as an individual. 655.69Faculty are usually available after class and during office hours. 295.67It is an enjoyable experience to be a student on this campus. 335.66My academic advisor is knowledgeable about requirements in my major. 755.66My understanding of God is being strengthened by classroom and/or campus experiences. 75.65The campus is safe and secure for all students. 415.65There is a commitment to academic excellence on this campus. CCCU SSI Averages (15 Highest Importance Scores)

Slide 48

Where are your students least satisfied? CCCU SSI Averages (15 Lowest Satisfaction Scores) Item No.Sat. ScoreDescription 213.66The amount of student parking space on campus is adequate. 384.32There is an adequate selection of food available in the cafeteria. 424.61There are a sufficient number of weekend activities for students. 174.68Adequate financial aid is available for most students. 114.70Billing policies are reasonable. 714.72Channels for expressing student complaints are readily available. 574.74I seldom get the run-around when seeking information on campus. 734.76Student activities fees are put to good use. 404.77Residence hall regulations are reasonable. 244.82The intercollegiate athletic programs contribute to a strong sense of school spirit. 364.88Security staff respond quickly in emergencies. 234.90Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air conditioning, telephones, etc. 674.95Freedom of expression is protected on campus. 664.95Tuition paid is a worthwhile investment. 194.96My academic advisor helps me set goals to work toward.

Slide 49

Where are your students expectations not being met? CCCU SSI Averages (15 Largest Performance Gaps) Item No.Gap ScoreDescription 212.37The amount of student parking space on campus is adequate. 381.79There is an adequate selection of food available in the cafeteria. 171.78Adequate financial aid is available for most students. 661.57Tuition paid is a worthwhile investment. 341.50I am able to register for classes I need with few conflicts. 231.44Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air conditioning, telephones, etc.) 111.37Billing policies are reasonable. 571.37I seldom get the run-around when seeking information on this campus. 361.37Security staff respond quickly in emergencies. 731.32Student activities fees are put to good use. 401.28Residence halls regulations are reasonable. 711.24Channels for expressing student complaints are readily available. 121.23Financial aid awards are announced to students in time to be helpful in college planning. 51.21Financial aid counselors are helpful. 691.18There is a good variety of courses provided on this campus.

Slide 50

Institutional Strengths The content of the courses within my major is valuable. 6.63/5.61 The instruction in my major field is excellent. 6.61/5.63 Nearly all of the faculty are knowledgeable in their field. 6.58/5.91 The quality of instruction I receive in most of my classes is excellent. 6.54/5.57 I am able to experience intellectual growth here. 6.52/5.77 My academic advisor is knowledgeable about requirements in my major. 6.50/5.66 It is an enjoyable experience to be a student on this campus. 6.50/5.66 Being on this campus is contributing to my spiritual growth. 6.46/5.57 My understanding of God is being strengthened by classroom and/or campus experiences. 6.44/5.66 The campus is safe and secure for all students. 6.43/5.65

Slide 51

Institutional Strengths (contd) My academic advisor is approachable. 6.40/5.73 There is a commitment to academic excellence on this campus. 6.40/5.65 The campus staff are caring and helpful. 6.39/5.75 This institution shows concern for students as individuals. 6.38/5.61 Students are made to feel welcome on this campus. 6.36/5.70 Males and females are treated with equal respect on this campus. 6.35/5.79 Given where I am spiritually right now, this campus is a good fit for me. Faculty are usually available after class and during office hours. 6.37/5.69 Faculty care about me as an individual. 6.30/5.69 This institution has a good reputation with the community. 6.23/5.84 On the whole, the campus is well-maintained. 6.22/5.78 This campus provides adequate opportunities for involvement in ministry. 6.12/5.79

Slide 52

Priorities for Intervention Tuition paid is a worthwhile investment. 6.52/4.95 I am able to register for classes I need with few conflicts. 6.49/4.99 Adequate financial aid is available for most students. 6.46/4.68 Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air conditioning, telephones, etc.) 6.34/4.90 Security staff respond quickly in emergencies. 6.29/4.88 Financial aid awards are announced to students in time to be helpful in college planning. 6.24/5.02 Faculty provide timely feedback about student progress in a course. 6.23/5.08 Financial aid counselors are helpful 6.21/5.02 There are adequate services to help me decide upon a career. 6.12/5.12 I seldom get the run-around when seeking information on this campus. 6.12/4.74 Library resources and services are adequate. 6.11/5.13 How satisfied are you that this campus demonstrates a commitment to meeting the needs of evening students? 4.98

Slide 53

Matrix Union University Student Satisfaction Index Matrix for Prioritizing Action Very Important Priorities for InterventionInstitutional Strengths 17. Adequate financial aid is available for most students. 79. Given where I am spiritually right now, this campus is a good "fit" for me. 23. Living conditions in the residence halls are comfortable (adequate space, lighting, heat, air, etc.) 75. My understanding of God is being strengthened by classroom and/or campus experiences. 66. Tuition paid is a worthwhile investment. 74. Being on this campus is contributing to my spiritual growth. 68. Nearly all of the faculty are knowledgeable in their field. 29. It is an enjoyable experience to be a student on this campus. 39. I am able to experience intellectual growth here. Very DissatisfiedAreas that have Low Status for StudentsConsider Redirection of ResourcesSatisfied 42. There are a sufficient number of weekend activities for students. 9. A variety of intramural activities are offered. 24. The intercollegiate athletic programs contribute to a strong sense of school spirit. 13. Library staff are helpful and approachable. Very Unimportant

Slide 54

How We Used Data Strategic Planning Union 2010 Eight Priorities with Goals, Outcomes, Assessment Techniques

Slide 55

How We Used Data: Strategic Planning Assessment Result SSI Seniors: Library Resources and services are adequate (weakness at **) Strategic Plan response Priority 3: Cultivate a climate of excellence and innovation based on our distinctive vision and mission with continuous and measurable improvement, in all aspects of University life. (Student Services Committee) The University will develop its library and computing resources and services to meet the scholarship needs of the entire learning community, consistent with the Technology Master Plan. (Associate Vice Presidents of Academic Resources and Information Services) The University will develop library physical facilities on both the Jackson and Germantown campuses which will serve as academic commons of learning. (Associate Vice President of Academic Resources)

Slide 56

How We Used Data: Strategic Planning Assessment Result CSS: 62% of our seniors did not have a course that included community service/service learning since entering college compare to other religious (39%) and all private (46%). Strategic Plan response Priority 3: Cultivate a climate of excellence and innovation based on our distinctive vision and mission with continuous and measurable improvement, in all aspects of University life. (Student Services Committee) The University will focus on student learning through additional opportunities in 1) international and intercultural experiences, 2) service-learning courses, and 3) excellence in the classroom through 21st century pedagogical strategies such as web enhanced learning. (Provost/Academic Deans/Quality Enhancement Planning Team)

Slide 57

Other uses of data Food Services improvements move to a block meal plan Bookstore manager now attends national merchandise meeting Marketing team Financial Aid hired additional staff

Slide 58

Who else requested data? Examples include: Minority Task Team First year experience task team Bookstore

Slide 59

Discussion Who are your audiences? How many times do you present it? How are the results tailored for each audience? Does your university have a limit to how many issues you tackle based on the data? Pros/cons? How accessible is your data on your campus? (e.g. website, hard copies, etc.)

Slide 60

Thank you