Transcript of Getting the data off the shelf Jimmy Davis, Ph.D. Vice Provost Kimberly Thornbury, Ph.D. Dean of...
- Slide 1
- Getting the data off the shelf Jimmy Davis, Ph.D. Vice Provost
Kimberly Thornbury, Ph.D. Dean of Students Union University
- Slide 2
- Where We Were What Got Us Started in Assessment Institutional
Strategic Planning Long Range Planning (began 1982) Five-Year
Planning & Institutional Research Office (began 1994) Regional
Accreditation (SACS) Institutional Effectiveness ongoing,
integrated, and institutional-wide planning and evaluation
processes provides evidence of improvement based on analysis of
assessment results
- Slide 3
- The deciding factor in assessing the effectiveness of any
institutional effectiveness is evidence of the extent to which it
achieves its goals and objectives. The necessity of seeking such
evidence continually is inescapable; one of the primary hallmarks
of faculty, administration, and governing boards is the skill with
which they raise questions about institutional effectiveness, seek
answers, and significantly improve procedures in light of their
findings. (Commission on Higher Educations Standards for
Accreditation, 1982, pp 17-18.)
- Slide 4
- Without assessment, student affairs is left only to logic,
intuition, moral imperatives, goodwill, or seredipity in justifying
its existence. (Upcraft and Schuh, Assessment in Student Affairs,
pg 12.)
- Slide 5
- We in student affairs can no longer presume that our jobs are
important and effective simply because we think they are or because
weve always done them or even because were doing the right thing.
There must be evidence gathered from a comprehensive assessment
program to answer these questions. (Upcraft and Schuh, Assessment
in Student Affairs, pg. 16.)
- Slide 6
- Survey Instruments How to choose How to focus How to
report
- Slide 7
- Why We Changed CAP = FOCUS YearFallSpring 2000-2001CIRP
2001-2002SSI/IRPCSS 2002-2003CIRP 2003-2004SSI/IRPCSS
2004-2005CIRPFaculty Survey 2005-2006SSI/IRPCSS, STI 2006-2007CIRP
2007-2008SSI/IRPCSS
- Slide 8
- Astins I-E-O Model I refers to Inputs The personal qualities
that students bring to the educational program E refers to
Environment The students actual experience during college O refers
to Outputs Talents that are developed in the educational program
Astin, A. W. (1993). Assessment for excellence: The philosophy and
practice of assessment and evaluation in higher education. Phoenix:
The Oryx Press.
- Slide 9
- What Surveys does Union University Employ? Outputs Alumni
Survey General Education College BASE Major Areas Major Field Tests
Inputs Cooperative Institutional Research Program ( CIRP) Freshmen
Survey Environment and Outputs College Student Survey (CSS) Student
Satisfaction Index (SSI) Adult Student Priorities Survey (ASPS)
National Survey of Student Engagement (NSSE) HERI Faculty
Survey
- Slide 10
- Group Discussion: What Survey Instruments Do You Employ?
- Slide 11
- How we collected the data CIRP: part of orientation schedule
SSI: (random group selected, come at 10:00 or 3:00 hour.) NSSE: our
first experience with online surveys CSS: the draw of food Faculty
survey: January workshop Adult Noel Levitz: Program directors
Alumni:
- Slide 12
- Group Discussion: How do you collect your data? (e.g. service
days, in class distribution?)
- Slide 13
- How we distributed the data Most people wont read raw data. The
importance of interpretation and presentation. Most people need
trends, themes, highs and lows. Most people need categories (we
used the SSI categories for SSI, NSSE, CSS).
- Slide 14
- Slide 15
- Slide 16
- Slide 17
- Assessment Plunge: What we looked for CSS: Looks are the
differences between men, women, and our students versus other
religious 4 year and all private four year schools. (6% difference
gets our attention.) NSSE: Looks at first year students and seniors
versus other CCCU institutions and masters schools, and also
compares data with all NSSE participants (5% difference gets our
attention. SSI: Looks at our students compared to private four year
schools. We look at items with a Performance Gap > 1.0, and if
our scores were less than NGM. We also look at the items that are
most important (already listed) as well as the items ranked low in
satisfaction.
- Slide 18
- Distribution Provost council January Faculty Workshop In the
Academy session SGA open student senate Student Life divisional
workshop/retreat Open for other divisional retreats President
received hard data
- Slide 19
- Feedback categories Academic Advising Effectiveness Campus
Climate Campus Life Campus Support Services Concern for the
Individual Instructional Effectiveness Recruitment and Financial
Aid Effectiveness Registration Effectiveness Responsiveness to
Diverse Populations Safety and Security Service Excellence Student
Centeredness Spiritual Growth Decisions Factors Affecting Decision
to Enroll Extent of Student Involvement in the Community
Overall/General
- Slide 20
- Academic Advising Effectiveness
- Slide 21
- Academic Advising Effectiveness assesses the comprehensiveness
of your academic advising program. Academic advisors are evaluated
on the basis of their knowledge, competence and personal concern
for student success, as well as on their approachability.
- Slide 22
- * CSS
- Slide 23
- Slide 24
- Slide 25
- * SSI
- Slide 26
- Slide 27
- Campus Life
- Slide 28
- Campus Life assesses the effectiveness of student life programs
offered by your institution, covering issues ranging from athletics
to residence life. This scale also assesses campus policies and
procedures to determine students perceptions of their rights and
responsibilities.
- Slide 29
- Strengths
- Slide 30
- * CSS
- Slide 31
- Slide 32
- Slide 33
- Slide 34
- Slide 35
- SSI Seniors *** better than ngm
- Slide 36
- * SSI *** better than ngm
- Slide 37
- SSI Seniors *** better than ngm
- Slide 38
- SSI Seniors *** better than ngm
- Slide 39
- Areas for Improvement
- Slide 40
- There was a 13% drop in this physical health rating from
freshman to senior year. (same students) * CSS
- Slide 41
- SSI Seniors *** better than ngm
- Slide 42
- SSI Seniors Same as ngm
- Slide 43
- SSI Seniors Same as ngm
- Slide 44
- SSI Seniors Same as ngm
- Slide 45
- Importance of context Helpful to compare your strengths and
weaknesses with the overall CCCU data
- Slide 46
- What is most important to your students? Item No.Imp.
ScoreDescription 86.63The content of the courses within my major is
valuable. 166.61The instruction in my major is excellent.
686.58Nearly all of the faculty are knowledgeable in their field.
586.54The quality of instruction I receive in most of my classes is
excellent 666.52Tuition paid is a worthwhile investment. 396.52I am
able to experience intellectual growth here. 336.50My academic
advisor is knowledgeable about requirements in my major. 296.50It
is an enjoyable experience to be a student on this campus. 346.49I
am able to register for classes I need with few conflicts.
746.46Being on this campus is contributing to my spiritual growth.
176.46Adequate financial aid is available for most students.
756.44My understanding of God is being strengthened by classroom
and/or campus experiences. 696.43There is a good variety of courses
provided on this campus. 76.43The campus is safe and secure for all
students. 256.41Faculty are fair and unbiased in their treatment of
individual students. CCCU SSI Averages (15 Highest Importance
Scores)
- Slide 47
- Where are your students most satisfied? Item No.Sat.
ScoreDescription 685.91Nearly all of the faculty are knowledgeable
in their field. 515.81This institution has a good reputation within
the community. 785.79This campus provides adequate opportunities
for involvement in ministry. 775.79Males and females are treated
with equal respect on this campus. 725.78On the whole, the campus
is well-maintained. 395.77I am able to experience intellectual
growth here. 25.75The campus staff are caring and helpful. 65.73My
academic advisor is approachable. 455.70Students are made to feel
welcome on this campus. 35.69Faculty care about me as an
individual. 655.69Faculty are usually available after class and
during office hours. 295.67It is an enjoyable experience to be a
student on this campus. 335.66My academic advisor is knowledgeable
about requirements in my major. 755.66My understanding of God is
being strengthened by classroom and/or campus experiences. 75.65The
campus is safe and secure for all students. 415.65There is a
commitment to academic excellence on this campus. CCCU SSI Averages
(15 Highest Importance Scores)
- Slide 48
- Where are your students least satisfied? CCCU SSI Averages (15
Lowest Satisfaction Scores) Item No.Sat. ScoreDescription 213.66The
amount of student parking space on campus is adequate. 384.32There
is an adequate selection of food available in the cafeteria.
424.61There are a sufficient number of weekend activities for
students. 174.68Adequate financial aid is available for most
students. 114.70Billing policies are reasonable. 714.72Channels for
expressing student complaints are readily available. 574.74I seldom
get the run-around when seeking information on campus.
734.76Student activities fees are put to good use. 404.77Residence
hall regulations are reasonable. 244.82The intercollegiate athletic
programs contribute to a strong sense of school spirit.
364.88Security staff respond quickly in emergencies. 234.90Living
conditions in the residence halls are comfortable (adequate space,
lighting, heat, air conditioning, telephones, etc. 674.95Freedom of
expression is protected on campus. 664.95Tuition paid is a
worthwhile investment. 194.96My academic advisor helps me set goals
to work toward.
- Slide 49
- Where are your students expectations not being met? CCCU SSI
Averages (15 Largest Performance Gaps) Item No.Gap ScoreDescription
212.37The amount of student parking space on campus is adequate.
381.79There is an adequate selection of food available in the
cafeteria. 171.78Adequate financial aid is available for most
students. 661.57Tuition paid is a worthwhile investment. 341.50I am
able to register for classes I need with few conflicts.
231.44Living conditions in the residence halls are comfortable
(adequate space, lighting, heat, air conditioning, telephones,
etc.) 111.37Billing policies are reasonable. 571.37I seldom get the
run-around when seeking information on this campus. 361.37Security
staff respond quickly in emergencies. 731.32Student activities fees
are put to good use. 401.28Residence halls regulations are
reasonable. 711.24Channels for expressing student complaints are
readily available. 121.23Financial aid awards are announced to
students in time to be helpful in college planning. 51.21Financial
aid counselors are helpful. 691.18There is a good variety of
courses provided on this campus.
- Slide 50
- Institutional Strengths The content of the courses within my
major is valuable. 6.63/5.61 The instruction in my major field is
excellent. 6.61/5.63 Nearly all of the faculty are knowledgeable in
their field. 6.58/5.91 The quality of instruction I receive in most
of my classes is excellent. 6.54/5.57 I am able to experience
intellectual growth here. 6.52/5.77 My academic advisor is
knowledgeable about requirements in my major. 6.50/5.66 It is an
enjoyable experience to be a student on this campus. 6.50/5.66
Being on this campus is contributing to my spiritual growth.
6.46/5.57 My understanding of God is being strengthened by
classroom and/or campus experiences. 6.44/5.66 The campus is safe
and secure for all students. 6.43/5.65
- Slide 51
- Institutional Strengths (contd) My academic advisor is
approachable. 6.40/5.73 There is a commitment to academic
excellence on this campus. 6.40/5.65 The campus staff are caring
and helpful. 6.39/5.75 This institution shows concern for students
as individuals. 6.38/5.61 Students are made to feel welcome on this
campus. 6.36/5.70 Males and females are treated with equal respect
on this campus. 6.35/5.79 Given where I am spiritually right now,
this campus is a good fit for me. Faculty are usually available
after class and during office hours. 6.37/5.69 Faculty care about
me as an individual. 6.30/5.69 This institution has a good
reputation with the community. 6.23/5.84 On the whole, the campus
is well-maintained. 6.22/5.78 This campus provides adequate
opportunities for involvement in ministry. 6.12/5.79
- Slide 52
- Priorities for Intervention Tuition paid is a worthwhile
investment. 6.52/4.95 I am able to register for classes I need with
few conflicts. 6.49/4.99 Adequate financial aid is available for
most students. 6.46/4.68 Living conditions in the residence halls
are comfortable (adequate space, lighting, heat, air conditioning,
telephones, etc.) 6.34/4.90 Security staff respond quickly in
emergencies. 6.29/4.88 Financial aid awards are announced to
students in time to be helpful in college planning. 6.24/5.02
Faculty provide timely feedback about student progress in a course.
6.23/5.08 Financial aid counselors are helpful 6.21/5.02 There are
adequate services to help me decide upon a career. 6.12/5.12 I
seldom get the run-around when seeking information on this campus.
6.12/4.74 Library resources and services are adequate. 6.11/5.13
How satisfied are you that this campus demonstrates a commitment to
meeting the needs of evening students? 4.98
- Slide 53
- Matrix Union University Student Satisfaction Index Matrix for
Prioritizing Action Very Important Priorities for
InterventionInstitutional Strengths 17. Adequate financial aid is
available for most students. 79. Given where I am spiritually right
now, this campus is a good "fit" for me. 23. Living conditions in
the residence halls are comfortable (adequate space, lighting,
heat, air, etc.) 75. My understanding of God is being strengthened
by classroom and/or campus experiences. 66. Tuition paid is a
worthwhile investment. 74. Being on this campus is contributing to
my spiritual growth. 68. Nearly all of the faculty are
knowledgeable in their field. 29. It is an enjoyable experience to
be a student on this campus. 39. I am able to experience
intellectual growth here. Very DissatisfiedAreas that have Low
Status for StudentsConsider Redirection of ResourcesSatisfied 42.
There are a sufficient number of weekend activities for students.
9. A variety of intramural activities are offered. 24. The
intercollegiate athletic programs contribute to a strong sense of
school spirit. 13. Library staff are helpful and approachable. Very
Unimportant
- Slide 54
- How We Used Data Strategic Planning Union 2010 Eight Priorities
with Goals, Outcomes, Assessment Techniques
- Slide 55
- How We Used Data: Strategic Planning Assessment Result SSI
Seniors: Library Resources and services are adequate (weakness at
**) Strategic Plan response Priority 3: Cultivate a climate of
excellence and innovation based on our distinctive vision and
mission with continuous and measurable improvement, in all aspects
of University life. (Student Services Committee) The University
will develop its library and computing resources and services to
meet the scholarship needs of the entire learning community,
consistent with the Technology Master Plan. (Associate Vice
Presidents of Academic Resources and Information Services) The
University will develop library physical facilities on both the
Jackson and Germantown campuses which will serve as academic
commons of learning. (Associate Vice President of Academic
Resources)
- Slide 56
- How We Used Data: Strategic Planning Assessment Result CSS: 62%
of our seniors did not have a course that included community
service/service learning since entering college compare to other
religious (39%) and all private (46%). Strategic Plan response
Priority 3: Cultivate a climate of excellence and innovation based
on our distinctive vision and mission with continuous and
measurable improvement, in all aspects of University life. (Student
Services Committee) The University will focus on student learning
through additional opportunities in 1) international and
intercultural experiences, 2) service-learning courses, and 3)
excellence in the classroom through 21st century pedagogical
strategies such as web enhanced learning. (Provost/Academic
Deans/Quality Enhancement Planning Team)
- Slide 57
- Other uses of data Food Services improvements move to a block
meal plan Bookstore manager now attends national merchandise
meeting Marketing team Financial Aid hired additional staff
- Slide 58
- Who else requested data? Examples include: Minority Task Team
First year experience task team Bookstore
- Slide 59
- Discussion Who are your audiences? How many times do you
present it? How are the results tailored for each audience? Does
your university have a limit to how many issues you tackle based on
the data? Pros/cons? How accessible is your data on your campus?
(e.g. website, hard copies, etc.)
- Slide 60
- Thank you