Educating Ourselves through Program Assessment – Continuously Improving our Service to Students...
-
Upload
oliver-jacobs -
Category
Documents
-
view
213 -
download
0
Transcript of Educating Ourselves through Program Assessment – Continuously Improving our Service to Students...
![Page 1: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/1.jpg)
Educating Ourselves through Program Assessment –
Continuously Improving our Service to Students Dr. Ann Groves Lloyd
Associate DeanStudent Academic Affairs
College of Letters & ScienceUniversity of Wisconsin-Madison
![Page 2: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/2.jpg)
Snapshot of Student Academic Affairs
Associate Dean
Deputy Assoc. Dean for Advising Deputy Assoc. Dean for Policy Deputy Assoc. Dean for Pathways
CCASCareer Services
FIGSLSACAAP
Assistant DeansFor Policy
American IndianAfrican AmericanSoutheast Asian
Chicano/aUndergrad Research Scholars
Summer Collegiate ExperienceWriting Fellows
![Page 3: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/3.jpg)
Letters & Science Student Academic Affairs Demographics• Approximately 18 units• Size varies from 15 staff members to one• Some units have assigned populations• Some units have a campus-wide mission• College encompasses nearly 17,000
students• Familiarity and use of tracking and
evaluations tools varied widely
![Page 4: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/4.jpg)
SAA – Vision for Assessment
• Each unit would assess program quality, productivity, need, and demand
• Continuously improve quality of programs offered by capturing voices of students and other constituents
• Enhance program planning and budgeting• Establish baseline data to explore longitudinal
trends and patterns• Satisfy institutional/state/national reporting and
review requirements
![Page 5: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/5.jpg)
Charge to Assessment Committee
• Develop instruments to measure student perceptions and overall quality of student experience
• Develop standardized expectations for unit directors/coordinators
• Provide staff with a practical, helpful tool• Begin systematizing data collection to acquire
retention and longitudinal data• Pilot project in place for 2002-03 academic year
![Page 6: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/6.jpg)
L&S SAA Assessment Committee
• Made up of a cross-section of staff in SAA
• Determined that a handbook would be the best route to go
• Developed timeline and content areas
• Split into teams to work on each section
![Page 7: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/7.jpg)
Factors for Success
• Involved those who will implement and use the Handbook
• Process involved on-going negotiation and compromise
• Wanted to take the assessment picture down to very small and practical increments
• Took existing systems into account• Developed flexible, adaptable plan• Encourage risk-taking
![Page 8: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/8.jpg)
Foundation
• Differentiates between tracking for annual reporting and collecting data for program improvements (Service Quality versus Student Learning Outcomes)
• Does not ask unit directors/coordinators to get into determining retention rates
• Next steps will be defining and measuring learning outcomes
![Page 9: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/9.jpg)
Handbook Introduction
• Two-hour session with unit coordinators/directors in May 2002
• Associate Dean discussed her vision for assessment for our division of student academic affairs
• Assessment Team walked everyone through the contents of the Handbook
![Page 10: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/10.jpg)
Handbook Contents
• Associate Dean’s vision for Assessment in Student Academic Affairs
• Ethics in assessment• Expectations for unit directors regarding
collecting usage statistics and program improvement data, developing annual reports, and conducting self-studies
• Primer on the basic tools of assessment• A resource section
![Page 11: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/11.jpg)
Ethics in Evaluation
• Ensure ethical treatment of research participantsAvoid harmInformed consentFreedom from coercion
• Ensure ethical collection of data• Ensure ethical interpretation and
presentation of data
![Page 12: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/12.jpg)
Primer on Statistical Concepts
• Includes brief discussion and cautions regarding interpretation of data
• Also addresses managing data, including issues of confidentiality and security
• Has changed significantly since evolution of on-line survey tools such as Zoomerang.com and Survey Monkey
![Page 13: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/13.jpg)
On-Going Assessments
• Capturing student names/IDs in ISIS and/or stand-alone databases (discouraged)
• Collect demographic information about program participants
• Capture information about advising appointments• Establish systems to regularly assess student satisfaction
with services• Obtain feedback for program improvement• Systematically gather data to assess retention and
persistence (done at the divisional and college level)
![Page 14: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/14.jpg)
Ways to Collect On-Going Data
• Phone calls• Emails• Advising contacts• Resource use• Workshop attendance• Events in general• Website hits• Number of students turned away from services
![Page 15: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/15.jpg)
Annual Reports
• Each SAA unit will submit an assessment plan in September each year.
• Each unit will submit an annual report due in January each year.
• Responsibility of unit director/coordinator• Snapshot of previous year’s
accomplishments and a look toward future initiatives and challenges
![Page 16: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/16.jpg)
Contents of Annual Report
• Table of Contents
• Executive Summary
• Mission Statement and Core Values
• Strategic Plan if applicable
• Overview of past year
• Future goals and initiatives
![Page 17: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/17.jpg)
Self Studies
• Ten-year cycle for self studies
• Dean’s office has determined expectations for self studies
• Additional details are outlined in the handbook
![Page 18: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/18.jpg)
Assessment 101
• Selecting Assessment Tools
• Statistical Concepts
• Interpretation
• Managing Data
• Surveys
• Interviews
![Page 19: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/19.jpg)
Selecting Assessment Tools
• Quantitative
• Quantitative
• Advantages and disadvantages of each
• Encourages multiple measures and methods
![Page 20: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/20.jpg)
Surveys
• Gathering demographic information
• Types of questions
• Ways to gather survey data
• Tips
• Survey Kit by Arlene Fink (Sage Publications)
![Page 21: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/21.jpg)
Interviews
• Role of human subjects committee
• FAQ prior to interviews
• Focus groups versus in-depth individual interviews
• Analyzing interviews
![Page 22: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/22.jpg)
Resource Section
• List of campus resources• List of web-based resources• Books on assessment• Article by Upcraft & Schuh from About Campus
(1998)• AAHE – 9 Principles of Good Practice for
Assessing Student Learning• Handbook can be found at:
http://wiscinfo.doit.wisc.edu/obpa/Assessment/SAA_Assessment.htm
![Page 23: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/23.jpg)
Where are we now?
• Got the tracking and evaluation systems established in all units
• Have had two sets of assessment plans and annual reports turned in
• Obtained feedback from unit directors/coordinators and began updating handbook
• SAA has spend the past nine months going through its own assessment and evaluation process
![Page 24: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/24.jpg)
Final Thoughts
• Assessment can be intimidating, but knowing where you are is essential to determining where you want to go!
• Just pick someplace to begin – don’t get overwhelmed by the big picture
• Develop a culture that embraces continuous improvement and inclusion of student voices in the planning
![Page 25: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/25.jpg)
Stretch and Take a Break!
Discuss with your colleagues your biggest challenge regarding
assessment
![Page 26: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/26.jpg)
Self-Evaluation of Assessment Practices
• Tool for evaluating “where you’re at” regarding assessment in your unit/department
• Identify strengths upon which to build
• Identify aspects to improve or enhance
• Catalyst for important conversation and action
![Page 27: Educating Ourselves through Program Assessment – Continuously Improving our Service to Students Dr. Ann Groves Lloyd Associate Dean Student Academic Affairs.](https://reader034.fdocuments.in/reader034/viewer/2022042821/56649ddb5503460f94ad285e/html5/thumbnails/27.jpg)
Composite GraphUnit Self-Evaluation of Assessment Processes
UW-Madison
A Role Model 4
Very Effective 3
Just Beginning 1
Not Happening 0
DRAFT 12.11.02
7 8
Link
ed to
Dec
isio
nM
akin
g
6
Con
tinuo
us
5
Res
ourc
es
4
Impl
emen
ted
Reg
ular
ly
3
Pro
cess
es p
art o
fA
sses
smen
t Pla
n
2
Eva
luat
ion
Pro
cess
es E
xist
1
BasicallyEffective 2