Data assimilation Derek Karssenberg, Faculty of Geosciences, Utrecht University.
DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services
description
Transcript of DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services
![Page 1: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/1.jpg)
![Page 2: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/2.jpg)
IT TAKES TWO:A SYSTEMATIC, EFFECTIVE, & PRACTICAL PROCESS FOR INTEGRATING ASSESSMENT & PROGRAM REVIEW
DEREK HERRMANN & RYAN SMITHIllinois State University
University Assessment Services
Association for Institutional Research, New Orleans, LA, 6-5-2012
![Page 3: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/3.jpg)
ABOUT ISU
• DRU• Total enrollment: 21,080• 35.5% of new students were
transfer students• 82.7% of degrees granted were
baccalaureate degrees
![Page 4: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/4.jpg)
ABOUT ISU
• Retention rate: 85.1%• Graduation rate: 69.8%• Student to Faculty ratio: 19:1• Organizational structure
– UAS is part of Academic Affairs– IR is part of Finance and Planning
![Page 5: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/5.jpg)
ABOUT ISU
• Illinois Board of Higher Education (IBHE)– All academic degree programs
required to have assessment plans on file
– All academic programs required to undergo a periodic and systematic program review
![Page 6: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/6.jpg)
THE PROBLEM
1. Shifting views of higher education
“Higher education is one of the greatest hopes for intellectual and civic progress in this country. Yet, it is seen by many as part of the problem rather than the solution.”
- C. Palomba & T. Banta, Assessment Essentials, 1999
![Page 7: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/7.jpg)
THE PROBLEM
2. Increasing Accountability
Higher Learning Commission
In the 11 pages of Criteria in the proposed pathways model, assessment is mentioned 19 times, with Teaching and Assessing for Learning as its own criteria.
IBHE
- C. Palomba & T. Banta, Assessment Essentials, 1999
![Page 8: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/8.jpg)
THE PROBLEM3. Changing
World
![Page 9: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/9.jpg)
THE PROBLEM
4. Assumptions about evaluation and assessment
“Assessment projects are simply getting in the way of good teaching… We spend far too much time and energy in trying to assess learning, an unattainable goal….I love teaching, however. It is a great pleasure to share my enthusiasm for (my discipline) and I absolutely love the chance to pick the minds of my bright students.”
- B. Moore, Washington State Board of Community & Technical Colleges, Using Assessment to Improve Instruction (video),
http://www.youtube.com/watch?v=BZ3USs16J3Y
![Page 10: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/10.jpg)
THE PROBLEM
![Page 11: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/11.jpg)
HOW COLLEGES ARE ORGANIZED
Board
President
VP
Dean
Director
Staff
VP
Dean
Director
Staff
VP
Dean
Director
Staff
![Page 12: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/12.jpg)
HOW THEY ARE IN REALITY
Random People
Random People
Board
President
Union Leaders
Planning Person
Political Hire
Students
VP
Academic Dean
I.R. Person
Faculty
Provost
Assess. Dean
Director
Adjunct Faculty
![Page 13: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/13.jpg)
BASIC ASSESSMENT MODEL
Establish Learning
Outcomes
Gather Evidence
Analysis
Make a Decision/Change
Mission/Goals
Accreditation, IBHE/ICCB, State of Illinois, Strategic plans, Tech. plans, enrollment mgmt. plans, facilities plans, financial plans, environmental scan data, IR data
Governance, Budgets, Resources, Boards, Politics, Org. culture, good and bad bosses, etc.
Education, training, & experience, life: health, spouses/partners, kids, family, friends, pets, etc.
![Page 14: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/14.jpg)
Establish learning goal
Students will be able to explain the
impact of leadership on their
development
Provide learning opportunities
Mentorship program with
university leaders
Assess learning (gather evidence)Students keep a reflective journal
Coded and analyzed by leadership staff
Use the results (decision)
Students report a lack of contact with
mentors
Mission
Process is the outcome
Failure of decision-making or making it meaningful
FAILURE OF DECISION-MAKING
![Page 15: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/15.jpg)
ADVANTAGES
![Page 16: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/16.jpg)
THE PROBLEM
How does one design processes that…
- Engage faculty and staff?- Can be used for true improvement and
change in today’s society?- Make information meaningful for decision-
making?- Meets accountability standards?- Are genuine, not just transparent?
![Page 17: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/17.jpg)
DISCLAIMER
![Page 18: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/18.jpg)
AT ISU
• 8-Year cycle that includes– Process for the Review of Academic
Assessment Plans (PRAAP)– Self-study– Program Review– Annual Update
• Many across campus involved in these processes
![Page 19: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/19.jpg)
AT ISU
• Year 1 – PRAAP– Chairs/program directors asked to
provide current assessment plans– Assessment Advisory Council
(AAC) reviews them using a rubric and completes a feedback form
![Page 20: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/20.jpg)
THE PRAAP RUBRICUndeveloped Developing Established Exemplary
Program goals and intended student learning outcomes
Systematic assessment of student learning
Feedback from key stakeholders
Use of results/”Closing the Loop”
![Page 21: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/21.jpg)
AT ISU
• Year 1 – PRAAP– Chairs asked to provide current
assessment plans– Assessment Advisory Council
(AAC) reviews them using a rubric and completes a feedback form
– UAS staff meets with chairs/program directors to discuss the feedback
![Page 22: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/22.jpg)
AT ISU
• Year 2 – Self-Study– Faculty and staff complete self-
study for program review– Assessment plan is used to collect
data
![Page 23: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/23.jpg)
AT ISU
• Year 3 – Program Review– Academic Planning Committee
examines Program Review documents
– Often programs submit follow-up reports based on assessment plans
![Page 24: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/24.jpg)
AT ISU
• Years 4-8 – Annual Update– Chairs asked to complete a short
form concerning their assessment activities
– Questions based on Higher Learning Commission’s (HLC) Fundamental Questions for Conversations Student Learning
![Page 25: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/25.jpg)
THE ANNUAL UPDATE FORM• Please provide a brief summary
of the previous year’s data which was collected related to your outcomes.
• What efforts were used during the past year to involve faculty and constituencies in assessment processes?
![Page 26: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/26.jpg)
THE ANNUAL UPDATE FORM• Please outline what action plans
have been developed based upon the assessment process.
• Please provide any additional comments or information.
• Have changes or improvements been made to the Academic Assessment Plan in the previous year?
![Page 27: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/27.jpg)
AT ISU
• Years 4-8 – Annual Update– If applicable, an updated
assessment plan is sent to UAS– Collection of forms provided to
programs to be used in the next cycle
![Page 28: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/28.jpg)
ADVANTAGES
1. Assessment is a separate from program review, but still leads the process.
- A culture of inquiry over a culture of evidence.- One is about performance, the other about
learning.
![Page 29: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/29.jpg)
ADVANTAGES
What does it mean?
Where do we go?
How well are we doing?
- Jane Chapman, Heartland Community College, It’s Eleven O’Clock: Do You Know Where Your Students Are?, 15th Annual Comm. College Assessment Fair, Bloomington, IL, 2011
A focus on what results mean, instead of what they are.
![Page 30: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/30.jpg)
ADVANTAGES
2. Value people’s time, resources, energy, history, thoughts, and opinions.
- Attitudes, beliefs, and engagement with assessment are just as important as processes.
- Faculty have control through the self-study and review process.
- Assessment & program review aren’t used to micro-manage or fix a problem that is not perceived to exist.
- Ideas, people, and dialog lead – technology follows.
![Page 32: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/32.jpg)
ADVANTAGES
3. There is consensus and agreement about the process, and it is used to make decisions.
- It’s more important to be genuine, rather than transparent.
- Assessment and program review are never goals – they’re processes.
![Page 33: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/33.jpg)
CONTINUOUS IMPROVEMENT
Nothing is ever perfect, but it can be healthy.
Incremental changes to rubric.
![Page 34: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/34.jpg)
QUESTIONS?COMMENTS?
![Page 35: DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services](https://reader036.fdocuments.in/reader036/viewer/2022062517/56813e5d550346895da85ef7/html5/thumbnails/35.jpg)
FOR MORE INFORMATIONhttp://assessment.illinoisstate.edu/program/resources.shtml
http://provost.illinoisstate.edu/academic/program-review.shtml