Assessment Cadre #3: “Assess How? Designing Assessments to Do What You Want”
What Are Outcomes and Why Should I Want to Assess Them?
description
Transcript of What Are Outcomes and Why Should I Want to Assess Them?
1
What Are Outcomes and Why Should I Want to Assess Them?Dr. Jim Wilson, AVP Academic Affairs Wilmington University
Dana Santoro, Data Analyst – Institutional ResearchWilmington University
2
Class of 2011
What Berlin wall? They never “rolled down” a car window They have grown up with bottled water Wal-mart has always been a larger retailer than Sears and has
always employed more workers than GM High definition televisions have always been available China has always been more interested in making money than
in re-education They never saw Johnny Carson live on television Food packaging has always included nutritional labeling
Courtesy of Beloit College Public Affairs
3
Things Have Changed!
Higher Education Research Institute (HERI) UCLA
40yr Trends Presentation
4
Things Have Changed!
Item 1970 1995 Today
Postage Stamp $0.06 $0.32 $0.39
Bread $0.24 $1.40 $1.39
Milk $1.32 $2.59 $4.29
Gas $0.40 $1.35 $2.60
Car $2,500.00 $12,800.00 $27,800.00
Income $6,933.00 $19,717.00 $43,200.00
House $40,000.00 $111,000.00 $264,000.00
5
Education: Costs
Wilmington University Tuition30 credits, Undergraduate
1971
Full year tuition: $1,500
TodayFull year tuition: $8,400 (New Castle, UG)
6
Education: Some Changes
Courtesy of Beloit College Public Affairs
Today Student-centered learning The “Net Generation” Distance Learning, Online databases
1967 Teacher-centered learning Classroom, Books, Library card catalogs Al Gore was only 20 years old
7
Education: Student Learning
A new “paradigm for higher education”(Suskie, 2006.)
Learning CenteredAssessment feedback helps faculty understand
What is and is not working effectively How to improve the curriculum as well as teaching and
learning strategies
8
Wilmington University Mission
9
What is Assessment of Student Learning?
“Good assessment must begin in the classroom and end there.” (Wolvord & Anderson, 1998, p. 149)
10
What is Assessment of Student Learning?
“Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations.” (Suskie, 2006, p.3.)
11
What is Assessment of Student Learning?
“Assessment is first and foremost a tool for faculty members to use as they do their very best to teach their students well.” (Middle States, 2003)
12
Assessment Is A Friend Who…
Honestly tells us how we are doing Helps to clarify what we are doing and why Helps us to make informed decisions Improves student learning Helps us to improve Brags about how well we are doing
13
Assessment Could Be a Foe!
Federal requirements Regional Accreditation Legislators Parents and Community groups Students Professional organizations
14
SummaryWhat is Assessment of Student Learning?
Process of gathering & evaluating informationQuantitative and qualitativeFrom multiple sources
To enhance student learning
15
Student Outcomes Assessment Framework
16
Three Assessment Areas
Student Learning
Teaching Effectiveness
Student satisfaction with their academic experience
17
Graduation Competencies
Division / Program Mission
Professional Standards --------
Benchmarks
PROGRAM COMPETENCIES
CECRAMRubrics, Test Embedded Questions, Clinical Evaluations
Data Collection, Analysis, Reporting, Decisions,Actions
Benchmarks Benchmarks
License & Certification Exams
Major Field Exams
• IDEA Results• Faculty Evaluations• GPA Reports
• Enrollment Data •ACT Survey Results
Assessment of Student Satisfaction
Course Work
Assessment of Teaching Effectiveness
Wilmington UniversityACADEMIC AFFAIRS ASSESSMENT PLAN
COLLEGE MISSION
College Outcomes
Assessment of Student Learning Outcomes
18
Program Improvement Cycle
19
Language of Assessment
Direct evidenceDirect evidence of student learning indicates
whether or not a student has a command of specific content
Examples: Homework assignments, research papers, rubrics
CECRAM (Course-Embedded Criterion-Referenced Assessment Measures)
20
Indirect EvidenceIndirect evidence of student learning is
correlational – data exists which indicates that students are probably learning.
Examples: Course grades, time spent on service learning or homework
Language of Assessment
21
Formative assessmentOngoing assessment that is intended to improve an
individual student’s performance.Purpose: to improve course content, provide
feedback to faculty.
Language of Assessment
22
Summative assessmentProvides a true gauge of “outcomes” of student
learning.Data is typically gathered near the end of program
completionExamples: course-embedded project/test/portfolio,
clinical evaluations, standardized exams
Language of Assessment
23
WU Language
“Good enough”The process is accurate and truthful (Suskie, 2004, p. 302)
“Closing the loop”Outcomes data has been analyzed and appropriate
changes have been made.
24
Elements of Assessing Learning
1. Articulated expectation of student learning at various levels.
2. A plan that describes student learning assessment activities.
3. Evidence that student learning assessment information is used.
4. Documented use of student learning assessment as a part of institutional assessment.
25
Background Information
Academic Affairs Assessment Plan adopted in 2003
Plan called for evaluation in 2006
26
MSCHE Outcomes Assessment Guidelines
Assessment should be:UsefulCost-effective, simpleReasonably accurate and
truthfulOrganized, systematized and
sustained.
Suskie, L. (2005)
27
Major Changes Related to Outcomes Assessment
Five Year Data Collection and Implementation Cycle
Revision of Graduation Competencies Representative Sampling Formal Reporting Cycle
Five Year Program Review
28
How to maintain momentum?
A 5 year cycle will be utilized
Year 1 – Implement findings from program
review
Year 2 – Data collection
Year 3 – Reflection, take appropriate action and
collect “other kinds” of data
Year 4 – Data collection
Year 5 – Program Review Report
29
What are Examples?(of closing the loop)
Program: B.S., Finance Weakness: Undergraduate Finance majors were
missing some important finance skills Source: ETS Major Field Exam Action
Adjust the content of FIN 305 Add a new requirement, FIN 306
Competency Disciplined Inquiry Finance program competencies
30
What are Examples?(of closing the loop)
Program: All UG Education programs Weakness: Students were unclear of required skills Source: Portfolio review Actions:
Curriculum changes Provide candidates with exemplars Ensure understanding of required skills
Competency Disciplined inquiry
*Standardized exams 31
Program: M.S., Community Counseling Weakness: Appraisal Techniques
Source: CECRAM, CPCE*, NCE* & surveys
Response: Integration of assessment with diagnosis and treatment planning. Psychopathology course will precede the appraisal course
What are Examples?(of closing the loop)
32
Program: B.S., Nursing Weakness: Student teaching projects - measurable
objectives were not reliably clear Source: CECRAM Actions:
Add content and a teaching video to NUR 425 – Community Health Practitioner
Added teaching content to NUR 330 Revised curriculum and developed a new course:
NUR 323 - Nurse as Teacher
What are Examples?(of closing the loop)
33
What is outcomes assessment of What is outcomes assessment of student learning?student learning?
Why should I support the assessment Why should I support the assessment process?process?
Conclusion
34
References Caffo, Betty, (2003). Striving for Excellence. Faculty Senate
Presentation, Wilmington College. Middle States Commission on Higher Education. (2006).
Characteristics of excellence in higher education. Philadelphia: Author.
Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker.
Walvoord, B.E., and Anderson, V.J. (1998). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass.