Using WIDS to Document How Colleges Assess Learning Outcomes.
-
Upload
ralph-lindsey -
Category
Documents
-
view
219 -
download
4
Transcript of Using WIDS to Document How Colleges Assess Learning Outcomes.
Using WIDS Using WIDS to to Document Document How How Colleges Colleges Assess Assess Learning Learning OutcomesOutcomes
Using WIDS Using WIDS to to Document Document How How Colleges Colleges Assess Assess Learning Learning OutcomesOutcomes
Overview
What is assessment? What are the drivers? How does curriculum fit in? How does WIDS help?
“Assessment of student learning is a participatory, iterative process that:
Provides data/information you need on your students’ learning Engages you and others in analyzing and using this
data/information to confirm and improve teaching and learning Produces evidence that students are learning the outcomes
you intended Guides you in making educational and institutional
improvements Evaluates whether changes made improve/impact student
learning, and documents the learning and your efforts.”
NCA Higher Learning Commission
From “Student Learning, Assessment and Accreditation: Criteria and Contexts”, presented at Making a Difference in Student Learning: Assessment as a Core Strategy, a workshop from the Higher Learning Commission, July 26-28, 2006.
Assessment of Learning
Informed judgments about achievement of intended learning outcomes
Based on evidence of what students can do and their capacity to apply what they know (data) at the completion of the learning experience (course, certificate, program, or other credential)
Valid, reliable, and fair
Built into the plan for teaching
Data-driven evaluation that is actively used for continual improvement of teaching and learning
Assessment of Learning
Sound Assessment
ValidValidOutcomes based on standards (industry)Measures intended outcomesMeasures application and critical thinking
ReliableReliablePerformance assessment based on consistent rubrics, scoring guides, and rating scalesConsistent process - each learner is assessed in same way as other learners
FairFairValidReliableLearners informed of expectations up front!Feedback to learners
Summative/Formative Assessment
What are the drivers?
1. Learning & continuous improvement of teaching and learning
2. Accreditation (NCA, AQIP, Industry)
3. Carl Perkins IV
HLC/AQIP asks 5 fundamental questions: How are your stated learning outcomes appropriate to your
mission, programs, students and degrees?
How do you ensure shared responsibility for student learning & assessment of student learning?
What evidence do you have that students achieve your stated learning outcomes?
In what ways do you analyze and use evidence of student learning?
How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
NCA Higher Learning Commission
From “Student Learning, Assessment and Accreditation: Criteria and Contexts”, presented at Making a Difference in Student Learning: Assessment as a Core Strategy, a workshop from the Higher Learning Commission, July 26-28, 2006.
The What: Both secondary and postsecondary must develop a process for measuring technical skills that lead to an industry-recognized credential, or state-recognized credential, or locally-developed credential
The Why: Both secondary and postsecondary education systems must provide, as a separate measure of accountability, the extent of skill development in Career and Technical Education (CTE) programs
The How: Both secondary and postsecondary education must develop valid and reliable assessments
Carl Perkins IV – The New Law
Postsecondary Indicators Technical Skill Attainment Credential, Certificate, or Diploma Student Retention or Transfer Student Placement Nontraditional Participation Nontraditional Completion
Carl Perkins IV – The New Law
Carl Perkins IV – The New Law
WTCS will move toward a balance of silver and gold assessment of learning outcomes.
Gold Standard (ex. NCLEX, ASE, Barber/Cosmetology)
External, third-party assessment that objectively measures student attainment of industry recognized skills upon graduation.
Silver StandardWTCS approved assessment that objectively measures student attainment of industry recognized skills upon graduation.
May be developed and/or implemented by the state system. May be developed and/or implemented locally according to system guidelines
and with industry (advisory committee) approval. Must be valid and reliable.
Bronze Non-assessment related indicators such as indirect measures.
GPA and/or Course Completion Program Completion – Graduation Teacher-developed exams that do not meet silver standard (not externally
approved).
Carl Perkins IV – The New Law
Summary of Standards for Assessment of Technical Skill Attainment
Plan• Establish outcomes• Build assessments• Design learning
DO• Deliver• Assess• Document
CHECK• Analyze assessment
data• Propose
improvements
ACT•Adjust design
•Adjust delivery
•Adjust assessment
Assessm
en
t Pro
cess a
nd
C
urric
ulu
m
The Learning Assessment Design Technology Tool Kit
Tra
cDat
B
lack
boar
d
iWeb
folio
eF
olio
Que
stio
nmar
k
Peo
ple
Sof
t B
anne
r D
atat
el
Assessment Mgt SoftwareAssessment Mgt Software(ex. TracDat)(ex. TracDat)
• Describe Outcomes (Learning/Org)• Build Assessment Plans• Track Continual Improvement
Activities• Document/Report Results
WIDSWIDS•Establish Learning Outcomes•Build Assessments•Design and Plan Learning
Enterprise SystemEnterprise System(ex. PeopleSoft DataTel or (ex. PeopleSoft DataTel or
Banner)Banner)•Manage Student Records•Manage Enrollment/Scheduling•Manage Budget•Etc.
Assessment SoftwareAssessment Software(ex. Questionmark)(ex. Questionmark)
• Author Q & A Assessments and Surveys
• Deliver Assessments/Surveys• Record Learners Performance
(In terms of outcomes)
Electronic PortfolioElectronic Portfolio(ex. iWebfolio/MSNCU eFolio)(ex. iWebfolio/MSNCU eFolio)
• Present evidence of individual learning/professional experience
• Document connection between artifacts and outcomes
• Owned by the student
WIDS Learning Design System populates assessment, electronic portfolio, enterprise, and assessment systems with learning outcomes and performance standards.
Assessment Management Software provides feedback loop for continual improvement of learning design using WIDS.
Web-based Electronic Portfolio System presents evidence of individual’s learning and professional experience.
Learning Outcom
es
Learning Outcomes
Lea
rnin
g O
utco
mes
Cur
ricul
um D
ocum
ents
Learning Outcomes
Rubrics
Student Learning
Docum
entationC
ontin
uous
Im
pro
vem
ent
Fee
dbac
k
Student Data
What do they need to be able to achieve?
Who are the learners?
How will we know and show when they’ve achieved it?
How will they get there?
3 of 4 components answer assessment questions
WHO
WHAT
WHEN
HOW
WIDS Model
Linked Competencies
(performance assessments)
Exit Learning Outcomes
Program Outcomes(summative assessment)
Core Abilities
(summative assessment)
Gen Ed Outcomes(summative assessment)
Job Task AnalysisJob Task Analysis(DACUM)(DACUM)
Job Task AnalysisJob Task Analysis(DACUM)(DACUM)
External StandardsExternal Standards(Research-based)(Research-based)
External StandardsExternal Standards(Research-based)(Research-based)
Drives Learning Design
Assesses Learning ResultsIntegrating the OutcomesIntegrating the Outcomes
Linked Competencies
(performance assessments)
Linked Competencies
(performance assessments)
Formative assessment of program outcome
Summative assessment of learning outcomes
External
Program-Level
Course-Level
Drives Learning Design
Assesses Learning Results
External
Program-Level
Course-Level
Dental Hygiene ExampleDental Hygiene ExampleDental Hygiene ExampleDental Hygiene Example
Graduates are competent in applying ethical, legal and regulatory concepts to the provision and/or support of oral health care services.
Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies
Respond to a request to perform a task that is not legally permitted to be delegated to a dental hygienist.
ADA Dental Hygiene Education Standard
Dental Hygiene Program Outcome
Dental Hygiene Competency Formative assessment of program outcome
Summative assessment of Learning outcomes
Challenges
How do we communicate WTCS long-standing commitment to industry input and standards?
How do we continue WTCS commitment to performance assessment whenever possible (versus testing?)
How do we ensure that we use quality assessments? (That we don’t regress to testing regurgitation of information?)
How do we ensure that we are doing what our model and assessment plans imply?
1. Create Program Design files for all programs
2. Develop Exit Learning Outcomes Core Abilities (mission level) Program Outcomes (discipline level) General Education Outcomes (Optional)
3. Determine summative assessments
4. Define criteria for summative assessments
5. Link courses to exit learning outcomes
Using WIDS Program Design
Design. Document. Align
6. Create Performance Assessment Tasks (with scoring guides and rubrics) to support summative assessment Embed in program courses, using the
program file as the master file Create a separate Summative
Assessment course file for program
Using WIDS Learning Design
Design. Document. Align
7. Generate Analyzer reports Learning Outcomes Matrix Program Course List Matrix Assessment Task Outcomes Matrix
Using WIDS Analyzer
Design. Document. Align
Learning Outcomes Matrix
Outcomes Courses
Documents where outcomes are addressed
Summative Assessments
This report requires the following: WIDS Program File - program outcomes; direct measures and WIDS Course Files - course title; program outcomes (linked to competencies-optional); and Performance Assessment Tasks that target or link to program outcomes.
Cla
ssifi
catio
n C
hart
Res
each
Stu
dy
Com
mun
ity D
enta
l Hea
lth P
ropo
sal
Com
mun
ity H
ealth
Den
tal P
rogr
am S
ervi
ce P
roje
ct
Ref
lect
ion
Inf
ectio
n C
lass
ifica
tion
Ana
lysi
s
HA
ZC
OM
Wor
kshe
et a
nd L
abel
s
Per
sona
l and
Pat
ient
Pro
tect
ion
Ski
ll D
emo
Tre
atm
ent
Roo
m R
epro
cess
ing
Sim
ulat
ion
Ins
trum
ent
Re
proc
essi
ng D
emo
Was
te M
anag
emen
t E
xam
Pat
ient
His
tory
Rol
e P
lay
Vita
l Sig
ns S
kill
Dem
onst
ratio
n
Em
erge
ncy
Eq
uipm
ent
Mai
nten
ance
Per
form
ance
Tes
t
Che
mic
al E
me
rgen
cy S
imul
atio
n an
d In
cid
ent
Rep
ort
Med
ical
Em
erge
ncy
Sim
ulat
ion
and
Inci
dent
Rep
ort
Inf
ectio
us E
xpos
ure
Sim
ulat
ion
and
Inci
dent
Rep
ort
Law
s an
d E
thic
s C
ase
Stu
dy A
ssig
nmen
t
Pro
fess
iona
l Dev
elop
men
t P
lan
Den
tal T
eam
Cha
rt
Con
fiden
tialit
y C
ase
Stu
dy
Inf
orm
ed C
onse
nt R
ole
Pla
y
Pro
fess
iona
l App
eara
nce
Che
cklis
t
Clin
ical
Eva
lua
tion
Pat
ient
Den
tal H
ygie
ne C
are
Pla
ns
Cas
e M
anag
em
ent
Eva
luat
ion
Off
ice
Sim
ulat
ion
Ethics and Professionalism: Incorporate into dental hygiene practice professional laws, regulations and policies established by the licensing state and regulatory agencies
L L L L L L L L T L L L
Ethics and Professionalism: Model dental hygiene professional code of ethics in a rapidly changing environment
L L L L L T L
Ethics and Professionalism: Pursue lifelong professional growth and development through self-directed learning, participation in professional organizations, and continuing education
Promoting Oral Health: Counsel clients/patients to reduce health risks L T L
Assessment Task Outcomes Matrix - Dental Hygiene Sample
Program Outcomes
Community Dental Health
Dental Health Safety Dental Hygiene Ethics and Professionalism
Dental Hygiene Process IV
Assessment Task Outcomes Matrix
Performance Assessment Tasks By Course
Shows the assessment and the link/target
Maximize the potential of WIDS to support assessment planning:
Create/maintain Program Outcome Summaries for all new and existing programs.
Develop/maintain Course Outcome Summaries for all courses.
Design Performance Assessment Tasks for course level (formative) and beyond-the-course (summative) assessments.
Use WIDS Analyzer to build Learning Outcome Matrices that document how outcomes are linked with programs and courses.
Present WIDS documentation as a central feature in accreditation.