Navigating Assessment Malissa Martin, Ed.D., ATC, CSCS Director, Athletic Training Education...
-
Upload
maximillian-dorsey -
Category
Documents
-
view
216 -
download
0
Transcript of Navigating Assessment Malissa Martin, Ed.D., ATC, CSCS Director, Athletic Training Education...
Navigating Navigating AssessmentAssessmentNavigating Navigating AssessmentAssessment
Malissa Martin, Ed.D., ATC, CSCSMalissa Martin, Ed.D., ATC, CSCSDirector, Athletic Training EducationDirector, Athletic Training Education
Department of Health SciencesDepartment of Health SciencesCollege of Mount St JosephCollege of Mount St Joseph
Cincinnati, OHCincinnati, OH
Outcomes• Review components of a Program
Assessment Plan• Develop and analyze components
of an assessment plan• Describe ways evidence is used to
enhance student learning……… ”Close the Loop”
Agenda
• 2:00-2:15 Review Components of Program• Assessment Plan
• 2:15-2:30 Learning Outcomes Indicators• 2:30-2:45 Performance Indicators and Measures • 2:45-3:00 Benchmarks• 3:00-3:15 Closing the Loop• 3:15-3:30 Summary
Assessment is Learning…….
• Students• Curriculum (didactic and clinical)• Institutional fit• Assessment process
Accreditation General Program
Assessment Questions
• How are your LOs appropriate to your mission, program, degree?
• What evidence do you have that your students are achieving your LOs?
• In what ways to do you analyze student learning?
• How do you evaluate and improve the effectiveness of your assessment of student learning?
• Where do you assess your LOs?
Assessment
• “Systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development”
• Marchese, 1987
Programmatic Assessment
• Provides information about groups of students
• Determines student’s ability to integrate individual courses/clinical experiences into a coherent whole
• “Academic Introspection” Denney, 1996
Characteristics of an Assessment Plan
• All LOs are addressed and assessed• Multiple assessment measures• Efficient procedures• Describe people, committees, process
involved• Plan “closes the loop”
Ball State College of BusinessBanta TW & Palomba CA (1999) Assessment Essentials Josey-Bass
Meaning of Assessment
• What should your graduates know, be able to do, and value?
• Have graduates acquired learning?• What are the contributions of the
program to student growth?• How can student learning be improved?
• Hutchings & Marchese, 1990
Common Language• See handout…..
Guided Outcome Questions….
• Where do students start?• What do they encounter along the
way?• Where do they end up?
“The Ideal Graduate”
Components of a Program Assessment
Plan
1. Expected Outcomes (learning outcomes)
SummativeWhat will students be able to do upon
completion of the program?
FormativeWhat will students be able to do along the way
Nichols Model
• 5 Column Assessment Report Model
• SACS
Bloom’s and Assessment Plan
• Varying degrees of learning
• Produce learning vs providing instruction (Barr and Tagg, 1995)
Group Exercise
• At your table, develop 2 program learning outcomes (LOs).
• Excluding– Students will successfully pass the
BOC examination and gain ATC credential
Components…
Where are outcomes addressed?
Courses (list)Clinical programService learning
Components…2. Criteria to assess outcomes
Performance Indicators
How will students meet LO-type of product
When will students meet LO-formative vs summative
Where will students meet LOdidacticclinical extra-curricular experience (service learning)
Components…
3. Measures (how will data be collected)
Portfolio SimulationsPerformance Case studiesWriting samples Poster presentationsOral presentations InterviewsTests Clinical evaluationsSimulationsFocus groupsSurveys
Qualitative and quantitative measuresDirect vs Indirect measures
Group Exercise
• Using your 2 Los, develop– Performance indicators for each
• How will students meet LO– Product
• When will students meet LO– Formative vs summative
• Where in curriculum will students meet LO– Course -Clinical Exp -Post-Grad
– Identify measures used to assess the LO
Performance Assessment
• Authentic assessment• Linked to curriculum/courses• Real examples of student work• Critical and creative thinking• Holistic feedback• Student motivation• Labor intensive
Key PointsPerformance Assessment
• Articulate skills to be examined• Design appropriate tasks to
demonstrate skills• Specify criteria for evaluating
performance• Develop reliable process for rating
performance• Training raters• Evaluating results
Current Trend
• Course-embedded assessment
• Ewell, 1991 and Wright, 1997
Components…
4. Program’s expected level of performance on each outcome (benchmarks)
Numerical score on national examMastery level on writing sampleMasterly level on oral/poster presentationMasterly level a behavioral performance Masterly level on a clinical evaluationHolistic score on ability to solve a case study
At what level is student expected to achieve the outcome?
Group Exercise
• Using your PIs:– Develop criteria or benchmarks for
each PI
Assessment Schedule
Entry into the programEnd of a specific semesterCompletion of a set of coursesCompletion on a certain number of creditsUpon graduationUpon employmentNumber of years after graduation
Should each outcome be assessed every year?
Who collects/analysis/interpret
s results?
Inside Evaluators Outside EvaluatorsStudent affairs AlumniWriting center Employer
Assessment committee BOCATEP committee GRE
Components…
5. Results of data for each outcome
What percentage (number) of students met the LO/PI at the criteria/benchmark established
May want to include sample size
Sharing Results
Who will see the results?
AdministrationFacultyStudentsOutside accrediting agencies
Components…
6. Closing the LoopDoes the assessment plan lead to improvements in academic and co-curricular programs?
Revise curricula, sequence of courses….
Design more effective student orientation
Revise handbook
Increase connection between academic and clinical program
Purchase additional therapeutic exercise equipment
Closing the Loop
• Question
• “What changes to your program have you made based upon results of your program assessment data?”
Closing the Loop
• Curricular changes– Course information (competencies)– Instructional delivery– Course sequencing– Instructional resources (textbooks,
anatomical models, library holdings)– Technology
Closing the Loop
• Clinical program– Clinical instruction– Clinical sites– Clinical rotations– Clinical opportunities– Clinical performances
Closing the Loop
• Space• Budget• Faculty development• Human resources
Group ExerciseClosing the Loop
• With results provided:
Develop a “closing the loop” plan
Assessing the Program Plan
• Follow-up on implemented changes
• How often?• Who?
Trending Data
• Why?
• See changes overtime……• Overtime learning evidence• Charts student’s chronological
achievement
• Lots of work on front-end!
Trending Data
• Semester to semester• Year to year• Each learning outcome (program vs
student)• Clinical evaluations (student)• Clinical evaluations (instructors)• Clinical site evaluations• Teacher evaluations
Key Points to Faculty Involvement
Responsibility
Resources
RewardsPalomba and Banta (1999)
Characteristics of a Good Assessment Plan
• Asks important questions• Reflects the institutional mission• Reflects program goals and Los• Linked to decision making about
– Curriculum -budget -resources
• Involvement of multiple individuals• Contains relevant assessment techniques
Characteristics of a Good Assessment Plan
• Indirect and direct evidence of learning• Reflects what is known about student
learning• Shares information with multiple
audiences• Leads to reflection of faculty, staff, and
students• Allows for continuity, flexibility, and
improvement in the assessment process
Navigating Assessment
Thank You……
Questions ?????????????