Unit Assessment System

Teacher Education at Purdue

March 4, 2012

What is our assessment philosophy ?

CONTINUOUS IMPROVEMENT

3

All components of teacher education at Purdue engage in continuous improvement by gathering and analyzing relevant data and making decisions based on data.

BALANCE

4

The unit balances central coordination of the overall assessment system with encouraging individual offices and programs to develop flexible, creative, assessment system components that fit their mission and foster continuous improvement.

What have we done since the last visit to improve our UAS?

6

THE FIRST THREE YEARS• CI Focus: Initial licensure program

candidate assessment system– Reviewed the literature and state systems to

identify clear criteria for instructional performance

– Created the Gate C & D rubrics to measure those performances using the identified criteria

– Worked with program faculty to align program key assessments with the unit rubrics

7

THE GATE D RUBRIC• FOUR SECTIONS PLUS A HOLISTIC SUMMATIVE ASSESSMENT

I. PLANNING INSTRUCTION (also used for Gate C)II. TEACHING AND LEARNING EXPERIENCESIII. ASSESSING LEARNERSIV. REFLECTING ON LEARNING AND TEACHING

• EACH SECTION HAS…– 3-5 unit items– Items to assess impact on P-12 student learning– Encouragement to programs to add discipline-specific programs

• FOUR POINT SCALE FOR PERFORMANCE RATINGS– Clear descriptions of target behaviors in the cells

• PROGRAMS ALIGN TO THE UNIT RUBRIC IN VARIOUS WAYS

8

THE GATE D RUBRICEngineering/Technology

EducationRubric Criteria #1 #2 #3 #4

Subject Matter Focus (I.3) 0 3 10 1

Sound Instructional Strategies (I.4, II.1) 0 6 8 0

Student Communication Opportunities (II.3)

0 6 8 0

Communicating Performance Expectations and Assessment Criteria (III.1)

0 8 6 0

Focus of Student Assessments (I.5)

0 3 10 1

Summative Assessment of Student Performance (III.1)

0 3 10 1

Rubric Criteria

Beginning

Developing

Proficient

Expert

I.1 Knowledge of Students’ Learning Needs and Interests 0 4 17 34

I.2 Knowledge of Community, School, and Classroom Factors NCTE Standards 1 (1.3), 3.0 (3.1.3)

0 6 14 34

I.3 Subject Matter Focus of the Unit/Lesson(s) NCTE Standards 3.0 (3.1.4, 3.1.5, and/or 3.1.7), 3.5 (3.5.1, 3.5.2, 3.5.3, and/or 3.5.4)

0 4 13 34

I.4 Strategies and Resources, Including Technology, to Support Student Learning NCTE Standards 2.0 (2.4, 2.5, and/or 2.6), 3.3 (3.3.1, 3.3.2, and/or 3.3.3) 4.0 (4.2, 4.3, and/or 4.6)

0 3 19 34

I.5 Focus of Student Assessments NCTE Standards 4.0 (4.10) 0 1 20 34

English Education

9

ACCURACY, FAIRNESS, & CONSISTENCY• CI Focus: Initial licensure program

candidate assessment system– Use the Gate C&D rubrics for all initial

programs– Define and use key assessments for all

programs– Audit transition points– Coordination

• Programs– Program convener• Multiple Section Courses – Course coordinator

10

THE PAST FOUR YEARS• CI Focus: The operational assessment/decision

system– Created to specify the decision system– Created a UAS annual report template – Implemented TEC change decisions

• CI Focus: Advanced programs for teachers– Reviewed and categorized all graduate programs (TEC)– Created task force to design a unit framework, providing

clear criteria for performance (TEC)– Currently, advanced programs for teachers are being

redesigned to the new unit framework (Programs)

11

CURRENT• CI Focus: Program Assessment Systems (Standard

2)

– March 2011: First SPA program reports in 20 years• Results comparable to national averages• Most concerns related to the assessment systems

– December 2011: First state program reports in 20 years• Two approved, one approved with conditions• Most concerns related to the assessment systems

– Fall 2011: TEC established Standard 2 at the program level as the continuous improvement focus (2011-12)

• Assessment Coordinator met with all TE Program Conveners • Program faculty are currently working to improve their assessment

systems

What are the features of our current unit assessment system?

OPERATIONAL DECISION SYSTEM

13

Purdue University Teacher Education Operational Assessment/Decision System Data Sources Office & Program Management Analysis Decision Makers

Registrar

Praxis (ETS)

Applicants

Program Completers

Candidate Information

Employers

Cooperating Teachers

Cooperating School Administrators

Instructor Data

Graduate Office

Office of Professional Preparation & Licensure

Office of Field Experiences

Business Office

Program Conveners

Office of Advising & Recruiting

Purdue Student Education Council

DATA

CO

LLEC

TIO

N &

AN

ALYS

IS

Office of Strategic

Assessment

UAS Annual Report

Teacher Education

Administration (ADLE,

NCATE Coordinator)

Teacher Education

Council

Dean of College of Education

DATA

CO

LLEC

TIO

N

COE Leadership Team

Program Convener

Council

Dean’s Advisory Council

Data Types Decision support data gathered include: stakeholder perceptions, candidate performances, demographic information, operational process information, etc. Data Gathering Methods Methods used to collect and store this data include: surveys, course evaluation forms, portfolio assessments, operational performance metrics, grades, etc. Data Storage This information is stored in office and departmental data repositories, both electronic and paper.

Key Reporting and information sharing Information/Consultation upon request Post-decision feedback and influence ETS: Educational Testing Services ADLE: Associate Dean for Learning & Engagement This chart depicts the way data is used to improve the Purdue Teacher Education Program.

Decisions are made and enacted by entities in green.

DATA SOURCES DECISION MAKERS ANALYSIS OFFICE & PROGRAM MANAGEMENT

Teacher Education

Deans

Provost

CANDIDATE ASSESSMENT SYSTEM

14

Unit and Program LevelsUnit Level includes the conceptual framework and related standards, systems for assessing and managing critical transitions, and aggregated performance data from unit wide rubrics

Program Level (a) applicable program standards; (b) program assessment systems based on program and unit guidelines for transitions and key assessments; (c) national or state program review

18

CANDIDATE ASSESSMENT SYSTEMOnePurdue Transition to Banner

– Facilitated grade reports for content assessment – Facilitated some aspects of Gate audits– DegreeWorks… coming soon…

Performance-based Assessment Technology– P3T3 grant (before 2005)– TaskStream, an outsourced solution (2006-present)– We are currently researching alternatives

• Inexpensive• Integrated with Banner, DegreeWorks, and BlackBoard• Excellent capabilities

Questions?