Using Accreditation to Increase the Evidence Base of Teacher

Education (i.e., Using evidence to move faculty toward evidence)

Sandra B. CohenDirector of Teacher Education

Curry School of Education

University of Virginiasbc7v@virginia.edu

University of Virginia Curry School of Education: Context

• Twenty program areas in Curry School– 9 program areas in TED

• Approximately 973 full-time students in Curry School of Education

• 528 in teacher education• 13/yr. in reading education on-grounds

– 70/yr. off-grounds• 30/yr. in administration/supervision on grounds

– 300/yr. off-grounds

• 100 full-time instructional faculty in Curry– 32 faculty directly involved in teaching in TED

• 33 Research & project faculty

Curry School Teacher Education (TED): Context• Bachelor of Arts/Master of Teaching (5-year dual degree program )

– College of Arts & Sciences & Curry School of Education• Leads to B.A., M.T., licensure & endorsement

• Graduating class of approximately 110 students/year• Average SATs - V: 628; Q: 627• Average overall GPA: 3.51• Average major GPA in College of Arts & Sciences: 3.26

• Post-graduate Master of Teaching Program (2-year)– Curry School only

• Leads to both M.T. degree and licensure & endorsement• Approximately 55 students/year• Average GRE: 1124

Curry School Teacher Education Framework: “Teacher as a Reflective Decision-Maker”

Accreditation Implementation:

Curry TED (TEAC) Claims:

• Curry graduates know their students and work with others to support them

• Curry graduates know their content• Curry graduates are able to align instruction and assessment

within a positive learning environment• Curry graduates demonstrate professional integrity and

continuous self-reflection

Rule #1: Keep it simple

Rule #2: Make it work for us (i.e., base work on Virginia Teaching Standards)

Claims:

Rule #3: Match research to TED claims

ClaimsTeacher Education Programs at the Curry School of Education

VIRGINIA TEACHINGSTANDARDS

CURRY SCHOOL CLAIMS MEASUREMENTS

Standard I: Knowledge ofStudents

Standard V: Collaboration &Communication

Our students/graduates knowthe progression andchallenges associated withchild and adolescentdevelopment, understand theimpact of families, schoolpersonnel, and communitieson pupil achievement, andwork effectively with othersto support student learning.

3 Open Portfolio3 Course grades3 CLASS3 Exit surveys3 Follow-up survey

Standard II: Content Our students/graduates knowthe content that they areteaching and are able toarticulate the majorunderstandings of the disciplinearea and the supportingconcepts.

3 Praxis II3 VRA3 Course grades3 CLASS3 Open Portfolio3 Major distribution/GPA3 Lesson Plan Review3 Exit survey

Standard III:Planning, Instruction, &Assessment

Standard IV: LearningEnvironment

Our students/graduates areable to develop alignedinstruction that meets thecognitive, social, andemotional needs of students.Aligned instruction is thecongruence of standards,classroom objectives, andinstructional approaches andassessment within a positivelearning environment.

3 Open Portfolio3 Course Grades (generalmethods & content pedagogyclasses)3 Lesson Plan Review3 TPR3 CLASS3 Collaborative AssessmentLogs3 Exit surveys3 EBI survey3 Follow-up graduate survey

Standard VI:Professionalism

Our students/graduatesdemonstrate professional integritybased on continuous reflection oftheir instructional behaviors. Weassert that Curry graduates areprepared to continue theirprofessional development throughself-learning and reflection andthrough mentoring practices.

3 Open Portfolio3 EDIS 788 Field Projects3 Follow-up graduate survey

Sample Measures for Accreditation:

• Open Portfolio (Curry’s own student teaching on-line evaluation system)

• Educational Benchmark Incorporated (EBI) • Exit Surveys• Lesson Plan Review• Employer Survey• GPA• Licensure Tests (PRAXIS, VRA, VCLA)• CLASS (Classroom Learning Assessment Scoring

System [Student teaching observation system])

Purposes for Faculty Research:

• Verification of course material (faculty personal reinforcement)

• Survey & Interview Research (programmatic support/change)

• Content Specific Research (professional development)

• Research on Teacher Education Constituencies (professional development, programmatic support/change)

• Research on Teacher Education Program (value-added, programmatic support/change)

Know your purpose for evidence…

Evidence can make you feel good

Evidence can make you question

Evidence can make you comfortable

Evidence can make you change

Evidence from the accreditation process lead us to establish…

Research Pathways

Accreditation measures

Entry & Exit Surveys

Life Inventory

Demographic data

Core of Common DataArchival data

May or May not align with TED claims

Faculty Research Doctoral Research

Research InitiativesIndependent research

Grant research

Curry Participant Pool5 hour commitment

oralternative activity

Field Placement Evaluations

Lesson Plan Analysis

Work Samples

Accreditation Measuresand Program Evaluation

TED Program InitiativesType Title Here

Align with TED claimsNo pool restrictions

Research Coordinating Council

Sample Items - Employer Survey (handout)

Table 18: Employer Survey Results

Percent responding Claim Please rate ______,a Curry

graduate, on his/her ability to: Not at all

(1)

Some- what

(2)

Okay

(3)

Well

(4)

Very well

(5) 1 Collaborate with colleagues and

staff in the building and across the division

0.0% 1.0% 3.9% 31.4%

62.7%

1 Work with parents and families to support student learning

1.0% 1.0% 8.8% 35.3%

53.9%

2 Demonstrate depth and breath of content knowledge

0.0% 1.0% 14.7% 19.6%

63.7%

3 Maintain a safe and purposeful learning environment

0.0% 2.0% 4.9% 31.4%

61.8%

Teacher Preparation

PoorlySomewhatAdequatelyWell Highly

Employers’ (Principals’) Satisfaction: Curry alums overall compared to other beginning teachers

Table 5: Comparison of Curry Students and non-Curry Students Major GPA Academic Major TED

Student Average GPA

Standard Deviation

N= A & S Student Average GPA*

Standard Deviation

N=

English 3.17 . 9 3.28 253 History 3.03 11 3.18 244 Psychology 3.08 26 3.18 318 Sociology 3.42 6 3,07 120 Spanish 3.34 6 3.38 89

Table 6: Praxis II Data by Endorsement Area (2004-2005: last Title II report year) Endorsement & Cut Score

Number taking Praxis II - UVA

Praxis II Pass Rates -UVA

Praxis II Pass Rate State-wide

Elementary (143) 53 100% 100% English (172) 19 100% 99% Social Studies (161) 19 100% 99% Earth/ Space Sci (156) 7 100% 100% Biology (155) 7 100% 97% Chemistry (153) 1 100% 100% Physics (147) 2 100% NA French (169) 3 100% 92% Spanish (161) 4 100% 97% Math (147) 2 100% 96% Health/PE (151) 3 100% 100%

Comparative Utility of 2 Tables (handout)

CLASS: Classroom Assessment Scoring System

Emotional Support

Instructional Support

ClassroomOrganization

Student Outcomes

CLASS Focus

CLASS Data—Student Teachers (Fall ’04, Fall ’05)

0

1

2

3

4

5

6

EmotionalSupport

ClassroomOrganization

InstructionalSupport

All TAs (n=169)Elementary (n=57)Secondary (n=80)

CLASS Domains

Ave

rage

Dom

ain

Scor

es

CLASS: 4th year Curry and A&S Students

* .05 or less** .01 or less

CLASSDomain

Curry (n=20)

Mean (sd)

A&S (n=18)

Mean (sd) Sig.Effect Size

Homogeneityof Variance

EmotionalSupport

4.42 (0.79) 3.87 (1.10) 0.50 *

ClassroomOrganization

5.65 (0.60) 4.98 (1.30) * 0.52 **

InstructionalSupport

3.30 (0.90) 3.02 (1.13) 0.25

Student Engagement

5.25 (0.49) 4.54 (1.14) * 0.62 **

Table 16: Graduate Exit Survey MT Class 2006

Claim and Question Number Dec 2005 n=31 Well and Very Well

May 2006 n=103 Well and Very Well

Claim 1: Knowledge of Students and Collabor ation Q12 Help students become self-motivated and self-directed 76% 64% Q12 Relate classroom learning to the real world 89% 83% Q12 Develop curriculum that builds on studentsÕ experiences 93% 78% Q14 Work with parents and families to better understand students and support their le arning

79%

62%

Sample Items - MT Graduate Exit Survey(handout)

Curry’s Working Process for Accreditation

• TEAC Working Committee– Instruct ALL faculty about TEAC process

• Discussion of assertion & claims in program areas• Committee development of claims• Faculty review of claims• Committee development of Quality Control & Audit Process• Faculty review of Quality Control• Audit system development & implementation by committee• Pull evidence from all TED related projects

– Integrate/align all measures/evidence into TED program • Committee writing of Inquiry Brief• Faculty review of Inquiry Brief

– Faculty discussions, discussions, discussions

Curry’s Growth (see handout)

• Process must be both dynamic and on-going– Emphasis may change,

but focal points remain the same

• Parallel attention needs to be given to process knowledge (accreditation) and program knowledge (evidence-based decision making)

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Looking Back: What We Learned About Faculty Involvement in Developing Evidence:

• Research on TED must be seen as integral to, & integrated with, TED program for faculty to get involved

• Faculty more likely to participate in research when they see a direct connection to their job/role

• Faculty will respond more to data-driven issues when leadership is also involved

• Faculty want to retain ownership of their program, courses, & students

• Reliability needs to be established before measures are considered core to the assessment program

Looking Back (continued):

• Select & implement measures that can be sustained

• Need to continually invite faculty participation in research---times, situations, & interests change

• Faculty feel positively about common rubrics when they have input into their design & implementation

• Faculty need support for research efforts (data collection & analysis, grant writing, publication assistance, etc.)

• Evidence does move faculty

UVA’s Current Focus on Accreditation & Evidence

• Integration of TED program and research agendas• Development of core assessments• Seek reliability of measures• Cultural change---Research Pathways• Refinement of the research pool procedures• Development of sustainable measures (CLASS, Open

Portfolio)• Alignment of TED measures with VA state system

(VITAL)• Increase support for faculty involvement in research

(Pathways) and annual accreditation reports