Assessing Teaching Behaviors:

Introduction to an Assessment Instrument

for Teacher Candidates and Reflection of Teaching Behaviors

(A professional development workshop for cooperating teachers.) Workshop Outline (Note that all the information will be viewed on PowerPoint and there will be handouts with the slides for the participants to take notes. This PowerPoint will be available on the Office of Partnerships website following the workshop.)

1. Introduction (10 minutes) a. Welcome b. Introduction of facilitators c. Criteria for Cooperating Teachers-Info d. Purpose and goals of the workshop e. Point participants to blank papers/cards on their tables. They can write questions

that arise during the session. f. Overview of leaning objectives g. K-W-L (K-brainstorm what participants know abut assessment of teaching

behaviors, W-what participants want to know (and then the L-what participants learned at the end of the workshop about assessing teaching behaviors)

2. Short background of the need for assessment and how the Feinstein School of Education and Human Development (FSEHD) developed the assessment form (15 minutes)

a. Conceptual framework of PAR (Planning-Action-Reflecting) b. How the RIPTS are aligned with the assessment instrument c. The assessment instrument-dissecting and examining the sections

i. Section One: Lesson Indicators 1. Planning (Planning Indicators, Implementation Indicators) 2. Action (Implementation Indicators, Content Indicators, Climate

Indicators, Classroom Management Indicators) ii. Section Two: Capsule Rating of Observed Lesson

iii. Section Three: Post Observation iv. Section Four: Ongoing Progress (Personal Behavior Indicators,

Technology Indicators)

3. Scoring Criteria (30 minutes) a. Statistics 101 assessment vocabulary

i. Introduction to some assessment terms: analytic holistic scoring, criterion versus norm referenced scoring, performance level rubric, scoring criteria, indicators, normative versus developmental

4/26/2011

1

ASSESSING TEACHING BEHAVIORS:

PROFESSIONAL DEVELOPMENT WORKSHOP

FOR ALL RIC COOPERATING TEACHERS

INTRODUCTION TO AN ASSESSMENT INSTRUMENT FOR

TEACHER CANDIDATES AND REFLECTION 

OF TEACHING BEHAVIORS

NOVEMBER 19, 2009 & DECEMBER 8th, 2009

JANUARY 28TH & FEBRUARY 2nd, 2010

WELCOMEFIRST……•Thank you for being here•Thank you for being here•Thank you for working with our RIC teacher candidates•Thank you for your passion for professional development and to improve•Delighted to welcome you•This is the first in a series of workshops for you (next one‐Teacher Candidate Work Sample‐replaces for you (next one Teacher Candidate Work Sample replaces Exit Portfolio and then technology?)•RIPITS workshop is also required of all CT’s!•Do communicate with the OPP about your needs

4/26/2011

2

“HOUSEKEEPING NOTES” Restrooms

Sign in at the attendance table

Make a name tag

Clearly sign roster, including all present and former names, need certification # as well (not Social Security #) and/or birthday/date.

Be sure your EMAIL is correct too!

Credit reports (CEU’s) will be issued at close of workshop (Sign out at the same table where you registered.)

RIC submits names to RIDE

INTRODUCTIONS

OUR FACILITATORS TODAY:

SUSAN GRACIASUSAN GRACIA

EILEEN C. SULLIVAN

And…

MEMBERS OF THE FSEHD MEMBERS OF THE FSEHD ASSESSMENT COMMITTEE

4/26/2011

3

CRITERIA FOR COOPERATING TEACHERS Possess a continuing contract

Possess a professional teaching certificate for the grade level or special subject in which the student teacher is assigned

Have taught a minimum of three full years as a certified teacher

Have taught at least one full year at the current assignment

Agree to accept no more than one full semester or two half semester student teachers per academic year (Special note‐full semesters for elementary ed)

Attend Rhode Island College, RIDE, or School Department sponsored Rhode Island Professional Teaching Standards training sessions (as as specified in the partnership agreement between your school department and Rhode Island College.

Make a commitment to develop their teaching in line with the Feinstein School’s Conceptual Framework and the Rhode Island Professional Teaching Standards

Demonstrate teaching excellence as determined by the School Department

Complete a minimum of three formal evaluations of your student teacher using the observation forms supplied by the OSPFP   At least one of these 

REQUIREMENTS FOR OUR RIC COOPERATING TEACHERS

forms supplied by the OSPFP.  At least one of these observations must be completed in conjunction with the college supervisor.

Complete and submit the final evaluation forms

Participate in the evaluation of your student teacher’s Teacher Candidate Work Sample (previously Exit Portfolio) according to the (previously Exit Portfolio) according to the guidelines set forth by student teacher’s teacher preparation program

Electronic submission‐TBD

4/26/2011

4

GETTING TOGETTING TOKNOW YOUKNOW YOU

THAT’S ME!

THAT’S ME!THAT’S ME!•Elementary education teachers?•Middle school?•High school?•Mentored a RIC student teacher before? Fall? Spring?•Eager to work with one of our students?Taken RIPTS training?•Used the new Teacher Candidate Observation and Progress Report this Observation and Progress Report this semester? Survey Monkey?•Positive experience?•How many from each district?•Any college supervisors?•Eager to learn about the instrument?

4/26/2011

5

LEARNING OBJECTIVES

To introduce you to an assessment instrument, which analyzes teaching behaviors and documents the growth gof these behaviors

To examine the scoring criteria and the six‐point rubric of the defined assessment instrument

To teach you to use an assessment  To teach you to use an assessment instrument with teacher candidates

LEARNING OBJECTIVES (Continued) To be exposed to the proper terminology of scoring

To discuss teaching behaviors, using the defined assessment rubric  in large and defined assessment rubric, in large and then small groups

To analyze a teaching video with respect to Implementation, Climate, and Classroom Management (sections of the assessment instrument)assessment instrument)

To reflect on how this instrument can assist you with working with student teachers and your own reflective teaching practices

4/26/2011

6

GRAPHIC ORGANIZERGRAPHIC ORGANIZERKK‐‐WW‐‐LL

What do I Know?

What do I need to and Want to learn?

What have I Learned?

KNOWWhat do you KNOW about teacher evaluation, assessment of teaching behaviors, and the use of instruments  ,to assess teaching?

EVERY TABLE RESPONDS!

WANT TO LEARN

Will use the learning objectives

for the workshop

4/26/2011

7

AGENDA

I. Short background on the need for a new assessment instrument, the Teacher Candidate Observation and Progress Report for Student Teaching

II. Scoring criteria

III. Scoring practice

IV. Making connections with the instrument

V. Wrap up and the L (what have I learned) of graphic organizer, evaluations, CEU’s issued

I. INTRODUCTION/BACKGROUND ON THE INSTRUMENT

Created in response to faculty concerns and desire to gather and use more meaningful data

Designed in collaboration with FSEHD faculty, with multiple opportunities for feedback

Piloted in Spring 2009

Drawn from models used nationally

Represents first time observation data are being used in our unit assessment system

4/26/2011

8

FIRST‐LET’S EXAMINE THE OPR Let’s examine the OPR (see handout of the report)

Front page sign‐off documents the three visits and submitted at end of experience (all sign)

Other pages‐note organization and rating scale at topOther pages note organization and rating scale at top Section One: Lesson Indicators Planning, Action) Section Two: Capsule Rating of Observed Lesson Section Three: Post Observation Section Four: Ongoing Progress (completed by 

Cooperating Teacher only)

Implemented three times during student teaching experience (   t b   l t d t th )(one must be completed together)

These three reports document the student teaching experience

Submission of the OPR (TBD for spring 2009) Electronically? (Fall 2010 will be Chalk and Wire)

TEACHER CANDIDATE OBSERVATION AND PROGRESS REPORT (OPR)

Designed to demonstrate candidate growth!

Organized around Conceptual Framework—Planning, Action, Reflection

Aligned with RIPTSg

Observation indicators (completed by CT and CS) related to:   Planning Action:  implementation, content, climate, classroom 

management Reflection Overall Capsule Rating

Includes goals for next observation

Progress indicators (completed by CT) related to: Professional behavior Use of technology Available technical resources

4/26/2011

9

ASSESSMENT VOCABULARY (SEE PACKET)

Analytic scoring

Assessment

Competency‐based (criterion‐referenced assessment)

Criteria

Criteria referenced (competency‐based assessment)

Holistic scoring

Implicit scoring

Indicators

Inter‐rater reliability

Norm‐referenced assessment

Outcomes

Performance level

Rating scales

Rubrics

4/26/2011

10

SCORING AND THE OPR:

Includes multiple, observable indicators to:   k   li it  h t i  b i g make explicit what is being assessed reduce the need for the assessor to write many, repetitive, general comments

Yields granular information about multiple dimensions of a performance 

4/26/2011

11

•7 point scale is designed to detect wide variation in performance by offering 7 possible ratings—and to show growth

SCORING CRITERIA

•Accommodates faculty’s frequent requests for “wider” scale (e.g., +, .5 ratings)

Think of 0, 1, 3, and 5 as the base ratings for unacceptable, developing, acceptable, and target.  

R ti   f      d 6  t  i   t  th   t it  Ratings of 2, 4, and 6 account give raters the opportunity to describe performance that is a little higher than basic performance for that performance level – the equivalent of a + or .5 on old rubrics

III. SCORING PRACTICE Watch the video clip

Rate instruction on the Implementation, Climate, and Classroom Management indicators (see blue handout)

Discuss your scores with people at your table

Make sure you discuss these indicators:

Implementation:  Indicator 2 

Climate:  Indicator 7

Classroom Management:   Indicator  3 Classroom Management:   Indicator  3

Keep note of factors that make it easy and/or challenging to evaluate the instruction you viewed

4/26/2011

12

TEACHER VIDEO ANALYSIS

TIME TO STRETCH

ACTIVE LEARNING

SIMPLE CONTRALATERAL & 

CROSSING‐THE‐MIDLINE ACTIVITIES

CROSS OVERS  & ELBOW TO KNEE

APPLE PICKING

4/26/2011

13

“I hear and I forget.

d bI see and I remember.

I do and I understand.”

(Chinese Proverb)(Chinese Proverb)

Large GroupDiscussion Questions

What are some factors that made it easy yto evaluate this sample of instruction?

What are some factors that made it challenging to score this sample of instruction?

How do the teaching behaviors in the video relate/compare to those in your discipline? 

4/26/2011

14

FSEHD ASSESSMENT COMMITTEE

CONSENSUS RATINGS

Seven faculty members, watched the entire video and then used the OPR to rate the teacher and then used the OPR to rate the teacher 

Used Survey Monkey

Then a 4‐hour discussion to determine the ratings on the blue sheet

Read through our ratings and commentary

Discussion/Reaction

ACTION‐IMPLEMENTATION INDICATOR 2.

COMMITTEE SCORE=3

The teacher candidate attends to individual student needs, including learning and , g gbehavioral issues.

Score of 3 because the teacher circulates around the room, answers questions as they arise (from some students), and seems to meet the needs of most students   Because meet the needs of most students.  Because he only interacts with some students, it is not clear that the needs of all students are attended to.

4/26/2011

15

ACTION‐CLIMATE INDICATOR 7.

COMMITTEE SCORE=3

There was a high proportion of student‐to‐student i i b h f h lcommunication about the content of the lesson.

Score of 3 because there was a good amount of student‐to student communication in the lesson. However, the teacher did a lot of talking, and it was a o , o o g,very didactic lesson. Additionally, there were some students who were not engaged and were not working or communicating with other students.

CLASSROOM MANAGEMENT‐ INDICATOR  3.

COMMITTEE SCORE=4

The majority of class time is spent devoted to j y pacademic tasks, and time is divided in a meaningful, constructive way.

Score of 4 because the majority of class time is devoted to academic tasks.  However, time is not always divided in a meaningful, constructive way, as the teacher tends to tell students the answers without allowing them to construct their own meaning.

4/26/2011

16

IF TIME ALLOWS: ACTION‐IMPLEMENTATION 

INDICATOR 7.   COMMITTEE SCORE=1

The teacher candidate’s questioning strategies are lik l  t   h  th  d l t  f  t d t likely to enhance the development of student conceptual understanding/problem solving (e.g., emphasized higher order questions, appropriately used “wait time,” identified prior conceptions and misconceptions).

Score of 1 because the teacher frequently asks multiple questions, does not wait for answers, rephrases his own questions a number of times, and then answers his own questions for the students.

IV. MAKING CONNECTIONS WITH THE OPR

Use of the OPR as a reflective practice of your own teaching

Could you use it to evaluate a lesson you teach?  What  Could you use it to evaluate a lesson you teach?  What would the ratings tell you?

10 minutes to take time to select one indicator representing one of your strengths as a teacher and one where you would like to improve

• SEE HANDOUT

4/26/2011

17

IV. MAKING CONNECTIONS WITH THE INSTRUMENT

V. WRAP UP The L (what have I learned) of graphic organizer

Workshop evaluation (bring it with you to  Workshop evaluation (bring it with you to drop off as you leave)

CEU’s distributed at the entrance area as you leave

See you at the next workshop on the Teacher Candidate Work Sample then we’ll be p

teaching you about Chalk & Wire

Assessment Vocabulary

Analytic scoring: Evaluating candidate work across multiple dimensions of performance

rather than from an overall impression (holistic scoring). In analytic

scoring, individual scores for each dimension are scored and

reported. For example, analytic scoring of the Observation and

Progress Report include scores on multiple indicators of each RIPTS

standard, rather than an overall impression of candidate performance

on each standard.

Assessment Any systematic method of obtaining information from tests and other

sources, used to draw inferences about characteristics of people,

objects, or programs; the process of gathering, describing, or

quantifying information about performance; an exercise-such as a

written test, portfolio, or demonstration-that seeks to measure a

candidate's skills or knowledge in a subject area.

Competency-

based assessment

(criterion-

referenced

assessment)

Measures an individual's performance against a predetermined

standard of acceptable performance. Progress is based on actual

performance rather than on how well learners perform in comparison

to others or how they might be expected to perform at a given time

point. The Observation and Progress Report is an example of a

competency-based (or criterion-referenced) assessment.

Criteria Guidelines, rules, characteristics, or dimensions that are used to

judge the quality of candidate performance. Criteria indicate what we

value in candidate responses, products or performances. They may

be holistic, analytic, general, or specific. Scoring rubrics and rating

scales are based on criteria and define what the criteria mean and

how they are used.

Criterion-

referenced

assessment

(competency-

based

assessment)

An assessment where an individual's performance is compared to a

specific learning objective or performance standard and NOT to the

performance of other candidates or the expected performance of a

candidate at a given point of time. Criterion-referenced assessment

tells us how well candidates are performing on specific outcomes

rather that just telling how their performance compares to other

candidates or expectations at a given time. In criterion-referenced

assessments, it is possible that none, or all, of the examinees will

reach a particular goal or performance standard.

Holistic scoring Evaluating candidate work in which the score is based on an overall

impression of candidate performance rather than multiple dimensions

of performance (analytic scoring).

Implicit criteria Unstated guidelines, rules, characteristics, or dimensions --not stated in the scoring rubric or rating scale--that are sometimes applied during the scoring process. Applying implicit criteria detracts from the validity and meaningfulness of the assessment data collected.

Indicators Measures used to track performance over time.

Inter-rater

reliability

The consistency with which two or more judges rate the work or

performance of test takers.

Norm-referenced

assessment

An assessment where candidate performance or performances are

compared to that or those of a larger group. The purpose of a norm-

referenced assessment is usually to sort candidates and not to

measure achievement towards some criterion of performance.

Outcomes Changes in learners, such as improvements in teaching skills, that

occur as a direct result of their participation in a teacher education

program; knowledge, attitudes, skills, etc., that the candidate

acquires as a result of a learning experience.

Performance level A standard description of a candidate's ability at a given level in terms of a standard or outcome; a profile of skill levels for a candidate can thus be assigned and used for placement, instructional, or reporting purposes.

Rating scales Values given to candidate performance. Evaluations are made on

predetermined criteria for documenting where learners fall on a

continuum of proficiency. Rating scales include numerical scales

and/or descriptive scales.

Rubrics Specific sets of criteria that clearly define for both candidate and

instructor what a range of acceptable and unacceptable performance

looks like. Criteria define descriptors of ability at each level of

performance and assign values to each level. Levels referred to are

proficiency levels which describe a continuum from excellent to

unacceptable product or performance.

Adapted from: Glossary of Useful Terms, http://www.sabes.org/assessment/glossary.htm

MAKING CONNECTIONS WITH THE FSEHD

OBSERVATION AND PROGRESS REPORT (OPR)

Task: To be reflective and examine your teaching behaviors using the OPR

1. Select one Planning or Action Indicator from the OPR that represents one of your strengths as a teacher. Write the indicator here:

_____________________________________________________________________________________________

2. List 2-3 reasons why you selected this indicator and how you and use exemplary practices with the indicator:

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

3. Now select another OPR indicator, which might be a priority for improvement with your teaching. Write the indicator here:

_____________________________________________________________________________________________

4. List 2-3 reasons why you selected this indicator and how you plan on improving in this area:

_____________________________________________________________________________________________

_____________________________________________________________________________________________

_____________________________________________________________________________________________

The next step would be to take time and reflect on your teaching behaviors, select two-three

indicators, and then develop a strategic improvement plan. Use S.M.A.R.T. (Specific, Measurable,

Attainable, Realistic/Relevant and Timely) goals!

4/26/2011

18

Questions and Answers

THANK YOU

STAY IN TOUCH WITH THE OPP

We have an “OPEN DOOR” policy!

Also-Check the OPP website on RIC for updated information for Cooperating Teachers

4/26/2011

19

“H l i t b tt th li “Helping to better the lives of others is the greatest

of all achievements.”Alan Shawn Feinstein

b. Introduction of the performance level headings (unacceptable, developing, acceptable, target)

0 Unacceptable

1-2 Developing

3-4 Acceptable

5-6 Target

Not present. The candidate does not include the indicator in his/her planning, action, or reflection.

Elements of the indicator are clearly present but are partially or ineffectively carried out. The candidate is developing an awareness and may be beginning to meet the knowledge, skills, and competencies needed to meet the needs of some learners.

Elements of the indicator are of good quality, but there is room for improvement. The candidate knows and demonstrates the methods, skills, and strategies needed to meet the needs of most learners.

High quality implementation of indicator. The candidate knows and consistently demonstrates the methods, skills, and strategies needed to meet students’ diverse needs and interests.

c. Indicators:

i. Goal was to include multiple, observable indicators to: 1) make explicit what is being assessed and 2) reduce the need for the assessor to write many, repetitive, general comments.

ii. Provide granular statistics about every dimension of a performance. d. Designed to detect wide variation in performance by offering 7 possible ratings

(Unacceptable:0; Developing: 1-2; Acceptable: 3-4; Target: 5-6) e. Accommodates faculty’s frequent requests for “wider” scale (e.g., + and .5

ratings). When we had ratings of 1-2-3 or 1-2-3-4, faculty often asked for ratings of 2+ or 2.5, etc.

f. Think of 0, 1, 3, and 5 as the base ratings for unacceptable, developing, acceptable, and target. Ratings of 2, 4, and 6 account give raters the opportunity to describe performance that is a little higher than basic performance for that performance level – the equivalent of a + or .5 on old rubrics

g. Criterion-based: candidates are measured against defined (and objective) criteria. Used to establish a person’s competence (whether s/he can do something).

iii. Contrast with Normative: Assessing candidate relative to other students undertaking the assessment

iv. Contrast with Developmental: Assessing candidate’s present performance against expected performance at a particular stage

v. Stress: We are assessing candidates relative to expected performance of a beginning professional teacher! The criterion is the same at every observation, but our expectations for scores will shift.

vi. Provide examples. E.g., a student teacher might have a number of developing ratings at his/her first observation. That’s ok at this stage of development. Our expectations for performance will grow as the candidate progresses, but our criterion will remain stable.

h. Goal is to capture candidate growth

i. If the same standard/criterion is applied at each observation in student teaching, we would expect to be able to detect growth in candidates’ teaching skills

4. Scoring practice (40 minutes) a. Implementation

i. View short clips of “medium level” classroom instruction. Some aspects of instruction are implemented well, and others aren’t. See clip #3 at: http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/using_RTOP_4.html

ii. Ask participants to rate instruction in this clip on the Implementation, Climate, and Classroom Management indicators

iii. Discuss individual and group ratings in small groups iv. Discuss in large group, highlighting the use of the same criterion for

student teachers no matter what point they are at in student teaching

5. Making connections with the assessment instrument (15) a. Review of how the workshop can assist the teachers with reflective practices of

their own teaching b. Individual and small group work sharing two ways the workshop will assist the

participants with assessing their own teaching behaviors and practices

6. Wrap Up and Session Evaluation (10) a. L (the L part of the graphic organizer of K-W-L)-participants share what they

learned about assessment b. Summarize main ideas of session c. Ask participants to fill out session evaluation which includes question about what

they need next in terms of guidance in using the Observation and Progress Report and reflection of their own teaching behaviors

(Total: 120 minutes)