nysciresourcespd.weebly.comnysciresourcespd.weebly.com/uploads/9/1/6/2/9162895/... · Web viewex....

Getting Started in Collaborative InquirySample Agendas, Protocols & Templates

Network 6032014-2015

FINAL DRAFT

0

Prior to Team Meeting #1- Complete A. Identify Team Leader, Content Team Members, and Meeting Information

o Tool: Template A – Team Information

- Artifacts from Prior to Team Meeting #1o Completed Template A (1 per team)

1

Template A – Teacher Team Information

SCHOOL: SCHOOL YEAR:

Teacher Team Facilitator Name Courses Taught Contact Info (Cell/Email)

Content Team Member’s Names Courses Taught Contact Info (Cell/Email)1.

2.

3.

4.

5.

6.

7.

8.

Meeting Day Time Location

2

Phase 1: Plan – Analyze & Strategize

Prepare for Team Meeting #1- Finalize agenda for Team Meeting #1- Materials

o Facilitator brings: Copies of Meeting #1 Agenda, Protocol B, Template B, Protocol C, Template C Post-Its, pens, chart paper (or whiteboard), markers (or dry erase markers)

o All teachers bring: Easily accessible class data from assessments given to date (unit exams, MOSL, exit slips, etc.)

3

Team Meeting #1

- 20 min: B. Look at Student Data to Identify Common Issues in Content Areao Tool: Protocol B – Looking at Regents Datao Tool: Template B – Looking at Regents Datao Goal: To surface learning needs that exist from different Regents exams within the same content area

- 25 min: C. Surface Common Student Needs & Determine Focus Areao Tool: Protocol C – Surfacing Common Learning Needso Tool: Template C – Determining Focus Issueo Goal: To compile learning needs, prioritize them, and determine a common learning student need

(“focus issue) to focus on in this inquiry cycle

- Artifacts from Team Meeting #1o Completed Template B’s from each teacher on the team with identified learning needs for each

Regents course represented on the inquiry team, and notes about what curriculum changes will be made for content-specific learning needs

o Copy of Protocol Co Post-its captured on pages with category names (or picture taken of post-its organized on chart paper)o Completed Template C (1 per team)

Protocol B – Looking at Regents Data

Time: 20 Minutes

Supplies: pens, chart paper (or whiteboard), markers (or dry erase markers), copies of Template B, copies of applicable Regents exams for each member of your team, copies of applicable RADI item analysis reports from ATS for each member of your team

Step 1 (3 min): Reading the ATS RADI report (item analysis)- Facilitator provides team with overview of how to read the RADI report.- As shown below in the sample RADI report below:

o Column 1 is question number (from the Regents exam)o Column 2 is the total number of students and the percent of all students that took the exam that chose

answer choice #1 for that question.o Column 3 is the total number of students and the percent of all students that took the exam that chose



answer choice #4 for that question.o Column 6 is the total number of students that took the exam.o Column 9 is the percent of all students that took the exam that chose the correct answer

The correct answer choice is indicated in Columns 2, 3, 4, or 5 with an asterisk.

- For the free response questions, as shown below:o Column 1 is question number (from the Regents exam)o Columns 2 through 8 provide the total number of students and the percent of all students that took the

exam that received that many points on that free response question; e.g. for question 39, a 4-point free response question, 70% of students (28 total) earned 0 points (“Ans 0”), 10% of students (4 total) earned 1 point (“Ans 1”), and so on.

Over

4

Step 2 (12 minutes): Finding themes in the data and documenting- 8 min: Identify the five most-missed multiple-choice questions from your Regents and determine possible

reasons for why students performed poorly on those questions.o Questions to ask:

“Across these questions, what themes do you notice?” “Are there any commonalities in the content, the skills, the style, or the structure of the

questions that students did poorly on?” “Did students tend to prefer one of the distractor choices over another? Why might this have

been?”- 4 min: Document findings and planned curricular changes

o Make notes on Template B regarding common themes that are emerging (see Template for different types of themes to look for).

o Write down how these findings will impact your curriculum this year and what changes you will make to the curriculum based on these findings.

Step 3 (5 minutes): Sharing out- Team members take turns sharing their most significant “A-ha!” moment or common theme and their planned

curricular changes for this upcoming year.- Facilitator captures shared findings on chart paper and these will help group with Activity C in which they

identify common student learning needs across classrooms.

5

Template B – Looking at Regents Data DATE:

SEARCHING FOR PATTERNS & TRENDSIn terms of supporting your inquiry work, you want to identify patterns and trends that emerge from looking at data.

Focus on commonalities that you notice when you look at frequently missed multiple-choice questions and free-response questions.

Look for any patterns or trends that may emerge when looking at: The content of the frequently missed questions; e.g. “Students had difficulty on questions related to meiosis and mitosis” or “Students performed poorly

on questions that involved specific dates of events.” The skills of the frequently missed questions; e.g. “Students performed more poorly on questions that required them to graph on axes that were not

labeled with intervals” or “Students had issues with questions that involved interpreting data from a graph.” The style or approach of the frequently missed questions; e.g. “Students had difficulty with questions that asked them to identify which option was not

correct” or “Students performed the worst on open-ended questions that required them to explain or justify.” The structure of the frequently missed questions; e.g. “Students had exceptional difficulty with question prompts that were more than two sentences

long” or “Students did poorly on open-ended questions that had a second part

These patterns define learning needs of your students and help to inform curricular changes (changes to what you teach) for this upcoming year and instructional changes (changes in how you teach) that can be implemented in your classroom and investigated during inquiry.

REGENTS ANALYZED: June 2014 Regents in _________________________________________________________________________

Major Finding – Pattern or Trend in the DataWhat is the learning need?

Evidence to Support FindingHow do you know this need exists?

1.

2.

6

3.

Curriculum Adjustments based on Content-Specific Learning NeedsWhat changes are you going to make to your curriculum this year based on the learning needs you identified?

Example (from Living Environment)Major Pattern or Trend

What is the learning need?Data to Support Finding

How do you know this need exists?ex. 1. Questions with longer reading stems were more difficult than short stem (15 words or less).

Q6, Q14, Q22, Q25, Q37 on average students scored 14% lower than short stem questions

ex. 2. Questions that required students to read graphs were more difficult than other non-graph questions.

Q35, Q38, Q39, Q40, Q41, Q50 on average student scored 9% lower than questions without graphs.

ex. 3. Writing responses that asked students to explain were most difficult than the other types of writing responses.

Q59, Q61, Q67-69 on average earned the least number of points for Part C.

Curriculum Adjustments based on Content-Specific Learning NeedsWhat changes are you going to make to your curriculum this year based on the learning needs you identified?

ex. The Regents questions used in each lesson in the unit will build in word length as each unit progresses. A multiple choice question annotation system will be established to process all in class and test questions. And students will assess their own and other student answers using the annotation system. Learning activity templates will include “explain tables” that over time will transition to full writes.

7

Protocol C – Surfacing Common Learning NeedsReferences for this protocol:

Activity 1 & Resource A in Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Time: 25 MinutesSupplies: post-its, pens, chart paper (or whiteboard), markers (or dry erase markers), copies of Template C

Step 1 (2 min): Overview- Goal of this activity: to identify common student learning needs across classrooms.- Focus on learning needs that are based on learning standards, not needs based on student

behaviors (see examples below)ExamplesFocus on standards-aligned learning needs, such as:- Students need to improve estimation skills.- Students perform poorly on questions with longer reading stems (from Regents).- Students lack the ability to determine key points from texts.- Students need to improve their ability to read and interpret graphs (from Regents).- Students have difficulty evaluating arguments.- Students need to develop the ability to evaluate other points of view constructively.- Students perform poorly on questions that ask them to explain or justify (from Regents).- Students need to strengthen their writing.Avoid needs base on behaviors or work habits, such as:- Students need to stay organized.- Students need to turn in their work on time.- Students need to value education.- Students need to stay focused.- Students need to complete their work in class.Step 2 (9 min): Needs Assessment- On separate post-it notes, write each learning need and a brief evidence statement that:

o Surfaced during data analysis, oro Is a learning need that you are aware of in your own classroom (but supported by evidence!)

- See samples of possible post-it notes above.- Post all post-its on chart paper (or whiteboard if chart paper is unavailable).- Facilitator questions:

o “What are the learning needs that surfaced from the data?”o “What is it that your students struggle with the most in class, and what is the evidence?”o “What do many of your students experience difficulty doing, and what is the evidence?”

Step 3 (9 min): Affinity Grouping- Have group members go up quietly and organize the post-its into categories that make sense to them.- Facilitator question:

o “What ideas go together? Feel free to move post-its next to others that are similar in some way.”- Label the groups with category names (using markers).- Facilitator question:

o “What category names make sense for these groups?”Step 4 (4 min): Prioritize and Choose a Focus Issue- Have group members choose the category of learning needs that should be prioritized and from within that group,

choose one common learning need to focus on – this will be the “focus issue” of the inquiry cycle.- Facilitator question:

o “Which learning needs can our group work on collaboratively?”o “Of those, which learning needs can we begin to address using instructional changes?”o “Of those, which is most important to us and why?”

Step 5 (1 min): Document your focus issue, rationale, and the evidence on Template C – Determining Focus Issue.

8

Learning Need: Students have difficulty with questions involving reading graphsEvidence: Regents exams analysis from 6/14

Learning Need: Students need to strengthen their ability to cite textual evidenceEvidence: MOSL baseline data for trait 4

Sample post-its

Learning Need: Students need to improve estimation skillsEvidence: Student responses in class reveal that they cannot “ballpark” guesses

- Also provide an overview of what is occurring at the next meeting

9

Template C – Determining Focus Issue DATE:

Identified focus issue (learning need) chosen for our inquiry work:

Rationale for choosing this learning need as the focus issue:

Evidence that supports this focus issue as a learning need that needs to be addressed:

10

Prepare for Team Meeting #2:- Finalize agenda for Team Meeting #2- Materials

o Facilitator brings: Copies of Meeting #2 Agenda, Protocol D, Template D, Protocol E, Template E

o All teachers bring: Copies of their rosters and grade books Knowledge of their students (e.g. ELLs, SWDs, lowest third, class grades, performance on

Regents exams, etc.)

11

Team Meeting #2

- 25 min: D. Establish a Target Group of Studentso Tool: Protocol D – Establishing a Target Groupo Tool: Template D – Our Inquiry Team’s Target Groupo Goal: To establish a target group of between 15 and 30 students

- 20 min: E. Establish a Baseline Assessmento Tool: Protocol E – Establishing a Baseline Assessmento Tool: Template E – Our Baseline Assessmento Goal: To determine a baseline assessment to use as a starting point prior to implementing an

instructional change strategy

- Artifacts from Team Meeting #2o Copies of Protocol D and Eo Completed Templates D and E (1 of each per team)o Pictures taken/printed of the chart paper (or whiteboard) with ideas from the criteria activity

Protocol D – Establishing a Target Group

Time: 25 MinutesSupplies: pens, chart paper (or whiteboard), markers (or dry erase markers), copies of Template D

Step 1 (2 min): Facilitator makes sure all attendees have signed in and received an agenda; person taking minutes begin.

Step 2 (1 min): Facilitator begins meeting by announcing that today we will establish criteria that will help us to choose a target group of between 15 and 30 students, and then use that criteria to choose students from our rosters for the team’s target group.

Step 3 (5 min): Facilitator discusses the purpose of a target group and the use of criteria to determine a target group.

Some points to mention: Purpose of choosing a target group: There will be many data pieces to analyze, so keeping said data restricted

to a set group of students will keep the work focused and streamlined (Imagine 6 group members trying each bringing student work samples from all 90 of their students to each team meeting for the purpose of drawing conclusions)

Purpose of setting criteria: Could you just randomly pick student names by taking turns throwing darts at your rosters? Although you technically could, it would be much more meaningful and impactful to choose a group of students that are actually demonstrating the need you have identified! Of course, there will likely be many more than 30 students across all of your group’s classrooms that are exhibiting the common learning need, and so you should develop some other criteria that are meaningful to your group and use those criteria to help you choose your target group of students. Setting criteria for choosing a target population will ensure that we are looking at a group of students with commonalities among them, and this will also keep the work focused.

Possible criteria (can range from simple to more complex): Demonstrates the identified learning need (focus issue); high attendance rate; students with disabilities (decide if half-time or full-time IEP); English Language Learners; students within the school’s lowest third; students within a subgroup chosen for MOSL purposes (e.g. ELLs, Blank and Latino males, Overage/Under-credited); students who have failed a particular Regents multiple times; students that are repeating a course; students who scored less than a certain percentage on an exam; students who scored less than a certain percentage on an exam and scored less than a certain percentage on a specific portion of an exam (example: students who scored between 55 and 64 on the ELA Regents but that also scored less than an 18 on the multiple choice section).

Step 4 (3 min): Facilitator gives the group time to independently decide on criteria they feel best suits their inquiry work.

A question that might surface: How many criteria? There is no “minimum” number of criteria, as criteria are really used as a method of

meaningfully and identifying a target group. Teachers might wonder “how many students in my classes should meet these criteria?” and encourage them to design the criteria first based on what is important to them, without worrying about how many students would qualify for the target group. If they ask “is there a recommended number?,” then recommend three criteria per teacher.

Step 5 (3 min): Teachers share their criteria, and facilitator writes responses down on chart paper, putting a check mark next to any repeated criterion.

Step 6 (3 min): Facilitator will review items with the team and have members vote on criteria that are the most meaningful to them and document these criteria with the rationales behind them on Template D – Our Team’s Target Group.

Over

12

Step 7 (5 min): Teachers apply the established criteria to their rosters (all rosters for all classes they teach!) in order to generate a target group overall of 15 to 30 students in total that satisfy the criteria. This means that, for a team of 5 members, each teacher should have 3 to 6 students overall (across all of their courses) that end up in the target group.

Some questions or concerns that might surface: There are no students of mine that fit all criteria. It is very important that every teacher, across all of his or her

classes, can identify at least a few students to be a part of the target group (otherwise, they will have no one to monitor and report on with the group). This may require being flexible with your criteria! For example, a teacher may find that the four criteria selected produces too stringent of a condition to allow for any students to satisfy all conditions, so they pick the three criteria that are most meaningful to them, which then allows for 10 students to qualify.

There are too many students that fit the criteria. It is important that students are equitably distributed (as much as possible) across the teachers’ classrooms. So, if you are a teacher with 15 students that fit all criteria but other teachers have between 3 and 6, then perhaps apply another criterion of your own to help reduce that number down to 6 or so. If all teachers seem to have too many students (e.g. 90 students across all teachers’ rosters), then consider applying an additional group criterion to that group of students. Remember that all things related to this inquiry process are flexible and you can and should adapt as needed!

Can they all be in one class? Should they be in different classes? There are no “shoulds” or “should nots” here; if you have five students on your rosters that satisfy the criteria, and they are all in the same class, then let that be the case. If it turns out that each of those five students is in a different one of your five sections, then let that be the case. Allow the satisfaction of criteria trump how they are organized across your classes. But if you are a teacher with, say, 15 students that satisfy the criteria and are looking for another criterion to reduce that number to between 3 and 6, you might then consider using the arrangement of those students across your courses (e.g. using “students in my period 5 Common Core Algebra I class” as a final criterion, with the rationale being something like “I want to specifically monitor students in the single Regents course that I teach”).

Step 8 (2 min): Document chosen students on Template D – Our Team’s Target Group.

Step 9 (1 min): Provide overview of the next meeting.

13

Template D – Our Team’s Target Group DATE:

Criteria for selecting our target group:

Rationale for using these criteria:

TARGET GROUPStudent Name Teacher who will collect data on this student Relevant notes on this student1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

14

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

15

Protocol E – Establishing a Baseline Assessment

Time: 20 MinutesSupplies: pens, copies of Template E, chart paper, markers

Step 1 (3 min): Facilitator discusses the purpose of the baseline.Some points to mention:

The baseline serves as a basis for comparison with future assessment data, and in conjunction with student evidence throughout the upcoming weeks and post-assessment data, will help your team determine the effectiveness of the instructional change strategy that your team will be implementing.

Baseline data is important because the team should compare students’ performance on last year’s exam to a more current assignment that assesses the same skills.

The point of the baseline is to assess where students currently are at the moment—they might have progressed, or even regressed, since last year’s assessment.

Some possible questions: Do all my students take the baseline or just the target population?

o Since you are most likely using class time to administer your baseline, you should give the baseline to all of your students; separating the target population students is not necessary, or even encouraged.

How is a baseline useful for all of my students? o As previously mentioned, it is important to note any progress students have made since last year’s state

assessment. Therefore, baseline data is useful to have for all students. Moreover, you will not deny your other students of quality instruction; the strategies and data analysis you use and do for your team are not exclusively for the students in your target population—the only difference is that you will share only the target population’s data and student work with the team.

Step 2 (5 min): Facilitator discusses some possible baselines.Some points to mention:

For the baseline assessment, you may use an existing assessment that will be or has been administered, including professionally prepared assessments. Some examples potentially include MOSL assessments, Performance Series, periodic assessments, school-based interim assessment, midterms, finals, unit tests, etc.

The baseline assessment needs to be a common assessment across classrooms that at least all students in the target group have taken or will be taking soon, although it will likely be an assessment that all students take (e.g. MOSL); the team should agree on a baseline assessment that all teachers on the team will administer in their classes.

You may also create your own baseline; to do this, it is typical to select a handful (5-10) of strategically chosen questions from the course’s end-of-year exam (i.e. the Regents) that align to your focus issue.

Regardless of the baseline you choose, it is important that the identified student learning need (focus issue) is represented in some form on the baseline! For example, if your focus issue is “students have difficulty justifying their answers in writing,” then your baseline assessment should provide you with some data on this learning need.

o Using an assessment such as the MOSL assessment or a formal departmental unit baseline should be done only if it is assessing the same skills that are aligned to your team’s focus. You should not use an assessment that isn’t aligned in any way to your focus issue.

Step 3 (4 min): Brainstorm: Group members individually brainstorm potential baseline assessments and rationales for why those assessments would make good baselines.

Step 4 (7 min): Facilitator elicits ideas from group, writing them on chart paper. Team will vote on which baseline assessment to use (or if it will be created). If it hasn’t been administered already, the team will decide when and how it should be given.

Step 5 (1 min): Person taking minutes will give a quick recap of meeting; provide overview of the next meeting.

16

Template E – Our Baseline Assessment DATE:

Brainstorm!What possible assessments might you use as a baseline? Consider MOSL assessments, state assessments, periodic assessments, or other common assessments. Will the entire assessment be the baseline, or will you use a specific part of it?

Chosen BaselineWhat is the baseline assessment your team will be use? (if you will create one, see the section below)

Why is your baseline assessment appropriate for determining the students’ abilities on the focus issue?

Has the baseline already been administered? If not, when will your team be administering it in their classes?

If you are creating a baseline assessment:What skills must the assessment address?

What will be the format of the exam? (i.e. multiple choice, short response, essay, mixed). *Note: The design of the exam is entirely up to you, so long as it is assessing the skills that are aligned to your focus. For example, if students are struggling with multiple choice questions, then it would be best to use that format for the baseline.

Why is this baseline assessment a better alternative than the exams that are already in existence? What is the rationale for creating your own baseline?

When will the baseline be created?

Other Points to Consider:

17

1. How will you monitor your students’ progress overtime after conducting the baseline assessment?

2. How often will you assess your students’ progress within the inquiry cycle? Why?

3. How can you do diagnostic assessments for NEW skills?

4. How do you know if your diagnostic assessment is providing an accurate baseline?

18

Sample E – Baseline AssessmentFocus Issue: Determining the Effect When a Familiar Situation is Changed

1. If the sequence of building blocks were changed, what effect could it most likely have on the protein?

__________________________________________________________________________________________________________________________

2. State what would most likely happen to one other population in this food web if all the squirrels and rabbits were suddenly killed by a viral disease. Support your answer. [1]

3. If this forest community experienced a severe lack of rain throughout the spring and summer seasons, state what effect this drought could have on the grouse population. Support your answer. [1]______________________________________________________________________________________

4. Identify one kind of bird that would show an immediate decrease in number if the flowering land plants were destroyed by an environmental change. Support your answer. [1]______________________________________________________________________________________

5. Different enzymes are secreted in each of the three locations. Ptyalin digests carbohydrates. Pepsin and trypsin both digest proteins. State how a fever of 40°C would most likely affect the activity of these enzymes and support your answer [1]

19


o Facilitator brings: Copies of Meeting #3 Agenda, Protocol F, Template F, Protocol G, Template G Chart paper (or whiteboard), markers (or dry erase markers)

Baseline Administration: At some point between meetings 2 and 4, ensure that the baseline assessment has been administered in all classrooms in which students in the target group are located; be sure to save the class data from these assessments and make copies of the assessments from the target group students to be analyzed in Meeting #5.

20

Team Meeting #3

- 25 min: F. Identify Your Team’s Instructional Change Strategyo Tool: Protocol F – Identifying Instructional Change Strategieso Tool: Template F – Our Instructional Change Strategyo Goal: To identify one common strategy that all teachers on the team will implement in all of their

classrooms in the coming weeks.

- 20 min: G. Develop Your Team’s Plan of Actiono Tool: Protocol G – Developing a Plan of Action o Tool: Template G – Our Inquiry Team’s Plan of Actiono Goal: To develop the team’s plan of action which contains a purpose statement, inquiry question to

be answered, and the explicit details of the instructional strategy to be implemented.

- Artifacts from Team Meeting #3o Copies of Protocol F and Go Completed Template F’s from each teacher on the team with brainstormed/researched ideas and

selected instructional strategy with rationaleo Completed Template G (1 per team)

Protocol F – Identifying Instructional Change StrategiesReference for this protocol:

Resource Appendix of Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Time: 25 MinutesSupplies: computers or laptops connected to the Internet, pens, copies of Template F, chart paper and markers

Note: The team will be spending part of the time researching strategies on the Internet, so this meeting should either be held in a computer lab, or all participants should bring a device that they can connect to wifi.

Step 1 (1 min): Review team’s work so far- Facilitator recaps the work so far:

o Surfaced learning needs in our different courses.o Determined a common learning need across all of our classes that we feel is important to address.o Established thoughtful criteria with which we chose a specific group of students that we will monitor in each

of our classes; each of us has several students that we will be monitoring over the coming weeks.o Determined an assessment that we will use as a baseline.

Step 2 (1 min): Overview of this activity- Facilitator provides overview of today:

o The goal is to select the instructional change strategy that we will implement in our classrooms in the coming weeks in order to improve student performance related to the focus issue.

o First is a brainstorming activity, followed by an activity to refine the list by researching online sources.o The session ends when we collaboratively choose one common strategy to implement in our classrooms.

Step 3 (10 min): Brainstorm strategies- Facilitator writes the following prompt on chart paper or the whiteboard:“Some possible instructional strategies that may improve our students’ ability to [fill in with your group’s focus issue] are ____________________________________”- 8 min: Teachers independently brainstorm as many strategies as they can that could fill in the blank, and document

their ideas on paper using Template F – Our Instructional Change Strategy.- Facilitator questions:

o “What are some instructional strategies you may have heard about that might work for our focus issue?”o “What are some instructional strategies you may have tried or are already doing in your classroom for other

purposes that we could build upon or expand in order to assist students with the focus issue?”- 2 min: Group members take turns sharing their ideas, and facilitator writes them on chart paper.

ExamplesOne instructional strategy that may improve our students’ ability to… is…

read generalized graphs - teaching and having students use a “four steps to reading a line graph” processparticipate effectively in collaborative discussions

- utilizing the “tell your partner, partner explains, you add” strategy- peer assess each other using a four-point discussion rubric

understand the main idea of a passage - having students use the six marks of understanding when reading a passageretain information from the lesson - utilizing Cornell notes in the classroomsolve equations involving multiple steps - using a flow-chart graphic organizer to facilitate solving equationsapply information to a new situation - using a “what if we changed the…?” strategy

- utilizing a “reverse, change the variable, or add information” strategyunderstand a concept in terms of an analogy - having students create a diagram and an analogy

- connecting the concept to lifedetermine the “most likely reason” - implementing the “concept, process, key words” strategy

- teaching a “ranking the answers” strategySource of some of these examples: Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Over

21

Step 4 (8 min): Research strategies- 8 min: Teachers independently browse internet resources to identify additional instructional strategies that may be

useful and document them on Template F – Our Instructional Change Strategyo Search tip: try searching for “instructional strategies in [subject area] to improve [focus issue]”

- 2 min: Group members take turns sharing what they found, and facilitator adds them to the chart paper.

Step 5 (4 min): Discuss and choose!- Group discusses the possible strategies and decides on a single strategy that:

o All teachers can and will implement across their classrooms.o Is manageable for each teacher to implement.

- Facilitator questions to help refine the list:o “Which strategies are manageable for us to implement?”o “Which strategies will likely have the biggest impact on student achievement?”o “Which strategies do we envision as being applicable in our classrooms every day?”o “Which strategies are research-based?”

Some questions that might surface:o Should we be using it in all of our classes and with all of our students? Yes and yes! Since the instructional

strategy should be one that is high-impact and the goal of inquiry to is to ultimately help all students to improve, then the strategy should be used across all classrooms and with all students. Keep in mind that the target group of students are those that will be monitored specifically (and given they are likely more needy than non-target group students, could potentially benefit more), but

o Should the instructional strategy be something that is used every day? As close to everyday as possible! Very likely your identified common learning need is one that occurs frequently enough throughout your courses (e.g. reading information from graphs, interpreting meaning, engaging in collaborative discussion, etc.) that you can use the strategy each day in your classes. Although there may be a day here and there where you don’t (e.g. test days), it is very important for your students to have routine practice with the strategy so that they can learn it at the same that you are practicing with its implementation.

o Does the instructional strategy have to be a change that is significant in scope and size? Absolutely not! In fact, strategies that are smaller and represent incremental changes in teaching practice are the best, especially when teams are just starting. This is because the process of implementing a change in the classroom and measuring its impact requires learning and practice, on the part of both teachers and students. As groups become proficient with the inquiry cycle and implementing common instructional changes, they could consider strategies that are larger in scope and size.

Step 6 (1 min): Document the team’s chosen instructional change strategy and rationale on Template F – Our Instructional Change Strategy.- Also discuss what will be taking place at the next meeting.

22

Template F – Our Instructional Change Strategy DATE:

Some possible instructional strategies that may improve our students’ ability to…Our focus issue:

Are…Brainstorming! (What ideas do you already have? What strategies have you heard about?)Strategy:

Strategy:

Strategy:

Strategy:

Researching! (What additional strategy ideas can you find?)TIP – consider searching for: instructional strategies in [subject area] to improve [focus issue]Strategy:

Source:

Strategy:

Source:

Strategy:

Source:

The instructional change strategy that we will implement in all of our classrooms is:Strategy:

Our team chose this strategy because:(You might consider manageability of implementation, potential for impact on student achievement, routine applicability across classrooms, extent to which strategy is research-based, and/or any other reasons important to you) Rationale:

23

Protocol G – Developing a Plan of ActionReferences for this protocol:Activities 5, 6, 10 & Resource C in Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Time: 20 MinutesSupplies: pens, copies of Template G

Step 1 (4 min): Develop the team’s purpose statement- Referencing the team’s focus issue and instructional strategy, the team collaborates to determine a purpose

statement of the team’s work.- The purpose statement should:

o Combine the focus issue with the change strategyo Be written using exploratory language, such as “explore,” “discover,” “understand,” “describe,”

“determine,” etc. Recall that the purpose of a team’s work is to investigate how making a specific change in teacher practice impacts student achievement in a particular area of focus.

Possible sentence frame for your team’s purpose statement:The purpose of this inquiry cycle is to [exploratory action; e.g. explore, discover, understand, describe, determine, etc.]

the impact of [instructional strategy] on students’ ability to [focus issue].

Step 2 (4 min): Develop the team’s inquiry question- Referencing the team’s focus issue and instructional strategy, the team collaborates to determine the inquiry

question that the team is attempting to answer in this inquiry cycle.- The inquiry question should:

o Combine the focus issue with the change strategyo Begin with “how” or “what”

Possible question frames for your team’s inquiry question:How does [instructional strategy] impact students’ ability to [focus issue]?

What is the impact of [instructional strategy] on students’ ability to [focus issue]?

Step 3 (10 min): Develop a shared understanding of the strategy- Referencing the team’s focus instructional strategy, the facilitator should guide the team to establish a shared

understanding of exactly what the instructional strategy looks like in practice.- Develop this shared understanding into a checklist containing 4 to 7 items that would be observable when the

strategy is being implemented.o These checklist items could include:

What the teacher is doing when the strategy is being implemented What students are doing when the strategy is being implemented The presence of any tools, resources, or materials (e.g. graphic organizers) when strategy is being

utilized The presence of any posted materials or written evidence of strategy (e.g. posters hung, strategy

written in lesson plan)o This checklist can be used in the data collection phase for teachers to self-assess their own implementation

and to provide one another with feedback on strategy implementation during intervisitations.

Step 4 (2 min): Document the team’s purpose statement, inquiry question, and shared understanding checklist on Template G – Our Team’s Plan of Action.- Also discuss what will be taking place at the next meeting.

24

Template G – Our Team’s Plan of Action DATE:

Our focus issue:

Our instructional strategy:

Our Team’s Purpose Statement:

Our Team’s Inquiry Question:

CHECKLIST: A Shared UnderstandingWhen our Instructional Strategy is implemented in our classrooms, it will look like: (include 4 to 7 items)

25

Phase 2: Implement – Take Action & Gather Evidence

Begin implementing the strategy! Once the baseline assessment has been administered (and copies of the assessment from the students in your target group have been made), it is time to implement the strategy across classrooms. Teachers can use the checklist developed as part of the team’s Plan of Action to guide their implementation. The first self-assessment of the strategy’s implementation takes place in Meeting #5.

Prepare for Team Meeting #4 - Finalize agenda for Team Meeting #4- Materials

o Facilitator brings: Copies of Meeting #4 Agenda; Protocol H; Templates H(A), H(B), and H(C)

26

Team Meeting #4

- 45 min: H. Develop a Data/Evidence Collection Plan and Timelineo Tool: Protocol H – Developing a Data/Evidence Collection Plan and Timelineo Tool: Template H(A) – Choosing Sources of Evidence and Datao Tool: Template H(B) – Our Timeline for Presenting and Analyzing Evidenceo Tool: Template H(C) – Checklist for Gathering and Submitting Evidenceo Goal: To establish what types of data sources and products every teacher will bring in for each of the

students in the target population and timeline for when each teacher will be responsible for providing the student evidence to discuss for each upcoming meeting.

- Prior to closing: Ensure teachers know what evidence they will be gathering in the coming weeks, how it will be gathered, and when they will be presenting their evidence to the group.

- Artifacts from Team Meeting #4o Copies of Protocol Ho Completed Templates H(A) and H(B) o Copy of Template H(C) (completed copies of this template can be submitted and captured by

teachers when they share their evidence)

Protocol H – Developing a Data/Evidence Collection Plan and TimelineReference for this protocol:

Activity 12 of Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Time: 45 MinutesSupplies: pens, chart paper or white board, markers, copies of Templates H(A), H(B), and H(C)

Step 1 (7 min): Facilitator discusses the purpose of evidence/data collection- Over the next several weeks, we will be implementing our instructional strategy and we want to be able to monitor

and discuss its impact on students on a regular basis. Although our post assessment will help to inform our conclusions in a quantitative way near the end of our study, we want to be able to discuss the growth of our students and analyze the impact of the strategy on an ongoing basis. In order to do this, we will need to gather evidence from our classrooms – evidence that will help us to assess whether our strategy is helping students to be more successful.

- There are so many forms of evidence and data that we could gather, and we want to gather different types of evidence from our students so as to have a more holistic view of their performance and attitude, as well as the impact of the strategy.

- Student work products are the most common evidence source (and definitely a necessity), but others we could consider include:

o Student surveys or reflections on the use of the instructional strategy.o Interviews with students in the target group around their experience thus far with the strategy.o Observations of students in target group as they work.o Conversations with students in the class as they engage with the strategy.o Other anecdotal evidence that you collect.o Any other data or evidence that your team wants to use that will provide your team with information

regarding the impact of the instructional strategy.- We will brainstorm which we think will be the most useful to us, and then decide as a group which data sources we

will use. Over the coming weeks, we will take turns bringing in evidence to share with the group and we will analyze the evidence together to see where students are struggling and discuss the current impact of the strategy.

Some possible questions: Is data collection more work for me?

o It is not! In fact, data-collection tools are meant to help you create a dashboard of your students, allowing you to understand their progress throughout a unit, as well as throughout the year.

Once I have collected data, what do I do with it? o As educators, we hear the term data-driven instruction all the time, but we often don’t know what that

means. By using data-collection tools, we are not only compiling and analyzing data, but we will use it to guide our next steps and revise curriculum accordingly.

Step 2 (5 min): Facilitator will ask the team to independently complete the brainstorm section of Template H(A). - Team members will brainstorm which sources the group should collect regularly to analyze.- Points to mention:

o These sources will be collected for each student in the target population. o You should choose sources that will provide the team with a complete and holistic view of the students.

Step 3 (10 min): Determine which products and sources will be collected and analyzed. - Facilitator will ask the teachers to share out their suggestions for sources and their responses to the questions on

Template H(A). The facilitator will chart the responses as the group shares out. - Once everyone has shared out, the team members will vote on the sources they will gather over the coming weeks.

Over

27

Step 4 (15 min): Use Template H(B) to determine a plan and timeline for collecting and analyzing evidence as a group.- 10 min: Discuss the plan for gathering the agreed upon evidence sources.

o For example, if teachers are going to be gathering exit slip samples from target group students, which exit slips and how often? If teachers will be conversing with students as part of the evidence collection, how will they document their notes (i.e. will they use a graphic organizer of some sort)? If student reflections are going to be collected, when should those be administered in classrooms?

- 5 min: Determine which teachers will be presenting their gathered evidence at which meeting.o If desired, use the rightmost column on Template H(B) to help streamline the evidence consolidation

process by providing a date by which the teacher should submit the documents with their evidence to the facilitator in order for the facilitator to make copies of the student work samples and other evidence for the group for the upcoming meeting. Use “N/A” if each teacher will bring his/her own copies.

- Points to considero If your team meets more than once a week, then you can focus on one teacher’s data per meeting. If you

are meeting only once a week, you are encouraged to focus on two teachers’ data per meeting. o It helps to have your upcoming team meeting dates mapped out already; the facilitator should ask people to

volunteer for the different presentation slots and try to accommodate requests, but if people do not volunteer, then random assignments of slots to teachers works as well.

Step 5 (5 min): Provide overview of Template H(C), if the group will be using it- Facilitator explain Template H(C) to the team by saying that it is the checklist with the final agreed upon

evidence/data sources, and that the team members should attach it to the documents that they are handing to him/her by the submission date on Template H(B).

- Team members can take a few minutes to populate their copies of the Template H(C) with the agreed upon sources and work products so that teachers can check them off as they collect them.

o Don’t forget to include blank copies of any tasks that produced student work samples!

Step 6 (3 min): Person taking minutes will recap the data sources and timeline and ensure that all group members know what they will be bringing and when!

28

Template H(A) – Choosing Sources of Evidence and Data DATE:

Independent Brainstorm!Which sources of data/evidence do you feel would be the best for the team to analyze?

Why are these sources of evidence/data important to developing a holistic view of the students?

How will this source provide evidence of the strategy’s impact?

Group’s final decision for sources of evidence to collect and analyze:(Multiple sources are ideal, and student work products should definitely be one!) Why did your group choose these sources?

Don’t forget to gather blank copies of any task that is generating student work samples!

29

Template H(B) – Our Timeline for Presenting and Analyzing Evidence DATE:

What is the plan for gathering the agreed upon data sources in each classroom? (e.g. use of a graphic organizer to capture observation notes)

TeacherGrade Level(s) / Course(s)

of target group students (from whom evidence will be collected)

Date that teacher will present evidence to team

Date by which evidence should be submitted to the facilitator

to make copies (“N/A” if teacher will bring his/her own copies)

30

Template H(c) – Checklist for Gathering and Submitting EvidenceFor use by teachers submitting documents to facilitator to make data analysis packets

Teacher’s name: _______________________________ Date of presentation/data review: __________________

Date of document submission to facilitator: ________________________

Please list and check off all final sources and products that you are attaching to this inventory sheet.Include all work samples from target group students, documentation of other evidence collected (per your team’s data collection plan), blank copies of tasks, etc.

Check box if item was gathered in for every one of your students in the target group

If not, why is source not present for every student?

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

______________________________________

__

__

__

__

__

__

__

__

__

___________________

___________________

___________________

___________________

___________________

___________________

___________________

___________________

___________________

Signature of teacher submitting evidence for team sharing: ________________________________________

Signature of facilitator receiving documents: _______________________________________

31

Sample H – Rubric-Based Formative AssessmentsSamples of formative assessments that could potentially be used as student work samples to share and analyze.

Sample rubric-based formative assessment from a high school Regents US History course

1. What is one claim that President Truman is making?

2. What evidence from the above text supports that claim?

3 – Meeting Standard 2 – Approaching Standard 1 – Significantly Below Standard Student develops claim that

demonstrates clear thinking about topic

Selected pieces of evidence clearly support claim

Student develops claim, but the connection to the topic is unclear

Selected pieces of evidence do not relate to claim

Student does not clearly develop a claim

Student does not support claim with relevant evidence

32

Sample rubric-based formative assessment from a high school Integrated Algebra course

3 – Meeting Standard 2 – Approaching Standard 1 – Below Standard 0 – Well Below Standard Correct interval is stated. Correct justification is

given.

One computational or conceptual error is made.

An appropriate justification is given.

Correct interval is stated, but the justification is missing or incorrect

Completely incorrect, irrelevant, incoherent, or blank.

Feedback/next steps:

33

Sample rubric-based formative assessment from a high school Regents Living Environment course

August 2014 Living Environment Regents

Base your answer to question 70 on the information below and your knowledge of biology.

Researchers monitoring the atmosphere of Earth report that the ozone shield has stopped shrinking and will most likely be on the mend over the next several decades. These findings were based on analyses of satellite records and instruments monitoring the surface of Earth. Scientists credit an international agreement to phase out the production of ozone depleting chemicals for this turnabout.

Current Question70 State one reason why the ozone shield is important. [1]

Allow 1 credit. Acceptable responses include, but are not limited to:— It shields living things from harmful radiation from the Sun.— protects us from UV radiation— protects us from DNA damage/mutation/skin cancer

Revised Question to be Graded with Rubric Below70. What inferences that can be made about the ozone shield and life on earth.

4 – Exceeds Standard 3 – Meeting Standard 2 – Approaching Standard 1 – Below Standard

Draws highly plausible inferences about the implications of the statements; provides convincing reasons for the inferences.

Large amount of Identified meanings, clues, and details that not explicitly stated.

Includes substantial connections between the text and the reader’s background knowledge in science.

In addition all descriptors in “3,” shows insights that connect additional information beyond the immediate text.

Draws plausible inferences about the implications of the statements; provides good reasons for the inferences.

Adequate identification of meanings, clues, and details that not explicitly stated.

Includes adequate connections between the text and the reader’s background knowledge in science.

Identifies assumptions in a text. Draws correct inferences from charts, graphs, and other pictorial representations.

Draws generally plausible inferences about the implications of the statements; provides reasons for some of the inferences.

Little identification of meanings, clues, and details that not explicitly stated.

Includes few connections between the text and the reader’s background knowledge in science.

Identifies assumptions and draws correct inferences that are simpler and more concrete.

Draws some plausible inferences about the implications of the statements; provides little justification for the inferences.

Poor or minimal identification of meanings, clues, and details that not explicitly stated.

Includes poor or minimal connections between the text and the reader’s background knowledge in science.

Cannot identity simpler assumptions.

34


o Facilitator brings: Copies of Meeting #5 Agenda, Protocol J, Template J(A), Template J(B), Protocol K, Template K

o All teachers bring: Numeric baseline assessment results for all of their classes

It is helpful if teachers have taken the time before the meeting to determine the average performance of all students in all of their classes on the baseline assessment

Knowledge of which students on their rosters are in the target group, are ELLs, are SWDs, and are in any other subgroup of interest

Copies of the baseline assessments for all students in target group (i.e. 3-6 papers per teacher)

35

Team Meeting #5

- 15 min: J. Self-Assessmento Tool: Protocol J – Self-Assessingo Tool: Template J(A) – Implementation Rubrico Tool: Template J(B) – Our Team’s First Self-Assessmento Goal: To develop a quick self-assessment rubric as a team and have group members individually self-

assess as to how aligned their implementation of the strategy was to the checklist.

- 30 min: K: Analyze Baseline Datao Tool: Protocol K – Looking at Student Datao Tool: Template K – Our Team’s Baseline Resultso Goal: To review baseline results from each classroom and discuss the implications that the results

have on the implementation of the instructional strategy.

- Prior to closing: Ensure teachers know who is providing student work and other evidence for the next meeting, in accordance with the data collection plan.

- Artifacts from Team Meeting #5o Copy of Protocol Jo Completed Template J(A) (1 per group)o Completed Template J(B) (1 per group)o Copy of Protocol Ko Completed Template K’s from each team membero Copy of a blank baseline assessmento The student work sample copies that were chosen by teachers and passed around

Protocol J – Self-AssessmentReferences for this protocol:

Activity 9 in Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Time: 15 MinutesSupplies: pens, copies of Template J(A), copies of Template J(B)

Step 1 (2 min): Brief check-in with implementation of the strategy so far- Some questions facilitator can ask:

o What successes and/or challenges have you experienced?o How have students responded to the instructional strategy?

Step 2 (8 min): Develop a three-point rubric for implementing the instructional strategy- Referencing the team’s Plan of Action from Activity G, the team collaborates to develop a rubric containing each of

the identified observable points outlined in the Shared Understanding Checklist. o This rubric will be used for teachers to self-assess themselves on their implementation of the instructional

strategy, as well as serve as a tool for intervisitations.- For each point, collaboratively develop language to define level 0 (not yet implemented), level 1 (partially

implemented), and level 2 (fully implemented).o Document this language on Template J(A) – Implementation Rubric.

Step 3 (2 min): Self-assessment- Allow each teacher to self-assess themselves using the defined rubric.

Step 4 (2 min): Share and capture team results- Facilitator leads group through the completion of Template J(B) – Our Team’s First Self-Assessment by documenting

each member’s self-assessment numeric results.

Step 5 (1 min): Next steps- Each team member then shares his or her own self-defined “next step” for refining and improving the

implementation of the strategy in his or her classroom; facilitator documents these next steps on Template J(B).

36

Template J(A) – Implementation Rubric DATE:Reference for this template: Activity 10 of Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Our Instructional Strategy

CHECKLIST: A Shared Understanding – Rubric of ImplementationObservable item from checklist Not yet implemented Partially implemented Fully implemented1.

0 1 2

2.

0 1 2

3.

0 1 2

4.

0 1 2

5.

0 1 2

6.

0 1 2

7.

0 1 2

37

Template J(B) – Our Team’s First Self-Assessment DATE:

Rubric of Implementation

Teacher Item 1(0 – 2)

Item 2(0 – 2)

Item 3(0 – 2)

Item 4(0 – 2)

Item 5(0 – 2)

Item 6(0 – 2)

Item 7(0 – 2) Teacher’s self-identified next step

TEAM TOTALS

38

Protocol K – Looking at Student DataAdapted from the “ATLAS – Looking at Data” protocol by National School Reform Faculty (NSRF)

Time: 30 MinutesSupplies: calculator, pens, copies of Template K, blank copies of the baseline assessment, teachers bring quantitative class results (e.g. their grade book) of their baseline assessment and copies of baseline assessment papers from their students that are in the target group (about 3 to 6 students)

Step 1 (8 min): Establish, share, and capture class averages- Provide teachers with time to calculate average performance on the baseline assessment for all classes, for their

students in the target group, for ELLs in their classroom, for SWDs in their classroom, and (optionally) for another subgroup of their choosing.

Step 2 (4 min): Review quantitative data- Facilitator provides time for team to look at the quantitative data and make inferences- Questions for facilitators:

o “What does this data suggest?”o “What assumptions can be made about the students?”o “What is the evidence that supports your interpretations?”

Step 3 (2 min): Review the baseline assessment- Review blank copies of the assessment and review why the baseline assessment is a good tool for measuring the

starting place of students around the focus issue.- Questions for facilitators:

o “What part(s) of this assessment pertain to our focus issue?”o “As we look at work samples, where should we focus our attention and why?”

Step 3 (6 min): Pass and review student assessment samples- Each teacher should have brought with them copies of the assessment of the students in the target group in his or

her classroom (from about 3 to 6 copied assessments).- 1 min: Facilitators instruct teachers to each choose two assessment samples that speak to them in some way

pertaining to the focus issue, and/or best demonstrate the purpose of the inquiry cycle.o Encourage teachers to pick two work samples that reveal something different about the focus issue, if

possible.o If your group is larger (e.g. 7 or 8 members), consider only choosing one work sample each.

- 5 min: Group passes around the work sample pairs, taking about a minute (give or take depending on the number of teachers in the group) to look at each pair of assessments; keep passing until teachers get theirs back.

o Facilitator encourages teachers to look for and note themes that emerge pertaining to student performance. The themes can relate to the focus issue or anything else; they should capture any themes they notice!

Step 4 (4 min): Discussion of themes and performance related to the focus issue- Teachers share out and document any themes that emerge regarding student performance from their review.- Teachers discuss what they notice about student performance and the focus issue specifically.

Step 5 (3 min): Discussion of implications for the instructional strategy- Teachers discuss what this baseline data reveals about the work ahead with the instructional strategy.- Question: “How would the instructional strategy have helped struggling students on this assessment?”

Step 6 (2 min): Discussion of any questions that this data raises- Question: “What questions do you have after looking at this data and work samples?”- Question: “Did anything surprise you?”

39

Step 7 (1 min): Document findings and provide overview of next meeting

40

Template K – Our Student’s Baseline Results DATE:Adapted from the “ATLAS – Looking at Data” protocol by National School Reform Faculty (NSRF)

Baseline Assessment: __________________________________________________ Max Possible Score: ______

Student Baseline Results

TeacherOverall

Average(All students)

Average of students in

target group

Average of ELLs

Average of SWDs

Average of

___________

Team Averages

What does this data suggest? What assumptions can you make?

Looking at Target Group Baseline Assessment SamplesWhat themes emerge across target group work samples with regard to student performance?

What do the work samples reveal about student performance and the focus issue?

What are the implications for the implementation of the instructional strategy?

Are there any questions that this data raises?

41


o Facilitator brings: Copies of Meeting #6 Agenda, Protocol L, Template L

o Specific teacher brings: Copies of student work samples from the students in their classrooms that are in the target

group, as outlined in the data collection plan. Copies of a blank task/assessment/activity/etc. that generated the work samples. Additional evidence of impact, as per the team’s data collection plan (e.g. notes from

conversations with students, observations of target group students, feedback forms, etc.) Any other relevant data or anecdotal evidence that they would like to share on the students.

Meeting Note: Since this is the first meeting at which the Looking at Student Work protocol is being utilized by the team, it is recommended that the group focus on student work/evidence from only one teacher’s classroom, to provide them more time to learn the protocol.

42

Team Meeting #6

- 45 min: L: Review Student Work & Other Evidenceo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impacto Goal: To look at student work samples and other evidence in order to begin qualitatively evaluating

the impact of the strategy thus far on student achievement.


- Artifacts from Team Meeting #6o Copy of Protocol Lo Completed Template L’s from each team membero One copy of the blank task (and rubric, if applicable) that generated the student worko One copy of each student work sample that was brought in by the teachero Copies of any additional evidence brought by teacher

Protocol L – Looking at Student WorkAdapted from the “Examining Student Work Protocol” from the NYCDOE Division of Teaching and Learning

and the “ATLAS – Learning from Student Work” protocol by National School Reform Faculty

Time: 45 minutesSupplies: pens, copies of Template L, copies of student materials brought by sharing teacher(s)Preparation note: The teacher(s) sharing evidence from their classrooms should have enough copies of each work sample for all teachers on the team; copies of a blank task (and rubric, if applicable) that generated the work samples should also be brought. In accordance with the team’s data collection plan, the sharing teacher(s) should prepare to share any additional evidence of impact and bring copies as they see fit (e.g. if the team is using a standard evidence collection form for holding conversations and observing students, then copies of that page with the teacher’s notes would be helpful to have).

Protocol note on times: When possible, it is recommended that student work samples from two teachers be reviewed in one session. Each step below has two times listed; the first time is the recommended time to spend on that step if only one teacher is presenting work from a task, and the second time listed is the recommended time to spend if two teachers are presenting. If two teachers are presenting, then complete all steps for the first teacher and then work through them again for the second teacher. For the first time using the protocol, work from one teacher should be reviewed to allow the group an opportunity to review and learn the protocol.

Step 1 (2-1 min): Teacher provides overview of the task- The teacher providing the student work gives a brief statement of the assignment to the group, including what

the students were asked to do and what the expected outcome was.

Step 2 (6-3 min): Review the task (and rubric, if applicable)- The teacher who brought the task and work samples distributes copies of the blank task (and rubric) to the team

members to review.- On Template L, teachers document notes about:

o What students need to know and be able to do in order to be successful with the tasko How the use of the instructional change strategy might help students to be successful with the tasko How the design of the task and rubric might influence student outcomes

Step 3 (8-4 min): Independent review of student work samples- Independently review each student work sample, one at a time, and make notes on your graphic organizer about:

o Student performance level (low, medium, or high)o How the instructional strategy was used by that student on the tasko The perceived impact of the strategy on the student’s performance within the focus issueo Student struggles with the use of the strategy or the task

Step 4 (8-4 min): Teachers share their notes- Teachers take turns sharing what they noticed about how each student in each category above.

Step 5 (2-1 min): Teacher shares any additional evidence of impact for the target group students (as applicable)- Teachers document notes on other evidence shared for each student (e.g. student observations or conversations).

Step 6 (8-4 min): Discuss common areas of success and areas of student struggles- Based on teacher findings, discuss themes that emerge pertaining to student success and struggle.

Step 7 (4-2 min): Discuss possible causes of struggle- Teams might consider the task design or topic, the implementation or utilization of the strategy, or anything else.

Step 8 (4-2 min): Identify next step for teacher- A next step for a teacher should be a very mall tweak that he or she can make quickly to the task, rubric, curriculum,

the implementation of the strategy, or anything else in order to help students to be more successful.

Step 9 (3-1 min): Discuss the implications on the action plan or strategy- If needed, discuss any modifications that the team plans to make to either the action plan or strategy.

43

Step 10: Document and collect student work samples as artifacts

44

Template L – Capturing Evidence of Impact DATE:

Teacher Sharing Student Evidence: ________________________________________________ Subject: ___________________________________

Task Review Task: _______________________________________________________________ Type of task: ______________________________________

In order to be successful, students need to know and be able to… How could the instructional strategy help students to be successful with this task?

How does the task reflect the focus issue? How might the design of the task (and/or rubric) impact student outcomes?

Review of EvidenceReview of Student Work Samples Review of Other Evidence Overall

Student #

PerformanceLevel

(High/Med/Low)

How was the instructional strategy used or applied by the

student?

Where did the student struggle in the use of the instructional strategy

or on the task in general?

What did other evidence (e.g. conversations, observations,

reflections) reveal about the strategy?

What is the perceived impact of the strategy on student

performance in the focus issue?

1

2

45

Student #

Performance Level

(High/Med/Low)

How was the instructional strategy used or applied by the

student?

Where did the student struggle in the use of the instructional strategy

or on the task in general?

What did other evidence (e.g. conversations, observations,

reflections) reveal about the strategy?

What is the perceived impact of the strategy on student

performance on the focus issue?

3

4

5

Common areas of struggle across evidence Possible causes for common struggles Common themes from all evidence regarding the strategy’s impact and the focus issue

Next Step for Teacher Modifications to Action Plan or Strategy, if necessary

46

Team Meeting #7 (with option)

- 22 min: L: Review Student Work & Other Evidence – Teacher Oneo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impact

- 22 min: L: Review Student Work & Other Evidence – Teacher Twoo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impact




o Facilitator brings: Copies of Meeting #7 Agenda, Protocol L, Template L

o Specific teacher(s) brings: Copies of student work samples from the students in their classrooms that are in the target



Meeting Option (Recommended): For this meeting, consider having two teachers bring in evidence to share.

47

Team Meeting #7

- 45 min: L: Review Student Work & Other Evidenceo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impact




o Facilitator brings: Copies of Meeting #8 Agenda, Protocol L, Template L Copies of the team’s finalized implementation rubric (Template J(A)) and a modified Template

J(B) for a mid-implementation self-assessmento Specific teachers brings:

Copies of student work samples from the students in their classrooms that are in the target group, as outlined in the data collection plan.

Copies of a blank task/assessment/activity/etc. that generated the work samples. Additional evidence of impact, as per the team’s data collection plan (e.g. notes from


Meeting Note: This meeting agenda provides time for two teachers to bring and share work.

Meeting Option (Recommended): For this meeting, consider starting the meeting (first 5 minutes) by having teachers complete a mid-implementation self-assessment using the implementation rubric on Template J(A) and a modified Template J(B).

48

Team Meeting #8 (with option)

- 5 min: J: Self-Assessment (mid-implementation)o Tool: Template J(A) – Implementation Rubrico Tool: Modified Template J(B) – Our Team’s Mid-Implementation Self-Assessment

- 20 min: L: Review Student Work & Other Evidence – Teacher Oneo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impact

- 20 min: L: Review Student Work & Other Evidence – Teacher Twoo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impact


- Artifacts from Team Meeting #8o Completed modified Template J(B) (1 per team)o Copy of Protocol Lo Completed Template L’s from each team membero One copy of the blank task (and rubric, if applicable) that generated the student worko One copy of each student work sample that was brought in by the teachero Copies of any additional evidence brought by teacher

Template J(B) – Our Team’s Mid-Implementation Self-Assessment DATE:



Item 2(0 – 2)

Item 3(0 – 2)

Item 4(0 – 2)

Item 5(0 – 2)

Item 6(0 – 2)

Item 7(0 – 2) Teacher’s self-identified next step

TEAM TOTALS

Where has our team showed the most improvement overall? What has been the most challenging component of the strategy to implement and why?

Are there any modifications that should be made to the rubric of implementation or to the checklist for implementation on Template J(A)?

49


o Facilitator brings: Copies of Meeting #8 Agenda, Protocol L, Template L, Protocol M, Template M

o Specific teachers brings: Copies of student work samples from the students in their classrooms that are in the target



Post Assessment Administration: Once the post-assessment is determined (and designed if needed), it should be administered prior to the meeting at which Activity N is explored (see note below).

The next meeting: Meeting #10, as written, is the session at which the results of the post assessment are examined (Activity O); however, if your team still has members that have not yet had the opportunity to bring in and share student work and other evidence from their classrooms, consider repeating the agenda and contents of Meeting #7 (one or two teachers per session) until all teachers have had a chance to share work from their classrooms.

50

Team Meeting #9

- 25 min: L: Review Student Work & Other Evidenceo Tool: Protocol L – Looking at Student Worko Tool: Template L – Capturing Evidence of Impact

- 20 min: M: Establish Post Assessmento Tool: Protocol M – Establishing a Post Assessmento Tool: Template M – Establishing a Post Assessmento Goal: To determine a post assessment to use as a starting point prior to implementing an

instructional change strategy

- Prior to closing: Ensure teachers know what they need to bring to the next meeting (i.e. post assessment results and final evidence, or more student work? See two boxed notes below).

- Artifacts from Team Meeting #9o Copy of Protocol Lo Completed Template L’s from each team membero One copy of the blank task (and rubric, if applicable) that generated the student worko One copy of each student work sample that was brought in by the teachero Copies of any additional evidence brought by teachero Copy of Protocol Mo Completed Template M

Protocol M – Establishing a Post Assessment

Time: 20 MinutesSupplies: pens, copies of Template M, chart paper, markers

Step 1 (3 min): Facilitator discusses the purpose of the post-assessment.Some points to mention:

The post assessment serves as the final data piece to determine students’ progress with respect to the focus via the use of the chosen strategy.

Post-assessment data is important because the team should compare students’ performance to the baseline, as well as to the other data sources and products throughout the process.

Post-assessment data will also inform the team as to where they need to continue to support students, as well as new areas of focus.

The baseline should be a common assessment that all students in the target group will take.A possible question:

Do all my students take the post-assessment or just the target population? o Since you are most likely using class time to administer your post-assessment, you should give it to all of

your students; separating the target population students is not necessary, or even encouraged.

Step 2 (5 min): Facilitator discusses some possible post assessments.Some points to mention:

Similar to the baseline (for more info on these points, see Protocol E):o You may use an existing assessment that will be or has been administered.o The post assessment needs to be a common assessment across classrooms.o You may also create your own post assessment.o The focus issue needs to be represented on the post assessment.

Ideally, the style and format of the post assessment is similar to that of the baseline.

Step 3 (4 min): Brainstorm: Group members individually brainstorm potential post assessments and rationales for why those assessments would make good baselines.

Step 4 (7 min): Facilitator elicits ideas from group, writing them on chart paper. Team will vote on which post assessment to use (or if it will be created). If it hasn’t been administered already, the team will decide when and how it should be given.

Step 5 (1 min): Person taking minutes will give a quick recap of meeting; provide overview of the next meeting.

51

Template M – Our Post Assessment DATE:

Brainstorm!What possible assessments might you use as a post assessment? Consider MOSL assessments, state assessments, periodic assessments, or other common assessments.

Chosen Post AssessmentWhat is the post assessment your team will be use? (if you will create one, see the section below)

Why is your post assessment appropriate for determining the students’ progress on the focus issue?

When will your team be administering the post assessment in their classes?

If you are creating a post assessment:What skills must the assessment address?

What will be the format of the assessment? (i.e. multiple choice, short response, essay, mixed). If the format is different from the baseline, what is the rationale behind choosing a different format?

Why is this post assessment a better alternative than the assessments that are already in existence? What is the rationale for creating your own post assessment?

When will the post assessment be created? Is the intended design of the assessment aligned to the baseline?

52

Phase 3: Assess – Analyze & Conclude


o Facilitator brings: Copies of Meeting #10 Agenda, Protocol N, Template N, Protocol O, Template O

o All teachers bring: Final pieces of evidence from their classrooms, in accordance to the data collection plan, to

share and review (may include additional student work samples, notes from interviews with students, results from student surveys or reflections, observations of students in class, classroom anecdotes, etc.)

Numeric post assessment results for all of their classes It is helpful if teachers have taken the time before the meeting to determine the

average performance of all students in all of their classes on the baseline assessment Copies of the baseline assessments for all students in target group (i.e. 3-6 papers per teacher)

53

Team Meeting #10

- 20 min: N. Analyze the Impact of the Strategyo Tool: Protocol N – Analyzing the Impact o Tool: Template N – Our Team’s Evidence of Impacto Goal: To review final evidence from each teacher’s classroom, combine it with all evidence reviewed

to date, and draw qualitative conclusions about the impact of the instructional strategy.

- 25 min: O. Analyze the Post Assessment Resultso Tool: Protocol O – Looking at Student Data o Tool: Template O – Our Students’ Post Assessment Resultso Goal: To review the quantitative results from the post-assessment and draw quantitative conclusions

with regard to how the instructional strategy impacted student performance.

- Artifacts from Team Meeting #10o Copy of Protocol No Completed Template N (1 per team)o Copies of any tangible final evidence brought and shared by teacherso Copy of Protocol Oo Completed Template O’s from each team membero Copy of a blank post assessmento The student work sample copies that were chosen by teachers and passed around

Protocol N – Analyzing the ImpactReference for this protocol:

Activity 12 of Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Time: 20 MinutesSupplies: pens, copies of Template N, teachers bring final evidence from their classrooms (as determined in their data collection plan)

Step 1 (3 min): Capture current thinking- Facilitator provides group with independent time to formulate and capture their current thinking on the qualitative

impact thus far of the instructional strategy.o Encourage them to write down statements that are supported by the evidence they have reviewed to date.

- A question that might surface:o Are we just considering impact on students in the target group? Nope! Any evidence-supported

statement or inference that you can make should be considered; the conclusion might be regarding the whole class (e.g. “the evidence shows that most students found the strategy to be useful, as evidenced by student surveys that we collected last week”), the target group (e.g. “students in the target group did a better job identifying the main idea, as evidenced by the quality of the work shared over the past few weeks”), or some other group. Just as long as the statement is backed up with evidence!

- Questions for each team member to consider (on Template N):o “Based on all evidence shared to date, describe your current thinking on the impact of the instructional

strategy on student performance pertaining to the focus issue.”o “What evidence has helped to inform your current thinking?”

- Possible sentence starters:o “The evidence shows that…” (on Template N)o “The evidence reveals that the impact on students/teaching/learning has been…”

Step 2 (3 min): Share current thinking- Facilitator leads teachers through a quick-share of their current thinking and the supporting evidence.

Step 3 (9 min): Share additional evidence- Each teacher takes 1-2 minutes to share additional evidence that they may have brought for this final round (such

evidence might include additional student work samples, notes from interviews with students, student surveys or reflections, observations of specific students by the teacher, classroom anecdotes, etc).

o As teachers share, they discuss what they believe the evidence suggests with regard to the impact of the instructional strategy.

o Listening teachers should be invited to share what they believe the evidence may also suggest and document on Template N whether the evidence shared by the teacher supports their own current thinking

Step 4 (3 min): Draw qualitative conclusions from the evidence- Facilitator leads the team to develop qualitative conclusion statements with regard to the impact of the instructional

strategy on student performance and the focus issueo Questions and sentence frames to consider:

“The evidence tells us…” (on Template N) “What has been the impact of the strategy on students’ ability to [focus issue]?” “What has been the impact of the strategy on your teaching practice?” “When students used the strategy to __________, they ___________” “When teachers implemented the strategy completely, ___________” “The use of the instructional strategy led to _________” “Students in the target group __________”

54

OverExamples

“The evidence tells us that all teachers shared their thinking aloud with their class to demonstrate how to support an opinion.”

“The evidence tells us that the quality of students’ responses depended on the prompts we provided.” “The evidence tells us that when we were explicit in asking students to provide reasons and support, they were

more likely to do so.” “The evidence tells us that students used personal experiences more than other types of support when backing

their opinions.” “The evidence tells us that descriptive feedback helped students improve the quality of their responses.” “The evidence tells us that the students in the target group improved.”

Modified from language in Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement, by Jenni Donohoo

Step 5 (2 min): Discuss implications of the conclusions on the instructional strategy- Is this a strategy that should be continued? Recommended to other departments?

55

Template N – Our Team’s Evidence of Impact DATE:Reference for this template: Activity 12 of Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Based on all evidence shared to date, describe your current thinking on the impact of the instructionalstrategy on performance of students (can be target group, whole class, etc.) pertaining to the focus issue. What evidence has helped to inform your current thinking?(“The evidence shows that…”)

Additional Evidence of Impact

TeacherType of Evidence Collected & Shared

(e.g. student work, interviews with students , student surveys or reflections, observations, classroom anecdotes)

What does this evidence reveal with regard to the impact of the instructional

strategy?

Does this additional evidence support your current thinking?

If so, how? If not, why not?

What qualitative conclusion statements can your team make? (“The evidence tells us…”) What are the implications of these conclusions on the instructional strategy?

56

Protocol O – Looking at Student DataAdapted from the “ATLAS – Looking at Data” protocol by National School Reform Faculty (NSRF)

Time: 25 MinutesSupplies: calculator, pens, copies of Template O, copies of completed Template K for comparison (containing baseline results), blank copies of the baseline assessment, teachers bring quantitative class results (e.g. their grade book) of their baseline assessment and copies of baseline assessment papers from their students that are in the target group (about 3 to 6 students)

Step 1 (8 min): Establish, share, and capture class averages- Provide teachers with time to calculate average performance on the baseline assessment for all classes, for their

students in the target group, for ELLs in their classroom, for SWDs in their classroom, and (optionally) for another subgroup of their choosing.

Step 2 (4 min): Review quantitative data- Facilitator distributes copies of baseline data and provides time for team to look at that as compared to the post

assessment data and draw inferences- Questions for facilitators:

o “What does this data suggest?”o “Which students performed better? Worse? About the same?” (use and X)

Step 3 (5 min): Pass and review student assessment samples- Each teacher should have brought with them copies of the assessment of the students in the target group in his or

her classroom (from about 3 to 6 copied assessments).- 1 min: Facilitators instruct teachers to each choose one assessment sample that they feel best demonstrates the

impact of the instructional strategy on student performance (could be either positive or negative).- 4 min: Group passes around the work sample pairs, taking about a minute (give or take depending on the number of

teachers in the group) to look at each assessments; keep passing until teachers get theirs back.o Facilitator encourages teachers to look for what the work samples reveal about student performance and

the focus issue.

Step 4 (4 min): Discussion of conclusions that can be drawn- Document group conclusions from the data and work sample analysis.

o Questions and sentence frames to consider: “The evidence shows…” (on Template O) “Students in the _______ subgroup increased their performance by ______% ” “When students used the strategy to __________, their performance improved by _______%” “From classroom to classroom, student performance overall ________” “Students in the target group (or other subgroup) __________” “The use of the instructional strategy led to _________”

- Group discusses alignment between qualitative conclusions (from Activity N) and quantitative conclusions (from Activity O)

o Question: “If there are any discrepancies, what may have caused them?”

Step 5 (4 min): Discussion of impacts of and implications for the instructional strategy- Questions to consider:

o “What are the implications of this data on the instructional strategy’s implementation?”o “Should the strategy be continued?”o “What modifications, if any, should be made?”

57

Template O – Our Student’s Post Assessment Results DATE:Adapted from the “ATLAS – Looking at Data” protocol by National School Reform Faculty (NSRF)

Post Assessment: __________________________________________________ Max Possible Score: ______

Student Post Assessment Results

TeacherOverall

Average(All students)

Average of students in

target group

Average of ELLs

Average of SWDs

Average of

___________

Team Averages

What does this data suggest?

In the table above, put a next to values that have increased since the baseline and an X next to those that have decreased.Which student groups performed better? Worse? About the same? Why might this be?

Looking at target group post assessments, what do they reveal about student performance and the focus issue?

What conclusions (quantitative and qualitative) can be drawn from the data above and work samples?Are these conclusions in alignment with the conclusions from Activity N?

What are the implications of this data on the instructional strategy’s implementation? Continued use?

58


o Facilitator brings: Copies of Meeting #11 Agenda, Protocol P, Template P Copies of conclusions documented on Templates N and O (for reference in Activity P) Copies of the team’s finalized implementation rubric (Template J(A)) and a modified Template

J(B) for a final self-assessment

59

Team Meeting #11

- 10 min: J: Self-Assessment (final self-assessment)o Tool: Template J(A) – Implementation Rubrico Tool: Modified Template J(B) – Our Team’s Final Self-Assessmento Goal: To draw conclusions about the relationship between the implementation of the strategy and

the impact of the strategy.

- 35 min: P. Conclude & Debriefo Tool: Protocol P – Drawing Conclusions & Debriefing o Tool: Template P – Our Team’s Conclusions & Next Stepso Goal: To finalize the conclusions from the team’s inquiry work, determine next steps for the

instructional strategy, and debrief the full cycle of inquiry for the purpose of improving it for the next time around.

- Artifacts from Team Meeting #11o Completed modified Template J(B) (1 per team)o Copy of Protocol Po Completed Template P (1 per team)

Template J(B) – Our Team’s Final Self-Assessment DATE:



Item 2(0 – 2)

Item 3(0 – 2)

Item 4(0 – 2)

Item 5(0 – 2)

Item 6(0 – 2)

Item 7(0 – 2)

TEAM TOTALS

What inferences can be made with regard to the relationship between the team’s implementation of the strategy and its impact on student performance?

Based on these inferences, what are the next steps for the team? How will this data inform the work of the team in the next cycle?

60

Protocol P – Concluding & Debriefing

Time: 35 MinutesSupplies: pens, copies of Template P, copies of the conclusions from Activities N & O for reference (Templates N & O)

Step 1 (4 min): Bringing it all together- Facilitator recaps the work of the team

o Surfaced learning needs in our different courseso Determined a common learning need across all of our classes that we feel is important to addresso Established a target group from whom to gather data, work samples, and evidenceo Determined a baseline assessmentso Decided upon an instructional change strategyo Developed a plan of action, a purpose statement, and an inquiry questiono Performed self-assessments on our own implementationso Looked at student work, data, post assessment results, and other evidence for the purpose of determining

the impact of our strategy on student learningo Analyzed the results and drew conclusionso Today we will finalize conclusions, formally answer our inquiry question, reflect, and determine next steps

Step 2 (4 min): Fulfilling the purpose statement- 1 min: Group members independently write thoughts about the extent to which the work of the group fulfilled its

purpose statemento Question: Did we stay focused on the purpose of our inquiry?

- 3 min: Group decides on final language to document on Template P.

Step 3 (12 min): Answering the inquiry question- 2 min: Group members independently write thoughts about the answer to the inquiry question.

o Refer group members to their own conclusions and to the conclusions documented on Templates N and O- 10 min: Group decides on final language to document on Template P.

Example 1The think-aloud is an effective strategy for teaching students how to form and support an opinion. Within a month, students in the target group improved in their ability to support an opinion. In addition to the think-aloud, opinion prompts impacted student learning. When explicitly asked for their opinion, students were more likely to state their opinion, provide detailed and relevant reasons for their opinions, and provide support for their reason. For prompts where an opinion was implicitly called for, about half of the students failed to provide sufficient support for reasons. The most frequent type of support students provided was personal experiences and the least frequent type of support used were comparisons or analogies. Every student indicated that the think-aloud strategy helped support their learning.

Example 2Teachers’ and students’ understanding and use of think-alouds increased. Teachers and students’ understanding of how to form and support an opinion increased. Levels of implementation varied from one classroom to the next. With practice, teachers and students became more effective developing and using think-alouds. When students were given opportunities to work with partners prior to working independently, the quality of their work increased.

Modified from language in Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement, by Jenni Donohoo

Step 4 (3 min): Draw conclusions about the instructional strategy

Step 5 (12 min): Reflection and next steps- 3 min: Reflecting on strengths and weaknesses as a group.- 3 min: Discussing the implications and value of the inquiry work on student learning and teacher learning.- 3 min: Planning Start/Stop/Continue for the next cycle

61

- 3 min: Groupthink about what’s next!

62

Template P – Our Team’s Conclusions & Next Steps DATE:References for this template:Activities 14-16, Resources J-K of Collaborative Inquiry for Educators: A Facilitator's Guide to School Improvement by Jenni Donohoo

Bringing it all togetherOur Focus Issue (Activity C): Our Instructional Strategy (Activity F):

Our Purpose Statement (Activity G): Our Inquiry Question (Activity G):

Finalizing our conclusionsTo what extent did our team’s inquiry work fulfill the Purpose Statement?

Reflecting on all reviewed student work, data, other evidence, and your conclusion statements, the answer to our team’s inquiry question is as follows:(Refer to your group’s conclusions from Activities O and P)

Over

63

What does the evidence suggest about continuing the instructional strategy?Should it be modified or adjusted, and if so, how?

ReflectingWhat were our group’s strength’s? What do we need to improve upon?

What are the implications of this work on student learning? On teacher learning?

For our next inquiry, we will…Start doing the following: Stop doing the following: Continue doing the following:

Based on what was learned during this inquiry, what are some initial thoughts for the next inquiry?What questions or other learning needs came up as you reviewed and analyzed evidence?

64

nysciresourcespd.weebly.comnysciresourcespd.weebly.com/uploads/9/1/6/2/9162895/... · Web viewex....

Documents

Transcript of nysciresourcespd.weebly.comnysciresourcespd.weebly.com/uploads/9/1/6/2/9162895/... · Web viewex....