Presentation For Gene S Revision 3

C E N TE R F OR E D UC ATI ON A L EF F E CT I V E N E S S , I N C .

Washington State UniversityAugust 12, 2009


Jack McCullough, Planning and Solutions Coach

The Center for Educational Effectiveness, Inc.


Group NormsParticipate in a positive manner.Actively listen to the viewpoints of your

colleagues. Disagree in a respectful manner.Take care of personal needs.Stay on task. Refrain from sidebar conversations.Have fun!


Today you will:

• Receive one set of Data Carousel Planning Templates per person and learn how to use it

• Share stories concerning common experience with data reviews• Refine your Theory of Action surrounding the use of data• Identify common concerns when proposing a data review• Receive one Data Primer and discuss how it might be used• Receive one Data Carousel Accelerator sample and discuss

how it might be used• Begin planning your data carousel/activity • Receive a list of potential resources.


“Even if you're on the right track,you'll get run over if you just sit

there."—Will Rogers


Creating the case for Common Language and Consistent

PracticeUsing Marvin’s Model

An engagement activity


A Culture of Inquiry

What comes to your mind?

Using Marvin’s Model


Remember adults have different learning styles and perspectives when working with data. It helps to remember the “beach ball.”


Data-analysis is like the ocean because…


Data

da·ta: noun; plural, but singular or plural

in construction, from the Latin, plural of datum. Factual information (as measurements or

statistics) used as a basis for reasoning, discussion or calculation.

Information output by a sensing device that includes both useful and irrelevant or redundant information and must be processed to be meaningful.

-Merriam Webster’s Collegiate Dictionary


“We are a society that is data rich but information poor.”

-Robert H. Waterman


Avoid the Drip

Data Rich Information Poor


Are these numbers considered data?

122 100

110 90 103

111 117


What kind of data are these?

122° 100°

110° 90° 103° 117°


We bring the meaning to the data.

122°F 100°F 110°F

90°F 103°F 117°F


Guiding Assumptions

1. Data have no meaning.Data are just information until people

organize, analyze and interpret meaning. Interpretation is subjective; data are

objective. Frames of reference influence the

meaning we derive from the data we collect and select.


2. Understanding should proceed planning.

Determine the desired outcome.Clearly define the problems.Cultivate collegial dialogue prior

to planning.


3. Knowledge is both a personal and a social construction.

Human beings are meaning-making organisms.

We sift through personal and social filters, forming beliefs and ways of knowing.

Individuals interact with information and with others shape new understandings about our world.


4. Cycles of inquiry

Inquiry, experimentation and reflection accelerate continuous growth and learning.

Learning occurs when we shift from professional certainty to conscious curiosity.

Constant pursuit of meaningful questions from thoughtful data analysis and ongoing monitoring of progress.


5. Norms of data-driven collaboration

Data alone leads to no action.Collective inquiry generates continuous

improvements.Meaning and action result from

professional learning communities that develop a shared commitment to improve student learning.

-Wellman, Bruce and Lipton, Laura.(2004).Data Driven Dialogue.


“Teachers blaze the path to knowledge when they purposefully use data as a

source for analyzing progress and proactively plan for improvement.”

Wellman & Lipton. (2004). Data Driven Dialogue.


School Improvement Planning:

Nine Characteristics

Of High

PerformingSchools

Nine Characteristics

Of High

PerformingSchools

Evaluateplan’s impacton student

achievement

Evaluateplan’s impacton student

achievement

Set and prioritizegoals

Set and prioritizegoals

Craft actionplans

Craft actionplans

Study and select research-based

practices

Study and select research-based

practices

Assess readinessto benefit


Collect sort andselect data


Build and analyzeportfolio

Build and analyzeportfolio

Implement andmonitor plan

Implement andmonitor plan

It is a Process


Benefits of data analysis

• It is more than solving a particular student learning problem

• School/District improvement teams become more efficient and effective

• Decisions making becomes ore collaborative

• Teachers develop more positive attitudes about their and their students’ abilities

• Educators feel more in charge of their own destinies

• Development of school wide culture if inquiry


Data Carousel Planning Template

CEE has created template sheets to assist your team in planning your data carousel activity.

You will spend time today using these sheets and will identify (today) many of your challenges for planning and executing a successful carousel.


Data Carousel Planning Template Steps

• Assess Readiness

• Planning Process

• Selection of Data

• Implementation

• Immediate Follow-up and Next Steps


Key Data Decisions– Depth & breadth of data– Carousel model– Presentation of data– Responding to Guiding Questions

Planning Process


Carousels are:

• A data sharing and exploration strategy

• A data analysis activity

• A process to identify needs and “next steps” in digging deeper

• Effective to engage multiple times per year


Carousel Models

• Traditional (i.e. SIP/SSIRG guide)

• Packet Method

• Large Chart Method

• Guided PowerPoint


Digging Deeper- Theme Carousel:Math Across the Curriculum

Needs, Goals == Researchand Action Planning

Theme -B Theme -C

Other Data Sources: EES, WASL Analysis, Local Assessments, Demographics

Basic Carousel

Information informs SIP Plan Steps 6,7, & 8 and next year’s revisions

School Performance Review Report

Chronology for Planning and Implementation


Data Carousel

• A means for engaging the entire staff in the process of data analysis• Typically 2-3 hours in length if done at one setting (My bias is not to

do it in one setting)• Intended to be a high level scan to determine trends, strengths and

concerns


• Arrangements– Space– Materials– Roles– Timeline/schedule– Food or snacks– Reminders– Distribute prep materials– Prepare facilitators

Implementation


• Time and People Decisions– Number– Stakeholders– Skills– Communication


• Data Training Requirements– 4 Domains of Data– Writing narratives– Types of carousel




Why are some schools successful and others not when implementing the same improvement strategies?

Readiness Guiding Question


• Willingness – attitudes, experiences, buy-in

• Process Skills– Decision-making– Conflict management– Problem-solving– Code of cooperation

• Roles We Play

Assess Readiness


Check Your Readiness

• Using the “Assessing General Readiness” worksheet discuss your school’s readiness to engage in the School Improvement Process and craft plans to respond to the challenges you foresee.


Basic or Initial Carousels

• All 4 Domains of Data• Designed to give large groups (i.e. all staff,

all certs, all certs+IA/ParaPros or greater “community stakeholders”) a broad view of information

• Contain “non-negotiables”


Process 1: Carousel

So, let’s say there are 4 tables for the 4 data groups…Staff are asked to look at the data and craft narrativesThey do this for about 20 minutesThen they move to the next tableRepeat until all data has been reviewed

Logistical Considerations:Who will be involved in the Carousel?What could you do to make it even more fun? A theme perhaps?Should staff be assigned tables?Snacks, meals and comfort of participants?


Pause and Reflect on what you saw and heard. What is running around in your head?


Where Do We Go From Here?

Teachers and principals alike assess student and teacher achievement early and often – and use the information to drive improvement rather than assign blame.

The key, however, is not simply that the successful schools have data – it’s who is using the data and how they use the data.

Beat The Odds (2006)


Guided Questions

• Help bring clarity

• Helps bring focus to more than one thing

• Helps bring focus to elements of leadership

• Guided Question Stem– “What evidence do I have…”


Well conceived guided questions should

• Inquire into the nature (what)

• Inquire into the quality (how well)

• Inquire into the frequency (how often)


Remember that with data analysis you are trying to define the problem, not solve it.


Triangulation -

Adding relevance and meaning through multiple data sources


Some guided questions to use when thinking about

Dr. Ken Jenkins UNC @Chapel Hill• Where are your widest

achievement gaps?• How persistent have

these gaps been?• Are there dramatic

difference from one year to the next? What might explain the differences?

• Are the gender difference worth noting?

• Is there any relationship you can determine between the population of free and reduced price lunch students and general student achievement?

• For High School, are there differences between major curriculum areas worth noting?

• What are the bright spots contained within the data?


Collect, Sort and Select DataCollect, Sort and Select Data

Has the team collected data from multiple indicators (i.e. student assessment, perception, demographic, school context)?

Has the team determined what data should be included in the school’s portfolio?

Has the team determined a process for allowing all stakeholders to analyze the data?

Has the team determined how the data will be displayed?


• Characteristics, Qualities and Types– 4 Domains– Formative– Summative– Longitudinal– Relevant– Reliable– Valid– Aligned with standards– Community sensitive

Selection of Data


Selecting Data

• From the data that has been collected you will need to purposefully select a subset for staff review.

• What questions do you want to investigate?• What do you believe the staff “cares about”?• Choose a reasonable (say 6-8 pages) amount for their

review.• What background knowledge will staff need to

interpret the data?


Demographics

Co

nte

xt

Pe

rcep

tion

s

Student Learning



Collecting Data




Collecting DataC

on

text

Pe

rcep

tion

s

Student Learning

Demographics

Free and ReducedESLSpecial PopulationsGenderEthnicityMobilityDropout Rates

Demographics

Guiding Questions:•Who are our students?•What trends do we see in our student population?•What trends do we see in our community?




Collecting Data

Demographics

Co

nte

xt

Student Learning

Pe

rcep

tion

s

Perceptions9 CharacteristicsTechnology

Guiding Questions:•How do the members of our school community feel about our school and district?•How satisfied are school community members with our educational programs?•What do the members of our school community perceive to be the strengths and needs of our school?




Collecting Data

Demographics

Co

nte

xt

Pe

rcep

tion

s

Student Learning

ContextHealthy Youth SurveySafe Schools DataDiscipline DataSchool Programs

Guiding Questions:•How successful are our programs in support of struggling learners?•What factors outside the school may be influencing student achievement?




Demographics

Pe

rcep

tion

sCo

nte

xt

Student Learning

Student Learning

WASLLocal AssessmentsClassroom Based AssessmentsGPA

Guiding Questions:•What evidence can we gather about our students’ learning?•What evidence can be gather about curriculum, instructional and assessment alignment to standards?•To what do we attribute our achievement trends?


Build and Analyze PortfolioBuild and Analyze Portfolio

•Has the team selected appropriate data from each domain?

•Is data displayed in a manner that is easy to interpret?

•Do staff members know how to craft narrative statements?

•Is there a process for engaging staff in review of data?

•Is there a model for reaching consensus?


School Portfolio

Grade 4 Reading Accountability Goals

55.158

6164

66.3

55.1

73.676.3 76.9

30

35

40

45

50

55

60

65

70

75

80

1997 1998 1999 2000 2001

SES TargetedScores

SES Scores

Elementary School Parents79 responses

Averages 5 = High 1 = Low

3.6

2.5

3.13.2

4.74.4

1.0

2.0

3.0

4.0

5.0

K-4 Class sizereductions

5-12 Class sizereductions

K-12 ExtendedLearning

Opportunities

AdditionalProfessionalDevelopment

Pre-Kindergarten

support

Improve SchoolBuildingFacilities

Total School - Staff167 responses

Averages 5 = High 1 = Low

4.04.4

3.2 3.2 3.23.7

1.0

2.0

3.0

4.0

5.0

K-4 Class sizereductions

5-12 Class sizereductions

K-12 ExtendedLearning

Opportunities

AdditionalProfessional

Development

Pre-Kindergarten

support

Improve SchoolBuildingFacilities

1 2 3 4 5A. Reduce class sizes in grades K-4 by hiring certificated

elementary classroom teachers 3 4 7 12 4.1

B. Make selected class size reductions in grades 5-12, such as middle school and high school writing classes 1 8 17 4.6

C. Provide extended learning opportunities for students in K-12, including but not limited to extended school year, extended school day, before and after school programs, special tutoring programs, weekend school programs, summer school, and all-day kindergarten

3 3 7 8 6 3.4

D. Provide additional professional development for educators5 3 7 7 4 3.1

E. Provide early assistance for children who need pre-kindergarten support 4 3 8 8 3 3.1

F. Provide improvements or additions to school building facilities directly related to class size reductions and extended learning opportunities

5 2 10 9 3.7

AveragePriority

Lowest Highest

District/Schools Baseline 1997

Goal 1998

Goal 1999

Goal 2000

Goal 2001

Fall City 65.1 68 70 72 73.8North Bend 53.4 56 59 62 65.1Opstad 60.2 63 66 68 70.2Snoqualmie 55.1 58 61 64 66.3

District 58.9 62 65 67 69.2

01020304050

6070

Reading Math Language

1999-2000 Grade 6 ITBS Scores

SVSD

State

Nation


• Data exploration during the carousel activity– Logistics – people, facility, movement of data,

# of copies, cost– Encourage open-mindedness


• During the carousel activity– Review why and process– Basic skill review– Allow all participants opportunity to see data– Narrative statements - process


Writing Narratives

• Keep it simple- Communicate a single idea.

• Make them short and easy to read

• Avoid Evaluation- Describe what you see, not what caused it or what to do about it


Criteria for Good Narratives

• Content– Describe building wide

performance– Describe trends in

performance over time– Describe high and low

performing groups– Compare performance in

your building with a benchmark for example statewide performance

• Format– Good Narratives

• Communicate a single idea about student performance

• Are short, clear sentences or phrases

• Are descriptive rather than evaluative

• Use everyday language that is easy to understand

• Are independent statement that incorporate numbers


Product 1: List of Concerns

• At the end of the Carrousel, the staff should have access to a list of concerns based on data

• You will need to determine the method for collecting concerns and returning them to staff


Process 2: Rating and Ranking

• The team should select a process for reaching consensus about the school’s priority concerns.

• We have used a rating and ranking activity– Staff is given printed copies of the concerns from the Data Carousel– They are asked to read for clarification (not allowed to lobby for or

against a concern)– They are also asked to eliminate any duplicates– Staff select their 5 greatest concerns– Staff assign points to their concerns (5 to 1) with 5 points assigned to

the greatest concern and 1 go the least– Public vote for each concern– Most points wins


Product 2: A prioritized list of concerns

• At the end of the Data Carrousel, the staff will leave with a list of prioritized concerns.

• Next step is typically a leadership team activity: Group concerns into themes and craft goal statements.

• This process results in a deeper understanding of the school’s data, allows for staff input regarding priorities, supports a transparent decision making process.


“I am tired of talk that come to nothing.It makes my heart sick when I remember all the good words and broken promises…”

Chief Joseph


Pre-Mortem ProcessLearning Improvement Team for Climbing Higher School/District

• You are a member of the school’s LIT charged with planning a data sharing activity with some “tough” data

• Reflect on the various “personalities” you might have to work with during the data review planning process

• Suggestions – Recall the following

• Principles of Adult Learning• 5 by 5 Whys


Comparison Perspective- 9 Characteristics of High Performing Schools Percent Positive Responses

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Readiness To Benefit

Clear & Shared Focus

High Standards & Expectations

Effective School Leadership

Collaboration, Communication &Community

Parent & Community Involvement

Supportive Learning Environment

Frequent Monitoring of Teaching &Learning

Focused Professional Development

Curriculum, Instruction, and Assessment

Sept 2008 May 2007

Note: Further from the center implies more positive responses

Copyright © 2006 Center for Educational Effectiveness. All Rights Reserved.


Comparison Perspective- Characteristics of High Performing Schools (Chart 1 of 2)

15%

19%

15%

22%

7%

13%

11%

18%

5%

7%

35%

32%

38%

42%

34%

32%

35%

38%

24%

32%

25%

29%

22%

26%

28%

28%

23%

23%

34%

41%

7%

12%

4%

6%

9%

16%

11%

12%

13%

14%

4%

7%

3%

2%

5%

8%

4%

6%

6%

4%

15%

1%

18%

3%

17%

2%

17%

3%

18%

3%

0% 20% 40% 60% 80% 100%

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Re

ad

ine

ss T

o B

en

efit

Cle

ar

& S

ha

red

Fo

cu

s

Hig

h S

tan

da

rds &

Exp

ecta

tio

ns

Effe

ctive

Sch

oo

l

Le

ad

ers

hip

Co

lla

bo

ratio

n &

Co

mm

un

ica

tio

n

AlmostAlways True

Often True

SometimesTrue

Seldom True

Almost NeverTrue

Missing



Comparison Perspective- Characteristics of High Performing Schools (Chart 2 of 2)

7%

15%

15%

19%

6%

10%

8%

8%

7%

13%

6%

9%

28%

38%

34%

38%

29%

30%

26%

35%

33%

39%

26%

34%

29%

32%

23%

27%

28%

34%

29%

33%

29%

33%

29%

37%

16%

12%

9%

10%

14%

18%

14%

14%

7%

9%

12%

12%

2%

1%

3%

4%

5%

6%

5%

6%

4%

2%

10%

3%

17%

1%

16%

2%

19%

2%

18%

4%

20%

4%

17%

4%

0% 20% 40% 60% 80% 100%

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Sept 2008

May 2007

Pa

ren

t &

Co

mm

un

ity

Invo

lve

me

nt

Su

pp

ort

ive

Le

arn

ing

En

vir

on

me

nt

Fre

qu

en

t M

on

ito

rin

g o

f

Te

ach

ing

& L

ea

rnin

g

Fo

cu

se

d P

rofe

ssio

na

l

De

ve

lop

me

nt

Cu

rric

ulu

m, In

str

uctio

n,

an

d A

sse

ssm

en

tD

istr

ict C

ha

racte

ristics

AlmostAlways True

Often True

SometimesTrue

Seldom True

Almost NeverTrue

Missing

Copyright © The Center for Educational Effectiveness, Inc., 2004



High Levels of Parent and Community Involvement

45%

53%

15%

32%

49%

17%

63%

70%

46%

43%

71%

27%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

This school encourages parent involvement

Our teachers effectively communicate student progressto parents

With important decisions we collaborate with parentsand the community

This school communicates effectively to families of allcultures

This school has activities to celebrate the cultures of itscommunity

The curriculum we teach reflects the cultures of thecommunity we serve

Sept 2008 May 2007


Most recent parent survey results

Parent & Community Involvement: Summary View

48%

47%

51%

45%

39%

34%

26%

35%

34%

29%

34%

36%

36%

38%

11%

11%

13%

11%

14%

16%

18%

2%

3%

3%

3%

4%

5%

6%

3%

3%

1%

3%

3%

4%

6%

2%

2%

4%

4%

4%

5%

6%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

This school respects the different culturesrepresented in our community

I feel welcome to visit this school at any time

When I share concerns with my child's teacher,he/she listens

This school has activities to celebrate differentcultures, including mine

This school schedules events at times convenientfor parents

Parents and families participate in importantdecisions about their children's education

I know many ways that I can be involved in thisschool

Almost Always Often True Sometimes True Seldom True Almost Never True Missing


Time to prepare…

• Take some time to review the readiness worksheet and consider the context of your data review.

• Craft some questions you would like to have your data address.

• Create your plan for engaging the staff in a Data Carrousel


Additional Resources

• Informing Practices and Improving Results with Data-Driven Decisions (August 2000-ECS (Education Commission of the States www.ecs.org Issued Paper)

• “The Flywheel Effect” by Timothy D. Kanold• “Buried Treasure-Developing a Management

Guide to Mountains of School Data”-January 2005 (Center for reinventing public education authored by Mary Beth Celio and James Harvey)

C E N TE R F OR E D UC ATI ON A L EF F E CT I V E N E S S , I N C . Source: “Addressing Barriers to Learning” Vol. 9, Number 4. Fall 2004.

From School Mental Health Project/Center for Mental Health in Schools, UCLA.


Questions?

• Don’t hesitate to call CEE – 425-283-0384 Sue is ext 1#, Greg is ext 2#, Jack at 425-444-6600 and Terry at ?

• OR you can email us:– [email protected]– [email protected]– [email protected]– [email protected]

Presentation For Gene S Revision 3

Education

Transcript of Presentation For Gene S Revision 3