Staff Perceptions of School Conditions What Are They? Do They Reflect Reality?

1The Center for Educational Effectiveness, Inc.

The Center for Educational Effectiveness, Inc.www.effectiveness.org

Organization

Lead

ersh

ip

InstructionStudentLearning


Organization

Lead

ersh

ip


Staff Perceptions of School Conditions

What Are They?Do They Reflect Reality?

WERAMarch 2008

Pete Bylsma [email protected]

Greg Lobdell [email protected]



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


• Field-based research, service, and data-centric tools to support School & District Improvement

• Partnerships with 500 Schools in 100 districts in WA• Assist all schools & districts in OSPI School & District

Improvement programs– What we do & how we do it varies based on serving districts from 80

students K-12, to districts over 30,000 K-12.

• WASL Analysis and “Educational Growth” repository – WASL growth data (student cohorts) for schools serving 380,000

students

• The largest database of school effectiveness information in the state of Washington (Nine Characteristics of High Performing Schools)– 40,000 Staff responses– 110,000 Students (30% from homes where English is not primary

language)– 38,000 Parents (30% from homes where English is not primary

language)

• Schools of Distinction: Creation and Design for OSPI and ongoing research into what’s happening in these buildings

Data and theCenter for Educational Effectiveness



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip




Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Partnership

• Expand Capacity• Provide Value-add services• Expertise in core areas• Partnership is critical

CEE Services

OSPI

ESDs

Districts

Schools & Classrooms

Demographics & Community CharacteristicsDemographics & Community Characteristics

PerceptualAcademic

AchievementContextual-Program and

Process



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Who We ServeWASHINGTON SCHOOLS

April 2006 WASL Data

0

0.5

1

1.5

2

2.5

3

3.5

4

0 10 20 30 40 50 60 70 80 90 100

Percent Poverty

Co

mb

ined

Rea

din

g /

Mat

h L

evel

In

dex

All Schools



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Who We ServeWASHINGTON SCHOOLS

April 2006 WASL Data

0

0.5

1

1.5

2

2.5

3

3.5

4

0 10 20 30 40 50 60 70 80 90 100

Percent Poverty

Co

mb

ined

Rea

din

g /

Mat

h L

evel

In

dex

All Schools CEE Clients



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Opening the CEE Repositories for Research

Activities• What?

– Educational Effectiveness SurveyTM: Staff, Student, and Parent editions

• Why?– Expands CEE capacity to get research

back into the field– Brings organizational effectiveness data

from 189,000 educational stakeholders in WA into critical conversations at all levels

– Improves our collective practice – impacts future research and development activities



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


The EES Staff Repository

• Demographics and descriptive variables– Respondents: Position, School Level,

Department, Length of service (both “this school” and “in education”), and ethnicity (listed as “optional”)

– Extensive research variables are attached at CEE as the data enters the repository

– N= 40,236 staff respondents as of March 1, 2008



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Position

Administrator-School

3%

Classified Support Staff

21%

Certificated Support Staff

7%

Certificated Teacher

64%

Para-Professional /

Instr. Aide5%

School Level

K-120%

High School24%

Elementary48%

Middle School23%

Junior High5%

Department or Level

Generalist K-3

17%Generalist 4-

68%

Other42%

Lang. Arts / Soc. Studies

14%

Electives8%

Math / Science

11%

Staff Descriptors



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Length of Service- In Education

0

2000

4000

6000

8000

10000

12000

< 1year

1 - 3years

4 - 5years

6 - 10years

11 -15

years

15+years

# of

Res

pond

ents

Length of Service- This School

0

1000

2000

3000

4000

5000

6000

< 1year

1 - 3years

4 - 5years

6 - 10years

11 - 15years

15+years

# of

Res

pond

ents

Staff Years of Service



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Ethnicity (optional) Other2%

African-American

1%

Asian / Pacif ic Islander

2%

American Indian / Alaskan Native

1%

Multi-racial2%

White / Caucasian

84%

Hispanic / Latino(a)

8%

Gender

Male, 6775, 26%

Female, 19383, 74%

Student Ethnicity: School-Level

Ethnic Majority44%

White Majority56%

Staff Ethnicity & Gender

School Makeup- Students



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


ESD by Respondents

ESD10526%

ESD1128%

ESD1134%

ESD11411%

ESD12127%

ESD12313% ESD171

5%

ESD1895%

ESD1011%

Geographic Segmentation

Large Central City1%

Rural, inside MSA17%

Rural, outside MSA10%

Small Tow n12%Large Tow n

5%

Urban Fringe of a Mid-Size City

10%

Urban Fringe of a Large City

22% Mid-size Central City23%

Title I Eligible

Eligible69%

Not Eligible31%

Title I Schoolwide

Schoolw ide52%

Not48%



Organization

Lead

ersh

ip



Organization

Lead

ersh

ip


Met AYP- Latest Year?

NO46%

YES54%

Federal School Improvement Step Number

Step 119%

Step 213%

Step 38%

Step 42%

Step 50%

None58%


Organization

StudentLearning


Organization

StudentLearning

The Challenge• Time is the educational leaders best friend and

worst enemy• Dissemination and use of broadly applicable

information and research takes far too long and often doesn’t reach the people who need it most

• Let’s not forget:– Ocam’s Razor: “one should not increase, beyond what

is necessary, the number of entities required to explain anything”

– Often shortened as: “all else being equal, the simpler solution is preferable” (The “K.I.S.” principle)

Staff Perceptions of School Conditions

What Are They?Do They Reflect Reality?

WERAMarch 2008

Greg LobdellPete Bylsma

Study Rationale• Self assessments are one element of the school

improvement process.• State Board of Education requires survey data from

staff, students, and family members to inform their annual school improvement plans.

• Staff typically examine their schools on the dimensions of effective schools and reflect on the results to address perceived shortcomings.

• No attempt to examine survey results across multiple schools, no norms when reviewing results

Concerns About Survey Results

• Self perceptions reflect self awareness• Self awareness may not reflect reality

Psychologists have documented our tendency to inflate ourselves and be blind to our shortcomings or suffer from “groupthink”

• If educators don’t have accurate perceptions of their condition, they won’t to identify their problems, which will lead to efforts not focused on the right solutionsNeeded changes won’t occur, outcomes may not improve.

Lead to discouragement, less effort to improve in the future, and a belief that external conditions are to blame for the problems

Phases of Learning

Unconscious Competence

Conscious Incompetence

Unconscious Incompetence

Conscious Competence

Conscious of Unconscious Competence

Matrix of Perceptions of School Quality

Unrealistic

Unaware of limitations

Realistic

Aware of success

Realistic

Aware of limits/success

Realistic

Aware of limitations

Unrealistic

Unaware of success

Staff Perceptions of School Quality (consciousness)

Student Outcomes adjusted for student characteristics

(competence)

HIGH

MEDIUM

LOW

LOW MEDIUM HIGH

Objectives and Scope

• Determine staff perceptions of school characteristics based on a school’s– Grade level (elementary, middle, high, multiple grades, alternative)– Size (number of staff)– Socioeconomic status (% eligible for free/reduced lunch)– Student performance (avg. % met on grade 4/7/10 reading & math

WASL)

• Results from 430 public schools in CEE database (only known data set across schools in multiple districts)– Not a representative sample: more likely to be poor, low performing– Schools enrolled about 25% of all students in the state– Surveys reflect the views of more than 19,000 staff– Only included results from first survey administration

• School-level analysis only

Survey Contents/Structure

• 86 statements covering 9 characteristics of high-performing schools (plus 14 other statements)

• Range of 5-16 statements per characteristic

• 5-point Likert scale1. Almost always true

2. Often true

3. Sometimes true

4. Seldom true

5. Almost never true

Survey Results: All Schools

3.55 3.58 3.66 3.73

3.98 4.02 4.08

3.803.823.79

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Com

munic

atio

n &

Colla

bora

tion

Fo

cuse

d P

rof.

Deve

lop

me

nt

Fa

mily

& C

om

munity

Invo

lvem

ent

Mon

itor

Teach

ing &

Le

arn

ing

Alig

ned

Sta

nd.,

Curr

.,In

st., A

ssess

.

Hig

h S

tanda

rds

&E

xpect

atio

ns

Suppo

rtiv

e L

ea

rnin

gE

nvi

ronm

ent

Cle

ar

& S

ha

red

Fo

cus

Effect

ive L

ead

ers

hip

ALL

CH

AR

AC

TE

RIS

TIC

S

Ave

rage

resp

onse

Scale: 1 Almost never true 2 Seldom true 3 Sometimes true 4 Often true 5 Almost always true

(All results are preliminary)

Survey Results By Grade Level

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Clear & SharedFocus

High Standards& Expectations

EffectiveLeadership

Commun. &Collaboration

Aligned Stand.Curr. Inst.Assess.

Monitor Teaching& Learning

FocusedProfessionalDevelopment

SupportiveLearning

Environment

Family &CommunityInvolvement

Ave

rag

e r

esp

on

se

Elementary Middle/Jr High High (Regular)

On every characteristic, elementary had highest scores, HS the lowest

Survey Results By Size

140120100806040200

Number of staff responses

5.0

4.0

3.0

2.0

1.0

Cle

ar

& S

ha

red

Fo

cu

s

Fit line for Total

Middle.Jr High

High

ElementaryGrade Level

R Sq Linear = 0.046

• Larger schools had lower perceptions, but only because HS were larger and had lower scores

• No variation by size at the grade level

Survey Results By SES• Weak negative relationship at elementary and middle

school levels (perceptions decline as poverty increases), no relationship for professional development

• Only one characteristic significant at HS level (PD, +)• SES had little effect in multiple regression

1.0000.8000.6000.4000.2000.000

Pct Low Income

5.0

4.0

3.0

2.0

1.0

Hig

h S

tan

dar

ds

& E

xpec

tati

on

s

Grade Level: Middle/Jr High

R Sq Linear = 0.177

Strongest Relationship Middle School Results for “High Standards and Expectations”(R-square = .177)

Most lines were very flat

Survey Results By WASLPositive relationship at elementary and middle school levels (perceptions increase as WASL increases), but no relationship for professional development or for any HS results

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Clear & SharedFocus

High Standards &Expectations

Effective Leadership Commun. &Collaboration

Aligned Stand. Curr.Inst. Assess.

Monitor Teaching &Learning

FocusedProfessionalDevelopment

Supportive LearningEnvironment

Family &CommunityInvolvement

Ave

rage

Sta

ff R

atin

g

Bottom 1/3 Middle 1/3 Top 1/3Results at elementary level

Survey Results By WASLStrongest relationship related to “Family and Community Involvement” at elementary and middle school levels

10080604020

WASL avg (reading & math), 1st survey yr

5.0

4.0

3.0

2.0

1.0

Fa

mil

y &

Co

mm

un

ity

In

vo

lve

me

nt

Fit line for Total

High Achieving Group

Middle Achieving Group

Low Achieving Group

ElementaryAchievement Level

R Sq Linear = 0.213

100806040200

WASL avg (reading & math), 1st survey yr

5.0

4.0

3.0

2.0

1.0

Fa

mily

& C

om

mu

nit

y In

vo

lve

me

nt

Fit line for Total

High Achieving Group

Middle Achieving Group

Low Achieving Group

Middle/Jr HighAchievement Level

R Sq Linear = 0.312

Perceptions and WASL Crosstabs• Schools grouped into three groups based on

achievement controlling for SES

• Schools grouped into three groups based on overall perception score for all characteristics

Student Outcomes (competence)

Staff Perceptions of School Quality (consciousness)

Hig h 6.3% 13.7% 14.5%n 24 52 55

M ed iu m 10.0% 12.4% 10.6%n 38 47 40

Lo w 14.8% 9.2% 8.4%n 56 35 32

Lo w M ed iu m Hig h

(N=379)

Summary and Implications

• On average, school staff rate themselves fairly high

• Most results seem to reflect actual conditions and are consistent with research on effective schools

– Elementary schools score the highest, high schools the lowest– Perceptions increase as performance increases

• Perceptions do not increase as school size decreases, not affected much by SES

• High schools and PD remain enigmas• Some schools appear to have inaccurate perceptions

• Need other assessments/surveys to gain less subjective data

• Schools can use these trends to compare themselves to others

• We need ways to compare similar schools to each other

Further Study Needed

We still need to understand …• Perceptions of alternative schools and those serving multiple grades

• What is happening with professional development activities

• Why similar schools generate such a wide range of perceptions

• How perceptions in the same school change over time

• How perceptions at the staff level (e.g., new vs. experienced teachers)

• What is happening in schools with unrealistically high & low ratings

• How staff perceptions compare to student and family views

• How the results are being used at the school and district levels

Staff Perceptions of School Conditions What Are They? Do They Reflect Reality?

Documents

Transcript of Staff Perceptions of School Conditions What Are They? Do They Reflect Reality?