Data Analysis - Tools and Processes · 2013. 10. 24. · Data Analysis - Tools and Processes...
Transcript of Data Analysis - Tools and Processes · 2013. 10. 24. · Data Analysis - Tools and Processes...
-
Data Analysis - Tools and Processes (School Level)
Food for Thought
How does your school use data to inform instruction and
improve student achievement?
Tuesday, February 21, 2012
Hawaii Department of Education Office of Curriculum, Instruction and Student Support
-
Webiquette
1. Before the meeting starts, close
any other applications running
on your computer.
2. Use the “Hand” icon
if you wish to speak
or have a question.
3. Mute your microphone
to eliminate ambient
noises.
4. Make sure that your
“Chat” box is set for
“Everyone.”
5. Use the “Chat” box
for questions.
6. Restrict the use of text-speak,
please respond using standard
English to text.
7. Your collaboration
is vital. Every
perspective
contributes to the
whole picture.
Muted
Unmuted
2
-
Group Norms for Webinar
Self-directed Learner
› Make personal connections to your position
Community Contributor
› Honor the expertise of ALL
Complex Thinker
› Synergize – Collective thoughts
Quality Producer
› Grow professionally
Effective Communicator
› Seek first to understand, then to be understood
Effective & Ethical User of
Technology
› Remove all other distractions
3
-
4
-
5
-
Hawaii’s Five RTTT Pillars
Improved Student
Outcomes
Data for School Improvement
Longitudinal Data System
Balanced Scorecard
Data Governance
Using data to inform instruction
Common Core Standards
Career & College Ready Diploma
Curriculum Framework
Common Instructional Materials
Formative Assessments
Interim Assessments
Summative Assessments
STEM
Focused support on
lowest-performing schools
-
Zones of School
Innovation
•Flexibility
•Great teachers and great
leaders
•Remove barriers to
learning
Performance-based
evaluation system
New Teacher Induction &
Mentoring
Incentives
Leadership development
Alternative pathways
Systems of Support to enable schools to do their best work – reprioritize and reorganize State resources;
establish Human Resources Unit in Zones of School Innovation; automate
6
-
Essential Question
How does data analysis help in school
improvement efforts?
7
-
Desired Outcomes
A common understanding of the various
purposes for analyzing data
An understanding of how to analyze data
using a variety of tools and processes
8
-
Data Analysis (School Level)
Agenda
Reason We Analyze Data
Basic Information We Use to Analyze Data
Processes We Can Use to Analyze Data
Finding Root Causes
9
-
Reason We Analyze Data
Why do we need to use data?
Why do we want to use data?
10
-
Why We Need to Use Data
-
Why We Need to Use Data
-
Why We Need to Use Data
-
Why We Need to Use Data
-
Why We Need to Use Data
-
Why We Need to Use Data
=
-
Why We Need to Use Data
•Formative Assessment / Instruction
•Data for School Improvement (DSI) as a formative assessment tool
•Using DSI Reports to inform instruction
•Deconstructing the Standards Process K-12
-
Why We Want to Use Data 1. Pick a number from 1-10.
2. Multiply that number by 9.
3. Add up the digits of the answer.
4. Subtract 5 from the number.
5. Find the letter that corresponds to the number (example: 1=A, 2=B, 3=C, etc.)
6. Think of a country whose name starts with that letter and write it down.
7. Take the 2nd letter in the country's name, and think of an animal whose name starts with that letter and write it down.
8. Write down the color of that animal.
9. Take the last letter of the country's name and write down an animal whose name starts with that letter.
10. Use the last letter of that animal and write down a fruit that starts with that letter.
18
-
Types of School Teams Principal
Counselor
Curriculum Coordinator
GL Reps/Dept Chairs
Kindergarten
First
Second
Teacher Teacher
Science
Math
Language Arts
Teacher Teacher
Leadership Team
Grade Level/
Department
Classroom
19
-
Take a Minute
How does your school use data to help
students and teachers succeed?
What types of data teams do you have at
your school?
20
-
State Goals
Vision of a Hawaii high school graduate is that
all public school graduates will:
Realize their individual goals and aspirations;
Possess the attitudes, knowledge and skills necessary to
contribute positively and compete in a global society;
Exercise the rights and responsibilities of citizenship; and
Pursue post-secondary education and/or careers without
need for remediation.
21
-
State Goals
22
Vision of a Hawaii high school graduate
There are six General Learner Outcomes (GLOs) that are the goals of standards-based learning in all content areas:
Self-Directed Learner: The ability to be responsible for one's own learning
Community Contributor: The understanding that it is essential for human beings to work together
Complex Thinker: The ability to be involved in complex thinking and problem solving
Quality Producer: The ability to recognize and produce quality performance and quality products
Effective Communicator: the ability to communicate effectively
Effective and Ethical User of Technology: the ability to use a variety of technology effectively and ethically.
-
Inverted Data Pyramid
Summative High Stakes Assessment
Other
23
N. Love, 2010
-
Data Pyramid: What kinds of data do
teams and coaches use?
Summative High Stakes
Assessments
Demographic, Process and
Perceptual Data
Benchmark Common Assessment
Formative Common Assessment
Formative Classroom Assessment
24
N. Love, 2010
-
Data Analysis (School Level)
Agenda
Reason We Analyze Data
Basic Information We Use to Analyze Data
Processes We Can Use to Analyze Data
Finding Root Causes
25
-
Using Data at the School Level
Where are we now?
• Requires measures and data
• What measures do you have available to help to determine what may be the problem?
Where do we want to go?
Where are we going?
• Determine the goal – know the target
• What are the needs of the school or the students?
How do we get there?
• Identify a strategy or process
• What will you put in place in order to achieve the outcome?
26
-
Too much data?
Diverts attention away from the primary purpose:
improving instruction.
Leads to overload – creating long, “comprehensive”
plans that few read.
Reveals too many things to address – so too many
goals and initiatives are created.
-
Multiple
Measures Demographics
Perceptions
Student Learning
Processes
28 V. Bernhardt
-
Demographic Data
Clarifies who our “clients” are.
Builds on the context of the school
Helps to predict future conditions to best serve the needs of our future students.
“Demographic information is crucial in data analysis as it helps us understand the context within which schoolwide change is planned and takes place.”
(V. Bernhardt, 1998, p-25)
29
-
Examples of Demographic
Data Number of students in the school
Number of students with special needs
Ethnicities of the students in the school
Number of graduates
Number of disadvantaged students
30
-
Tools –Finding Data Demographic
(Type any other places that you get this type of data into the Chat box)
Longitudinal Data System (LDS) http://employees.hidoe.k12.hi.us
-
Tools –Finding Data Demographic
(Type any other places that you get this type of data into the Chat box)
United States Census http://quickfacts.census.gov/qfd/states/15000.html
-
Tools –Finding Data Demographic
(Type any other places that you get this type of data into the Chat box)
Hawaii Department of Business, Economic Development & Tourism http://hawaii.gov/dbedt/info/census/
-
Tools –Finding Data Demographic
(Type any other places that you get this type of data into the Chat box)
School Documents Online http://iportal.k12.hi.us
-
Perceptual Data A view, judgment or appraisal formed in the mind about a
particular matter.
A belief stronger than impression and less strong than positive knowledge.
A judgment one holds as true.
“ In organizations, if we want to know what is possible . . .we need to know the perceptions of the people who make up the organization.”
V. Bernhardt, 1998,pg. 41
35
-
Examples of Perceptual Data
Observations
Person-to-person interviews
Telephone surveys
Focus groups
Parent surveys
36
-
Tools –Finding Data Perceptions
(Type any other places that you get this type of data into the Chat box)
School Quality Survey (SQS) http://arch.k12.hi.us
-
Process Data
Programs can include a wide variety of offerings, from
specially funded programs to academic curricular
sequences to extracurricular programs.
38
-
Examples of Process Data (Type any other places that you get this type of data into the Chat box)
Grant data
Program data
Comprehensive Needs Assessment (continuous improvement process)
Curriculum mapping
Data Teams
39
-
Student Learning Data Most important type of data to focus on.
Annual Large-Scale Assessment Data
Periodic Assessment Data
Ongoing Classroom Assessment Data
40
-
Examples of Student Learning Data
Hawaii State Assessment (HSA)
Terra Nova
DIBELS/DIBELS Next
Reading Inventories
Classroom Assessment Data
41
LDS
-
Tools –Finding Data Student Learning
(Type any other places that you get this type of data into the Chat box)
Accountability Resource Center Hawaii (ARCH) http://arch.k12.hi.us
-
Tools –Finding Data Covers Demographic, Perceptual, and Student Learning
(Type any other places that you get this type of data into the Chat box)
School Status & Improvement Report (SSIR) http://arch.k12.hi.us
-
Disaggregating Data
44
-
Disaggregating Data
Typically, student achievement data are reported for
whole populations, or as aggregate data. It is not,
however, until the data are disaggregated that
patterns, trends and other important information are
uncovered.
** Disaggregated data simply means looking at test
scores by specific subgroups of students.
45
-
Disaggregating Uncovers . . .
Achievement gaps are differences in academic
achievement amongst different groups of students.
It is important to examine these differences in order to
find ways that we can address some of the inequities.
The disaggregated data and the dialogue that arises
can transform beliefs and practices.
46
-
Trend Data
Data that shows a pattern over time.
Time is the variable over which one constant is being compared.
The more years of data that you have, the more reliable are the trends and patterns.
Statistically three years of data just barely indicates a trend. Five years provides more confidence to your inferences.
N. Love, 2008
47
-
Several Ways to Disaggregate
Data Gender
Socio-Economic Status
Mobility
Special Education and Disability
ELL – English Language Learners
Grade level
Classroom or course
48
-
Take a Minute
What student data/information does your
school use to make decisions?
49
-
Data Analysis (School Level)
Agenda
Reason We Analyze Data
Basic Information We Use to Analyze Data
Processes We Can Use to Analyze Data
Finding Root Causes
50
-
Using Data at the School Level
Where are we now?
• Requires measures and data
• What measures do you have available to help to determine what may be the problem?
Where do we want to go?
Where are we going?
• Determine the goal – know the target
• What are the needs of the school or the students?
How do we get there?
• Identify a strategy or process
• What will you put in place in order to achieve the outcome?
51
-
Problem Solving Processes
52
-
Root Cause Analysis - Process
Evaluate Programs
Improvement Planning
Determine Root Causes
Conduct Data Analysis
Define Problems
Organize Teams
53 P. Preuss, 2003
-
Structured Collaboration
BUILD
• Foundation
IDENTIFY
• Student Learning Problems
VERIFY
• Causes
GENERATE
• Solutions
RESULTS
54
N. Love, 2010
-
The Problem Solving Cycle
1. Identify the Problem
2. Describe Hunches and Hypotheses
3. Identify Question and
Data
4. Analyze Multiple
Measures
5. Analyze Political Realities
6. Develop Action Plan Resolution
7. Implement Action Plan
8. Evaluate Implementation
9. Improve the Process and
System
55
V. Bernhardt,
2011
-
Step 2: Analyze Data to
Prioritize Needs
Step 3: Establish SMART Goals
Step 4: Select
Specific Strategies
Step 5: Determine
Results Indicators
Step 6: Monitor and
Evaluate Results
Step 1: Conduct a Treasure
Hunt
Decision
Making for
Results –
Doug Reeves
56
-
Inquiry Cycle Annenberg Institute for School Reform
Establish Desired
Outcomes
Define the Questions
Collect and Organize the
Data
Make meaning of the data
Take Action
Assess and Evaluate actions
57
-
Data Analysis (School Level)
Agenda
Reason We Analyze Data
Basic Information We Use to Analyze Data
Processes We Can Use to Analyze Data
Finding Root Causes
58
-
Using Data at the School Level
Where are we now?
• Requires measures and data
• What measures do you have available to help to determine what may be the problem?
Where do we want to go?
Where are we going?
• Determine the goal – know the target
• What are the needs of the school or the students?
How do we get there?
• Identify a strategy or process
• What will you put in place in order to achieve the outcome?
59
-
What are the Causes?
60
-
“Data helps us get to the root
causes of a problem so we solve
the problem and not just the
symptom.”
V. Bernhardt
61
-
What is Root Cause?
The deepest underlying cause,
or causes, of positive or negative
symptoms within any process
that, if dissolved, would result in
eliminating or substantial
reduction of the symptom.
62
P. Preuss, 2003
-
Levels of Root Cause
External Level
Systemic Level
Programmatic or Process Level
Incident/Procedural Level
• Families
• Communities
• Supporting agencies
• Leadership
• Values/Beliefs
• Instructional Process
• Time
• Staff Development
• Students
• Teachers
• Incidents
63
-
When is a Cause a Root Cause?
Would the problem
have occurred if the
cause had not been
present?
Will the problem
reoccur as the result of
the same cause if the
cause is corrected or
dissolved?
Will correction or
dissolution of the cause
lead to similar events?
If yes, then it is a contributing
cause.
If no, then it is a
root cause
64
-
Tools for Root Cause Analysis
The Five Whys (Roots Toyota Corporation)
The Questioning Data Process (P. Preuss)
Causes for Student Learning Problems (N. Love)
System Planning Process (P. Preuss)
The Diagnostic Tree Process (P. Preuss)
65
-
The Five Whys – Why is it happening? Problem: 10th grade is not making significant progress in reading
Why? • Students not progressing are not doing work
Why?
• Students not progressing miss multiple days a quarter
Why? • Students miss instruction because they are not in class
Why? • Students are not in class because they babysit siblings
Why?
• Elementary school and high school schedules are different
66
-
Questioning Data Process (P. Preuss)
Step 1: Ask “What do you see in this data
set?”
Step 2: Ask “What questions do you
have about what you see?”
67
-
Possible Causes for Student
Learning Problems (N. Love) • Did we teach it? In enough depth? Placed in the right sequence?
Frequently enough? Curriculum:
• Did we use a variety of research-based instructional approaches? Are we sharing successful practices? Did we reteach using a different approach to individuals or groups who didn’t yet get it?
Instruction:
• Do we use ongoing formative assessment to explore student thinking and built on it in our instruction? Communicate to students how to improve? Help them self-assess?
Assessment:
• Did we examine attitudes or practices that might contribute to achievement/relationship/teaching gaps? Equity:
• Did we identify students who need additional help and provide them with it?
Individual Assistance:
• What knowledge/skills would help us improve student achievement? Teacher
Development:
68
-
System Planning Process –
EXAMPLE (P. Preuss) • 18 of 20 IEPs for high school students with
disabilities lack post secondary goals. Where are we
now?
• Post secondary goal statements must reflect goals after leaving high school
Where are we going?
• Involve the student. career interest Inventory
How will we get there?
• Making time for students to explore options after leaving high school
What is holding us back?
69
-
Sample Diagnostic Tree (P. Preuss)
HSA Math Scores below standard
Math Achievement Score in Grade 3
Student Demographics
Curriculum Instruction
Math Achievement Score in Grade 4
System Processes
Organizational Culture
Math Achievement Score in Grade 5
70
-
Data Analysis (School Level)
Agenda
Reason We Analyze Data
Basic Information We Use to Analyze Data
Processes We Can Use to Analyze Data
Finding Root Causes
71
-
How does data analysis help in
school improvement efforts?
My answer:
It allows us to see everything that may affect
student learning
It allows school level leaders identify ways to
support student learning
It complements classroom (instructional) level
data teams
It allows us to predict success!
72
-
Question & Answer
73
-
Thank you for joining us!
A recording of this webinar will be posted
on the Standards Toolkit website.
If there are any questions, please e-mail: Dewey Gottlieb, Mathematics Specialist
Monica Mann, Acting Administrator
Petra Schatz, Language Arts Specialist, or
Derrick Tsuruda, Science Specialist
74