Post on 17-Jan-2016
Measuring Child and Family Outcomes Conference Arlington, VA July 30, 2010
Looking for Patterns in Child Outcomes Data:
How to Examine Data for Red Flags
Donna Noyes, New York Part C Program
Lauren Barton, ECO at SRI International
Cornelia Taylor, ECO at SRI International
Session Overview
• Why do pattern checking?
• What are we looking for?
• Example: New York Part C Data
• Activity: Getting started looking for patterns
2Early Childhood Outcomes Center
What is Pattern Checking?
• A process of slicing and displaying your child outcomes data in different ways
• Reveals relationships between variables
• Does this data look similar to or different from
– Data for various subgroups
– Overall data observed in other states
– The relationship or pattern you would expect to see
3Early Childhood Outcomes Center
Searching for Red Flags
Searching for patterns - sometimes • Warning signs… • Look more closely, can this be
right?– Missing data?– Data quality?– Data analysis?– Program itself?
Early Childhood Outcomes Center 4
Early Childhood Outcomes Center 5
Available on the ECO website www.the-eco-center.org, ECO resources,
Quality Assurance, Data Quality (called the Pattern Checking Table)
Why Undertake Pattern Checking?
• Makes a case that your data are valid (or not yet)• Uncovers information about the quality of data –
confidence and/or concern• Spotlight on areas for further investigation • Provides clues about where to target resources
– Quality improvement– Program improvement
6Early Childhood Outcomes Center
Outcomes Measurement Approach
• Pattern checking is a process
• It works with any outcomes measurement approach
• Ways you split the data may change
• Still looking for data to show the same relationships
• What you do with the red flags you find may differ
Early Childhood Outcomes Center 7
Caveat!!
• Remember this is only weighing the pig
• Weighing the pig does not make it fatter
• Need to take what you learn from the analysis and do something with it
8Early Childhood Outcomes Center
Using Data
Evidence
Inference
Action
The numbers, facts
What they mean from thinking about the facts in context
Implications, what to do
Early Childhood Outcomes Center 9
State Example: New York Part C Program
10Early Childhood Outcomes Center
Pattern Checking Activity
11Early Childhood Outcomes Center
Example: Distribution of Progress Categories in Two States
Early Childhood Outcomes Center 12
Distribution of COSF Ratings at Entry in Two States
Early Childhood Outcomes Center 13
Progress Categories by Gender
14Early Childhood Outcomes Center
A B C D E
Males (n=312)
1.6% 28.4% 29% 23% 18%
Females(n=299)
1.0% 14% 26.5% 31% 27.5%
Crosstab of Entry and Exit Ratings on Outcome 1 (could use scores)
15Early Childhood Outcomes Center
Exit Outcom
e 1
Entry Outcome 1
1 2 3 4 5 6 7
1 7 1 1 0 0 0 2
2 20 16 4 1 2 0 0
3 33 36 26 5 8 1 0
4 16 28 39 1 4 12 4
5 22 62 72 40 24 9 8
6 34 63 106 64 93 26 11
7 20 39 58 44 44 31 14
Early Childhood Outcomes Center 16