Data BasicsData Basics

Clif St. Germain, Ph.D.

One of the most critical problems our schools face is the fragmentation overload and incoherence resulting from uncritical and uncoordinated acceptance of anything that seems to work.

Here is the problem:

Selecting instructional programs and processes “that work” is not enough to warrant inclusion as a viable strategy for improving the academic performance of our students.

What teachers and principals need to guarantee sustainable

progress is a process for identifying strategies and programs that create a

“multiplier effect” in that they impact academic

performance beyond what can be expected by average

progress.

This is where the appropriate use of data can be quite powerful.

If our goal is to create classrooms in which all

students can improve their learning potential and

demonstrate learning progress on end of year standardized

tests then…

Priority one must be to disaggregate relevant testing data to determine the level of impact learning support programs have upon different performance groups in existing student population.

Session One

Seminar objectives:

Revisit the basic principles of data diagnostics

Compare and contrast types of tests and data reporting scales

  Practice reading comprehensive testing histories

Review promising learning support strategies for disaggregated performance sub-groups

Work with sample testing histories to practice creating Within Group Learning Support Profiles for classroom teachers

Staying Clear About the Purposes of Data Disaggregation

Experts suggest that test data, appropriately used, provide schools (and school leaders) with a reliable mechanism for selecting appropriate learning support programs.

The decisions schools make in this regard significantly impact their

ability to show aggregate gains on standardized measures of

achievement. Knowing the appropriate intervention to select

is priceless.

Getting Out Ahead of Test Results

Getting Out Ahead of Test Results

For example, if a school’s data indicated a large number of students in the Special Needs group and that school did not provide extensive reading support it would be reasonable to expect that school to show no gain. Similarly, if a school provided reading support for all students, even Gifted students, the effort would be largely wasted.

The message here is knowing how to select and implement differentiated learning support programs is vital to showing across the board learning gains on state-mandated measures of achievement.

Getting Out Ahead of Test Results

Data As A Steering Mechanism Data As A Steering Mechanism for Accelerating Student for Accelerating Student Academic Progress Academic Progress

•Honor the natural variance of student abilities•Emphasize progress over attainment•Align school-wide learning support with within-group population distributions (i.e. students needing reading support)•Monitor within group year to year student progress

What is value-added assessment?

Value-added assessment is a new way of analyzing test data that can measure teaching and learning. Based on a review of students' test score gains from previous grades,

researchers can predict the amount of growth those students are likely

to make in a given year.

What is value-added assessment?

Thus, value-added assessment can show whether particular students -

those taking a certain Algebra class, say - have made the expected

amount of progress, have made less progress than expected, or have been stretched beyond what they could reasonably be expected to

achieve.

What is value-added assessment?

Using the same methods, one can look back over several years to

measure the long-term impact that a particular teacher or school had on

student achievement.

How is value-added assessment different from traditional measures of student performance? Student performance on assessments can be measured in two very different ways, both of which are important.

How is value-added assessment different from traditional measures of student performance? In the past, students and schools have been ranked solely according to attainment. The problem with this method is that attainment is highly linked to the socioeconomic status of a student's family.

For example, according to Educational Testing Service, SAT scores rise with every $10,000 of family income. This should not be surprising since all the variables that contribute to high-test scores correlate strongly with family income.

How is value-added assessment different from traditional measures of student performance? In contrast, value-added assessment measures growth and answers the question: how much value did the school staff add to the students who live in its community? How, in effect, did they do with the hand society dealt them?

If schools are to be judged fairly, it is important to understand this

significant difference. (Source: Theodore Hershberg Ph.D.)

NRT Data Reporting Scales

NRT Data Reporting Scales

NRT Data Reporting Scales

National Percentile Rank (NPR)

National Percentile Rank (NPR)

National Percentile Rank (NPR)

National Percentile Rank (NPR)

The percentile rank magnifies small differences in the middle raw scores while reducing differences in very high and very low raw scores. For example, a difference of only three raw score points might be necessary to move from a percentile rank of 50 to a percentile rank of 55, while a difference of seven raw score points might be necessary to move from a percentile rank  

National Stanine (NS)Stanines (Standard-nine) express test results in nine equal steps ranging from 1 (lowest) to 9 (highest).The average stanine is a score of 5. Stanines, which are similar to percentile ranks, are relatively easy to use because they are all one-digit numbers. The national stanine (NS) shows a student’s standing within the group of students in the same grade who took the test at the same time of year.

National Stanine (NS)Stanines for groups are useful for broadly identifying areas of academic strengths and weaknesses that might be represented by a set of test scores. A group of students with a stanine of 4 for Reading Total for example, shows that these students share a reading level that is slightly below average.

Stanine Percentile% of

Students9 - Highest Level 96 - 99 4%

8 – High Level 90 - 95 7%

7 – Well Above Average 78 -89 12%

6 – Slightly Above Average

60 - 77 17%

5 - Average 41 - 59 20%

4 – Slightly Below Average

23 - 40 17%

3 – Well Below Average 11 - 22 12%

2 – Low Level 5 -10 7%

1 – Lowest Level 1 - 4 4%

Within Group Distributions

National Stanine (NS)Stanines are less precise than percentile rank scores.

For example, percentile ranks of 24 and 40 both represent a stanine of 4. However, PRs of 23 and 24 are consecutive PRs that represent stanines of 3 and 4, respectively.

Reading Comprehension Sub-test

Standard Score (SS)

Standard Score (SS)

See handout for Sample IOWA NRT Survey Battery Score (Overall) /Year to Year Expected Progress

Questions:What do you know about a student in 5th

grade with a Standard Score of 263? Can you determine this student’s NPR,

Stanine?

Is this student a high, average or low performer?

 Why do you think there are some spaces

with empty scores?

Questions:What do you know about a 6th grade

student with a standard score of 216?

Can you determine this student’s predicted level for 7th grade?

What questions do you have about reading this score progression chart?

Comprehensive Testing History:

The Louisiana State Department of Education publishes student testing

histories for all students. These histories list the entire testing history of an

individual student. In those instances where the most recent testing data do not

provide ample diagnostic information to make an informed judgment of appropriate

learning support it is recommended that teachers print a complete

history.

Comprehensive Testing History:

Session Two

Within Group Learning Support ProfilesYour students can be disaggregated into five performance groups according to standardized academic indicators and overall literacy scores (survey battery).

A Note to Principals and Teachers

Each performance group generally represents students with similar academic needs. Similar to the Response to Interventions Model, the performance groups represented here can be expected to improve their collective performance with appropriate instructional support and progress monitoring.

A Note to Principals and Teachers

Note Well: Instructional interventions suggested for one group are not necessarily appropriate for others, nor are they intended to be substitutes for regular classroom learning experiences. Performance Groups with substantial percentages should be given considerable attention as you formulate your school improvement priorities.

Questions you might ask include the following: Is the percentage of special needs students

similar to the number of students presently approved for special academic services?

Students in the targeted assistance group are predicted to experience much higher levels of failure without additional time to complete basic school tasks. How can we arrange for these students to pair their reading, language and/or math classes with regular support linked to these subjects?

Questions you might ask include the following: Students in the maximum impact group, because

they have gaps in their prior knowledge due to poor organizational skills, or limited active working memory capacity, tend be underachievers. How can we guarantee that these students learn how to become more efficient learners?

Students in the gifted alert group are students who learn rapidly but whose development is asynchronous (uneven). In these children there is a great gap between mental abilities and physical and emotional development. For this reason these students are more vulnerable.

Promising Learning Support Disaggregated by Performance Groups

Special Needs Group: These students score in the well-below average range on the iLeap Survey Battery Score/Over All (NPR 1-10)

A check-list of “best practice” strategies known to enhance the performance of special needs students includes the following:

A check-list of “best practice” strategies known to enhance the performance of special needs students includes the following:

A check-list of “best practice” strategies known to enhance the performance of special needs students includes the following:It cannot be avoided that these students require intense elemental help. If their test results are accurate, expecting these students to achieve significant increases in performance in one year of school is not supportable. The goal with these students is long-term, incremental progress in all areas with intense emphasis on reading and basic levels of literacy. The social and emotional well-being of these students is also important to monitor. Providing on-going counseling support for these students is a must.

Promising Learning Support Disaggregated by Performance Groups

Targeted Assistance: These students score in the below average range on the iLeap Survey Battery Score/ Over All (NPR 11-22)

Promising Learning Support Disaggregated by Performance Groups – Targeted Assistance

The good news for this group is that they can be expected to show general improvement of their performance with effort and targeted assistance. Research supports providing these students with a double block of instruction in language and mathematics. The additional time should be used to guarantee that these students have the support necessary to successfully complete their language and mathematics assignments.

Recent initiatives using decoding reading skills programs have shown remarkable results with this performance group.

Suggestions for helping the Targeted Assistance group improve their performance in school:

Suggestions for helping the Targeted Assistance group improve their performance in school:

Suggestions for helping the Targeted Assistance group improve their performance in school:

Promising Learning Support Disaggregated by Performance Groups

Maximum Impact: These students score in the slightly below average to average range on the iLeap Survey Battery Score/ Over All (NPR 23-49)

Promising Learning Support Disaggregated by Performance Groups – Maximum Impact

This group of students, because they also represent a sizable percentage of your student population, will have a significant impact on school performance indicators used by most accountability systems.  

Promising Learning Support Disaggregated by Performance Groups – Maximum ImpactThese students are most often very capable learners who can improve their performance with a limited amount of instructional intervention. The most common barrier to their success is their lack of “with-it-ness.” They don’t know “how” to be successful in school. This means that they have not learned how to determine the difference between what is essential to know and what is less important. Over the years they have failed to develop efficient strategies for learning in school. The result is that their prior knowledge is sketchy at best. Therefore, they must work harder to initiate meaningful learning.

Promising Learning Support Disaggregated by Performance Groups – Maximum Impact

Here are a few suggestions for helping these students improve their performance

in school:

First check their reading comprehension scores to rule out a reading problem.

If the reading comprehension score is above the 24th percentile one or several of the following strategies are recommended:

If the reading comprehension score is above the 24th percentile one or several of the following strategies are recommended:

If the reading comprehension score is above the 24th percentile one or several of the following strategies are recommended:

If the reading comprehension score is

below the 24th percentile, reading

support will be required.

Promising Learning Support Disaggregated by Performance Groups

Class Strength/Gifted: These students score in the above to well above average range on the iLeap Survey Battery Score/ Over All (NPR 51-88/Gifted Alert NPR 89-99 respectively)

Promising Learning Support Disaggregated by Performance Groups – Class Strength/Gifted

These are students who thrive in challenging situations and are motivated by recognition for the quality of their intellect. However, many of these students “fall through the cracks.” Any student in the Gifted Alert or Class Strength category who is not experiencing success in school represents a missed opportunity to add a contributing member to your school’s learning community.

Promising Learning Support Disaggregated by Performance Groups – Class Strength/GiftedThese students require daily opportunities to question ideas, express personal opinions, solve challenging problems, hypothesize and direct their own learning. Frequent opportunities to go beyond the textbook, conduct research and stretch the limits of their thinking will keep these students meaningfully engaged in school.

The challenge for teachers is to create lesson formats that provide opportunities for these high performers to engage in the following:

The challenge for teachers is to create lesson formats that provide opportunities for these high performers to engage in the following:

The challenge for teachers is to create lesson formats that provide opportunities for these high performers to engage in the following:

It should be noted here that all students require a caring teacher

who is aware of the academic needs of each individual student and is

committed to providing high engagement learning activities,

personalized support, reinforcement and feedback as needed.

Classroom Learning Support Profile

This profile is designed to draw teacher attention to the specific type of learning support warranted by the performance levels of the students in this class. It is in no way intended for ability grouping of students.  

Classroom Learning Support Profile

Directions: Use the individual student’s Overall Performance score reported as a Stanine (NS) or a National Percentile Rank (NPR) equivalent to determine the appropriate performance range for each student. Then place the student’s name in the appropriate learning support group.  

Closing Table Talk Discussion

Internal Accountability

Value-added student performance data drives learning decisions

We…

We use Data to place students into classrooms

We…

We use Data to design learning support programs that are aligned to our student needs

We…

Our teachers know where our students are now and can describe exactly what meaningful progress looks like

Our goal is to use data to create classrooms that are more likely to succeed

Nobody falls through the cracks

The contents of this PowerPoint presentation were developed under a grant from the US Department of Education, #H323A110003. However those contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government.