Report Leveled Literacy Intervention

23
Fountas and Pinnell Leveled Literacy Intervention Spring 2011 Program Evaluation Leveled Literacy Intervention 1

Transcript of Report Leveled Literacy Intervention

Page 1: Report Leveled Literacy Intervention

Fountas and PinnellLeveled Literacy Intervention

Spring 2011Program Evaluation

Leveled Literacy Intervention

1

Page 2: Report Leveled Literacy Intervention

Fountas and Pinnell Leveled Literacy Intervention Evaluation Report

Table of Contents

I. Executive Summary pg. 3

II. Introduction to the Report pg. 7

a. Purpose of the Evaluation pg. 7

b. Audiences for the evaluation report pg. 7

c. Limitations of the evaluation pg. 8

d. Overview of report contents pg. 8

III. Focus of the Evaluation pg. 8

a. Description of the evaluation objects pg. 8

b. Evaluative questions used to focus the study pg. 9

c. Information needed to complete the evaluation pg. 9

IV. Brief Overview of evaluation plan and procedures pg. 10

V. Presentation of evaluation results pg. 12

a. Summary of evaluation findings pg. 13

b. Interpretation of evaluation findings pg. 13

VI. Conclusions and recommendations pg. 14

VII. Appendices pg. 15

Leveled Literacy Intervention

2

2

Page 3: Report Leveled Literacy Intervention

Executive Summary

Purpose

The Fountas and Pinnell Leveled Literacy Program began at Lyman Hall Elementary

School three years ago. The LLI Program is available for students in kindergarten through

fifth grade. The Leveled Literacy Intervention System is a small-group, supplementary

intervention designed for children who find reading and writing difficult. LLI is designed to

bring children quickly up to grade-level competency—in 14 to 18 weeks on average. LLI

serves those students who need intensive support to achieve grade-level competency.

These children are the lowest achieving children in the classroom who are not receiving

another supplementary intervention. Each lesson in the LLI system also provides specific

suggestions for supporting English language learners who are selected for the program.

An evaluation of the Leveled Literacy Intervention Program was conducted for a few

reasons. The program has been in place for three years and has never been evaluated. Doing

an evaluation was a good way to decide what is working with the program, what may need

to be changed, or what is not working with the program. Several questions were addressed

in the evaluation. The main questions addressed and researched for this program

evaluation is as follows: How many students recover before 14 weeks, during the 14-18

week period or do not recover at the completion of 18 weeks. What are the parent, teacher

and staff perceptions about the Leveled Literacy System?

Information Collection

When completing the LLI Program evaluation, vital information was gathered. The

primary source of collecting data for this evaluation was through formal DRA’s (directed

reading assessments) and informal DRA’s known as cold reads. The information gathered

Leveled Literacy Intervention

3

Page 4: Report Leveled Literacy Intervention

through these assessments were then used to answer evaluation questions related to

whether students are reading below grade level or reading on grade level. Interviews were

also conducted with parents, teachers, and staff members to better understand the

perceptions about the LLI Program.

Students had to answer the following questions when completing the DRA and cold

reads. As well as answering these questions students receive a percentage on their reading

ability. (see appendices DRA)

Look at the title and the pictures and tell me what you think is going to happen in

the book? (Prediction)

Start at the beginning and tell me what happened in the book? (Retelling)

What was your favorite part and why?

What connection did you make while reading the book?

By having students answer these questions many conclusions were made. Below is a list of

findings. A more detailed list with explanations can be found in the full evaluation report.

Findings

Out of 5 students evaluated for the LLI Program 0 of the students were on grade

level before 14 weeks.

Out of the 5 students evaluated for the LLI Program 0 of the students were on grade

at 14 weeks.

Out of the 5 students evaluated for the LLI Program 0 of the students were on grade

level after the 18 week completion of the program.

The findings from the parent, teacher and staff surveys were unanimous. The main concern

expressed through the survey was that the program benefits middle class society as well as

those students with a good English vocabulary. The students within this program are

Leveled Literacy Intervention

4

4

Page 5: Report Leveled Literacy Intervention

students who speak Spanish as their first language and English as their second. Their

families are below the poverty line, they are considered migrant students who move for

work and they do not receive help from home. The teachers and staff suggested piloting a

program that is specifically geared toward low end, second language learners.

Judgments

Overall the students and staff were very willing to help in data collection. The

evaluation indicates that the program is not recovering students to grade level reading

within the specified time, 14-18 weeks. The evaluation also shows that students are

becoming frustrated with their reading skills/abilities and are then choosing not to read.

Recommendations

After viewing all of the data, a list of recommendations has been made. Please refer

to the entire report for more information and explanations.

Continue to use the LLI structure of even days reading books and odd days

writing about the books, 5 minutes reread, 5 minutes phonics work, 15

minutes new book or writing depending on day and 5 minutes word work.

Seeing Star boxes can add to classroom word work to allow for

differentiation, more exposure and use of nonsense words (more

explanation is given under recommendations later in the report).

Instead of using LLI books use Lyman Hall Leveled Books developed for English

Language Learners on their independent reading level and instructional reading

level.

Leveled Literacy Intervention

5

Page 6: Report Leveled Literacy Intervention

Before taking a warm read or formal DRA allow the student to read a warm

read book (independent level) to get their confidence built and then

administer the cold read or formal DRA.

The LLI Program at Lyman Hall Elementary School is not working well and needs to

be improved. From the feedback and data collected it is obvious that the program is not

recovering students to grade level within the 14-18 week time line. With the

recommendations provided, the LLI Program can in turn reach below grade level students

and have them continue to grow in their reading levels.

Please review the full report for more information and data that reflects the information

stated in the Executive Summary.

Leveled Literacy Intervention

6

6

Page 7: Report Leveled Literacy Intervention

Introduction

Purpose of Evaluation

The main purpose of the Leveled Literacy System evaluation was to determine whether

the program was meeting the objectives that had been outlined in the description of the

program. This evaluation evaluates if the students receive the amount of support needed in

order to benefit from the system’s objectives. The main question that arises during this

evaluation was, are the students being able to close the reading gap after completing the

program? The evaluation of the Leveled Literacy System was conducted to research this

question to determine if this program was benefiting the student’s reading level. Data was

collected from administration and staff members who have first-hand experience with this

literacy program. This data gave us a picture of how this program was working within the real

world setting.

Audiences of the Report

This evaluation can help all staff members that are involved with the Leveled Literacy

System. Teachers who use this intervention system to improve their student’s reading level are

the targeted audience who will benefit the most from the outcome of this evaluation. Other

groups of people like parents and other school systems can use this evaluation as an insight as

to how the program works and how effective the program is. Each person who reviews this

Leveled Literacy Intervention

7

Page 8: Report Leveled Literacy Intervention

evaluation will be able to see the positives and negatives, as well as the areas in which

improvements can be made.

Limitations of the Evaluation

This evaluation only had several limitations. The main limitation was the teachers did

not have an universal recording system where all administrators could see the process of the

students who participated in the leveled literacy intervention. The teachers track their students

on their own and keep their recordings for their records. If there was a progress monitoring

sheet for each teacher to record the students’ progress then the administrators and other

teachers could easily keep track of each student and his progress within the system. This

progress monitoring sheet would be saved on the school drive and would be created to use for

the recording of the Level Literacy System’s outcomes.

Overview of the Report Contents

This report includes a description of the Fountas and Pinnell Leveled Literacy System

and how it is used to increase the reading levels of struggling students. The audience of this

evaluation will learn what questions were important when evaluating the system. Included is

also a description of the evaluation plan and procedures which were used to evaluate the

system. A summary and interpretation of the findings from the research of the level literacy

system are also included as a part of the evaluation. In the final section, recommendations and

conclusions are made for this intervention program.

FOCUS OF THE ELVAUATION

Description of the evaluation object:

The evaluation focuses on the Fountas and Pinnell Leveled Literacy Intervention System.

This intervention program is designed for a small group setup. The system is not a reading

Leveled Literacy Intervention

8

8

Page 9: Report Leveled Literacy Intervention

program, but a supplementary program to be used in addition to the reading program that is

already in place for that school system. The small group who received the supplementary

program are those students who need intense support to reach grade level competency.

Therefore, the students who receive the leveled literacy intervention are low-achieving children

who are not receiving other forms of supplementary intervention.

The leveled literacy intervention is a short term program that has been proven to bring

students up to grade level competency in reading in fourteen to eighteen weeks. It is broken

into three systems that support different levels of learners. Students are tested using the

running record testing program to determine which system (orange, green, or blue) that the

child needs to begin the appropriate support instruction.

Each system contains a collection of at least seventy books. For each book, there is a

lesson for the teacher to follow. Within each lesson, there is an emphasis on reading skills as

well as writing skills. The specific components of each lesson are listed below:

Comprehension strategies

Attention to fiction and nonfiction texts

Specific work on sounds, letters, and words (and how words “work”)

Vocabulary practice

Fluency and phrased reading

Writing about reading to practice writing strategies

Evaluative questions used to focus the study:

How many students recover before 14-18 weeks, during the 14-18 week period

or not recover at the completion of the 18 week?

What are the parent, teacher, and staff perceptions about the leveled literacy

system?

Leveled Literacy Intervention

9

Page 10: Report Leveled Literacy Intervention

Information needed to complete the evaluation:

Before beginning the evaluation information had to be collected on the Leveled Literacy

Intervention Program. Individual student’s formal and informal DRA assessments were collected

to evaluate their progress throughout the program. This is the greatest source of information for

us to use as we measured the effectiveness of the program. The formal DRA assessment was

given twice during the program and the informal DRA assessment was given every two weeks

with a total of sixteen assessments. Depending upon the score on these assessments

determined whether the student would then revisit the current level or progress to a new level

and which level they would progress to.

Brief Overview of Evaluation Plan and Procedures

Before this evaluation began, there were extensive steps to complete the background

research for the Leveled Literacy System. Research was completed to gain information

about the Leveled Literacy System. There were documents that were collected to

understand the objectives and goals of the program. Documented outcomes of students

who completed the program were analyzed to determine exactly how the program is

working in a particular school system. The main focus of the evaluation was discussed

based on items collected. The evaluation questions were formed after these items had been

analyzed.

The second step of the evaluation was to complete an evaluability assessment. This

assessment is a tool that serves a communication between the stakeholders and evaluators

to make sure the program is worthy of the evaluation. Everyone that was involved in this

evaluation was a part of the evaluability assessment. Teachers and parents were able to

share concerns that were taken into consideration for the assessment. Including all persons

involved allowed the evaluators to see if the program was meeting the needs of all.

Leveled Literacy Intervention

10

10

Page 11: Report Leveled Literacy Intervention

After reviewing the evaluability assessment, the decision was made that the

evaluation would be completed. The idea was discussed that the outcome of this evaluation

would help school systems across the state to determine if the program would work based

on the specific needs. Documentation has been withheld from testing, parent and teacher

surveys, and student achievement to provide the information needed throughout the

process.

After the evaulability assessment was complete, the next step of the evaluation was

to create the standards and criteria. Each standard was used to evaluate the success of each

question in the evaluation. The standards and criteria that were produced followed along

with the Georgia Performance Standards for the specific grade level that it was being used

in. The standards also align with the evaluation questions that were produced at the

beginning of the evaluation.

A spreadsheet was created to organize all information that summarized the

evaluation plan. Within this spreadsheet, one could determine the evaluation questions,

information required, design, information source, method of collecting information,

sampling, information collection procedures, schedule, and analysis procedures. It was

determined that the data collected would come from teacher information about the success

of students within a specific grade level.

Finally, the plan was placed into action when the data collection process began.

Students were monitored as they completed the Leveled Literacy System with their

homeroom teacher. This tool was used to generate a specific percentage of students who

were closing the reading gap before the 14 week period, during the 14 – 18 week period, or

not closing the gap at all. We compared these results with the results of the teacher surveys

to obtain any relation between the program outcomes and the administers.

Leveled Literacy Intervention

11

Page 12: Report Leveled Literacy Intervention

Presentation of Results

Summary of Findings

Conclusions were drawn based on test scores from Running Records and Developmental

Reading Assessments. Feedback from parents and teachers were also used to make

conclusions. Zero percent of students between the 14-18 weeks of intervention, closed the

reading gap to meet grade level peers and grade level standards according to the test results.

When conducting the online survey, we had lack of parent response, and ninety-five

percent of teachers responded. Ninety-five percent of teachers felt that the Fountas and Pinnell

program was not an efficient reading intervention program. As mentioned earlier the report the

program is too repetitive and students lack motivation for moving up levels. It was also stated

that students were frustrated and bored with the Leveled Literacy Intervention program causing

lack of motivation to build fluency and comprehension. A first grade teacher stated that

students need hands-on learning manipulatives and visuals to manipulate diphthongs, diagraphs

and letter sounds to build fluency. Another first grade teacher, expressed concerns with the lack

of background knowledge and vocabulary for second grade content due to lack of parent

involvement. Zero percent of parents participated in the online survey of the Fountas and

Pinnell Literacy Program.

Interpretations of the Findings

The evaluation indicates that the program is not helping students close gaps in order to

meet grade level reading competency. Because students are bored with the repetitive nature of

Leveled Literacy Intervention

12

12

Page 13: Report Leveled Literacy Intervention

the program, they are unable to view themselves as successful grade level readers. It is also

mentioned that the program is designed for middle class students who speak only English. The

main concern expressed through the survey was that the program benefits middle class

society as well as those students with a good English vocabulary. The students within this

program are students who speak Spanish as their first language and English as their second.

Another concern is that when using the program, teachers do not have the opportunity

to make the learning meaningful. Research states that when students internalize and make

connections to their reading they are more likely to build fluency and comprehension. Due to

the students’ lack of exposure to reading content and vocabulary, they are unable to make

connections. These students’ first need to understand why it’s important to be able to read,

then they will be more likely to internalize and connect to the content; lack of parent

involvement creates a huge barrier for these students.

Conclusions and Recommendations

After reading the data, a list of recommendations with explanations has been made. Since

teachers believe the root of the problem is the focus on students who speak one language and

the repetitive nature of the program, suggestions have been made to incorporate more hands

on, visual learning.

1. Teachers can incorporate graphic organizers to aid in teaching comprehension

strategies. Use Lyman Hall Leveled Books developed for English Language Learners on their

independent reading level and instructional reading level; then have students create graphic

organizers in small groups to connect to the content.

Comprehension strategies can be taught through making hands on material that can

help students visualize and connect to the content. For students to find the plot of the story,

they could use paper to create a house. At the top of the house, they could write or draw about

Leveled Literacy Intervention

13

Page 14: Report Leveled Literacy Intervention

what happened in the beginning of the story, in the middle of the house they could write or

draw about the middle of the story and at the bottom of the house they could write or draw

about what happened at the end of the story; students have now created a graphic organizer.

Before assessing comprehension by using a cold read or formal DRA allow the student

to read a warm read book (independent level) to get their confidence built and then

administer the cold read or formal DRA.

2. Teachers could incorporate Seeing Stars boxes into daily reading rotations. The teacher

holds up a nonsense word (not a real word) from the box. She then has students use letter tiles

to make the word. A student reads the word and then the teacher asks the student to delete or

add letters to create a new word. At first, the goal is for the teacher to have the students

create words they can easily recognize to build self-esteem then, the teacher has the students

build more difficult words as they become more comfortable. Seeing star boxes allow teachers

to individualize instruction for phonemic awareness, sight words and spelling, which also builds

grade level fluency. On Monday, the teacher would get a baseline and on Friday, teachers will

assess the phonemic awareness skill worked on for the week. For example if the teacher worked

on long vowels she would get a baseline on Monday to see what the students already know.

Throughout the week, the teacher would focus on long vowels using Seeing Star Boxes during

reading rotations. On Fridays, she would assess words the students have never seen to get the

results and help guide her instruction for the next week. If the students needed more work on

long vowels, she would continue to work long vowels and create new words for the following

week to assess.

Leveled Literacy Intervention

14

14

Page 15: Report Leveled Literacy Intervention

Appendices

DRA short form for recording reading accuracy and comprehension.

Leveled Literacy Intervention

15

Page 16: Report Leveled Literacy Intervention

Leveled Literacy Intervention

16

16

Page 17: Report Leveled Literacy Intervention

Leveled Literacy Intervention

17