A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR...

A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES

TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH

SCHOOLS IN TEXAS

ARTHUR L. PETTERWAY

Submitted to the Graduate School

Prairie View A&M University

in partial fulfillment of the requirements for the degree of

DOCTOR OF PHILOSOPHY

May, 2007

Dr. Will itsonis

Dr. Camille ibson Dr. David Herrington

F Dr. . Paul M Dean, The Whitlowe R. Green

I

. Dou las Hermond HO Aid- ie.

ck_

e of Education

A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES


SCHOOLS IN TEXAS

A Dissertation

by

ARTHUR L. PETTERWAY

Approved as to style and content by:

L. M. Paul Mehta, Chair

Dr. William Parker Dean, Graduate School

May 2007

iii

ABSTRACT

A Mixed - Method Analysis of the Impact of High Stakes Testing on

English Language Learners in Major Urban High Schools in Texas

April, 2007

Arthur L. Petterway: B.A. – Dillard University

M.Ed., Prairie View A&M University

Dissertation Chair: Dr. M. Paul Mehta

Ample research has been conducted on the intrinsic validity of

standardized assessments, and on the factors affecting the assimilation

and integration of English language learners (ELLs). The reliability of

these assessments as a universal tool to measure student learning, and

as a basis for determining school performance needed closer

examination.

The purpose of this study was to determine the impact of high-

stakes testing on ELLs. This was shown in both the quantitative and

qualitative dimensions of the study. Data obtained from Texas Education

Agency (TEA) were used to determine whether there was a relationship

between the percentage of ELLs enrolled in a school and the percentage

of all students passing the 10th grade Texas Assessment of Knowledge

and Skills (TAKS) tests in the core areas of English Language Arts and

Mathematics given in 2003, 2004, 2005, and 2006. The qualitative

aspect of this study explored what certified English as a Second

Language( ESL) teachers, non-certified ESL teachers who teach ELLs,

iv

administrators, and district ESL personnel viewed as the impact that

high stakes standardized assessments had on ELLs, ESL curriculum,

and instruction in ESL classrooms.

This study determined the impact of high-stakes testing on ELLs

using the explanatory design of mixed method analysis. Data of 173

major urban high schools obtained from the Texas Education Agency

(TEA). It was determined through the Pearson correlation computations

using the Statistical Package for the Social Sciences (SPSS) that there

was a significant relationship between the percent of ELLs enrolled in a

school and the percent of all students passing the 10th Grade TAKS tests

in English Language Arts and Mathematics. In the qualitative portion of

the study, the views and opinions of district ESL personnel were

gathered. Principals, assistant principals, ESL and non-ESL teachers

took part in an on-line, open-ended questionnaire; one-on-one

interviews; and focus groups. The focus groups addressd the purposes

of statewide testing; its intended consequences; problems and changes

created by TAKS, and the recommendations to improve ESL curriculum

and instruction.

The results of the study affirmed the expected outcome that a

significant relationship existed between the percentage of ELLs enrolled

in a school and the percentage of all students passing the 10th grade

TAKS tests in both core areas of English Language Arts and

Mathematics. The regression analysis predicted that as the percentage of

v

ELLs in a school increased, the performance on the statewide, high-

stakes testing in terms of all students passing the 10th grade TAKS tests

decreased. Respondents of the study considered TAKS as a tool to gauge

knowledge in the different core areas. English language learners were

expected to have at least average scores on TAKS. There was a difference

in the expected and actual results; respondents observed dismal or

failing performance of ELLS in the actual results in TAKS. This was

evident by the high failure rate of ELLs in their respective schools. Higher

dropout rate and lower graduation rate of ELLs were problems

encountered due to TAKS. Respondents favored a different test for ELLs,

possibly given at a later date after ELLs had studied in the country for at

least several years. Respondents believed that interventions were needed

to help ELLs perform better. Both the school and the home, together with

the community, have to be involved in preparing ELLs for their present

and future roles in the American society.

Results of this study may provide valuable data to district and

school administrators to develop strategies that will improve the

performance of ELLs on the statewide, high-stakes testing and to develop

assessments that truly measure learning without the nullifying effect of

linguistic and cultural bias. The study may also help to enhance the

reliability of standardized assessments as a tool to determine

accountability for student performance.

vi

DEDICATION

This dissertation is dedicated in humble gratitude to my Lord and

Savior, Jesus Christ, in whom I move, trust, and have my being.

Through HIS divine wisdom and purpose, HE gave me my parents:

Bob Stevenson Petterway

November 23, 1923-September 28, 1992

and

Myrtice Lee Petterway

February 10, 1927-February 15, 1959

They are now in Glory with HIM sharing this divine blessing that HE has

bestowed upon me.

vii

ACKNOWLEDGEMENTS

I would like to acknowledge those without whom this work would

not have been possible. First and foremost I wish to acknowledge the

blessings bestowed upon me by my Lord and Savior Jesus Christ.

Among these blessings are the kind souls I mention here.

First, I would like to acknowledge my dissertation committee,

Dr. M. Paul Mehta, Dr. William A. Kritsonis, Dr. Douglas Hermond,

Dr. David Herrington, and Dr.Camille Gibson. They have been thorough,

fair, understanding, demanding, and, most of all, dedicated to academic

excellence in all phases of this work. I would especially like to thank

Dr. M. Paul Mehta for taking time, along with his duties as the Dean of

Education, and carrying the baton of being my committee chairman and

Dr. Robert Marshall, my former committee chairman (who would not let

Me fall off of my bicycle and taught me how to eat an elephant) for the

long hours they have spent and the endless patience they have shown as

they have guided me through this endeavor. I also wish to thank

Dr. Kritsonis for his passion for making sure that I get published before I

receive my PhD and taking on the duties of head cheerleader for this

project. Thanks goes to Dr. Hermond for serving as lead statistician and

for not allowing me to attempt to eat the whole pie and limiting me to a

small slice. Many thanks also to Dr. Herrington who guided me in tying

up several loose ends. I am also grateful to Dr. Gibson for taking time out

of her very busy schedule to offer her support and encouragement.

viii

Additional thanks go to Dr. William Parker, Dean of Graduate Studies,

for believing in me and giving me the opportunity to prove myself

beginning with my pursuit of my Masters Degree.

I also gratefully acknowledge all of my professors and thank them

for the wisdom and knowledge they have so generously shared. I would

further like to thank all of the faculty and staff of Prairie View A&M

University who have contributed to my achievement in countless ways. I

also wish to acknowledge the unwavering encouragement of my student

cohort as we shared the joy and pain of this incredibly challenging

pursuit.

I would like to thank my principal, Mrs. Linda Llorente, for her

understanding and support as I pursued this dream. She has been

abundantly patient and understanding of the demands this work has

placed on my time and energy. I also wish to acknowledge the

contributions of my peers and colleagues at Austin High School.

Although it is virtually impossible to name all who have contributed to

the completion of this work, I feel that there are several who must be

thanked by name. I would like to thank Andy Lamboso and Rhodora

Maligad who helped with proofreading and typing along with Kathy Koch,

Betty Shaw, Debbie Kubiak, and Raul Asoy who helped with proofreading

and editing.

Finally, I wish to acknowledge the precious prayers offered by the

righteous to strengthen and uphold me through this challenging time,

ix

with a special thanks to my most avid prayer warrior, my aunt, Mrs. Sina

Gunnels, and her chicken with a snuff cup under its wing.

ix

TABLE OF CONTENTS

Page

ABSTRACT ..................................................................................... iii

DEDICATION ................................................................................. vi

ACKNOWLEDGEMENT .................................................................. vii

TABLE OF CONTENTS ................................................................... ix

LIST OF TABLES .......................................................................... xiv

CHAPTER I. INTRODUCTION .......................................................... 1

Statement of the Problem....................................................... 3

Purpose of the Study ............................................................. 6

Research Questions ............................................................... 7

Quantitative .................................................................. 7

Null Hypothesis One ...................................................... 7

Null Hypothesis Two ...................................................... 7

Qualitative .................................................................... 7

Description of the Research Design ........................................ 8

Assumptions ......................................................................... 9

Limitations of the Study ........................................................ 9

Delimitations of the Study .................................................... 10

Definition of Terms ............................................................... 10

Significance of the Study ...................................................... 12

Organization of Study ........................................................... 13

CHAPTER II. REVIEW OF LITERATURE ......................................... 15

x

Page

No Child Left Behind (NCLB) ................................................. 15

Historical Perspective .................................................. 15

Description of the Key Factors ..................................... 18

Expectations for Parents .............................................. 20

Response to NCLB ....................................................... 21

Adequate Yearly Progress (AYP) ............................................. 22

Purpose and Support to NCLB ..................................... 22

Changes and Updates.................................................. 24

AYP and Limited English Proficient (LEP) Students ................ 30

Definition of English Language Learners (ELLs) ........... 30

Issues and Other Considerations of LEP ....................... 34

High Stakes/Statewide Testing ............................................. 43

Principles of Testing Programs ..................................... 44

Accountability in Testing ............................................. 50

Effects of High Stakes Testing on Student Motivation .... 52

Other Considerations of Assessment on Testing ............ 56

Related Studies .................................................................... 59

Summary ............................................................................. 61

CHAPTER III.METHODOLOGY. ...................................................... 63

Introduction .................................................................................. 63

Research Questions .............................................................. 64

Quantitative ................................................................ 64

xi

Page

Null Hypothesis One .................................................... 64

Null Hypotheses Two ................................................... 65

Qualitative .................................................................. 65

Research Methods ................................................................ 66

Research Design................................................................... 67

Quantitative ................................................................ 68

Qualitative .................................................................. 68

Pilot Study ........................................................................... 68

Population and Sample ......................................................... 70

Quantitative ................................................................ 70

Qualitative ................................................................. 71

Instrumentation ................................................................... 72

Instruments ................................................................ 72

Validity ....................................................................... 74

Reliability ................................................................... 74

Research Procedures ............................................................ 75

Quantitative ................................................................ 75

Qualitative ................................................................. 75

Data Collection and Recording .............................................. 76

Quantitative ................................................................ 76

Qualitative ................................................................. 76

Data Analysis ....................................................................... 78

xii

Page

Quantitative ................................................................ 78

Qualitative .................................................................. 79

Summary ............................................................................. 80

CHAPTER IV ANALYSIS OF DATA .................................................. 83

Findings .............................................................................. 85

Quantitative Research Question .................................... 85

Null Hypothesis One ..................................................... 86

Null Hypothesis Two ..................................................... 86

Qualitative Research Question ..................................... 100

Discussion ........................................................................ 141

Summary .......................................................................... 145

CHAPTER V SUMMARY, CONCLUSION, IMPLICATIONS AND

RECOMMENDATIONS ................................................................. 147

Summary .......................................................................... 147

Conclusions ...................................................................... 150

Implications ...................................................................... 151

Recommendations for Further Study .................................. 154

REFERENCES ............................................................................. 157

APPENDICES .............................................................................. 177

Appendix A IRB .................................................................. 178

Appendix B Consent Form .................................................. 183

Appendix C Interview Questions .......................................... 186

xiii

Page

Appendix D On-Line Questionnaire ..................................... 191

Appendix E Letter to Participants ........................................ 196

Appendix F Request for Extant Data from T.E.A... ................ 198

VITA ........................................................................................... 201

xiv

LIST OF TABLES

Table Page

4.1.1 Means and Standard Deviations of ELLs Enrolled in School

and All Students Passing the 2003 10th Grade TAKS for

English Language Arts and Mathematics .............................. 87



English Language Arts and Mathematics ............................... 87







4.1.1.1 Comparison of Results in 10th Grade English Language

Arts TAKS ......................................................................... 89

4.1.1.2 Comparison of Results in 10th Grade Mathematics TAKS...... 90

4.2.1 Pearson Correlation: 2003 10th Grade TAKS for English

Language Arts and Mathematics............................................ 91





xv

Table Page



4.2.5 Coefficients for Percentage of All Students Passing the

2003 10th Grade English Language Arts TAKS ........................ 93


2003 10th Grade Mathematics TAKS ...................................... 94












2006 10th Grade Mathematics TAKS .................................... 100

4.3 Distribution of Respondents by Gender .................................. 102

4.4 Distribution of Respondents by Age ........................................ 102

4.5 Distribution of Respondents by Professional Position ............... 103

4.6 Distribution of Respondents by Highest Degree Earned ........... 103

xvi

Table Page

4.7 Distribution of Respondents by Years of Experience in

Education ............................................................................. 104

4.8 Distribution of Respondents by Certifications Held .................. 105

4.9 Why is TAKS Given as a Statewide Test to ELLs? .................... 107

4.10 What are the Anticipated Results of Statewide Testing

for ELLs? ............................................................................ 111

4.11 What are the Actual Results of Statewide Testing

for ELLs? ............................................................................ 115

4.12 What are the Intended Consequences of TAKS for ELLs? ....... 118

4.13 What Has Happened to ELLs Because of TAKS? .................. 122

4.14 What Problems Have Occurred for ELLs Due to TAKS? ......... 126

4.15 What Changes Have Occurred for ELLs Due to TAKS? .......... 129

4.16 What Recommendations are suggested for Improvement

of ELLs Performance on TAKS?............................................. 132

4.17 What are the Recommendations, with Greatest Value,

are offered for ELLs Success on TAKS? ................................. 136

1

CHAPTER I

INTRODUCTION

For years, the English language learners (ELLs) have been

subjected to educational systems that did not expect them to rise to the

same standards as their native English-speaking peers (Winograd, 2002).

Although that it can take several years to acquire the second language

skills needed to be successful in school (Collier, 1989), too often English

language learners born in the U.S. are still in English as a second

language (ESL) classes and far behind their grade level peers in the

content areas by the time they reach high school (Freeman & Freeman,

2002).

One factor that should be considered in this failure to reach grade

level requirements is that language may constitute an element of self-

identity. It is possible that minority groups are insistent on retaining

their ethnic language as their “first.” English proficiency then would be a

mere elective instead of an indispensable learning tool. If this is the case,

schools are being held accountable for the consequences of a socio-

cultural phenomenon that is beyond their limited powers to address.

Public schools are under close scrutiny. Since they are supported

by public funds, there is an increasing demand for accountability. The

No Child Left Behind Act (NCLB) now requires all students to be

accounted for in any state‟s assessment system, although that has not

always been the case (Abedi, 2004). School districts are now required to

2

clearly demonstrate they deserve, and effectively utilize public funding. In

itself, this is not a disturbing trend. Institutions that are wholly or partly

supported by public funds should be accountable. This is essentially a

consequence of democracy. A government that is created by, and for the

people, is so unlike an aristocracy that is not required to serve a

constituency beyond the guarantee of protection from marauders or

invading armies. The U.S. system of government empowers the state to

undertake measures that guarantee the common good. This goes beyond

the guarantee of physical safety, since the term “common good” has a

wider application, and implies a calculated sensitivity to every citizen‟s

pursuit of happiness. While education is not categorized as a

fundamental right, it is perceived as primary among a bundle of values

essential for every person‟s quest for self-fulfillment and happiness. This

explains why there is little argument about whether the government

should be involved in education at all, and whether this is an endeavor

better left to the private sector (Abedi, 2004).

The government‟s involvement in education opens a wide avenue

for the analysis and evaluation of results. In today‟s world, it is not

enough that public schools have adequate facilities, although this

constitutes one level of analysis. It is important that schools are safe and

teachers are qualified, although in the hierarchy of priorities considered

for evaluating schools, these outcomes are not standard. Schools are

judged principally based on the amount of learning that takes place in

3

their classrooms. As an internal act, the evidence of learning is analyzed

from scores students obtain on standardized assessments.

Institutions are now facing an ever-increasing demand for

accountability. There is pressure from every conceivable corner to make

public schools accountable to their stakeholders. This means that it is

not enough for students to learn in school; it is equally important that

learning should occur in ways that are measurable. If students are

unable to demonstrate what they have learned, it is presumed that no

learning took place at all. The time when public schools are allowed to

operate without proven success is over. It is appropriate to inquire about

the valid manifestations of success and learning, and how they may

actually be measured. Cultural construct renders school rankings flawed

to a certain extent since they become less accurate as a measure of the

faculty and administration‟s performance. Instead, they become

unintended indicators of the ethnicity of the students to which schools

cater (Abedi, 2004).

Statement of the Problem

High stakes assessment systems are meant to bring attention to

the needs of ELLs, who are most at risk of not reaching the educational

goals set for them (Anderson, 2004). But what results do statewide

accountability tests really produce for ELLs (Anderson, 2004)?

Assessment systems usually produce both positive and negative

consequences (Anderson, 2004). The positive and negative consequences

4

of assessments are what is called „washback‟ (Alderson & Wall, 1993), or

how the results of an assessment affect the stakeholders taking the test

(Anderson, 2004).

While quantifiable washback effects such as increased dropout

rates or increased referral to Special Education have been researched,

assessment washback is more complicated than numbers alone can tell

(Anderson, 2004). Students who qualify for Special Education may be

allowed to take alternative assessments in lieu of the state assessments

such as the Texas Assessment of Knowledge and Skills (TAKS). It is

interesting to note that while the numbers of African-American and

Hispanic students are over-represented in Special Education, about eight

to nine percent of ELLs are identified as receiving Special Education

services in the United States (D‟Emilio, 2003; Zehler, Fleischman,

Hopstock, Pendzick, & Stepherson, 2003). While these assessments are

not on grade level, schools are expected to demonstrate that, based on

students‟ scores on alternative assessments, improvement in academic

performance is taking place.

Data are needed that tell us more about the full range of intended

and unintended consequences occurring in schools today (Anderson,

2004). Since school rankings affect student and faculty morale, they

serve more as a force for the preservation of the status quo than a force

for improvement in student performance. A school that works hard to

ensure that learning occurs, and that its students progress academically,

5

but which has a large proportion of ELLs, will risk being ranked as

underperforming because the measure used to evaluate its performance

is blind to this important demographic reality.

One way to get at these data is by talking with the stakeholders at

the schools. Educators are the ones who deal directly with the impact of

high stakes assessments, but are overlooked in research. While teachers‟

opinions are often cited as anecdotal evidence that a problem exists,

their expert observations often go unrecorded in any systematic way

(Anderson, 2004).

Standardized assessments are a measure for holding schools

accountable for student learning. At the present time, schools in Texas

are ranked Exemplary, Recognized, Acceptable or Underperforming,

depending on the performance of their students in the Texas Assessment

of Knowledge Skills (TAKS). This produces a vicious cycle since

exemplary schools attract the best students who may leave

underperforming schools to seek what is perceived to be a higher quality

of instruction in higher ranked schools. These labels tend to have a self-

fulfilling effect, or at least they make it difficult for underperforming

schools to achieve higher performance scores on standardized tests,

since they face the additional burden of surmounting language barriers

and a history of low performance.

Related to this concern is the prevailing system of voluntary

segregation in most zones and districts. Some schools have either a

6

predominant population of White, Hispanic, or African-American

students. Each of these student groups is given the same tests, and yet

they have varying degrees of proficiency in the language in which the

assessments are given. It begs to be asked whether these assessments,

in fact, measure learning and whether they are linguistically and

culturally neutral. The implication is that these students will be able to

answer the test questions even if they do not have equal exposure to

cultural references that may frame some of the test questions.

This study is intended to explore what educators perceive as the

consequences of statewide assessment for ELLs and what they observe

as actually occurring (Anderson, 2004).

Purpose of the Study


stakes testing on ELLs. This was shown in both the quantitative and

qualitative dimensions of the study. Data obtained from TEA were used

to determine whether there is a relationship between the percentage of

English language learners enrolled in a school and the percentage of all

students passing the 10th Grade TAKS tests in the core areas of English

Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To

support the quantitative aspect, this study explored what certified ESL

teachers, non-certified ESL teachers who teach ELLs, administrators,

and district ESL personnel viewed as the impact that high stakes

7

standardized assessments have on ELLs, ESL curriculum and

instruction, and what they observed as actually occurring.

Research Questions

Quantitative

Is there a relationship between the percentage of English language

learners enrolled in a school and the percentage of all students passing

the 10th grade TAKS tests in the core areas of English Language Arts

and Mathematics given in 2003, 2004, 2005, and 2006?

Hypotheses

H01: There is no statistically significant relationship between the

percentage of English language learners enrolled in a school and

the percentage of all students passing the 10th grade TAKS tests in

English Language Arts given in 2003, 2004, 2005, and 2006?

H02: There is no statistically significant relationship between the



Mathematics given in 2003, 2004, 2005, and 2006?

Qualitative

The major question addressed by this study was: What are the

anticipated and observed consequences of the statewide testing,

specifically TAKS, on ESL curriculum and instruction as viewed by

certified ESL teachers, non-certified ESL teachers who teach ELLs,

school administrators, and district ESL personnel?

8

This major question was explored using the following probes:

1. Why is TAKS given as a statewide test?

2. What are the intended consequences of this statewide testing? (Or

what has happened because of TAKS?)

3. What problems have occurred related to or because of TAKS?

4. What changes were caused by this statewide testing?

5. What are your recommendations to improve this statewide testing?

6. What needs to be done for the ESL students to improve their

performance in general and specifically for this statewide test?

Description of the Research Design

The study analyzed the issues and challenges faced by ELLs and

the public schools that serve them. Quantitative data for this research

were gathered from the Texas Education Agency (TEA) regarding the

percentage of ELLs and the performance of 10th grade students from the

major urban high schools in Texas on the Texas Assessment of

Knowledge and Skills (TAKS) tests in English Language Arts and

Mathematics for 2003, 2004, 2005, and 2006. Qualitative data were

derived from one-on-one and focus group interviews and an on-line

questionnaire focusing on respondents‟ views and opinions about the

various ways that standardized assessments impact ELLs.

9

Assumptions

Fraenkel and Wallen (2003) stated that an assumption is anything

taken for granted rather than tested or checked. This study is no

different and the following assumptions were made: (a) that the first

language of the ELLs is Spanish and they have varying degrees of fluency

in the English language; (b) that the ESL curriculum is appropriate for

the mastery of the TAKS test for the ELLs; (c) that the on-line open-ended

qualitative questionnaire will be completed by the respondents on time;

and (d) that the respondents in the focus groups will truthfully express

their views and opinions regarding issues or concerns brought to the

group.

Limitations of the Study

Limitations of the study included several factors: mainly the

qualitative questionnaire and the manner in which respondents gave

their responses. The questionnaire may have vague questions open to

more than one interpretation. The pilot study helped in streamlining the

questionnaire to remove or modify such vague issues or concerns.

Another limitation may have been the manner in which the respondents

answered the question. For one reason or another, they may not have

truthfully answered some of the questions. The respondents may or may

not have completed the questionnaire due to no ready access to a

computer or they just did not want to complete the questionnaire. These

non-respondents became part of the mortality factor involved in the

10

study. Responses to the open-ended questions became difficult to classify

under a certain category. This was facilitated through the Non-

Numerical, Unstructured Data, Indexing Searching & Theorizing Vivo-

“Nudist Alive” (NVivo) software system (Version 7.0) and by the focus

group interviews where the respondents helped determine the category of

such responses.

A factor that may have been encountered in the quantitative

dimension of the study was the lack of intended data for the study.

Diligent efforts were made to gather data from available sources.

Delimitations of the Study

The questions for the on-line qualitative questionnaire may have

been a delimitation of the study. The pilot study contributed to the

improvement of the qualitative tool. Another delimitation may have been

the choice of participants, especially in the focus groups. The “snowball

technique” addressed this issue. Better interaction happened with added

„quality‟ members to the focus groups.

Qualitative data are available and the inclusion of the quantitative

aspect of the study provided a challenge and an opportunity to determine

if certain factors of the study have any impact on the ELLs.

Definition of Terms

Content Standards are broad descriptions of the knowledge, skills, and

understandings that schools should teach and students should acquire

in a particular subject area (McLaughlin & Shepard, 1995).

11

English Language Learners (ELLs) is the preferred term to describe a

student whose native language is other than English (Chamot &

O‟Malley, 1994). These students require instructional modifications, and

eventually take the TAKS after two years of enrollment in the school

districts.

High Stakes Assessment is an assessment in which student promotion

(i.e., high school graduation) can be denied if the scores do not reflect

competence (NCBE, 1997).

Limited English Proficient (LEP) refers to a student with a language

background other than English, and whose proficiency in English is such

that the probability of academic success in an English-only classroom is

below that of an academically successful peer with an English-language

background (CCSSO, 1992).

No Child Left Behind Act (NCLB) of 2001 (PL – 107 – 110). It is the

reauthorization of the Elementary and Secondary Education Act (ESEA).

Opportunity-to-learn (OTL) Standard defines the level and availability of

programs, staff and other resources sufficient to enable all students to

meet challenging content and performance standards (McLaughlin &

Shepard, 1995).

Performance Standards are concrete examples and explicit definitions of

what students have to know and be able to do to demonstrate that such

students are proficient in the skills and knowledge framed by the content

standards (McLaughlin & Shepard, 1995).

12

Standardized Assessments include the Texas Assessment of Knowledge

and Skills (TAKS) and the State and Locally-Developed Alternative

Assessment (SLDAA) for students who are exempted from the TAKS. A

standardized assessment is a measurement of what students know and

can do (McLaughlin & Shepard, 1995).

Standards-based Reform requires setting standards of performance in

academic subject areas as a means of improving the substance of school

curricula and increasing the motivation and effort of students, teachers,

and school systems and thereby improving student achievement

(McLaughlin & Shepard, 1995).

Washback shows how the results of an assessment affect the

stakeholders taking the test (Alderson & Wall, 1993).

Significance of the Study

Expected outcome of this study may possibly provide additional

valuable data for writers or researchers in regard to biases in

standardized assessments that may encourage school districts to develop

assessments that truly measure learning without the nullifying effect of

linguistic and cultural bias. Additionally, this study enhances the

reliability of standardized assessments as a tool in determining

accountability where the performance of English language learners is

concerned.

13

Organization of the Study

Chapter I identifies the problem this study addresses: the impact

of high stakes assessments on the curriculum and instruction of English

language learners. It includes the hypotheses and research questions of

the present study. Included are the definitions of terms valuable to the

study.

Chapter II includes the review of literature about the essential

conditions and factors regarding the NCLB Act, the AYP implications for

concerned schools, high-stakes, statewide assessments and the

implications and challenges they present to the preparation and

education of ELLs. The information reveals the difficulties that English

language learners face when taking these high stakes assessments, the

possible positive and negative consequences and possible “washback”

related to the assessments.

A mixed methods study is identified and expounded in Chapter III.

Quantitative data for this research were gathered from the Texas

Education Agency regarding the percentage of ELLs and the performance

of major urban high schools in Texas in the statewide test (Texas

Assessment of Knowledge and Skills) for 2003, 2004, 2005, and 2006.

Qualitative data were derived from an on-line, open-ended questionnaire

and interviews that focused on the respondents‟ views and opinions

about the varied ways standardized assessments impact English

language learners.

14

Results of the study are presented in detail in Chapter IV.

Quantitative results include the available data collected from Texas

Education Agency. Results of computations employing the Statistical

Package for the Social Sciences (SPSS) statistical package, (Version 14.0)

are shown in tabular presentations and explanations regarding the

relationship among the variables are included. Qualitative results include

the participants‟ views and opinions on the impact of high stakes testing

on English language learners and the information collected from the on-

line, open-ended questionnaire, individual and focus group interviews.

Major findings of the study are discussed in Chapter V. Impact of

high stakes standardized assessments on English language learners are

also summarized. Other relevant factors that influenced this study are

presented, as well as recommendations for future research.

15

CHAPTER II

REVIEW OF LITERATURE

Key issues and concerns about the No Child Left Behind Act (NCLB) of

2001 and the Adequate Yearly Progress (AYP) are major parts of the review

of related literature. Included are the principles and accountability involved

in high-stakes testing and the descriptions and accommodations given to

the ultimate beneficiary of the efforts exerted by the federal and state

policymakers, the school and district administrators – the learners,

specifically, the English language learners who strive to be better citizens of

this country. Short description of related studies on statewide testing and

English language learners (ELLs) are given to show their tie-in with this

study.

No Child Left Behind (NCLB)

Historical Perspective

The NCLB Act of 2001 (PL – 107 -110), is the reauthorization of the

Elementary and Secondary Education Act (ESEA). The ESEA was first

passed in 1965 with the goal of improving the U. S. educational system by

providing better education for students in poverty through an increase in

services to them. The ESEA provided federal funds for schools but did not

require accountability in the use of those funds. In 2003, the Center of

Educational Policy clarified why accountability was not part of ESEA in

1965: “At that time, the federal role in education was marginal, most state

education agencies had very limited authority and capabilities, and local

16

people were extremely wary that more federal aid would bring federal

control” (p.iv).

The National Assessment of Educational Progress (NAEP) was initiated

as a federal testing program at about the same time when ESEA came into

existence. NAEP was tasked to report how the nation‟s students were

performing on selected items at the three grade levels --- 4th, 8th and 12th.

Brennan (2004) reported that there were fears that the NAEP might become

a “high-stakes federal testing program” found in some European countries.

He explained that, “to help preclude that possibility, it was written into law

that NAEP could not report scores for individual students” (p.2). The NAEP

evolved through the 1980s and early 1990s from a reporting of item scores

to test scores and then, on a trial basis, to a reporting of scores that

addressed achievement levels (below basic, basic, proficient, and advanced).

It is currently used to confirm state NCLB testing results which, according

to Brennan, “is the de facto elevation of NAEP to a federally-mandated high-

stakes testing program” (p.9).

Through the NCLB Act, policymakers in Washington seek to raise

academic achievement in the nation by requiring schools to assess all

students on specified content areas and report their progress toward

proficiency. Focus of NCLB is on core academic subjects as defined in the

law: “The term „core academic subjects‟ means English, reading or language

arts, mathematics, science, foreign language, civics, and government,

17

economics, arts, history, and geography” (U.S. Department of Education,

2002).

The premise of NCLB is that our nation‟s schools are failing. Thus, the

purpose of NCLB is raising the achievement of all students and eliminating

the achievement gap among students differentiated by race, ethnicity,

poverty, disability, and English proficiency. Since this Act redefines the

federal role in education policy that has traditionally been a state

responsibility, it merits the attention of educators, parents and citizens.

Because the NCLB Act has an impact on the teaching and the learning of

the core content areas, including languages, language educators need to be

informed about it.

If a roomful of educators were asked which word or phrase best sums

up No Child Left Behind (NCLB), many would say accountability. Others

might propose student achievement, proficiency or raised expectations. But

perhaps the most accurate word to encapsulate the United States‟ most

ambitious federal education law – which proposes to close achievement gaps

and aims for 100% student proficiency by 2014 - is testing. Certainly, the

focus on holding schools accountable for student achievement on

standardized tests sets NCLB apart from previous versions of the law.

(Guilfoyle, 2006).

18

Description of the Key Factors

There are four key elements in the NCLB Act (Rosenbusch, 2005):

(1) Accountability. States are required to establish a definition of

student proficiency in the core academic subjects of Reading/Language

Arts, Mathematics and Science through prescribed indicators and set a

timetable to bring all students in all subgroups up to the defined levels of

proficiency by 2013-2014. The school must report to parents their child‟s

progress in each targeted academic subject annually, and the state is

required to report the results of students‟ performance on the annual tests

for every public school to parents and the community. Schools that fail to

meet state-defined AYP toward their defined goals for two years are

identified as needing improvement. Schools that have not met AYP after four

years are subject to restructuring or reconstitution.

(2) Testing. States must develop and administer annual tests that

define the proficiency that all students are expected to reach in

Reading/Language Arts, Mathematics, and Science. States must include a

sample of students in fourth and eighth grades in a biennial NAEP in

Mathematics and Reading to verify state assessments.

NCLB requires that by School Year (SY) 2005-2006, each state must

measure every child‟s progress in Reading and Mathematics in each of

grades 3 through 8 and at least once during grades 10 through 12. In the

meantime, each state must meet the requirements of the previous law

reauthorizing ESEA (the Improving America‟s Schools act of 1994) for

19

assessments in Reading and Mathematics at three grade spans (3-5; 6-9;

and 10-12). By SY 2007-2008, states must have in place Science

assessments to be administered at least once during grades 3-5; grades 6-9;

and grades 10-12. States must ensure that districts administer a test of

English proficiency to measure oral language, Reading and Writing skills in

English to all limited English proficient students, as of SY 2002-2003.

Students may still undergo state assessments in other subject areas (i.e.,

History, Geography, and Writing skills), if and when the state requires it.

NCLB requires assessments only in the areas of Reading/Language Arts,

Mathematics, and Science.

(3) Teacher Quality. Public elementary and secondary school teachers

who teach core content areas are required to be “highly qualified,” which is

defined as having full state certification (may be attained through alternate

routes specified by the state), holding a bachelor‟s degree, and having

demonstrated subject matter competency as determined by the state under

NCLB guidelines. States are required to develop a plan by the end of 2005-

2006 to ensure that every teacher is highly qualified to teach in his or her

core content area.

(4) Scientifically-Based Research. The NCLB Act requires that all

educational decisions be informed by scientifically-based research as

defined in the legislation. The NCLB Act funds for Reading First Grants, for

example, are to be used for methods of reading instruction backed by

scientifically-based research.

20

Expectations for Parents Due to NCLB (from Collegeboard.com)

(1) New standards for students will require that beginning 2005,

students in grades 3 through 8 must be tested in Mathematics and English

to ensure they are meeting state standards. Students in Grades 10 through

12 will be tested at least once. By 2007, states will begin testing students in

Science as well. Results of the yearly tests will be known to parents. NCLB

requires that school districts provide parents with an annual “report card”

that shows how well students in each school performed. The information is

broken down by race, ethnicity, gender, disability status, and other

categories so that parents will know how well each school is doing in

educating minority students or those with disabilities.

(2) By the end of SY 2005-2006, teachers must be “highly qualified” in

the subjects they teach. States will determine what skills teachers must

have to be “highly qualified”, but the requirements could include a degree in

the subject they teach or extra training. States must provide annual report

cards about teacher certifications, including the percentage of classrooms in

the state not taught by highly qualified teachers. Principals must also

maintain information about whether or not their school‟s teachers meet the

requirements.

(3) Each year, schools must increase the number of students who

achieve state standards. At the end of 12 years, all students should be able

to pass the tests. Schools that fail to achieve this progress will be targeted

21

for improvements that could include increased funding or staff and

curriculum changes.

(4) NCLB requires school districts to notify parents if the child‟s

school has been identified as needing improvement as a result of failing to

increase the number of students meeting state standards.

(5) About half of all public schools receive funding to help students

from low-income families. If such a school is targeted for improvement and

fails after two years, parents can choose to transfer their child to another

school or enroll in free tutoring. Parents have this choice for as long as the

school fails to adequately perform.

Response to NCLB (Rosenbusch, 2005)

NCLB has engendered controversy that is centered in part on the

increased role of the federal government in educational policy. A majority of

Americans believe that decisions about what is taught in public schools

should be made at the local level by the school board (61%), rather than at

the state level (22%) or the federal level (15%) (Rose & Gallup, 2003).

Results of a 2004 survey indicate that they disagree with “the major

strategies NCLB uses to determine whether a school is or is not in need of

improvement” (Rose & Gallup, 2004, p.2). For example, 83% of those

surveyed believe that testing only in English and Mathematics will not yield

a fair picture of the school, 73% say it is not possible to judge a student‟s

proficiency in English and Mathematics on a single test, and 81% are

concerned that basing decisions about school on students‟ performance in

22

English and Mathematics will mean less emphasis on art, music, history

and other subjects.

In the U.S. Department of Education, there is support for high

standards and high expectations for every child, but the NCLB focus on

standardized testing is resulting in a narrowing of the curriculum and a

“sorting of students” (Marshak, 2003, p.229) and “could halt the

development of truly significant improvements in teaching and learning”

(Lewis, 2002, p.179). The National Education Association supports the

NCLB Act in its goal but views it as an obstacle to improving public

education because of its focus on “punishment rather than assistance”, and

“mandates rather than support for effective programs” (National Education

Association, n.d.).

Adequate Yearly Progress (AYP)

Purpose and Support to NCLB

The No Child Left Behind Act of 2001 (NCLB; Public Law No. 107-110,

115 Stat. 1425, 2002), the most recent reauthorization of the Elementary

and Secondary Act of 1965, holds states using federal funds accountable for

student academic achievement. States are required to develop a set of high-

quality, yearly student assessments that include, at a minimum,

assessments in Reading/Language Arts, Mathematics and Science. Each

year, they must report student progress in terms of percentage of students

scoring at the “proficient” level or higher. This reporting is referred to as

adequate yearly progress (AYP). A state‟s definition of AYP should also

23

include high school graduation rates and an additional indicator for middle

schools to reach the “proficient” level or higher, which must be no more

than 12 years after the start date of the 2001 – 2002 school year, provided

that the first increase occurs within the first 2 years (Abedi, 2004).

AYP will be reported for schools, school districts, and the state for all

students. In addition, AYP must be reported for the following subgroup

categories of students: (a) economically disadvantaged students, (b)

students from major racial and ethnic groups, (c) students with disabilities,

and (d) students with limited English proficiency (LEP). According to the

educational statistics for 2000 – 2001 school year, the total number of

students labeled as LEP in the nation‟s public schools is more than 4.5

million or 9.6% of total enrollment; (National Center for Education Statistics

[NCES], 2002).

States are continuing to find new ways to calculate AYP under the

NCLB, in order to increase the number of schools and districts that meet

the student achievement targets set by law. Over the past few years, the

U.S. Department of Education (ED) has allowed states to make many

changes in the way they determine AYP, including the following: (1)

confidence intervals, which make allowances for natural fluctuations in test

scores and essentially bolster a school‟s or subgroup‟s percentage of

students scoring at proficient levels; (2) performance indices that allow

schools to get “partial credit” for the performance of students below the

proficient level; (3) retesting, which allows students to retake a different

24

version of the same test and permits schools to use a student‟s best score to

count toward AYP, and (4) increased minimum subgroup sizes, which mean

that in many schools, subgroups do not get counted for AYP purposes. The

changes have the effect of making it easier for the schools to make AYP,

early indications are that the number of schools not making AYP has leveled

off, despite predictions that this number would increase as proficiency

targets rose (Olson, 2005).

Changes and Updates

In NCLB‟s original conception, determining AYP for a subgroup of

students, a school, or a district was already fairly complicated. States had to

establish, for every year between 2003 and 2014, a set of ever-increasing

state targets in terms of the percentage of students scoring at the proficient

level or above on annual tests, with a final goal of 100% proficiency in 2014.

If at least 95% of the students in each subgroup are tested, and if all

students and subgroups meet the state proficiency targets, the school or

district makes AYP. The school has to meet targets for an additional

academic indicator, such as the graduation or attendance rate. The law has

a “safe harbor” provision: if a school or subgroup fails to meet the state

targets, it could still make AYP if it reduces the number of students who are

not proficient from the previous year by 10%, and meets its additional

academic indicator.

Some other state changes that have been approved are briefly

summarized below (Center on Education Policy, 2005):

25

Minimum subgroup size. To make AYP, schools and districts must

meet achievement targets for each significant subgroup of students enrolled,

such as African-American students, low-income students, or students with

disabilities. Higher minimum subgroup sizes mean that in many schools,

subgroups do not get counted for AYP purposes.

Thirteen states increased their minimum subgroup sizes in 2004; ten

more did so in 2005. The trend is away from a single minimum size and

toward larger subgroup sizes, different subgroup sizes for different

subgroups and/or purpose, and the use of formulas for determining

subgroup sizes. Georgia is one state that uses a formula approach. Its

subgroup size varies according to the size of the school; the minimum size is

either 40 students or 10% of a school‟s student population, whichever is

greater, with a cap of 75 students.

Participation averaging. NCLB requires 95% of the students in every

school and every subgroup within a school to take each subject test

required by the Act. If this test participation requirement is not met, the

school cannot make AYP even if its test scores meet state targets. In March

2004, the Department relaxed this requirement, allowing states to average

their participation rates over two or three years, so that a 94% participation

rate one year could be balanced by a 96% participation rate the following or

previous year. In 2005, six states changed their accountability plans to

incorporate this new policy, in addition to the 32 that did so last year.

26

English language learners. Initially the U.S. Department of Education

(ED) required all English language learners to be tested with the same

grade-level tests as other students. In response to state and local criticism,

the Department revised its policy in February 2004 to allow states to exempt

immigrant students who are in their first year of enrollment in a U.S. school

for less than one year from taking the regular state English Language Arts

tests. These students still have to take an English language proficiency test

and a Mathematics test, but the results need not count toward AYP. When

calculating AYP for the subgroup of English language learners, states can

also count the progress of former English language learners for two years

after they reach English proficiency. Six more states adopted these changes

in 2005, in addition to the 36 states that did so in 2004.

Extra time is given for students with disabilities and English language

learners to graduate. In 2005, eight states received approval from ED to

count students with disabilities and/or English language learners as

graduating on time even if they need extra years of high school. Seven states

received permission to do this in 2004. For students with disabilities, their

individualized education plans would need to call for extra years of high

school beyond age 18. English language learners can be counted as

graduating on time if it takes five years, or as determined on a case-to-case

basis (Center on Education Policy, 2005).

Identifying districts for improvement. In 2005, ED approved

amendments requested by 13 states to identify a district as being in need of

27

improvement only when it does not make AYP in the same subject and

across all three grade spans (elementary, middle and high school) for two

consecutive years. In 2004, 18 states made this change. California

attempted to have ED accept a relatively lenient method that exempted

districts where low-income students reached a certain level on state tests.

ED rejected that method, and California settled on the grade span approach

instead (Davis & Sack, 2005).

Annual measurable objectives. Eleven states changed their annual

score targets in 2005; four states did so in 2004. For example, Florida was

allowed to change its schedule of annual measurable objectives so that

targets would increase in smaller increments annually, rather than in large

increments every three years (Olson, 2005); Virginia did so as well. Several

other states, including Alabama, Alaska, New Mexico, and North Carolina,

changed their annual targets because they were introducing new

assessments.

NCLB is a demanding law. The achievement goals are ambitious, and

the burden on states and districts of declaring schools in need of

improvement and then imposing sanctions on them is high. To try to meet

these demands, states have a strong incentive to keep the numbers of

schools and districts not making AYP as low as possible. Unable to change

the fundamental requirements written into the law, states are using

administrative methods to lessen the numbers of schools and districts not

making the AYP – confidence intervals, indexing, and other techniques.

28

Education Secretary Margaret Spellings has been more flexible than

her predecessor in policies regarding students with disabilities, and in

granting special exemptions to some districts in the areas of school choice

and supplemental educational services (tutoring). Secretary Spellings has

decided to allow the Chicago school district to provide tutoring despite the

fact that the district has been identified for improvement (Gewertz, 2005).

This exemption was then extended to New York City, Los Angeles, Boston,

Memphis, Anchorage, and Dayton. This was a regulatory change.

Secretary Spellings went further with four districts in Virginia by

suspending a key element of the law itself, invoking a clause in NCLB that

allows the Secretary of Education to do so. Her action exempted these

districts from the law‟s requirement that they provide school choice before

tutoring (Olson, 2005). Secretary Spelling‟s letter to Virginia officials

indicates that this is a pilot program intended to raise the numbers of

students receiving supplemental educational services (Spellings, 2005). In

addition, districts in the five states most affected by Hurricane Katrina were

allowed to postpone, for one year, the consequences that follow when a

school is in need of improvement, such as tutoring, restructuring, and

corrective action (Olson & Davis, 2005).

ED‟s willingness to make adjustments based on state and local

experience is commendable. But on the downside, parents in many states

would now find it difficult to understand what it means when a school does

or does not make AYP, and what criteria were used to determine this

29

success or failure. For example, parents in Pennsylvania may see a report

card that indicates that their child‟s elementary school has made AYP, but

might wonder whether the school is improving or whether it simply made

AYP as the result of what might be seen as a new “loophole” in the law. The

parents probably would not understand that the school may have made AYP

through the use of a 95% confidence interval, safe harbor with a 75%

confidence interval, or the Pennsylvania Performance Index as a second safe

harbor. In other states, parents of English language learners, students with

disabilities, or other subgroups may not realize that raising the minimum

subgroup sizes means that their children no longer count for AYP purposes

at the school level. They might not realize that the use of confidence

intervals allows for considerable leeway in a subgroup‟s test scores not

available to larger groups of students, and that this is occurring despite the

assertion that improving achievement for subgroups is a major focus of the

law.

Other drawbacks to the increasing complexity may contribute in the

difficulty of discerning clear trends in the number of schools and districts

not making AYP, because the rules governing AYP keep changing every year.

Amid these changes, it is impossible to determine whether an increase in

the number of schools making AYP within a state is due to better teaching

and learning or NCLB rule changes. The constant rule changes, particularly

the use of large confidence intervals and ever-increasing minimum

subgroup sizes, may raise questions about whether the law is being watered

30

down so much that it shortchanges the very groups of disadvantaged

children that it aims to help. Public support may wither if the

implementation of the law is perceived as deceptive or confusing.

As states continue to learn from one another about the new types of

flexibility that ED is allowing, and as state achievement targets continue to

rise until 2014, changes in AYP policies are likely to occur at a more rapid

pace, at the expense of the public‟s ability to understand these changes.

More transparency is needed at both the state and federal levels. States

must fully and clearly explain their rationales for requesting changes to

accountability plans. Once changes are approved by ED, they should be

explained in such a way that the public understand how AYP is determined.

At the federal level, ED should more systematically and promptly

publicize its decisions about what types of changes to state accountability

plans are and are not acceptable, and why. The current process of granting

changes does not help state officials learn from other states‟ experiences,

nor does it help them understand how ED is interpreting the intent of the

law.

Adequate Yearly Progress (AYP) and Limited English Proficient (LEP)

Students

Definition of English Language Learners (ELLs) and LEP

Limited English Proficient (LEP) students are students who lack

sufficient English skills to participate in a regular education, all-English

speaking classroom. English Language Learner (ELL), according to Rivera

31

and Stansfield (1998), is a positive way to refer to any LEP student in

English.

NAEP does not provide a definition of the LEP population; instead it

presents criteria for the inclusion of LEP students. NAEP inclusion criteria

indicate that: A student who is identified on the Administration Schedule as

LEP and who is a native speaker of a language other than English should be

included in the NAEP assessment unless: (a) the student has received

Reading or Mathematics instruction primarily in English for less than 3

school years including the current year , and (b) the student cannot

demonstrate his or her knowledge of Reading or Mathematics in English

even with an accommodation permitted by NAEP (NCES, 2001).

Due to the importance of LEP subgroups in NCLB accountability and

reporting, NCLB provides an operational definition of LEP (NCLB, 2002).

According to this definition: The term „limited English proficient‟, when

used with respect to an individual, means an individual (a) who is aged 3

through 21; (b) who is enrolled or preparing to enroll in an elementary

school or secondary school; (c) who was not born in the United States or

whose native language is a language other than English; who is a Native

American or Alaska Native, or native resident of the outlying areas; and who

comes from an environment where a language other than English has had a

significant impact on the individual‟s level of English language proficiency;

or who is migratory, whose native language is a language other than

English, and who comes from an environment where a language other than

32

English is dominant; and (d) whose difficulties in speaking, reading, writing,

or understanding the English language may be sufficient to deny the

individual the ability to meet the State‟s proficient level of achievement on

State assessments described in section 111(b)(3); the ability to successfully

achieve in classrooms where the language of instruction is English; or the

opportunity to participate fully in society.

The term “English language learner” (ELL) is a recent designation for

students whose first language is not English. This group includes students

who are just beginning to learn English as well as those who have already

developed considerable proficiency. The term reflects a positive focus on

what these students are accomplishing – mastering another language- and

is preferred by some researchers to the term “limited English proficient”

(LEP), the designation used in federal and state education legislation and

most national and state data collection efforts (August & Hakuta, 1997;

LaCelle-Peterson & Rivera, 1994).

The ELL population is highly diverse, and any attempt to describe the

group as a whole, as with any diverse group of people, is bound to result in

inaccurate generalizations. While this group of students share one

important feature - the need to increase their proficiency in English - they

differ in many other important respects. ELLs are a diverse cross-section of

the public school student population. The primary language, cultural

background, socio-economic status, family history, length of time in the

33

United States, mobility, prior school experiences, or educational goals of any

student in this group can distinguish him or her from any other ELLs.

ELLs represent a rapidly growing, culturally and linguistically diverse

student population in the United States. In 2000-2001, LEP students

comprised nearly 4.6 million public high school students. The majority were

Spanish speakers (79.0%), followed by Vietnamese (2.0%), Hmong (1.6%),

Cantonese (1.0%), and Korean (1.0%). Since the 1990-1991 school year, the

limited English proficient population has grown approximately 105%, while

the overall school population has increased by only 12%.

English learners matriculate in schools throughout the nation, but

most frequently in large urban school districts in the Sun Belt states, in

industrial states in the Northeast, and around the Great Lakes. This trend

is changing as immigrants move to more affordable suburban and rural

areas and to areas where language-minority families are relative newcomers,

such as the Midwest. More than half (56.1%) reside in four states alone:

California (32.9%), Texas (12.4%), Florida (5.6%) and New York (5.2%)

(Kindler, 2002). English learners represent one in four K – 12 students in

California schools (California Department of Education, 2000).

This population includes recent immigrants as well as children born

in the United States. In the 2000-2001 school year, more than 44% of all

LEP students were enrolled in Pre-K through Grade 3; about 35% were

enrolled in Grades 4 – 8; and only 19% were enrolled at the high school level

(Kindler, 2002). Many LEP students attend schools where most of their

34

peers live in poverty. There are numerous differences among English

learners; for example, Spanish-speaking families tend to have lower parental

educational attainment and family incomes than Asian-or Pacific-language

families (August & Hakuta, 1997).

Many criteria are used across the nation for identification of ELLs.

Among the most commonly used criteria are Home Language Survey results

and scores from English proficiency tests. There are reasons to believe that

the Home Language Survey results may not be valid because of parents‟

concern over equity in education for their children, parents‟ citizenship

issues, and communication problems (Abedi, 2004b). Similarly, there are

concerns about the validity of current English proficiency tests, such as the

Language Assessment Scales and other commonly used assessments

(Zehler, Hopstock, Fleischman & Greniuk, 1994). Criterion-related validity

coefficients, or the correlation between English proficiency tests and other

existing valid measure of English proficiency, are not strong, explaining less

than 5% of the common variance (Abedi, 2003). Finally, in terms of content

and construct validity, there is little evidence that the contents of the

existing English proficiency tests align sufficiently with commonly accepted

English language proficiency standards, such as standards by Teachers of

English to Speakers of Other Languages (Bailey & Butler, 2003).

Issues and Other Considerations of LEP

Disaggregated progress reports by subgroups mandated by the NCLB

legislation will monitor the nation‟s goal of having “no child left behind.”

35

However, there are major issues in this disaggregated reporting among

different subgroup categories (students who are economically

disadvantaged, students from major racial and ethnic groups, students with

disabilities, and LEP students). NCLB requirement for subgroup reporting

may give the impression that students in the subgroup categories start the

achievement race at about the same level and can progress with other

students at about the same rate. This might be an overly optimistic view of

the situation of less advantaged learners. By focusing this discussion on

the consequences for schools enrolling LEP students, we see how putting

into practice the policy may produce invalid assessment and unreliable

reporting while exacerbating the burdens of current educators. Following is

a discussion of some challenges in AYP measurement and reporting for LEP

students.

Results of research on the assessment of LEP students suggest a

strong confounding of language and performance. LEP students exhibit

substantially lower performance than non-LEP students in subject areas

high in language demand. Studies suggest that the large performance gap

between LEP and non-LEP may not be due mainly to lack of content

knowledge. LEP students may possess the content knowledge but may not

be at the level of English language proficiency necessary to understand the

linguistic structure of assessment tools. Strong confusion of language

factors and content-based knowledge makes assessment and accountability

36

complex for LEP students and, very likely, students in other targeted

groups.

Because of the strong effect of language factors on the instruction and

assessment of LEP students, they lag far behind native English speakers.

This leads to huge initial differences. LEP students start with substantially

lower baseline scores. More important, unless LEP students‟ English

language proficiency is improved to the level of native English speakers-

which is not an easy task- they will not be able to move at the same rate on

the Adequate Yearly Progress line as do native English speakers.

NCLB cannot have much of an effect on the initial performance

differences between LEP and non-LEP students. A more sensible question

here is whether or not NCLB can provide enough resources to schools with a

large number of LEP students to help them increase these students‟

language proficiency to a sufficient extent that they can progress with their

native English speaker peers in both instruction and assessment.

Inconsistency in LEP classification across and within states makes

AYP reporting for LEP students even more complex. If students are not

correctly identified as LEP, how can their AYP be reliably reported at a

subgroup level? Although NCLB attempts to resolve this issue by providing a

definition for this group, its criteria for classifying LEP students may face

the same problems as the existing classification system (Abedi, 2003;

Zehler, Hopstock, Fleishman & Greniuk, 1994).

37

Inconsistency in the classification of LEP students may lead to more

heterogeneity in the LEP subgroup. With a more heterogeneous population,

larger numbers of students are needed to provide the statistically reliable

results required by NCLB. The population of LEP students in many districts

and states is sparse. In many states, there may not be enough students in a

district or school to satisfy even the minimum number of 25 students

suggested in the literature (Linn, Baker & Herman, 2002). Other researchers

have argued that even 25 students may not be enough to provide

statistically reliable results and have proposed a minimum group size of 100

students (Hill & DePascale, 2003). Considering a small number of LEP

students in many districts and states, the small group size for LEP reporting

would be another obstacle in regard to reliable AYP reporting.

The LEP subgroup suffers from yet another major problem related to

AYP reporting: The lack of stability of this group. In many states and

districts across the nation, LEP students‟ level of English proficiency is

reevaluated regularly, and if they reach a proficient level of English

proficiency, they move out of the LEP subgroup. While this helps the more

English-proficient students receive more appropriate instruction and

assessment, it results in the LEP subgroup continuing to be low-performing.

The students in this group will always be labeled as underachievers, and

schools with large number of LEP students will be stuck in the “need for

improvement” category.

38

Some states with substantial numbers of LEP students have

expressed concern over this issue. They have proposed ideas and negotiated

with the federal government to ease the level of possible negative impact

that this situation may have on school, district, and state accountability.

For example, Indiana and Delaware will continue to include exited LEP

students in the LEP subgroup for 2 years after they have been determined to

be proficient in English. Georgia plans to include LEP students as long as

they still receive services through the English for Speakers of Other

Languages program, even if they have met exit criteria (Erpenbach, Forte-

Fast & Potts, 2003). In California, students re-designated as LEP will remain

in the LEP category until they reach the proficient or above level on the

California Standards Test in English-language arts for 3 consecutive years

(California Department of Education, 2003). However, the question of

whether this policy will provide a long-term solution to the problem of LEP

subgroup instability or serve only as a temporary relief remains

unanswered.

The measurement of the academic achievement of LEP students is

much more complex than what the NCLB legislation conceives. A fair

assessment of students in the four targeted subgroup categories requires

much more serious consideration than is outlined in the law. Despite

attempting to solve the age-old problem of heterogeneity among LEP

students, the NCLB seems to perpetuate it, thereby leaving more room for

children to be left behind.

39

On the other hand, NCLB‟s attention to students in the four subgroup

categories in general and to the LEP population in particular is a step in the

right direction. Considering that Title III of NCLB requires assessment of

LEP students‟ English proficiency on an annual basis and providing support

to states to develop reliable and valid measures of students‟ proficiency is

promising. Any decisions concerning assessment for all subgroups,

particularly LEP students, must be informed by results of research and

experience in the education community.

Currently, several tests for measuring students‟ level of English

language proficiency exist. Some of these tests have been used for many

years by different states and districts. In spite of the existence of such tests,

states are developing new English language proficiency tests with funding

through NCLB‟s Enhanced Assessment Instruments. A reasonable

explanation for this might be that states did not find that the existing tests

provided reliable and valid measures of students‟ level of English language

proficiency as required by NCLB. If this is the reason for the development of

the new tests, then the test developers should be aware of problems in the

existing tests to avoid the same problems in the new tests.

For example, a careful review of some of the most commonly used

language proficiency tests concluded that the tests differ considerably in

types of tasks and specific item content and are based on different

theoretical emphases prevalent at the time of their development (Zehler,

Hopstock, Fleischman & Greniuk, 1994). This suggests that in the case of

40

some of the existing tests, the English language proficiency domain was not

operationally defined before the test development process. This and similar

studies and reviews should inform the development process of new tests.

For example, it is imperative this domain be operationally defined before any

effort in developing an English proficiency test. This definition should be

based on current developments in the areas of psycholinguistics,

developmental psychology, education, linguistics, and psychometrics.

Content standards for English for speakers of other languages should also

be considered (Bailey & Butler, 2003).

In analyzing data from the administration of existing language

proficiency tests, researchers have expressed concerns about the reliability

and validity of these tests, the adequacy of the scoring directions, and the

limited populations on which test norms are based. For example, analyses

of several large data sets from different locations across the nation have

shown validity problems in predicting LEP classification and lack of power

in identifying different levels of English language proficiency among the LEP

student population (Abedi, 2003; Abedi, Leon, & Mirocha, 2003). Those

involved in the development of new English language proficiency tests

should learn from such research and should conduct more analyses on the

wealth of data that exists in this area. To be considered valid and reliable

measures of English language proficiency, as outlined in the NCLB, new

tests must first go through a rigorous validation process. Otherwise, there

41

may not be a reasonable justification to spend the limited NCLB resources

on English language proficiency test development (Abedi, 2003).

As a final thought, assessment and accountability of LEP students

cannot be pursued in isolation of other important factors. An effective

education system for LEP students that may lead to a successful AYP

outcome should include at least three interactive components: (a)

classification, (b) instruction, and (c) assessment. A problem in any one of

these components may affect the other two. For example, a student

misclassified as LEP student may be assigned a different curriculum and

thus receives inappropriate instruction. Alternately, inappropriate

instruction may result in low performance that may in turn result in

misclassification. While each component has a unique role, they share

common ground - the effect of language factors or barriers. Unnecessary

linguistic complexity of assessment may threaten the validity and

equitability of assessment among LEP students. Complex linguistic

structure of instruction may negatively affect LEP students‟ ability to

understand classroom instruction, and invalid assessment of students‟ level

of English proficiency may result in misclassification. In a positive light,

valid assessment may provide diagnostic information that can inform

instruction and classification (Abedi, 2003).

An effective way to help LEP students reach proficiency in the AYP

model is to consider the broader picture using the interactive model. The

following are few critical needs:

42

1. Improve current LEP classification and assessment. There is a need to

establish a common definition of English language proficiency and

substantially improve the validity of LEP instruments. Among other

things, validity of LEP assessment can be enhanced by avoiding

cultural biases and reducing unnecessary linguistic complexity of

assessments.

2. Improve monitoring of progress. Schools need effective and valid data

collection methods that can be used to monitor LEP progress at every

stage of a student‟s education. Weaknesses must be quickly

addressed with appropriate instructional strategies.

3. Improve teacher quality. LEP students need teachers who are well

qualified in both language development and content, each of which

plays a crucial role in LEP student achievement. The federal

government can play a key role in this process by funding and

encouraging programs that improve teacher capacity in this dual role.

Teachers of LEP students should receive training in content delivery,

language sheltering, and the teaching of the academic language.

4. Consider redesignated LEP students as part of the LEP subgroup that

established the baseline score. State plans allowing redesignated

students to remain in the LEP subgroup for only a limited time are

temporary fixes. While new LEP students are added to the subgroup,

redesignated students should be retained for AYP reporting. This

“semicohort” approach to tracking LEP students allows the progress of

43

redesignated students to be counted toward subgroup AYP progress

(Abedi, 2003).

Based on the results of the research, policymakers, lawmakers, and

decision makers are urged to take appropriate action to correct the

inequities resulting from the NCLB in regard to the subgroups targeted by

the legislation, particularly the LEP student subgroup. What is encouraging

is that states, in collaboration with the federal government, are taking steps

to remedy some of these issues. The hope is that these continued efforts will

bring more fairness into the assessment of and accountability for LEP

students (Abedi, 2003).

High Stakes / Statewide Testing

The 2001 reauthorization of the Elementary and Secondary Education

Act (ESEA), also known as the No Child Left Behind (NCLB), carries testing

and accountability requirements that will substantially increase student

testing and hold all schools accountable for student performance. This

legislation marks a major departure from the federal government‟s

traditional role regarding elementary and secondary education. It requires

that states administer Reading and Mathematics tests annually in grades 3

– 8 and during one year in high school starting in 2005 – 2006. These

requirements will affect almost 25 million students each school year

(National Center for Education Statistics, 2002).

NCLB requires states to meet adequate yearly progress (AYP) goals to

ensure school accountability for student achievement on state tests. Schools

44

that fail to achieve AYP goals face demanding corrective actions, such as

replacement of school staff, implementation of new curriculum, extension of

the school day or academic year, parental choice options, and, finally,

complete reorganization.

Today‟s widespread implementation of standards-based reform and

the federal government‟s commitment to test-based accountability ensure

that testing will remain a central issue in education for the foreseeable

future. Test results can provide useful information about student progress

toward meeting curricular standards. But when policymakers insist on

linking test scores to high-stakes consequences for students and schools,

they often overlook lessons from the long history of research (Abrams &

Madaus, 2003).

Current emphasis on testing as a tool of education reform continues a

long tradition of using tests to change pedagogical priorities and practices.

In the United States, this use of testing dates back to 1845 in Boston, when

Horace Mann, then Secretary of the Massachusetts State Board of

Education, replaced the traditional oral examination with a standardized

written essay test. Internationally, high-stakes testing extends as far back

as the 15th century in Treviso, Italy, where teacher salaries were linked to

student examination performance (Madaus & O‟Dwyer, 1999).

Principles of Testing Programs

A 1988 examination of the effects of high-stakes testing programs on

teaching and learning in Europe and in the United States (Madaus, 1988)

45

identified seven principles that captured the intended and unintended

consequences of such programs. Current research confirms that these

principles still hold true for contemporary statewide testing efforts.

Principle 1: The power of tests to affect individuals, institutions,

curriculum, or instruction is a perceptual phenomenon. Tests produce large

effects if students, teachers, or administrators believe that the results are

important. Policymakers and the public generally do believe that test scores

provide a reliable, external, objective measure of school quality. They view

tests as symbols of order, control and attainment (Airasian, 1988).

Today‟s high-stakes testing movement relies on the symbolic

importance of test scores. Forty-eight states currently require schools to

provide the public with “report cards” (Edwards, 2003). Goldhaber and

Hannaway (2001) found that the stigma associated with a school receiving a

low grade on the state report card was a more powerful influence on Florida

teachers than were the school-level sanctions imposed for poor test results.

Principle 2: The more any quantitative social indicator is used for

social decision making, the more likely it will be to distort and corrupt the

social process it is intended to monitor. In other words, placing great

importance on state tests can have a major influence on what takes place in

the classrooms, often resulting in an emphasis on test preparation that can

compromise the credibility or accuracy of test scores as a measure of

student achievement.

46

We can assess whether this principle still applies today by examining

the relationship between rising state test scores and scores on other

achievement tests. Both old and new studies of this relationship (Amrein &

Berliner, 2002; Haladyna, Nolen & Haas, 1991; Klein, Hamilton, McCaffrey

& Stecher, 2000; Linn, 1998) show that improvements in the state test

scores do not necessarily reflect general achievement gains.

We can find examples of this second principle in two recent surveys of

teachers‟ opinions. In one national study, roughly 40% of responding

teachers reported that they had found ways to raise state-mandated test

scores without, in their opinion, actually improving learning (Pedulla,

Abrams, Madaus, Russell, Ramos & Miao, 2003). Similarly, in a Texas

survey, 50% of the responding teachers did not agree that the rise in Texas

Assessment of Academic Skills (TAAS) scores “reflected increased learning

and high-quality teaching” (Hoffman, Assaf & Paris, 2001, p. 488).

Principle 3: If important decisions are based on test results, then

teachers will teach to the test. Curriculum standards and tests can focus

instruction and provide administrators, teachers, and students with clear

goals. A substantial body of past data and recent research confirms that as

the stakes increase, the curriculum narrows to reflect the content sampled

by the test (Jones et al., 1999; Madaus, 1991; McMillan, Myran, &

Workman, 1999; Pedulla et al., 2003; Stecher, Barron, Chun & Ross, 2000).

New York State, where the state department of education is requiring

schools to spend more time on the NCLB-tested areas of Reading and

47

Mathematics, provides an example on how such pressure encourages

schools to give greater attention to tested content and decrease emphasis on

non-tested content. According to one school principal, “the art, music, and

everything else are basically out the window… something has to go”

(Herszenhorn, 2003).

Principle 4: In every setting where a high-stakes test operates, the

examination content eventually defines the curriculum. Pressure and

sanctions associated with a state test often result in teachers using the

content of past tests to prepare students for the new test. Several studies

have documented that an overwhelming majority of teachers feel pressure to

improve student performance on the state test. For example, 88% of

teachers surveyed in Maryland and 98% in Kentucky believed that they were

under “undue pressure” to improve student performance (Koretz, Barron,

Mitchell & Keith, 1996a, 1996b). As an outgrowth of this pressure, the

amount of instructional time devoted to specific test preparation often

increased.

Studies have found that teachers are spending a sizable amount of

instructional time and using a variety of test-specific methods to prepare

students for their state tests (Herman & Golan, n.d.; Hoffman, Assaf, &

Paris, 2001). In North Carolina, 80% of elementary teachers surveyed “spent

more than 20% of their total instructional time practicing for the end-of-

grade tests” (Jones et al., 1999, p. 201). A national survey found that

teachers in high-stakes states were four times more likely than those in low-

48

stakes setting to report spending more than 30 hours a year on test

preparation activities, such as teaching or reviewing topics that would be on

the state test, providing students with items similar to those on the test,

and using commercial test-preparation materials from previous years for

practice (Pedulla et al., 2003).

Principle 5: Teachers pay attention to the form of the questions of

high-stakes tests (short-answer, essay, multiple-choice, and so on) and

adjust their instruction accordingly. A wide variety of research confirms that

test format does influence instruction in both positive and negative ways.

Studies in states that require students to formulate and provide

written responses to test questions show an increased emphasis on teaching

writing and higher-level thinking skills (Taylor, Shepard, Kinner &

Rosenthal, 2003). For example, in Kentucky, 80% of teachers surveyed

indicated that they had increased their instructional emphasis on problem

solving and writing as a result of the portfolio-based state test (Koretz,

Barron, Mitchell, & Keith, 1996a).

In several studies, teachers have reported decreases in the use of

more time-consuming instructional strategies and lengthy enrichment

activities (Pedulla et al., 2003). A study found that the format of the state

test may adversely affect the use of technology for instructional purposes:

One-third of teachers in high-stakes states said that they were less likely to

use computers to teach writing because students were required to construct

handwritten responses on the state test (Russell & Abrams).

49

Principle 6: When test results are the sole or even partial arbiter of

future education or life choices, society treats test results as the major goal

of schooling rather than as a useful but fallible indicator of achievement.

Almost 100 years ago, a chief inspector of schools in England described this

principle in a way that resonates today: Whenever the outward standard of

reality (examination results) has established itself at the expense of the

inward, the ease with which worth (or what passes for such) can be

measured is ever tending to become itself the chief, if not sole, measure of

worth. And in proportion as we tend to value the results of education for

their measurableness, so we tend to undervalue and at last ignore those

results which are too intrinsically valuable to be measured (Holmes, 1911).

In the next five years, almost half of U.S. states will require students

to pass a state-mandated test as a requirement for graduation (Edwards,

2003). As a result, a passing score on the state test is the coin of the realm

for students, parents, teachers, and administrators. The social importance

placed on state test scores ensures that students‟ successful performance

on the state test is the ultimate goal for schools. Local press coverage on

school pass rates and anecdotal evidence that scores on the state test may

influence local real estate sales show the importance of test performance as

a surrogate for education quality.

Principle 7: A high-stakes test transfers control over the curriculum to

the agency that sets or controls the examination. State standards-based

reform efforts leave the details and development of testing programs to state

50

departments of education and whomever the department contracts with to

construct the test. This system shifts the responsibility for determining

curricular priorities and performance standards away from local school

administrators or classroom teachers and often results in a one-size–fits-all

curriculum and test.

Falmouth, Massachusetts, provides a recent noteworthy example of

how a high-stakes state test can override local control. Under the threat of

losing state funding and the licensure of the school principal and

superintendent, the Falmouth School Committee reversed a decision to

award diplomas to special-needs students who failed the Massachusetts

state examination, thus shattering the hopes of a student seeking

admittance to a nonacademic culinary degree program (Myers, 2003).

Accountability in Testing

No one denies the importance of accountability. The relationship

between test scores and accountability, however, is not as simple as some

people think. The seven principles formulated in 1988 have been acted out

in state after state in the past 15 years and clearly reveal the serious flaws

in the practice of using a single high-stakes measure to hold all students

and schools accountable.

Cut-off scores that place students in such performance categories as

needs improvement, basic, proficient, or advanced are arbitrary. The

subjective methods used to categorize students into performance categories

often lack validity (Horn, Ramos, Blumer & Madaus, 2000). Most

51

policymakers and the public do not understand the psychometric

underpinnings of the tests. Issues that might seem trivial to them, such as

the assumptions made when running computer programs that produce

scaled scores, and even basic decisions about rounding, have significant

consequences when categorizing students.

Like any measurement tool that produces a number, test scores are

fallible. Yet most state laws do not consider margin of error when

interpreting students‟ scores. Misguided executive decisions, poorly

conceived legislation, understaffing, unrealistic reporting deadlines, and

unreasonable progress goals can cause numerous errors in test scores

(Rhoades & Madaus, 2003).

One single test can only sample knowledge and cannot give a full

picture of what students know and can do. As an illustration, Harlow and

Jones‟s (2003) interviews with students showed that on the science portion

of the Third International Mathematics and Science Study (TIMSS), the

students had more knowledge about concepts than their written answers

had demonstrated for more than half of the test questions. Conversely, the

interviews suggested that for one-third of the items, students lacked a

sound understanding of the information assessed even though they had

given the correct response.

A fundamental principle in social science research is to always use at

least two methods when studying social science phenomena because relying

on only one method can produce misleading results. We need to enhance

52

state testing programs by including multiple measures of student

achievement. Measuring in a variety of ways does not mean giving students

multiple opportunities to take the same test, but rather incorporating other

methods of measurement or additional criteria, such as teacher judgments,

when making decisions about grade promotion and graduation (Harlow &

Jones, 2003).

As districts, schools, and teachers respond to federal and state-based

accountability policies, we must step back from a blind reliance on test

scores. We need to acknowledge that tests, although useful, are also fallible

indicators of achievement. We also need to recognize that when test scores

are linked to high-stakes consequences, they can weaken the learning

experiences of students, transform teaching into test preparation, and taint

the test itself so that it no longer measures what it was intended to measure

(Harlow & Jones, 2003).

Effects of High-Stakes Testing on Student Motivation and Learning

Current generation of policymakers did not invent high-stakes testing.

Tests of various sorts have determined which immigrants could enter the

United States at the turn of the 20th century, who could serve in the armed

forces, who was gifted, who needed special education, and who received

scholarships to college. But the NCLB Act of 2001 aims to make high-stakes

testing more pervasive than ever before, mandating annual testing of

students in grades 3 – 8 in Reading and Mathematics.

53

Federal legislators who overwhelmingly passed this act into law

apparently assumed that high-stakes testing would improve student

motivation and raise student achievement. Because testing programs

similar to those required by NCLB already exist in many states, we can put

that assumption to the test.

Eighteen states currently use examinations to grant or withhold

diplomas: Alabama, Florida, Georgia, Indiana, Louisiana, Maryland,

Minnesota, Mississippi, Nevada, New Jersey, New Mexico, New York, North

Carolina, Ohio, South Carolina, Tennessee, Texas and Virginia. Most of

these states also attach to their state assessments a broad range of other

consequences for students, teachers, and schools. The experiences of these

states can help predict how the new nationwide program of high-stakes

testing will affect student achievement.

Unfortunately, the evidence shows that such tests actually decrease

student motivation and increase the proportion of students who leave school

early. Further, student achievement in the 18 high-stakes testing states has

not improved on a range of measures, such as the National Assessment of

Educational Progress (NAEP), despite higher scores on the state‟s own

assessments. (Amrein & Berliner, 2003).

High-stakes testing assumes that rewards and consequences attached

to rigorous tests will “motivate the unmotivated” to learn (Orfield &

Kornhaber, 2001). The “unmotivated” are usually identified as low socio-

economic students in urban schools, often African Americans and Latinos.

54

Researchers have found that when rewards and sanctions are attached to

performance on tests, students become less intrinsically motivated to learn

and less likely to engage in critical thinking. In addition, they have found

that high-stakes testing cause teachers to take greater control of the

learning experiences of their students, denying their students opportunities

to direct their own learning. When the stakes get high, teachers no longer

encourage students to explore the concepts and subjects that interest them.

Attaching stakes to tests apparently obstruct students‟ path to becoming

lifelong, self-directed learners and alienates students from their own

learning experiences in school (Sheldon & Biddle, 1998).

Dropout rates are climbing throughout the United States and many

researchers hold high-stakes testing at least partly to blame (Rothstein,

2002). Some researchers found that dropout rates were 4 to 6 percent

higher in schools with high school graduation examinations. Another study

reported that students in the bottom quintile in states with high-stakes

testing were 25% more likely to drop out of high school than were their

peers in states without high-stakes testing (Jacob, 2001). Researchers in yet

another study found that failing these tests significantly increased the

likelihood that even students with better academic records would drop out

(FairTest & Massachusetts CARE, 2000).

More and more teenagers are exiting formal schooling early to earn a

General Educational Development (GED) credential (Murnane, Willett, &

Tyler, 2000). Although young people who have earned such alternative

55

degrees do not technically count in dropout statistics, many of them

undoubtedly left school because of their concerns about passing rigorous

graduation tests.

Students who repeat a grade are significantly more likely to drop out

of school (Goldschmidt & Wang, 1999). In states where promotion to the

next grade hinges on passing the state exams, high-stakes testing policies

contribute to higher dropout rates in the long run. Even before they actually

take the test, struggling students are more likely to be retained in grade if

they attend schools in high-stakes testing environments. By holding low-

achieving students back, schools ensure that these students have more of

the knowledge necessary to perform well on high-stakes testing the next

year and keep low-performing students‟ test scores out of the composite test

performance in the grades in which high-stakes testing matter.

In Texas, students from racial minorities and low socio-economic

backgrounds are being retained in Grade 9 at very high rates before taking

the Texas Assessment of Academic Skills (TAAS) in Grade 10. Many teachers

retain students if they doubt their potential to pass TAAS the following year.

McNeil (2000) estimated that half of all minority students enrolled in Texas

high schools are technically enrolled as freshmen. Although some of them

are 9th graders for the first time, thousands of others have been retained in

the 9th grade once or even twice. Other researchers (Haney, 2000, 2001;

Klein, Hamilton, McCaffey & Stecher, 2000; Yardley, 2000) have verified her

numbers. In 1998, one in every four African American and Latino 9th

56

graders in Texas was retained (Fisher, 2000). After these students are

retained, thousands of them drop out of school.

Common problems of high-stakes testing programs are quite likely to

affect the breadth and depth of student learning. If schools narrow the

curriculum they teach; make heavy use of drill activities tied to the state

test; cheat by over-identifying language minority and special education

students and then keeping these students from taking these tests; retain

poorly performing students in grade; and encourage those who are at least

likely to pass the state‟s test to drop out, then scores on state tests will

almost certainly go up. But have students really learned any more than they

did before high-stakes testing policies were instituted (Fisher, 2000)?

Other Considerations of Assessment and Testing

Although NCLB now requires all students to be accounted for in any

state‟s assessment system, this has not always been the case (Anderson,

2004). In the past, groups of students such as English language learners or

students in Special Education were systematically excluded from large scale

assessments (State, 1999), or their scores were not reported (Thurlow,

Neilson, Tellucksingh, & Ysseldyke, 2000).

In the 2002-2003 school year, the Texas Assessment of Knowledge

and Skills (TAKS) replaced the Texas Assessment of Academic Skills (TAAS)

as the primary statewide assessment program. TAKS is designed by

legislative mandate to be more comprehensive than its predecessors and

encompasses more of the state-mandated curriculum, the Texas Essential

57

Knowledge and Skills (TEKS), at more grade levels than TAAS did. The high

school level assessments, administered at Grades 9, 10 and 11, are

grounded in the high school TEKS curriculum. By law, students for whom

TAKS is the graduation testing requirement must pass exit level tests in four

content areas – English Language Arts, Mathematics, Science, and Social

Studies – in order to graduate from a Texas public high school (Technical

Digest, 2003-2004).

In Texas, there is evidence that the numbers of black and Hispanic

students in Special Education rose between 1994-1998 while the state

implemented its statewide testing program which excluded some Special

Education students from the reported scores (Haney, 2000). It is interesting

to note that while the numbers of African-American and Hispanic students

are over-represented in Special Education, about eight to nine percent of

English language learners are identified as receiving Special Education

services in the US (D‟Emilio, 2003, June; Zehler, Fleischman, Hopstock,

Pendzick, & Stepherson, 2003).

Labeling schools can have an impact on teacher and student morale

(Anderson, 2004). Certainly, poor test scores or poorly explained

assessment systems can result in decreased student motivation (Lane &

Stone, 2002). Teachers have also reported that the high-stakes nature of

some assessments can have a negative impact on student morale (Flores &

Clark, 2003). Although some teachers have reported that their English

58

language learners can reach the high standards set for them, they may need

more time than other students (Hood, 2003).

For English-language learners, the additional requirements of an exit

examination could increase dropout rates (Anderson, 2004). Hispanic

students, many of whom are English-language learners, have higher

dropout rates than the population as a whole (Barro & Kolstad, 1987;

Kaufman, Alt, & Chapman, 2001).

In another study, teachers reported that increasing emphasis on test

scores cause them to dislike their jobs (Hinde, 2003). In a study examining

the discussion and journal entries from teachers, Flores and Clark (2003)

found that teachers were not against accountability and viewed it as distinct

from statewide testing, but also thought that an over-emphasis on testing

resulted in unbalanced curriculum and inappropriate instructional

decisions. In order for teachers to make specific changes to instruction, the

assessments needs to be clear as to what skills are being assessed (Popham,

2003). Some teachers may react to low test scores of English language

learners by teaching to the test while others may ignore the impact of the

test scores all together (Alderson & Wall, 1993).

Testing cannot be divorced from socio-cultural, economic, and

psychological issues (Solano-Flores & Trumbull, 2003). ELLs, for instance,

may not score any differently in an assessment even when allowed to use a

dictionary (Albus, Thurlow, Liu & Bielinski, 2005). This is further

complicated by the theory forwarded by Wang and Koda (2005) that ELLs as

59

a group may have diverse styles in developing English Language proficiency.

Therefore modifications are adapted to teach ELLs the academic content of a

lesson, and at the same time support the acquisition of a new language

(Major, Fitzmaurice, Bunta, & Balasubramanian, 2005).

A study of washback from a test in Hong Kong demonstrated that

change in the assessment could change the ways in which teachers and

students interacted (Cheng, 1999). School Administrators and teachers, as

well as students, need to be motivated to change the way learning takes

place and also be invested in demonstrating achievement on the

assessments in order for washback to instruction to take place and be

successful (Lane & Stone, 2002).

Related Studies

In a study on “Intended and Unintended Consequences of Statewide

Testing for ESL Curriculum and Instruction”, Anderson (2004) examined

what positive or negative impact assessment systems have on the

curriculum and instruction of English language learners in one Midwestern

school district. The researcher used focus groups and interviews to obtain

views of educators on the consequences of statewide testing for ELLs.

Positive consequences that were identified included more teacher

collaboration, changes in curriculum and instruction, better alignment

between ESL and content area curricula and more focus on reading and

writing. Negative consequences included student and teacher frustration,

more teaching to the test occurring, and a narrowed curriculum. Educators

60

in the study also identified problems with the accountability system and

made recommendations for how it could be improved (Anderson, 2004).

Another study on “Inclusion of Students with Limited English

Proficiency in National Assessment of Educational Progress (NAEP):

Classification and Measurement Issues” conducted by Abedi (2004) reported

the major concerns over classification and measurements for students with

limited English proficiency (LEP). Issues included the poor operational

definition of English proficiency construct and validity concerns on the

existing language proficiency tests. The study discussed issues concerning

the classification of ELLs and elaborated on factors that impact decisions to

include ELLs in NAEP assessments. With funding through a competitive

bidding process authorized under the NCLB section on Enhanced

Assessment Instruments, there are national efforts underway to develop

English proficiency tests that can be used to provide valid measures of

students‟ level of English proficiency (Abedi, 2004).

Wall (2000) made a microethnographic case study entitled “A Case

Study of Secondary School Efforts Toward English Language Learner

Success in a Standards-Based Reform System.” This study was designed to

describe and interpret the site-based decision-making process of a

collaborative study group of high school educators as they focused on the

appropriate participation of ELLs in a district wide, standards-based, reform

initiative. The research question which guided the study was: From what

perspectives and with what outcomes does a collaborative group of site-

61

based, high school educators deliberate the participation of ELLs in a

standards-based reform system which mandates high stakes assessments?

Three themes emerged from the study: (a) personal discovery, (b) informed

action, and (c) instructional advocacy. These themes suggested phases of

sociolinguistic accommodation through which educators progress in their

reform-based deliberations regarding appropriate approaches to support

ELLs in a high-stakes assessment system (Wall, 2000).

This study on the impact of high-stakes testing on ELLs in major

urban high schools in Texas showed quantitatively how the percentage of

ELLs enrolled in a school affects the school‟s performance in the State‟s

assessment. Qualitatively, it gathered the input and feedback of educators

on the different concerns included in the study: (a) purpose of TAKS, (b)

changes caused by TAKS, (c) consequences of TAKS, (d) recommendations to

improve TAKS, and (e) needs of ELLs.

Summary

As stated in chapter I, the purpose of this study was to determine the

impact of high-stakes testing on ELLs. This was shown in both the

quantitative and qualitative dimensions of the study. Data obtained from

TEA were used to determine whether there is a relationship between the

percentage of English language learners enrolled in a school and the

percentage of all students passing the 10th grade TAKS tests in the core

areas of English Language Arts and Mathematics given in 2003, 2004, 2005

and 2006. To support the quantitative aspect, this study explored what

62


administrators, and district ESL personnel viewed as the impact that high

stakes standardized assessments have on ELLs, ESL curriculum and


The mandates and key elements of the NCLB were geared towards

improving the achievement of students in the different public schools of the

United States. The measure of adherence was channeled through the AYP

that the different schools and districts of the different states monitor and

report. High-stakes testing became the measuring stick that gauged the

achievement of students in the different core subject areas. Issues and

concerns were centered on the ELLs regarding the different moves and

accommodations given to this special subgroup of learners. Feedback

regarding the issues and concerns of the different studies and researches

included both positive and negative dimensions.

63

CHAPTER III

METHODOLOGY

Introduction

Standardized testing and assessments have become necessary

facets of American education. Consequently, accountability testing is

currently implemented in practically every state in the U.S. Since the

purpose of this increased level of accountability is to ensure that all

students are receiving a quality, standards-based education, it is

important to document the consequences of the system to ensure that

the intended reforms are taking place. One of the goals of the

accountability system should be to document any negative opposing

impact that could occur so that interventions can be developed so that

these consequences can be minimized.


stakes testing on ELLs. This is shown in both the quantitative and

qualitative dimensions of the study. Data obtained from TEA were used

to determine whether there is a relationship between the percentage of

English language learners enrolled in a school and the percentage of all

students passing the 10th grade TAKS tests in the core areas of English

Language Arts and Mathematics given in 2003, 2004, 2005 and 2006. To

support the quantitative aspect, this study explored what certified ESL

teachers, non-certified ESL teachers who teach ELLs, administrators,

and district ESL personnel viewed as the impact that high stakes

64

standardized assessments have on ELLs, ESL curriculum and


The study also addressed concerns regarding the validity of

student evaluations, and the common inferences made about student

performance in these assessments. There is a need to know how public

schools that have diverse student attributes can be held accountable on

the basis of one uniform and universal standard. Since the standardized

assessments are given in English, schools with predominantly Hispanic

populations may already be at a disadvantage through no fault of their

own. What needs to be examined is whether standardized assessments

facts are free from linguistic and cultural bias as viewed by teachers of

ELLs.

Research Questions

Quantitative




and Mathematics given in 2003, 2004, 2005 and 2006?

Null Hypotheses

HO1: There is no statistically significant relationship between the



English Language Arts given in 2003, 2004, 2005 and 2006.

65




Mathematics given in 2003, 2004, 2005 and 2006.

Qualitative

The major question answered by this study was: What are the

anticipated and observed consequences of statewide testing specifically,

TAKS, on ELLs, ESL curriculum and instruction as viewed by certified

ESL teachers, non-certified ESL teachers who teach ELLs, school

administrators, and district ESL personnel?



2. What are the intended consequences of this statewide testing?

(Or what has happened because of TAKS?)



5. What are your recommendations to improve this statewide

testing?

6. What needs to be done for the ELLs to improve their


66

Research Methods

Both descriptive and comparative research techniques were

employed in the explanatory design of the mixed methods study.

Fraenkel and Wallen (2003) stated that Creswell describes the two types

of mixed methods.

1. In a triangulation design, the researcher simultaneously collects

both quantitative and qualitative data, compares results, and

then uses those findings to see whether they validate each other

(p. 443).

2. In an explanatory design, the researcher first collects and

analyzes quantitative data, and then obtains qualitative data to

follow up and refine the quantitative findings (p. 443).

For this study, the explanatory design was used. Quantitative data

for this research were gathered through TEA to determine if a

relationship existed between the percentage of English language learners

enrolled in a school and the percentage of all students passing the 10th

grade TAKS tests in the core areas of English Language Arts and

Mathematics given in 2003, 2004, 2005 and 2006. Qualitative data were

obtained through the on-line, open-ended questionnaire and individual

and focus group interviews about the varied ways in which standardized

assessments impacted ELLs.

For the qualitative research component, the study used the cross-

sectional, open-ended questionnaire. A cross-sectional, open-ended

67

questionnaire collects information from a sample that has been drawn

from a predetermined population. Furthermore, the information is

collected at just one point in time, although the time it takes to collect

the data may take anywhere from a day to a few weeks or more (Fraenkel

& Wallen, 2003).

The study also utilized descriptive research methods. Isaac and

Michael (1995, p. 46) describes this type of research as: “to describe

systematically a situation or area of interest factually or accurately.” A

survey study also falls under the classification of descriptive research.

Van Dalen (1979) lists the purpose of survey studies:

1. To collect detailed factual information that describes existing

phenomena.

2. To identify problems or justify current conditions and practices.

3. To make comparisons and evaluations.

4. To determine what others are doing with similar problems or

situations and benefit from their experience in making future

plans and decisions.

Research Design

Since the study utilized the explanatory design of the mixed

methods, the investigator first gathered quantitative data from Texas

Education Agency (TEA) regarding the major urban high schools in

Texas. TEA records personnel assisted in accessing and retrieving data

from the TEA website. Qualitative data were obtained through the on-

68

line, open-ended questionnaire and individual and focus group

interviews; views and opinions of the respondents were gathered and

collated to validate and support the quantitative data.

Quantitative Data

From the Texas Education Agency, the following data regarding the

urban high schools were gathered: the percentage of English language



and Mathematics given in 2003, 2004, 2005 and 2006.

Qualitative Data

Qualitative data were obtained using an on-line, open-ended

questionnaire given to principals, assistant principals, ESL district

personnel, ESL certified teachers and non-ESL certified teachers who

were purposively sampled for the study and through the individual and

focus group interviews using open-ended questions about the varied

ways in which standardized assessments impact ELLs.

Pilot Study

Two Houston Independent School District schools, not included in

the main study were selected for the pilot study. Quantitative data were

obtained regarding the schools’ percentage of English language learners

enrolled in a school and the percentage of all their 10th grade students

passing TAKS in the two core areas of English Language Arts and

69

Mathematics. This was for the four school years starting with the first

school year 2002 – 2003, when TAKS was administered.

During the pilot study the on-line questionnaire underwent pre-

testing with three basic considerations: (1) administered the pre-test

under conditions comparable to those anticipated in the final study; (2)

analyzed the results to assess the effectiveness of the trial questionnaire

to yield the information desired; and (3) made appropriate additions,

deletions, and modifications to the questionnaire (Isaac & Michael,

1995).

Qualitative data resulting from an on-line open-ended

questionnaire on the six different concerns listed below were tabulated

combining the results from the two schools. Results were categorized

using the NVivo software package but the categories were modified based

on the expert opinion of the respondents belonging to the focus groups.

The frequencies for the responses by the different respondents (teachers,

school administrators and district ESL personnel) pertaining to the

different categories were tallied and percentages were computed. Listing

of categories was based on the total frequencies; those categories

identified most by the respondents were listed first followed by those with

lower frequencies. The different concerns included the following: (1)

Purpose of TAKS; (2) Consequences of TAKS; (3) Problems Related to

TAKS; (4) Changes Caused by TAKS; (5) Recommendations to Improve

TAKS; and (6) Needs of ELLs.

70

Results of the focus group and one-on-one interviews were

validated against the results of the on-line questionnaire and provided

explanation or support for the answers given. The categories for the

different responses were affirmed or modified by the focus groups.

Population and Samples

Quantitative Data

The TEA provided the data on the percentage of English language



and Mathematics given in 2003, 2004, 2005 and 2006. The researcher

used purposive sampling in selecting schools for this study. Purposive

sampling is based on the assumptions that the investigator wants to

discover, understand, and gain insight and therefore must select a

sample from which the most can be learned (Merriam, 1998).

While not a random sampling of high schools, the sample is not

intended to create results that can be generalized to all major urban high

schools in the U.S. A purposive sampling was used in order to provide a

representative sample of the major urban high schools in Texas in order

to gain in-depth insight into what impact might be occurring. The impact

that might emerge from this study might occur in other high schools, but

it is important to take into account the characteristics of the high

schools as well as the assessment system in the state in order to

71

extrapolate from the findings and make comparisons with other

situations (Patton, 1990).

Qualitative Data

The on-line, open-ended questionnaire was given to the principals,

assistant principals, certified ESL teachers and non-ESL certified

teachers handling ELLs of the selected schools and to the district

personnel:

Total

1) ESL Teachers 30

2) Non-Certified ESL Teachers 30

3) Principals 10

4) Assistant Principals 20

5) District ESL Personnel 8

Total 98

The different focus groups consisted of ESL certified and non-ESL

certified teachers handling ELLs. One-on-one interviews involved the

selected principals and the selected district ESL personnel. The same

schools and district personnel who answered the on-line questionnaire

were included in the focus groups and one-on-one interviews. Selection

of the participants in the focus group interviews utilized the snowballing

technique. Participants will identify others whose input or experience will

also be valuable to the study (Krathwohl, 1993).

72

Since the researcher has the obligation to respect and protect the

rights and wishes of the research participants, the following actions were

done: (1) the researcher protected anonymity of the participants by using

computer-given codes for the responses; and (2) the researcher informed

the participants about the purpose of the survey.

The security of the raw data gathered through the records sections

of TEA and the selected schools, responses to the on-line questionnaire

and the transcripts of the interviews was assured in order to protect the

anonymity of the participants and to uphold the trustworthiness of the

study.

The above concerns regarding trustworthiness and confidentiality

of data or information were shared with the participants when the

researcher contacted them through e-mail, telephone, mail, or in person.

Instrumentation

Quantitative data were accessed and retrieved from the TEA

website regarding the major urban high schools in Texas. Data were

organized for computations utilizing the SPSS software package, Version

14.0.

The on-line, open-ended questionnaire provided one of the bases

for the qualitative data. The triangulation method included categorizing

the responses to the online, open-ended questionnaire into emergent

themes, interviewing the focus groups of teachers and assistant

73

principals and one-on-one interviews with the principals and district ESL

personnel.

According to Fraenkel and Wallen (2003) personal interview is

probably one of the most effective ways there is to enlist the cooperation

of respondents in a survey; rapport can be established, questions can be

clarified, unclear or incomplete answers can be followed up and so on.

Patton (1990) expounds that the purpose of interviews is to gain

access to those areas of the participants’ experiences or thought which

cannot be observed. Consequently, interviews will play a significant role

in data collection, a role which generally cannot be duplicated by other

means (Dexter, 1970).

Fraenkel and Wallen (2003) cite the following advantages of open-

ended questions in survey research (a) allows more freedom of response;

(b) easier to construct; and (c) permits follow-up by interviewer. But there

are also disadvantages: (a) responses tend to be inconsistent in length

and content across respondents; (b) both questions and responses may

be subject to misinterpretation; and (c) responses are harder to tabulate

and synthesize. However, these disadvantages can be minimized through

the use of the NVivo software package, expert help from the focus groups

in classifying categories, follow-up interviews with the focus groups and

one-on-one interviews.

74

Validity and Reliability

For validity and reliability, the following expert opinions were

considered. “Validity, I mean truth: interpreted on the extent to which an

account accurately represents the social phenomena to which it refers”

(Hammersley, 1990, p. 57). “Reliability refers to the degree of consistency

with which instances are assigned to the same category by different

observers or by the same observer on different occasions” (Hammersley,

1990, p. 67). The triangulation method involving the analysis of the

qualitative data, collation of data from the on-line questionnaire and

interviews assured the validity and reliability of the survey questions and

the explanatory design of the mixed methods study.

For the quantitative dimension of the study, validity and reliability

measurements were derived from the TAKS report prepared by TEA.

Validity is a process of collecting evidence to support inferences made

from scoring results of an assessment. In the case of TAKS, test results

are used to make inferences about the students’ knowledge and

understanding of the Texas Essential Knowledge and Skills (TEKS). Test

reliability indicates the consistency of measurement. TAKS test

reliabilities are based on internal consistency measures, in particular on

the Kuder-Richardson Formula 20 (KR-20) for tests involving

dichotomously scored (multiple choice) items and on the stratified

coefficient alpha for tests involving a mixture of dichotomous and

polytomous (essay-prompt and short answer) items.

75

In order to build trustworthiness in the qualitative aspect of the

study, four different criteria were considered to meet this need:

(1)credibility, which aims to produce findings that are believable and

convincing; (2) transferability, which attempts to apply findings in one

setting to other contextually similar settings; (3) dependability, which

addresses the question concerning which findings are consistent with

those of other similar investigations; and (4) confirmability, which

ensures that both the process and the product are auditable (Isaac &

Michael, 1995).

Research Procedures

Quantitative

After appropriate permissions for data gathering were obtained,

records personnel of TEA were contacted and arrangements made as to

process and assistance regarding acquisition of data for the study. The

dry-run or pilot study with the two HISD schools facilitated the above

process.

Qualitative

The questions in a survey, and the way they are asked, are of

crucial importance (Fraenkel & Wallen, 2003). The authors refer to Floyd

Fowler who points out that there are four practical standards that all

survey questions should meet:

1. Is this a question that can be asked exactly the way it is

written?

76

2. Is this a question that will mean the same thing to everyone?

3. Is this a question that people can answer?

4. Is this a question that people will be willing to answer, given the

data collection procedures? (Fowler, 1984).

After the questionnaire was refined based on the suggestions of the

focus groups during the pilot study, the questionnaire was placed on-line

to respondents of the study. Prior to this, the researcher contacted the

respondents in person, by phone, by email or mail. Furthermore, the

researcher arranged dates with the different schools and districts for the

focus group and one-on-one interviews.

Data Collection and Recording

Quantitative

The data for the major urban high schools regarding the

percentage of ELLs enrolled in a school and the percentage of all Grade

10 students passing the TAKS tests in English Language Arts and

Mathematics during the four years when TAKS was administered,

starting school year 2002 – 2003 were obtained from the TEA website.

Qualitative

An on-line, open-ended questionnaire was answered by the

principals, assistant principals, ESL teachers, and non-certified teachers

handling ELLs of the selected major urban high schools in Texas. District

ESL personnel were also requested to answer the same questionnaire.

The focus groups offered expert opinions regarding the categories to use

77

in classifying the responses to the questionnaire. Further clarification

was requested from the principals and the district ESL personnel during

the one-on-one interviews.

Results of the questionnaire were placed in categories suggested by

the focus groups after initial classification was done through the NVivo

software system. Transcripts of the interviews and focus groups were

entered into the NVivo software system (version 7.0) and coded according

to themes that emerged in the data. NVivo provides a sophisticated way

of electronically organizing interview transcripts for analysis and

classification into themes and allowed the researcher to work with a large

amount of transcript data. The themes that emerged from the data were

compiled and compared between high schools. While NVivo was a

valuable sorting tool that allowed the researcher to code, sort, and recall

data in different ways, the researcher developed and created codes for

the responses gathered. The analysis was done by the researcher using

NVivo’s capabilities to sort out the complexities of the rich data from the

interviews and focus groups. A program such as NVivo can help the

researcher ensure that the qualitative data were well-organized

(Weitzman, 2000).

One of the strengths of collecting qualitative data is the richness of

the information that can be collected and which can capture a theme in a

more complete way than the researcher may be able to summarize. This

evidence directly from the data was used to show a clear connection

78

between the data and the identified themes (Marshall, 1990). The rich

description of the themes from the participants’ own words also aids in

verifying that the themes identified are those that the participants

actually voiced (Creswell, 1998; Krueger & Casey, 2000).

The researcher triangulated quantitative data analysis, qualitative

data analysis, and interviews in order to strengthen the credibility of the

survey study. By combining multiple observers, theories, methods, and

data sources, researchers can hope to overcome the intrinsic bias that

comes from single-method, single-observer and single-theory studies

(Denzin, 1970). With the mix of analyses, the author has better tools to

discuss the impact of statewide testing on ELLs, ESL curriculum and

instruction.

Data Analysis

Quantitative

Descriptive statistics and analyses were performed to test each

variable. After the data were examined and properly inputted, the next

step was to compute for Pearson r correlation coefficients using the SPSS

statistical package and test for statistical relationship at p < 0.05. For

other analyses, the predictor variable is the percentage of ELLs enrolled

in a school and the outcome variables were the percentages of all

students passing the Grade 10 TAKS tests in English Language Arts and

Mathematics. For each of the years under study, two separate Pearson r

correlations were computed; the percentage of ELLs enrolled in a school

79

was compared with the English Language Arts results to determine if

they have significant relationship and the other comparison was with the

Mathematics results. The SPSS computations showed three different

results in tabular form: (1) the means and the standard deviations of the

percentage of ELLs enrolled in a school and the percentages of all

students passing the TAKS tests in English Language Arts and

Mathematics during the four years under study; (2) Pearson r correlation

coefficients to determine if there was significant relationship between the

percentage of ELLs enrolled in a school and the percentages of all

students passing the English Language Arts and Mathematics TAKS

tests given in 2003, 2004, 2005 and 2006; and, (3) regression analysis

using the percentage of ELLs enrolled in a school as the predictor

variable to predict the percentage of students passing in the 10th grade

TAKS tests in the core areas of English Language Arts and Mathematics.

Qualitative

The information for the qualitative portion of the study included

the emergent themes shown as categories in the frequency distribution

table. The frequency distribution is a table in which all score units are

listed in one column and the number of individuals receiving each score

appears as frequencies in the second column (Isaac and Michael, 1995).

Frequencies were tallied and percentages were computed. Categories

with higher percentages were listed first followed by those with lower

percentages. An overview preceded each table giving the emergent

80

themes mostly cited by the respondents. Anecdotal records followed the

tables - these are the views and opinions of the respondents regarding

the different concerns included in the study.

Summary

In this study the researcher considered the aspects of procedural

consistency, neutrality of findings, and truth value to assure the study of

trustworthiness. “Valid inquiry in any sphere… must demonstrate its

truth value, provide the basis for applying it, and allow for external

judgments to be made about the consistency of its procedures and the

neutrality of its findings or decisions” (Erlandson, 1993).

Quantitative data that were sourced as aggregate data from the

TEA website included the percentage of English language learners


grade TAKS tests in the core areas of English Language Arts and

Mathematics given in 2003, 2004, 2005 and 2006.

Qualitative data were collated from the responses of selected

respondents to the on-line questionnaire regarding the anticipated and

observed consequences of the statewide testing, specifically TAKS, on

ESL curriculum and instruction as viewed by ESL teachers, school

administrators and district ESL personnel. Interviews were conducted

with the focus groups and one-on-one interviews involved the principals

and district ESL personnel.

81

Presentation of data included: (a) the quantitative data analysis on

the correlation between the percentage of ELLs enrolled in a school and

the percentage of all students passing the 10th grade TAKS in English

Language Arts and Mathematics and the regression analysis using the

percentage of ELLs enrolled in a school as predictor variable and the

percentage of all students passing the 10th grade TAKS in ELA and

Mathematics as outcome variables; (b) qualitative data analysis

classifying responses to the on-line, open-ended questionnaire as

different emergent themes; and (c) anecdotal records from the interviews

with the different focus groups, principals, and district ESL personnel.

The relationship between the percentage of ELLs enrolled in a

school and the percentage of all students passing the 10th grade TAKS in

each of the core areas of English Language Arts and Mathematics was

determined using the SPSS program for Pearson r correlation. The

regression analysis resulted to linear regression equations predicted the

percentage of all students passing in 10th grade TAKS in English

Language Arts and Mathematics using the percentage of ELLs enrolled in

a school as predictor variable.

Emergent themes were categorized through the NVivo software

package and suggestions of the focus groups. The anecdotal records

expressed the views and opinions of the respondents regarding the

following and focused on the ELLs: (a) the purpose of the statewide, high

stakes TAKS; (b) intended consequences of TAKS; (c) problems related to

82

TAKS; (d) changes caused by TAKS; (e) recommendations to improve

performance in TAKS; and, (f) the needs of ELLs.

83

CHAPTER IV

ANALYSIS OF DATA


stakes testing on English Language Learners (ELLs). This was shown in

both the quantitative and qualitative dimensions of the study. Both

quantitative and qualitative dimensions of the study provided the status

of high-stakes testing as it affected ELLs and how it influenced efforts in

schools to improve performance of students, particularly ELLs. Data

obtained from Texas Education Agency (TEA) were used to determine

whether there was a relationship between the percentage of ELLs


grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core

areas of English Language Arts and Mathematics given in 2003, 2004,

2005, and 2006. To support the qualitative aspect, this study explored

what certified English as a Second Language( ESL) teachers, non-

certified ESL teachers who teach ELLs, administrators, and district ESL

personnel viewed as the impact that high stakes standardized

assessments had on ELLs, ESL curriculum and instruction, and what

they observed as actually occurring.

With the No Child Left Behind (NCLB) Act of 2001, student

achievement has been placed in the forefront of quality education.

Aligned to this effort is statewide testing aimed at assessing students‟

performance and status of the school. In this chapter, the relationships

84

between the percentage of English language learners enrolled in a school

and the percentage of all students passing the 10th grade TAKS tests in

the core areas of English Language Arts and Mathematics given in 2003,

2004, 2005, and 2006 are shown. The results of the Statistical Package

for the Social Sciences (SPSS) computations show the: (1) means and

the standard deviations of the percentage of ELLs enrolled in a school

and the percentages of all students passing the 10th grade TAKS tests in

English Language Arts and Mathematics during the four years under

study; (2) Pearson r correlation coefficients between the percentage of

ELLs enrolled in a school and the percentages of all 10th grade students

passing the English Language Arts and Mathematics TAKS tests given in

2003, 2004, 2005, and 2006; and, (3) regression analysis using the

percentage of ELLs as the predictor variable.

Following this quantitative dimension of the study is the qualitative

information regarding TAKS and its effects on the ELLs - its intended

purpose and consequences, problems encountered related to this

statewide testing, changes that occurred due to TAKS and

recommendations to improve the performance of ELLs in general and

specifically on this high-stakes test. The tabulated results of the open-

ended, on-line questionnaire and the supporting explanation gathered

through the individual and focus group interviews comprised the

qualitative portion of this study. ESL district personnel, principals,

85

assistant principals, ESL-certified teachers and non-ESL certified

teachers handling ELLs provided the needed information.

Schools with ELLs are asked to account for their performance in

line with NCLB. School and district administrators may get feedback

both from the numerical results of high-stakes testing and from inputs of

teachers and other personnel responsible for handling ELLs. Since the

presence of ELLs in schools is a reality administrators have to face, it

may be beneficial to be aware of the views and opinions of concerned

personnel in the school system regarding interventions that may improve

the curriculum and/or instruction of ELLs.

Findings

For the quantitative portion of the study, data for the 173 urban

high schools were obtained from Texas Education Agency; however, for

the year 2005, data available were only for 155 high schools. The data

gathered were collated and the SPSS software package was utilized for

the needed computations. The following research question was the focus

of the quantitative dimension of the study:

Research Question - Quantitative





86

Two null hypotheses were formulated to answer the above

question:




English Language Arts given in 2003, 2004, 2005, and 2006.




Mathematics given in 2003, 2004, 2005, and 2006.

The results are presented in the following order: (a) the descriptive

statistics (mean, standard deviation, number of cases); (b) the Pearson r

correlation coefficients; and, (c) regression analysis showing the linear

equations which can be used to predict the outcomes in the 10th grade

TAKS tests in English Language Arts and Mathematics.

Tables 4.1.1 to 4.1.4 show that the average percentage of ELLs in

the urban high schools of Texas during the four years under study

starting from 2003, ranged from 6.94% to 8.30 %; the average percentage

of ELLs enrolled in a school was 7.79%.

87

Table 4.1.1

Means and Standard Deviations of ELLs Enrolled in School and All

Students Passing the 2003 10th Grade TAKS for English Language Arts

and Mathematics.

Mean Std. Deviation N

ELLs Enrolled

In School

08.31 08.98 165

All Students

Passing ELA

62.87

18.11

151

All Students

Passing

Mathematics

61.85

20.84

151

Table 4.1.2



and Mathematics.


ELLs Enrolled

In School

08.30 09.64 147

All Students

Passing ELA

68.28

16.24

130

All Students

Passing

Mathematics

53.57

22.44

132

88

Table 4.1.3



and Mathematics.


ELLs Enrolled

In School

7.62 10.80 147

All Students

Passing ELA

59.39

20.13

135

All Students

Passing

Mathematics

47.68

23.35

134

Table 4.1.4



and Mathematics.


ELLs Enrolled

In School

6.94 9.94 167

All Students

Passing ELA

78.05

13.68

155

All Students

Passing

Mathematics

50.13

22.40

152

89

Tables 4.1.1 through 4.1.4 reflect the descriptive statistics showing

the percent of all students passing in English Language Arts were:

62.87% in 2003, 68.28% in 2004, 59.39% in 2005 and 78.05% in 2006.

The descriptive statistics showing the results of all students passing in

Mathematics were: 61.85% in 2003, 53.57% in 2004, 47.68% in 2005

and 50.13% in 2006.

Based on the above tables, Tables 4.1.1.1 and 4.1.1.2 compare the

results of 10th grade TAKS in English Language Arts and Mathematics.

Table 4.1.1.1

Comparison of Results in 10th Grade English Language Arts TAKS

Year

Percent

ELLs

Enrolled

in

School

Means-All

Students

Passing

Current

Year

Means-All

Students

Passing

Previous

Year

Difference

(+/-)

Percent

Increase

(Decrease)

2003 8.30 62.87 -- -- --

2004 8.29 68.28 62.87 +5.41 08.61

2005 7.62 59.39 68.28 -8.89 -13.02

2006 6.94 78.05 59.39 +18.66 031.42

90

Table 4.1.1.2

Comparison of Results in 10th Grade Mathematics TAKS

Year

Percent

ELLs

Enrolled

in

School

Means-All

Students

Passing

Current

Year

Means- All

Students

Passing

Previous

Year

Difference

(+/-)

Percent

Increase

(Decrease)

2003 8.30 61.85 -- -- --

2004 8.29 53.57 61.85 -8.28 -13.39

2005 7.62 47.68 53.57 -5.89 -10.99

2006 6.94 50.13 47.68 +2.45 05.14

Tables 4.2.1 to 4.2.4 show the obtained Pearson r correlation

coefficients for the correlations between percent of ELLs enrolled in a

school and the percent of all students passing in 10th grade TAKS test in

English Language Arts and Mathematics. The obtained Pearson r

correlation coefficients between the percentage of ELLs enrolled in a

school and the percentage of all students passing in 10th grade TAKS

were: -0.349 in 2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006

for English Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in

2005 and -0.356 in 2006 for Mathematics. All the computed Pearson r

correlation coefficients were significant at the 0.05 level, two-tailed. We

therefore reject the null hypotheses. All the obtained Pearson r

correlation coefficients were negative for both content areas; this inverse

relationship indicated that as the percentage of ELLs increased, the

91

percentage of all 10th grade students passing in each of the core areas of

English Language Arts and Mathematics decreased.

Table 4.2.1

Pearson Correlations: 2003 10th Grade TAKS for English Language Arts

and Mathematics.

Pearson Correlation

Percent of All

Students

Passing ELA

Percent of All

Students

Passing Math

Percent of ELLs Enrolled -.349(**) -.293(**)

Sig. (2-tailed) .00000 .00000

** Correlation is significant at the 0.05 level (2-tailed).

Table 4.2.2


and Mathematics.

Pearson Correlation

Percent of All

Students

Passing ELA

Percent of All

Students

Passing Math


Sig. (2-tailed) .00000 .00000


92

Table 4.2.3


and Mathematics.

Pearson Correlation

Percent of All

Students

Passing ELA

Percent of All

Students

Passing Math


Sig. (2-tailed) .00000 .00000


Table 4.2.4


and Mathematics.

Pearson Correlation

Percent of All

Students

Passing ELA

Percent of All

Students

Passing Math


Sig. (2-tailed) .00000 .00000


A regression analysis for the data in 2003 is shown in Tables 4.2.5

and 4.2.6. The correlation coefficient between the percentage of ELLs

enrolled in a school and the percentage of all students passing in 10th

grade TAKS in English Language Arts for 2003 is -0.349. The Pearson r


of all students passing in 10th grade TAKS in Mathematics is -0.293.

Both correlation coefficients are significant at p < 0.05, two-tailed.

93

The percentage of all students passing in 10th grade English

Language Arts TAKS can be predicted using the linear regression

equation: Ŷ = 68.71 – 0.70X, where X is the percentage of ELLs enrolled

in a school. For example, using the percentage of ELLs data for 2003

(X = 8.30), the predicted value for percent of all students passing 10th

grade English Language Arts is equal to 68.71 – 0.70(8.30) or 62.90.

Actual result was 62.87. The percentage of all students passing in 10th

grade Mathematics TAKS can be predicted using the linear regression


in a school. A similar example for Mathematics gives a predicted value of

67.49 – 0.68(8.30) or 61.85. Actual result was 61.85.

Table 4.2.5

Coefficients for Percentage of All Students Passing the 2003 10th Grade

English Language Arts TAKS.

Model Unstandardized

Coefficients

Standardized

Coefficients

T Sig.

B Std.

Error

Beta

1 (Constant) 68.71 1.89 36.37 .000

Percent ELLs 0 -.70 0.16 -.349 0-4.55 .000

a. Predictor: Percentage of ELLs Enrolled in a School

b. Outcome variable: Percentage of All Students Passing in 10th Grade

English Language Arts TAKS

94

Table 4.2.6


Mathematics TAKS.


Coefficients

Standardized

Coefficients

T Sig.

B Std.

Error

Beta

1 (Constant) 67.49 2.22 30.43 .000

Percent ELLs 0 -.68 0.18 -.293 -3.74 .000



Mathematics TAKS

Results of the regression analysis for the data in 2004 are shown

in Tables 4.2.7 and 4.2.8. The correlation coefficient between the

percentage of ELLs enrolled in a school and the percentage of all

students passing in 10th grade TAKS in English Language Arts for 2004

is -0.392. The Pearson r between the percentage of ELLs enrolled in a

school and the percentage of all students passing in 10th grade TAKS in

Mathematics is -0.351. Both correlation coefficients are significant at

p < 0.05, two-tailed.






95

grade English Language Arts is equal to 73.76 – 0.66(8.29) or 68.29.






Table 4.2.7




Coefficients

Standardized

Coefficients

T Sig.

B Std.

Error

Beta

1 (Constant) 73.76 1.74 42.46 .000

Percent ELLs 0 -.66 0.14 -.392 -4.83 .000



English Language Arts

96

Table 4.2.8


Mathematics TAKS.


Coefficients

Standardized

Coefficients

T Sig.

B Std.

Error

Beta

1 (Constant) 60.34 2.42 24.90 .000

Percent ELLs 0-.82 0.19 -.351 0-4.28 .000



Mathematics TAKS

A regression analysis for the data in 2005 is shown in Tables 4.2.9

and 4.2.10. The correlation coefficient between the percentage of ELLs

enrolled in a school and the percentage of all students passing in 10th

grade TAKS in English Language Arts for 2005 is -0.297. The Pearson r


of all students passing in 10th grade TAKS in Mathematics is -0.382.

Both correlation coefficients are significant at p < 0.05, two-tailed.





(X = 7.62), the predicted value for percent of all students passing

97

10th grade English Language Arts is equal to 63.61 – 0.55(7.62) or 59.42.






Table 4.2.9




Coefficients

Standardized

Coefficients

t Sig.

B Std.

Error

Beta

1 (Constant) 63.61 2.03 31.28 .000

Percent ELLs 0-.55 0.15 -.297 0-3.59 .000




98

Table 4.2.10


Mathematics TAKS.


Coefficients

Standardized

Coefficients

t Sig.

B Std.

Error

Beta

1 (Constant) 53.97 2.29 23.54 .000

Percent ELLs 0-.82 0.17 -.382 -4.74 .000



Mathematics TAKS

To show that the percentage of ELLs enrolled in a school affected


English Language Arts and Mathematics, the regression analysis results

are shown in Table 4.2.11 and Table 4.2.12. The correlation coefficient


of all students passing in 10th grade TAKS in ELA for 2006 is -0.398. The

Pearson r between the percentage of ELLs enrolled in a school and the

percentage of all students passing in 10th grade TAKS in Mathematics is

-0.356. Both correlation coefficients are significant at p < 0.05, two-

tailed.




99



grade ELA is equal to 81.85 – 0.55(6.94) or 78.03. Actual result was

78.05. The percentage of all students passing in 10th grade Mathematics

TAKS can be predicted using the linear regression equation:

Ŷ = 55.70 – 0.80X, where X is the percentage of ELLs enrolled in a

school. A similar example for Mathematics gives a predicted value of


Table 4.2.11




Coefficients

Standardized

Coefficients

t Sig.

B Std.

Error

Beta

1 (Constant) 81.85 1.24 66.28 .000

Percent ELLs 0-.55 0.10 -.398 -5.37 .000




100

Table 4.2.12


Mathematics TAKS.


Coefficients

Standardized

Coefficients

t Sig.

B Std.

Error

Beta

1 (Constant) 55.70 2.08 26.78 .000

Percent ELLs 0-.80 0.17 -.356 -4.67 .000



Mathematics TAKS

Research Question – Qualitative

After the refinements were done during the pilot study, an open-

ended, on-line questionnaire was given to the principals, assistant

principals, ESL district personnel, ESL certified teachers and non-ESL

certified teachers who were purposively sampled from urban high schools

in Texas.

The major question answered by this study was: What are the

anticipated and observed consequences of the statewide testing

specifically, TAKS, on ESL curriculum and instruction as viewed by


school administrators, and district ESL personnel?

101



2. What are the intended consequences of this statewide testing? (Or

what has happened because of TAKS?)






Before the above questions for the qualitative study are answered,

the following tables give the demographic information regarding the

respondents.

Demographic Characteristics of the Respondents

Demographic information regarding the respondents included

gender, age, current position, highest degree earned, years of experience

in public education and certification(s) achieved.

The open-ended, on-line questionnaire for the qualitative aspect of

the study was intended for 98 respondents. Six principals responded,

together with 9 assistant principals, 6 ESL district personnel, 15 ESL

certified teachers and 19 non-ESL certified teachers – a total of 55

respondents, for a response rate of 56%.

Table 4.3 shows that 69% of the respondents were female and 31%

were male.

102

Table 4.3

Distribution of Respondents by Gender.

Gender Frequency Percent (%)

Female 38 69

Male 17 31

N=55

The age of the respondents is shown in Table 4.4. With respect to

age, 31% of the respondents were between 41 – 50 years old and 51 – 60

years old. The group of 31 – 40 years old comprised 22% of the

respondents; 9% of the respondents were older than 60 years old and the

youngest between 21 – 30 years old was 7% of the group.

Table 4.4

Distribution of Respondents by Age.

Age Frequency Percent (%)

21-30 04 07

31-40 12 22

41-50 17 31

51-60 17 31

> 60 05 09

N=55

It is shown in Table 4.5 that the non-ESL certified teachers

comprised 35% of the respondents and 27% were ESL certified teachers.

The administrators accounted for the remaining 38%: 16% were

103

assistant principals, 11% were principals, and 11% were ESL district

personnel.

Table 4.5

Distribution of Respondents by Professional Position.

Current Position Frequency Percent (%)

Principal 06 11

Assistant Principal 09 16

ESL District Personnel 06 11

ESL Certified Teacher 15 27

Non-ESL Certified Teacher 19 35

N=55

Table 4.6 displays that 53% of the respondents have Master‟s

degrees, 40% have Bachelor‟s degrees and 7% have Doctorate degrees.

Table 4.6

Distribution of Respondents by Highest Degree Earned.

Highest Degree Frequency Percent (%)

Bachelor‟s Degree 22 40

Master‟s Degree 29 53

Doctorate Degree 04 07

N=55

Considering years of experience in public education, it is shown in

Table 4.7 that 40% of those who participated in the study had more than

20 years of public school experience. Thirty-one percent had 11 – 20

104

years of public school experience, 16% had 5 years or less, and 13% had

been in the public schools for 6 – 10 years.

Table 4.7

Distribution of Respondents by Years of Experience in Education.

Years of Experience Frequency Percent (%)

1 – 5 09 16

6-10 07 13

11-20 17 31

> 20 22 40

N=55

Table 4.8 displays the certifications of the ESL district personnel,

principals, assistant principals, ESL certified and non-ESL certified

teachers who handle ELLs that participated in this study. Some of these

administrators and teachers had more than one certification. Based on

total number of respondents, 47% had certification in English Language

Arts, 20% in Mid-Management, 18% in Mathematics, 16% in

Principalship, and 13% in Special Education.

105

Table 4.8

Distribution of Respondents by Certification.

Certification Frequency

English Language Arts 26

Mid-Management 11

Mathematics 10

Principal 09

Special Education 07

Bilingual 02

Educational Diagnostician 02

Biology 02

Sociology 02

Supervisor 02

Reading 02

Librarian 02

Superintendent 02

History 02

Technology 01

Marketing

Counseling

1

1

Vocational Home Economics 01

Social Studies Composite 01

N=55

106

The findings regarding the qualitative portion of the study were

presented as follows: (1) the nine open-ended responses to the on-line

questionnaire; (2) frequency tables showing the emergent themes,

frequencies and percentages; and, (3) anecdotal views and opinions of

district ESL personnel, principals, assistant principals, ESL-certified

teachers and non-certified ESL teachers handling ELLs regarding the

issues or concerns.

Answers of some respondents belonged to more than one

emergent theme; the total number of answers may have exceeded the

total number of respondents. The percentage shown after the total

responses given for each emergent theme was computed based on total

number of respondents.

Why is Texas Assessment of Knowledge and Skills (TAKS) given as a

Statewide Test?

Why is TAKS given as a statewide test? Responses are shown in

Table 4.9. Of the 55 respondents, 40% viewed TAKS as a tool to gauge

knowledge in the core academic areas. Twenty-nine percent (29%) of the

respondents considered TAKS as a means to determine the school‟s

effectiveness and performance (Exemplary, Recognized, Academically

Acceptable, or Academically Unacceptable), while 16% regarded TAKS as

an instrument to appraise the effectiveness of the state curriculum. Only

7% indicated that this statewide test is mandated by law and is aligned

with NCLB.

107

Table 4.9

Why is TAKS Given as a Statewide Test to ELLs?

Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach (15) NonTeach (19) Tot/%(55)

Gauge Skills/TEKS 3 3 3 6 7 22/40

School Accountability 0 2 6 3 5 0 16/29

Assess State Curri-Std 1 1 0 3 4 00 9/16

Mandated by NCLB 1 0 1 0 2 04/7

Com Schools/Districts 1 0 1 1 0 0003/5

Generate Revenue 1 0 1 0 1 0003/5

Test College Prep 0 0 1 1 0 00 2/4

Political Appeasement 0 0 0 1 0 00 1/2

No Response 0 0 0 0 1 00 1/2

108

One ESL district personnel (EDP-03) gave this opinion about TAKS:

“It is administered to measure the knowledge and skills of Texas

students,” and “it provides a comparative data table for critics.” An

assistant principal (AP-06) supported this view: “TAKS is to help gauge

students‟ basic skills” and “to make schools accountable for students‟

learning.”

Two high school principals (P-06 and P-04) referred to the TAKS as

an instrument of accountability: “…currently, TAKS is the State

Accountability System test in order to comply with NCLB.”

A more comprehensive response regarding the purpose of TAKS

was given by non-ESL certified teacher (non-ESL-14) who handles ELLs:

“At the state level, the TAKS is the state education agency‟s tool for

determining the accreditation status of schools and school

districts…With the enactment of the No Child Left Behind Act, the

TAKS is now being used also as a tool, among other indicators, for

determining the attainment or non-attainment of an adequate

yearly progress (AYP). It is expected that a school carrying

standards-based curriculum and intervention, especially one that

is predominantly attended by minority populations, would show

improving performance until the targeted year 2013 - 2014 when

all children have met standards, which is the essence of the phrase

„No Child Left Behind.‟ The TAKS performance is disaggregated into

subgroups, so that again, there is a way of determining whether

109

achievement gaps are being bridged. Much of the accountability for

student improvement on knowledge and skills in the four academic

areas falls on schools. The other indicators of adequate yearly

progress are completion rate and attendance.”

In addition to being an instrument of accountability, respondents

suggested that assessment results provide data for comparative analysis.

According to one assistant principal (AP-09), TAKS is given “to

compare/contrast statewide similarities and differences” in achievement

levels “between schools” and across districts.

During one of the focus group sessions, teachers gave these

comments regarding TAKS:

“The test administration is the same for all students. We have to

follow the guide from the state. But, getting the kids ready for the

test is different. We don‟t do anything differently, but we should.

It‟s going to take more than what we are doing right now to ensure

these kids (ELLs) graduate with the same diploma, meeting the

same standards, as the general population.”

Knowing what TAKS is for, does not necessarily translate into

unanimity in terms of agreeing on what schools must do to respond to

the challenge that the test brings. Respondents‟ opinions however,

indicate the sense that schools need to adapt, and do things differently if

they expect to achieve some level of parity between the TAKS

achievement scores of ELLs and their non-ELL counterparts.

110

Anticipated Results of ELLs on English Language Arts and Mathematics

TAKS

Table 4.10 shows the responses to the following question: What

are the anticipated results on the statewide testing, specifically English

Language Arts and Mathematics portion of the Texas Assessment of

Knowledge and Skills (TAKS), on English language learners (ELLs)?

Almost half (49%) of those who responded expected ELLs to have at least

average scores in TAKS. However, 16% observed ELLs‟ performance to be

lower than non-ELLs, and 15% projected low results for ELLs because of

their lower English language skills.

111

Table 4.10

What are the Anticipated Results of Statewide Testing for ELLs?

Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)

Average Scores 1 2 3 3 7 16/29

Everyone Pass/Meet AYP 2 1 3 3 2 0 11/20

Lower than Non-ELLs 1 2 0 3 3 009/16

Low: Language Barrier 1 1 1 3 2 08/15

Curriculum Improvement 2 0 0 0 2 0004/7

Dismal 0 0 1 0 1 0002/4

Cause Dropouts 1 0 0 1 0 00 2/4

Gap in Scores to Lessen 1 0 1 0 0 00 2/4

Team Effort Improvement 0 0 0 0 2 00 2/4

Political 0 0 0 1 0 00 1/2

Modified Diploma 0 0 0 1 0 00 1/2

Not Sure 0 0 0 1 0 00 1/2

112

One high school principal (P-04) anticipated that the: “results of

statewide testing are that all students will be expected to meet minimum

state standards.” Another high school principal (P-02) expressed with

some sense of exasperation that:

“English language learners tend to score extremely low on the ELA

and Mathematics portions of the TAKS. The state and the federal

governments expect us to work miracles with these groups of

students. If a student is in the U.S. for a year or two, he/she needs

extra remediation to get a significant gain in TAKS scores. He may

not necessarily pass the TAKS, but he will make gains. Keep in

mind the NCLB ruling. Making any gains is not enough. The

students must perform at 70% or better on all four sections of

TAKS. At this point, all administrators in poverty-stricken

areas are in fear of losing their jobs due to the low TAKS scores.

We are indeed under extreme pressure!”

One principal (P-05) indicated that the TAKS “will have a negative

impact on the English language learners because of their difficulty in

reading and writing in English.” The same negative outlook was

expressed by an assistant principal (AP-03) who bluntly predicted that:

“ELL students will fare worse than the general population.”

A non-ESL teacher handling ELLs (Non-ESL-14) proposed a

measure to address the feared, and somewhat expected under-

achievement of ELL Learners:

113

“Title I programs are instituted to address the intervention needs

of ELLs so that they perform well in statewide testing. The said

federal assistance is aimed at incrementally improving the

performance of disadvantaged groups, which include the ELLs and

the ethnic population they comprise. In my particular school,

intervention translates into identifying those who are at risk in this

group, breaking down their previous performances into specific

areas and conducting the needed mentoring.”

During a focus group interview, one member of the group

expressed this view regarding the anticipated results TAKS :

“I think that if we are going to require ELLs to be as successful on

TAKS as any native English speaker then those students need

specific language classes in school. ELLs need English classes

whose sole purpose is teaching them to understand and speak

English, with basic grammar skills included. In addition to a

language class, they do still need a literature-based class in which

they are learning to read and analyze text, as well as write both

creatively and analytically. Also, having tutoring available, either

built into the school day, or after school, during which ELLs

practice speaking and listening in English would do a world of

good. Teachers would not need to be the only tutors either, we

could tap into a college‟s community service group, or even take

advantage of peer tutoring.”

114

Actual Results of ELLs in English Language Arts and Mathematics TAKS

Table 4.11 summarizes the responses for the question: What are

the actual results of TAKS, specifically English Language Arts and

Mathematics, on ELLs? Responses were related to how the ELLs in the

schools actually performed on TAKS. Results were more negative than

positive.

Twenty-two percent (22%) observed dismal or failing performance, 18%

noticed performance of the ELLs to be lower than non-ELLs and 11%

mentioned that low results frustrate ELLs.

115

Table 4.11

What are the Actual Results of Statewide Testing for ELLs?


Dismal/Failing 0 1 5 1 5 12/22

Lower than Non-ELLs 3 1 1 3 2 0 10/18

Average Results 0 2 2 2 3 009/16

No Response 0 0 0 4 4 08/15

Frustrated ELLs 3 0 0 2 1 0006/11

Better than Average 0 2 1 0 0 000 3/5

Don‟t Know 0 0 0 0 2 00 02/4

Gap Narrowed 0 0 0 0 2 00 02/4

Showed Areas to Improve 0 0 0 1 0 00 01/2

Political 0 0 0 1 0 00 01/2

Teacher Teach TAKS 0 0 0 1 0 00 01/2

116

Actual results in TAKS by ELLs are shared by an assistant

principal (AP-03): “ELL students do fare worse than the general

population.” The same observation was given by a non-ESL certified

teacher (Non- ESL- 03):

“Actual results show that ELLs perform poorer than native

speakers and naturally so for they are very much handicapped as

far as their understanding of the intricacies of the language is

concerned. Their scores result in schools being „below standard‟, so

to speak.”

A non-ESL certified teacher (Non-ESL-14) offered a contrasting

view by sharing the actual experience in their district:

“I do not have the exact statistical report as far as the

performances of ELLs are concerned, but my school district came

up with a report lauding the improvement across grade levels in

Mathematics and English Language Arts among ELLs, except in 9th

Grade Mathematics. The district superintendent also mentioned

about the narrowing of the achievement gap, and cited the

percentage points of improvement.”

Intended Consequences of TAKS for ELLs

Table 4.12 summarizes the responses to the question: What are

the intended consequences of this statewide testing as it relates to ELLs?

With respect to the intended consequences of TAKS, 18% of the

respondents expressed the understanding that statewide testing is

117

intended to eventually result in ELLs performing as well as the rest of the

students and 18% agreed that TAKS is a tool to assess ELLs. Eleven

percent (11%) of the respondents expressed confidence that ELLs can

improve academically and eventually join the mainstream, and 11%

demonstrated awareness that TAKS may improve the graduation rate.

Teachers (9% of the respondents) expressed their wish for an opportunity

to teach ELLs individually with the end in mind of improving their

performance in TAKS.

118

Table 4.12

What are the Intended Consequences of TAKS for ELLs?


Same Level of Expectations 1 2 0 4 3 10/18

Assess ELLs 1 1 1 4 3 0 10/18

ELLs Improve-Mainstream 2 0 1 1 2 0006/11

Improve Graduation Rate 0 1 1 2 2 006/11

ELLs Learn English ASAP 0 0 2 2 2 00 06/11

School Accountability 1 1 1 1 1 0000 5/9

Individual Teaching of ELLs 0 0 0 2 3 00 005/9

Equal Learn-Opportunities 0 1 1 0 1 0 003/5

Improve Teacher Practices 0 0 1 1 1 00 003/5

Not Sure 1 0 1 0 1 0 03/5

No Response 0 0 0 0 1 00 1/2

119

An assistant principal (AP- 09) stated that statewide testing is

intended to ensure equality of learning opportunities for minority groups.

Another assistant principal (AP-03) saw the tests as eventually resulting

in the elevation of standards and the improvement of ELL students‟

English language skills. A number of non-ESL certified teachers held

similar views:

“With this statewide testing, ELLs can show how much they have

gotten into the „mainstream‟ of the American educational system.

In so doing, the government will get an „assurance‟ that graduates

of this educational system (or future adults of this society) are

competent and can function well in… this system” (Non-ESL-03).

“The ultimate objective of this educational reform is for everyone...

particularly minority populations, to be educationally competent so

that they become more and more socio-economically integrated

into the American mainstream” (Non-ESL-14).

While the respondents demonstrated competent awareness of the

intended consequences of the TAKS, a few were quick to point out the

challenges that testing poses. A principal (P-02) stated:

“I agree with the No Child Left Behind. We care about each and

every student, but you can‟t compare a 5th generation student with

a student who has only experienced a month or two in the U.S.A.

and cannot speak the English language or may not have had

formal education in his/her home country. These students are an

120

extremely difficult challenge. This challenge is rarely seen in the

suburban schools.”

On the other hand, one non-ESL certified teacher (Non-ESL-09)

observed that the intended consequence of the TAKS:

“Is to let ELL students stay in high school for more than 4 years

because they can not graduate without passing the TAKS exit

level…it should serve as a challenge to ELL kids to strive more and

work harder because …we all agree that at some point,

Mathematics in itself is a language…so anybody who has this class

is learning a new language… ELLs should not use language as a

barrier to learning Mathematics.”

An English teacher of a focus group expressed this opinion

regarding how statewide assessment affects ELLs:

“Statewide assessments that are intended for non-ESL students

have detrimental effects on the ELL populations. Many are indeed

good students, but they get their results back and only see failure.

The educator also sympathizes because he/she sees firsthand how

far the student has come, but there is no state test for progress. It

becomes frustrating for all parties involved. Sometimes

administrators assume the teacher is not doing his/her job

because the scores come back so low.”

121

What Has Happened to TAKS, As It Relates to ELLs

What has happened because of TAKS, as it relates to ELLs?

Responses given to this question are shown in Table 4.13. Twenty-six

percent (26%) of the respondents reported that there is a high failure rate

among ELLs. Other respondents observed that ELLs experienced a

diminished self-esteem because of low scores (reported by 11% of the

respondents) and tests caused frustration and exasperation on the part

of ELLs (also seen by 11% of the group). There is also pressure on ELLs

and the schools they attend to improve performance as noted by 11% of

the respondents. Eleven percent (11%) of the respondents reported that

there is now an overwhelming emphasis on test performance in their

campuses.

122

Table 4.13

What Has Happened to ELLs Because of TAKS?


Higher Failure Rate-ELLs 1 1 5 3 4 14/26

Dropout Rate Increased 2 1 1 1 2 0 7/13

ELLs Have Low Self-Esteem 0 1 1 1 3 006/11

Pressure on School/ELLs 0 1 1 1 3 06/11

Emphasis Test Performance 1 0 0 3 2 00 6/11

Frustration/Exasperation 1 1 0 2 2 00 6/11

Focus on Needs of ELLs 1 1 0 1 1 00 4/7

Teachers Teach to TAKS 0 0 0 2 2 00 4/7

Continue Testing Changes 0 0 1 0 2 00 03/5

ELLs Repeat TAKS Often 0 0 0 2 0 00 02/4

Acceptable Test Scores 0 1 0 1 0 00222/4

Get Rid of TAKS 0 0 1 1 0 00 2/4

No Response 0 0 0 1 1 00 2/4

123

District personnel (EDP-03) noted that “many of the ELL students

have dropped out, fallen behind and in some instances stopped caring.

Those who successfully passed the TAKS are concerned with their family

and friends, so they share the failure. An assistant principal (AP-02)

stated that TAKS has resulted in an “increase in dropout rate, low self

esteem.”

An ESL teacher (ESL-01) has observed that “many ELLs fear TAKS

and do not think they will ever be able to pass it… many drop out of

school altogether.” Another ESL teacher (ESL-05) noticed something

similar: “Every year ELLs either pass and exit the program, or fail… they

experience a sense of failure.” “The test has caused frustration and

exasperation on the part of many students because the stakes are too

high. They are only judged on this one test” (ESL – 10). Similarly, a non-

ESL teacher (Non- ESL- 03) observed that:

“Because of TAKS (and its results), ELLs have become the “trail

enders” (the ones at the end of the trail). Not performing well in

these tests and being “left behind” by the “better” students has

given them lower self-esteem and feelings of frustration. Failing the

test one too many times may result in feelings of frustration and,

possibly, rejection on their part.”

An ESL teacher (ESL-07) observed that “ELLs have been

penalized, a large portion must re-take TAKS multiple times in order to

124

pass, and schools have been held accountable for the failure of that

population of students.”

A more optimistic response was expressed by a non-ESL teacher

(Non- ESL-14) regarding what happened to ELLs because of TAKS:

“School administrators and educators have become more aware of

the standards and expectations that students must reach for them

to become competent human resource in the workplace and

society. Not only has there been a gap among ethnic populations,

with the ELLs at the disadvantaged end of the spectrum; there has

also been a gap between competencies of exiting high school

students and the actual skills and scholastic aptitude required in

postsecondary education. Falling on the shoulder of the academic

workforce, the TAKS has pushed these school stakeholders to

intensify their efforts of addressing the needs of ELLs. Student

expectations and academic objectives are being laid out more

clearly, and their assessments more meticulously scrutinized with

students, as part of instruction, in order for them to become more

aware and self-reflective in their learning. There is a culture of

concern for statistical performance, broken down furthermore into

numbers vis-à-vis chunks of knowledge and skills in a specific

subject area. Even ELLs have become familiar with the language of

testing.”

125

Problems of ELLs, Due to TAKS

What problems have occurred, if any, to ELLs related to or because

of TAKS? Responses are summarized in Table 4.14. Of the 55

respondents, 24 % mentioned the higher dropout rate as one of the

problems experienced by ELLs due to TAKS. At least one from each group

expressed this problem. Graduation rate decreased for ELLs due to this

statewide test - this was echoed by 22% of the respondents. Another

negative aspect is perceived to be the low esteem of ELLs as seen by 13%

of the group; 15% of the respondents realized the need to support ELLs.

126

Table 4.14

What Problems Have Occurred for ELLs Due to TAKS?


Higher dropout rate 3 2 1 4 3 13/24

Low graduation rates 2 0 1 4 4 0 11/20

Support for ELLs 0 1 1 4 2 008/15

Low self-esteem 0 1 1 3 2 07/13

Instruction is test oriented 1 0 0 2 1 00004/7

Increased achievement gap 0 0 2 1 1 000 4/7

Ostracized due to low scores 0 0 1 0 2 00 03/5

Tutorials/staff development 0 1 0 1 1 00 03/5

True ability not measured 0 0 2 0 1 00 03/5

No problem 0 2 0 0 1 00 03/5

Limited reading ability 0 0 0 0 2 00 02/4

Rush ELLs to learn English 0 0 0 0 1 01001/2

No Answer 0 0 0 0 1 01001/2

127

Problems experienced by principals included the following: “Some

have dropped out; others have done well enough to finish high school but

fear college. Others have lost all hope and courage to continue” (P-03).

“The results of TAKS have caused the English language learners to feel

as if they are failures” (P-05).

A non-ESL teacher (Non-ESL-03) observed the same problem:

“ELLs … may just disengage themselves from any school activities and

studies, and may, ultimately, check out of school.” This was echoed by

an assistant principal (AP-07) who observed that ELLs “leave high school,

complete all credits but do not receive diploma due to failing statewide

exam.” Though challenged, a principal (P-02) remained optimistic:

“Some of our English language learners‟ hopes and dreams of

passing the state exams never become a realization, but we don‟t

give up on them. We keep them and continue to work with them

individually so that they may succeed.”

The ELLs‟ performance in TAKS was explained by a non-ESL

teacher (Non-ESL-14):

“ELLs enroll into their grade levels not quite equipped with the

prerequisite skills to perform” according to standards. “Even with

the administration of the TAKS in the native language of the ELLs,

there still exists an academic gap, because the testing language is

not essentially native but academic, an orientation with which

ELLs are not familiar. Only 4 years old, the TAKS as a federal tool

128

for adequate yearly progress is battling against a culture (focused

on) economic survival. The TAKS measures scholastic aptitude and

academic achievement - attributes that make up the profile for

college readiness, or postsecondary education.”

It was also observed that ELLs are more likely to need employment

to survive. ELLs predominantly come from the working class, and they

view attendance in school as a bureaucratic legal requirement rather

than as an opportunity to develop skills necessary for social mobility.

Changes Which Occurred as a Result of Statewide Testing, as They

Relate to ELLs

Table 4.15 displays the responses to the question: What changes

have occurred as a result of statewide testing as it relates to ELLs? Due

to the low performance of ELLs, 35% of those who responded to the

questionnaire saw the need to focus and intervene on behalf of ELLs.

Although 16% observed no changes because of TAKS, 15% realized that

there will be more pressure at home and school to help ELLs pass TAKS.

Thirteen percent (13%) of the respondents affirmed their commitment to

be accountable for the ELLs.

129

Table 4.15

What Changes Have Occurred for ELLs Due to TAKS?


Focus/Intervention for ELLs 2 3 2 4 8 19/35

None 0 0 3 3 3 0 9/16

More pressure: school/home 1 1 1 3 2 008/15

Accountability for ELLs 1 2 2 1 1 07/13

Teacher development 0 1 1 2 1 000 5/9

Uncertain 2 0 0 2 1 000 5/9

Emphasis: test performance 2 0 0 0 1 00 3/5

Positive pressure to perform 0 0 0 1 1 00 2/4

Instruction is test oriented 0 0 0 0 2 00 2/4

Fewer Exemptions 1 0 0 1 0 00 2/4

Need effective leadership 0 0 1 0 2 00 1/2

Not welcome: public schools 0 0 0 0 1 010 1/2

130

In response to what should be done regarding the expected

changes, a non-ESL teacher (Non-ESL-11) suggested that “the changes

that have occurred include more intervention programs in classrooms,

on campuses and in districts.” This move was also supported by another

non-ESL teacher (Non-ESL-14):

“There is a growing number of ELLs being motivated to perform

according to standards. These are the ones who can actually see

beyond high school graduation upon passing the exit-level TAKS.

Opportunities for postsecondary education exist along with

standardized testing, and so the high school journey for these ELLs

seeing the long-term meaning of the TAKS gives them a greater

sense of direction. The TAKS has also created positive pressure

among this group of ELLs in that they have become aware of

specific academic competencies they are expected to attain,

thereby creating in them greater self-efficacy and confidence for

postsecondary education.”

An assistant principal (AP-06) observed that:

“The changes that have occurred in some schools have been

teachers receiving more staff development in how to instruct

students whose first language is not English. More responsibility

has been placed on teachers in all classes to help strengthen the

students‟ ability to pass the test.”

131

A non-ESL teacher (Non- ESL- 09) also saw that “changes in

teaching strategies are affected…teachers should be able to teach using

English as a medium of instruction and yet accommodate the ELLs in

their classes.”

A principal (P-02) offered this strategy to cope with changes due to

TAKS:

“We are focused on each child. The teachers are aware of their

strengths and weaknesses. Teachers meet with administrators to talk

about the students and what strategies they will use to help the ELL

attain the level of academic achievement that the state and the school

require.”

Recommendations to Improve Performance of ELLs in Statewide Testing

Respondents gave their recommendations on how to improve the

overall performance of ELLs in TAKS; these are shown in Table 4.16.

Recommendations involved: (1) different/fairer test; and, (2) better

assistance from the school through the teachers and the curriculum.

Twenty percent (20%) of the respondents clamored for a different and fair

test, while 9% opted for a test to be given 5 to 7 years after ELLs started

schooling in the country. Eighteen percent (18%) recognized the need for

better prepared teachers; 15% requested for modifications in teaching;

and 11% proposed a paced curriculum for ELLs.

132

Table 4.16

What Recommendations are suggested for Improvement of ELLs Performance on TAKS?


Different/fairer test 1 0 0 5 5 11/20

Better prepared teachers 1 3 0 4 2 0 10/18

Teaching modifications 2 2 1 1 2 008/15

Paced curriculum 2 0 1 0 3 06/11

Test ELLs later (5-7years) 0 0 1 2 2 000 5/9

Increase overall performance 0 0 1 2 2 000 5/9

Intense English programs 0 1 3 0 0 444 4/7

Bilingual teachers for ELLs 0 0 0 0 3 00 3/5

Monitor ELLs 0 1 1 1 0 00 3/5

Test preparation classes 1 0 0 0 1 00 2/4

Smaller classes-create PLCs 0 0 1 1 0 00 2/4

Involvement of parents 0 0 0 0 2 010 2/4

133

ESL district personnel (EDP-03) recommended: “State legislators

(individuals who formulate the TAKS test) need to speak with ELL

teachers as well as the ELL students” to develop a more appropriate test

which measures achievement.

Two ESL teachers (ESL-06 and ESL-07) offered the following

changes in the administration of the TAKS to ELLs:

“There should be varying levels of the TAKS test for ELLs. When

students fail the TAKS, it‟s not because they are incapable of

passing…but the test is not a very ELL-friendly exam. If TAKS is

„supposed‟ to be a basic skills test for non-ELLs, then a test should

be devised to test the basic skills that ELLs have acquired since

they have learned English. It is not fair that a student that has

been speaking English their entire life takes the same test as a

person that has only been speaking English 3 or more years in

school.” “Have an „alternate‟ state test for the students who enter

the country/school at such a late age, which they realistically

cannot be expected to have enough of a grasp of English to pass

the TAKS. This of course would be a small portion of the ELLs.”

A non-ESL teacher (Non-ESL-16) proposed “that ELLs be given at

least 5 years of residency before they are given a statewide test. Teachers

need to be certified to teach ELLs so that the “problem” can be addressed

properly.”

134

An assistant principal (AP-02) recommended this plan of action:

“allow them more time to learn the English language before

administering the TAKS Exit Level Test; schedule them in intense

English/Reading classes for 75% of the school day.”

An even more drastic move was suggested by a non-ESL teacher

(Non-ESL-17) who suggested that we “do away with TAKS as a state test”

altogether.

Another non-ESL teacher (Non- ESL-14) recommended the

following regarding TAKS:

“Curriculum and instruction must translate standards into

classroom experience that takes into account their socio-cultural

milieu, their prior knowledge and skills and their linguistic

repertoire. It would help if those who are academically lagging

behind, or enrolled at a level several years above their actual level,

are given remedial courses so that they do not necessarily have to

study material that is incoherent to them, but would bring them

up from where they are to where they are supposed to be. Smaller

classes or intervention groups would perhaps ensure that greater

attention is given to the individual ELL‟s need. Since intrinsic

motivation for academic study is undermined by the survival

mode, perhaps a culture of career orientation can be instituted

where professionals from the community could conduct career

activities in schools. This will indirectly motivate towards ELLs

135

seeing the significance of standardized testing. Increased

involvement of parents is a big factor in this intrinsic motivation.”

Recommendations by a focus group included the following: (1)

“Much, much smaller classes for one thing, but it never happens because

it always comes down to money”; (2) “Home and community push for

excellence. Adult business tutors brought in on a weekly basis”; and (3)

“I believe that technology, certain computer programs, may be beneficial

to the ELL population.”

Recommendations of Greatest Value for ELLs‟ Success on Statewide

Testing

From the recommendations given, the respondents identified what

will be of greatest value for ELLs‟ success on statewide testing. These

suggestions are given in Table 4.17. Intervention for ELLs was

recommended by 27% of the respondents, while 16% emphasized quality

instruction, and 13% suggested a modified test. A more intensive English

program was proposed by 9% of the group, while 9% also observed the

need for help from home or for ELLs to have meaningful tutoring in

school.

136

Table 4.17

What are the Recommendations, with the Greatest Value, are offered for ELLs Success on TAKS?


Interventions 2 3 1 3 6 15/27

Quality Instruction 1 2 1 0 5 0 9/16

Modified Test 0 0 1 3 3 007/13

Intensive Programs for ELLs 1 0 2 2 0 0 5/9

Tutoring 0 0 0 2 3 0005/9

More ESL Teachers 0 0 0 2 2 0004/7

Practice Test 2 0 1 0 1 00 4/7

5 or More Years Residency 0 0 0 1 3 00 4/7

Bilingual/ESL Classes 0 1 2 0 0 00 3/5

None/Don‟t Know 0 1 1 0 1 00 3/5

Smaller Class 0 0 0 1 0 00 1/2

Modified Diploma 0 0 0 1 0 00 1/2

Effective School Leadership 0 0 1 0 0 00 1/2

137

A principal (P-03) considered this scenario as beneficial to the

ELLs:

“Better trained ESL teachers, more training for the traditional

teachers and training for administration on an assets-based model

rather than deficit. Also setting up more parent and community

involvement. Parents do care and want the best for their children

but they need to feel comfortable coming to school and made to feel

welcomed.”

An assistant principal (AP-09) suggested that: “Bilingual/ESL

classes should be continuously offered. Certified Bilingual/ESL teachers

should be recruited to offer even more assistance to these students.” This

suggestion finds support from an ESL teacher (ESL-03) who stated that:

“ELLs must be taught the skills that will allow them to assimilate into a

new society at a rigorous level. The test should reflect their needs and

also be more culturally sensitive.”

A non-ESL teacher (Non-ESL-14) averred that: “The remedial

intervention program for ELLs whose academic skills fall below their

actual grade level would greatly help in an immediate manner.” Yet

another non-ESL teacher (Non-ESL-17) suggested that “ESL teachers

and regular teachers focus on teaching skills that would be most

beneficial to the students based on their future aspiration (work or

college after high school).”

138

Regarding statewide testing, an assistant principal (AP-06)

proposed that:

“The test should be written in the language the student speaks.

Are we testing for academic skills? Or, are we trying to hold some

schools more accountable than others when giving tests? Schools

that have a higher minority rate must have teachers who are more

specialized in special education or bilingual education.”

During one of the interviews, a principal stated a rather grim

scenario involving ELLs:

“Standards are going to be difficult for them to achieve due to

their lack of language and academic skills. It takes an ELL student

five to seven years to acquire language skills adequate for the rigor

of high school curriculum. These students eventually pass but not

before causing tremendous challenges on our respective campuses.

These students show up on our „dropout lists.‟ They have a huge

impact on all our student data. For the most part these students

come from dysfunctional homes. The father is in Mexico. Mother is

here trying to raise a house full of kids- often forced to move from

place to place due to financial difficulties. These students seldom

stay in one school very long. I think the family unit should move

here together and stay until the student graduates from high

school.”

139

Another principal offered a radical suggestion regarding what can

be done to handle ELLs regarding the issue of statewide testing:

“If I could, I would send them (ELLs) all away from this school.

They would all be placed in a school designed to introduce,

develop, and build their language and culture skills, they bring our

scores down. That shouldn‟t be any surprise. Their skills are weak.

Their language is weak. Many of these students develop discipline

and attendance problems. Our ISD provides a „new comers‟

program for ESL students. The campus is a separate campus-away

from the main high school. Students remain on the „new comers‟

campus for about six months. Then, when they are ready, the

students are transitioned onto the main campuses. They ruin our

state scores; and wreak havoc on AYP scores. We have a large

population of these kids. Our school data indicates that we have

19 percent ESL students. That does not, however, include the

monitored kids. Most ESL students eventually pass the state exam

but we will never get above „acceptable‟ status if we continue to try

to teach and educate these kids on our campuses, within the

regular population. The faculty and staff must work ten times

harder than schools with few ESL kids.”

140

A principal explained how a strategy worked in their campus to

help ELLs:

“ELL students who continue to be monitored are placed in the

appropriate English Language Arts course and are served by

certified ESL instructors. As with all of our students, tutorial

assistance is provided one hour each week after school for the four

key academic content areas. Tutorials begin the second week of

school and continue throughout the year (generally increasing in

number prior to TAKS testing in the spring). Teachers closely

monitor student performance and parents are contacted at the first

sign of difficulty. Informal parent teacher conferences are held, and

the student is encouraged to take every advantage of the

assistance provided by the instructional staff. Students are held

accountable for their study time. Note that our students are

expected to study approximately one and a half hours per school

night. Students and parents sign a contract to this effect prior to

being accepted at the school. If informal parent conferences are not

successful in providing the assistance and motivation to improve

academic success, the Intervention Assistance Team is employed.”

ELLs enrolling in a school during anytime during the year is a

reality school administrators and teachers have to face in public schools.

141

They have to be ready to map out a course of action for each ELL to

provide guidance and education during the stay of that student in their

school.

Discussion

Research Question - Quantitative





The descriptive statistics showing the means of the 10th grade

TAKS for English Language Arts and Mathematics do not indicate

improvement in performance despite the decrease in the percent of ELLs

enrolled in a school. Percent ELLs was 8.30% in 2003, 8.29% in 2004,

7.62% in 2005 and 6.94% in 2006. However, it is in the computations for

Pearson r correlation that the significant relationship was determined.

Using the SPSS software package for the TEA data of the sampled

schools, the obtained Pearson r correlation coefficients were: -0.349 in

2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006 for English

Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in 2005, and

-0.356 in 2006 for Mathematics. All of the computed Pearson r

correlation coefficients were significant at the 0.05 level, two-tailed. All of

the obtained Pearson r correlation coefficients were negative for both

content areas; this inverse relationship indicated that as the percentage

142

of ELLs increased, the percentage of all 10th grade students passing in

each of the core areas of English Language Arts and Mathematics

decreased. A regression analysis using the percentage of ELLs enrolled in

a school as a predictor variable resulted to a linear regression equation

which can predict the percentage of all students passing in 10th grade

TAKS in English Language Arts and Mathematics.

Any school which has ELLs should be aware of the number of

these types of students. If there is a surge in the influx of ELLs sometime

during the year, the school should carefully plan on how to prepare them

for future testing. Both curriculum and instruction for ELLs may be put

under review for improvement to better address this growing population

of ELLs.

Research Question - Qualitative

What are the anticipated and observed consequences of the

statewide testing specifically, TAKS, on ESL curriculum and instruction

as viewed by certified ESL teachers, non-certified ESL teachers who

teach ELLs, school administrators, and district ESL personnel?



Respondents gave the following answers: (1) TAKS is given as a tool

to gauge knowledge in the core areas or what is supposed to be taught;

(2) TAKS is considered as a means to determine the school‟s status

(Exemplary, Recognized, Academically Acceptable, or Academically

143

Unacceptable); (3) TAKS is regarded as an avenue to assess the state

curriculum or standards; and (4) this statewide test is mandated by law

and is aligned with NCLB. The reasons why TAKS is given have been

understood by the different school personnel.

2. What are the intended consequences of this statewide testing?

Respondents expressed the understanding that statewide testing is

intended to eventually result in ELLs performing as well as the rest of the

students. Respondents are confident that ELLs can improve academically

and eventually join the mainstream; they are also aware that TAKS is a

requirement for graduation. Teachers expressed their wish for an

opportunity to teach ELLs individually with the end in mind of improving

their performance in TAKS. School personnel affirmed their commitment

on improving the situation of ELLs in school.


Respondents identified the following as problems encountered by

ELLs due to TAKS: (1) higher dropout rate; (2) graduation rate decreased

for ELLs; and (3) low self-esteem of ELLs. These problems are rooted on

the difficulty ELLs face in passing the TAKS tests.


Due to TAKS, schools experienced the negative reality that there is

a high failure rate among ELLs. Other unpleasant realities included low

self-esteem because of low scores and tests caused frustration and

exasperation on the part of ELLs. There is mounting pressure on the

144

school and ELLs and emphasis is placed on test performance. Changes

require action to improve the situation of the school, particularly those

aspects which are related to the ELLs. Low scores have to be improved

and ELLs need counseling on ways on how to have a positive view of

anything one meets in life.


Recommendations involved two main factors: (1) deferment of the

test, possibly a different but fair test; and, (2) better assistance from the

school through the teachers and the curriculum, modifications in

teaching and possibly a paced curriculum for ELLs. Since one

requirement of NCLB is testing and results indicate progress, there is

need for school personnel to continually monitor ELLs and devise

strategies to better help these students.



The following recommendations were seen as more helpful to ELLs:

Specific interventions for ELLs, quality instruction, a more intensive

English program, help from home and meaningful tutoring in school.

Both the school and the home, together with the community have to be

partners in getting involved with ELLs.

145

Summary

Since the researcher utilized the explanatory design of mixed

methods, both quantitative and qualitative aspects were considered in

this study.

From the data gathered from TEA and after utilizing the SPSS

software package for Pearson r correlation, the obtained results indicate

that there existed significant, negative relationships between the percent

of ELLs enrolled in a school and the percentage of 10th grade students

passing in the English Language Arts and Mathematics TAKS tests given

in 2003, 2004, 2005, and 2006. The null hypotheses were then rejected.

The regression analysis provided the linear regression equations to

predict the percentage of all students passing in 10th grade TAKS tests in

ELA and Mathematics using the percentage of ELLs enrolled in a school

as the predictor variable.

For the qualitative dimension of the study, the respondents were

mostly certified in English Language Arts, Mathematics, and Special

Education for teachers and some process of administrative certification

for ESL district personnel, principals and assistant principals.

As a statewide test, TAKS was seen as a tool to gauge knowledge

and skills in the core areas. According to the respondents, students who

took TAKS were expected to pass and have at least average scores.

Unfortunately, most of the respondents saw dismal or low results in their

respective schools, especially among ELLs. Consequently, focus had been

146

centered on this group of students and recommendations included

testing them at a later date by using a modified test, and having

interventions to improve the overall performance of ELLs.

Administrators and teachers realized the need to improve the

quality of instruction and provided interventions especially geared

towards improving the academic performance of ELLs. It is hoped that

both instruction and curriculum for ELLs were directed towards

improving the plight of ELLs.

147

CHAPTER V

SUMMARY, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS

Summary


stakes testing on English Language Learners (ELLs). This was shown in

both the quantitative and qualitative dimensions of the study. Both

quantitative and qualitative dimensions of the study provided the status

of high-stakes testing as it affected ELLs and how it influenced efforts in

schools to improve performance of students, particularly ELLs. Data

obtained from Texas Education Agency (TEA) were used to determine

whether there was a relationship between the percentage of ELLs


grade Texas Assessment of Knowledge and Skills (TAKS) tests in the core

areas of English Language Arts and Mathematics given in 2003, 2004,

2005, and 2006. To support the qualitative aspect, this study explored

what certified English as a Second Language( ESL) teachers, non-

certified ESL teachers who teach ELLs, administrators, and district ESL

personnel viewed as the impact that high stakes standardized

assessments had on ELLs, ESL curriculum and instruction.

Review of literature included the important consideration of the No

Child Left Behind Act (NCLB). NCLB holds states using federal funds

accountable for student academic achievement. States are required to

develop a set of high-quality, yearly student assessments that include, at

148

a minimum, assessments in Reading/Language Arts, Mathematics and

Science. NCLB requires states to report Adequate Yearly Progress (AYP)

for all students and for subgroups, including students with limited

English proficiency (Abedi, 2004). Phrases such as “student

achievement,” “proficiency,” “raised expectations” and “testing” are

implications of NCLB. Certainly, the focus on holding schools

accountable for student achievement on standardized tests sets NCLB

apart from previous versions of the law. (Guilfoyle, 2006).

The focus of the study is the ELLs. The term “English language

learner” is a recent designation for students whose first language is not

English. This group includes students who are just beginning to learn

English as well as those who have already developed considerable

proficiency. The driving force behind including English language learners

in statewide accountability testing is the legislation requiring it. In order

to continue to receive Title I funds through NCLB, states must set high

standards for all students and implement accountability systems to

measure progress towards those standards. NCLB specifically states that

English language learners must be included in statewide accountability

testing, that their scores must be disaggregated so that it can be seen

how they are achieving as a subgroup, and that the assessment system

must accommodate their linguistic needs (“NCLB”, 2002).

149

High-stakes testing -- using standardized scores to impose

consequences affecting teachers and students – has been embraced

widely in recent years as a way to hold educators and students

accountable for their performance. Experts say the movement is one of

the most significant shifts in U.S. education in decades (Whoriskey,

2006).

The goal of statewide accountability testing for English language

learners (or for all students for that matter) is to improve standards-

based practices. The intended “washback” of including English language

learners in standards-based assessment has been described as providing

“the leverage needed to raise expectations for English language learners,

and the emphasis on higher level skills should improve the quality of

teaching provided to them” (Lachat, 1999, p.60), “feedback that will allow

instructional leaders to improve instructional programs” (Lacelle-

Peterson & Rivera, 1994, p.64), and will ideally “…help students reach

the standards by (a) influencing what is taught and how it is taught (i.e.,

„washback‟ to instruction), (b) providing data to guide instructional

modifications, and (c) targeting resources to schools they are most

needed” (Rivera & Vincent, 1997, p.336). In addition, Mehrens (2002)

states that large-scale assessments have two major purposes: to drive

reform and to gauge if reform policies have had an impact on student

learning. These goals are especially important for English language

150

learners who often face socio-economic, cultural, and linguistic

challenges to academic achievement.

Demographic Data

Total respondents who answered the on-line questionnaire totaled

55 – 35% are non-ESL certified teachers and 27% are ESL-certified

teachers. The administrators accounted for the remaining 38%- 16% are

Assistant Principals, 11% are Principals and 11% are ESL District

Personnel.

Conclusions

The analysis of the quantitative data in Chapter IV led the researcher

to draw the following conclusions:

1. The descriptive statistics showing the means of the 10th grade

TAKS for ELA and Mathematics do not indicate improvement in

performance despite the decrease in the percent of ELLs enrolled in

a school.

2. All the obtained Pearson r correlation coefficients to determine

whether there is a relationship between the percentage of English

language learners enrolled in a school and the percentage of all

students passing the 10th grade TAKS tests in the core areas of

English Language Arts and Mathematics given in 2003, 2004,

2005, and 2006 were all significant at the 0.05 level, two-tailed.

Both null hypotheses were rejected.

151

3. The negative Pearson r correlation coefficients implied that as the

percentage of ELLs in a school increased, performance on both

English Language Arts and Mathematics decreased.

4. The linear regression equations may be used to predict outcomes

in 10th grade TAKS tests in ELA and Mathematics using the

percentage of ELLs enrolled in a school as the predictor variable.

The analysis of the qualitative data in Chapter IV led the researcher to

draw the following conclusions:

1. Texas Assessment of Knowledge and Skills (TAKS) were perceived

by respondents as a tool to gauge knowledge in the core areas.

2. ELLs were expected to have at least average scores on TAKS.

3. There was a difference in the expected and actual results.

Respondents observed dismal or failing performance of ELLS in

actual results in TAKS. This was evident by the high failure rate of

ELLs in their respective schools.

4. Higher dropout rate and lower graduation rates of ELLs were

problems encountered due to TAKS.

5. Respondents favored a different test for ELLs.

6. Respondents believed that interventions were needed to help ELLs

perform better.

Implications

The research data gathered in the course of this study suggested

that while there was a common perception that ELLs performed poorly

152

on high stakes testing, there was no unanimity among professionals in

the field of education regarding the viability of options that might be

considered in addressing the low achievement level of ELLs. This was not

necessarily relevant, although it suggested that the appreciation of the

problem and its causes lent itself to biases and distortions depending on

the personal circumstance and perspective of those presenting these

options. It was clear from the study that schools needed to do things

differently, if they expected ELLs to perform better on standardized

assessments. The major implications of the study were as follows:

1. The performance of schools in high stakes testing was affected by

the size and proportion of ELLs taking the test. At the same time,

ELLs were not evenly distributed across campuses. The

performance of schools on standardized tests was influenced to a

degree by the voluntary segregation in many districts of ethnic

groups who speak English only as an adopted language. This was

a phenomenon that was beyond the power of school districts to

address, and that required wide coordination among various

government agencies to develop an appropriate policy response.

2. The extended deferment of standardized tests administered in

English to ELLS should be considered. The primary goal of these

tests was to measure learning, that might be more accurately

accomplished if the assessment was done in the language the

student was most proficient. There was the expectation that the

153

student will eventually be proficient in English as well. Since a

second language is acquired in degrees, it might be reasonable to

assume that ELLs would not readily have the same facility for

English as a native speaker. Administering the test in English

before the ELL student was ready for it would compromise the

stated goal of measuring learning as accurately as possible.

3. Learning is transmitted through communication. Due to the

unique linguistic characteristic of ELLs, unique strategies,

modifications, and instructions need to be used to maximize their

capacity to learn concepts and skills. It is futile to assume that

ELLs will learn the same way as native speakers of the English

language. It follows that education professionals need the

specialized training and support to be able to facilitate learning for

ELLs.

4. The Language Proficiency Assessment Committee (LPAC) in each

campus needs to take a more active role in monitoring the progress

of ELLs and devising specific plans to properly respond to the

requirements designed specifically for ELLs. They need to

undertake a regular evaluation of instruction and curriculum for

ELLs and communicate findings and recommendations to all

stakeholders – school administrators, teachers, parents and the

ELLs.

154

5. Interventions to improve the situation of ELLs should include

specific action plans to devise a more intensive English program in

schools and a continued emphasis on quality instruction

employing strategies suggested by educational experts who have

extensively researched on such courses of action.

Recommendations for Further Study

Based on the results of the study, the researcher recommends the

following concerns for further study:

1. A study to determine what additional supports are needed to

ensure that English language learners will pass high-stakes tests.

2. A study to identify what data are needed to make fair high-stakes

decisions about English language learners (like subject grades,

samples of class work and recommendations of teachers and

counselors).

3. A study to determine the reasons why English language learners

scored lowest among student groups in the Texas Assessment of

Knowledge and Skills tests in the core areas of English Language

Arts and/or Mathematics.

4. A study to explore different approaches in school campuses

regarding handling of English language learners in terms of

instruction, curriculum and other pertinent or related aspects

(such as some sort of evaluation - academic, social, financial, etc.)

155

that may guide administrators and teachers to effectively handle

English Language learners.

5. A study to determine the performance of 10th grade English

language learners compared to non-English language learners and

non-classified students based on the different objectives of the

Texas Assessment of Knowledge and Skills tests in either or both

Mathematics and English Language Arts.

6. A study to determine the impact of high stakes testing on English

language learners as viewed by parents and students.

7. A study to explore different instruments to measure academic

performance of English language learners.

8. A study to determine if there is significant difference between

performance in the different core areas of English language

learners belonging to different language groups.

This study affirmed the expected outcome that a significant

relationship existed between the percentage of ELLs enrolled in a school

and the percentage of all students passing the 10th grade TAKS tests in

both core areas of English Language Arts and Mathematics. The

regression analysis predicted that as the percentage of ELLs in a school

increased, the performance in the statewide, high-stakes testing in terms

of all students passing the 10th grade TAKS tests decreased. The

respondents of the study considered the Texas Assessment of Knowledge

and Skills (TAKS) as a tool to gauge knowledge in the different core areas.

156

English language learners were expected to have at least average scores

on TAKS. There was a difference in the expected and actual results;

respondents observed dismal or failing performance of ELLS in the actual

results in TAKS. This was evident by the high failure rate of ELLs in their

respective schools. Higher dropout rate and lower graduation rate of

ELLs were problems encountered due to TAKS. Respondents favored a

different test for ELLs, possibly given at a later date after ELLs had

studied in the country for at least several years. Respondents believed

that interventions were needed to help ELLs perform better. Both the

school and the home, together with the community have to be involved in

preparing ELLs to be better prepared for their present and future roles in

the American society.

157

REFERENCES

Abedi, J. (2004). The No Child Left Behind Act and English Language

Learners: Assessment and Accountability Issues. Educational

Researcher, 33(1), 4-14.

Abedi, J. (2004). Will You Explain the Question? Principal Leadership

(High School Ed.), 4(7), 27-31.

Abedi, J., Hofstetter, C. & Lord, C. (2004). Assessment Accommodations

for English Language Learners: Implications for Policy-Based

Empirical Research.

Abedi, J. (2004). Inclusion of Students with Limited English Proficiency

in NAEP: Classification and Measurement Issues. CSE Report 629

CRESST/ University of California, Los Angeles.

Abedi, J. (2004). Will You Explain the Title? Principal Leadership (High

School Ed.), 4(7), 27-31.

Abedi, J., Leon, S., & Mirocha, J (2003). Impact of student language

background on content-based performance: Analyses of extant

data (CSE Tech. Rep. No. 603). Los Angeles: University of

California, National Center for Research on Evaluation, Standards,

and Student Testing.

Abrams, L.M., & Madaus, G.F. (2003). The Lessons of High-Stakes

Testing. Educational Leadership, 61(3), 31-35.

158

Airasian, P. W. (1988). Symbolic validation: The case of state-mandated,

high-stakes testing. Educational Evaluation and Policy Archives,

10(4), 301-313.

Albus, D., Thurlow, M. L., Liu, K. K., & Bielinski, J. (2005). Reading Test

Performance of English Language Learners Using an English

Dictionary. The Journal of Educational Research, 98(4), 245-254.

Albus, D., Thurlow, M. L., & Liu, K. K. (2002). 1999-2000 Participation

and Performance of English Language Learners Reported in Public

State Documents and Web Sites (LEP Projects Report 3).

Minneapolis, MN: University of Minnesota, National Center on

Educational Outcomes.

Alderson, J. C., & Wall, D. (1993). Does Washback Exist? Applied

Linquistics, 14(2), 115-129.

Amrein, A. L., & Berliner, D. C. (2003). The Effects of High-Stakes

Testing On Student Motivation and Learning. Educational

Leadership, 60(5), 32-38.

Anderson, M. E. (2001). Item Response Differences Among Spanish-

Speaking ESL Students and Native English-Speaking Students

on a Standardized Reading Test. Unpublished Manuscript.

Anderson, M. E. (2004). Intended and Unintended Consequences of

Statewide Testing for ESL Curriculum and Instruction. (UMI No.

3137152).

159

Anderson, M. E., Thurlow, M. L., Swierznin, B., Liu, K. K., & Bielinski, J.

(2000). Bilingual Accommodations for Limited English Proficient

Students on Statewide Reading Tests: Phase (No. 31 Minnesota

report). Minneapolis, MN: University of Minnesota, National Center

on Educational Outcomes.

Artiles, A. J., Rueda, R., Salazar, J. J, & Higareda, I. (2005). Within

Group Diversity in Minority Disproportionate Representation:

English Language Learners in Urban City Schools. Exceptional

Children, 71(3), 283-300.

August, D., & Hakuta, K. (1997). Improving Schooling for Minority

Children: A Research Agenda. Washington, D.C.: National

Academy Press.

Barro, S.M., & Kolstad, A. (1987). Who Drops Out of High School?

Findings from High School and Beyond. Washington, D.C.: Center

for Education Statistics, Office of Research and Improvement, U.S.

Department of Education.

Brennan, R. L. (2004). Revolutions and Evolutions in Current

Educational Testing. Center for Advanced Studies in Measurement

and Assessment: CASMA Research Report #6.

California Department of Education. (2000). California demographics

data. Retrieved February 24, 2003, from

http://www.cde.ca.gov/demographics/

160

Center on Educational Policy (2005). From the capital to the classroom:

Year 3 of the No Child Left Behind Act. Washington, DC: Author.

Chamot, A. U., & O’Mailley. J.M. (1994). The CALLA Handbook:

Implementing the Cognitive Academic Language Learning Approach.

Reading, MA: Addison-Wesley.

Cheng, L. (1999). Changing Assessment: Washback on Teacher

Perceptions and Actions. Teaching and Teacher Education, 15,

253-271.

Cheng, L. (2003). Looking at the Impact of a Public Examination Change

on Secondary Classroom Teaching: A Hong Kong Case Study.

Journal of Classroom Interaction, 38(1), 1-10.

Cizek, G.J. (2003). When Teachers Cheat. Education Digest, 68(6), 28-31.

Cobb, T. (2000). One Size Fits All? Francophone Learners and English

Vocabulary Tests. Canadian Language Review, 57(2), 295-324.

Collier, V.P. (1989). How Long? A Synthesis of Research on Academic

Achievement in a Second Language. TESOL Quarterly, 23(3), 509-

531.

Coltrane, B., (2002).English Language Learners and High-Stakes Tests:

An Overview of the Issues. (November, 2002). Center for Applied

Linguistics.

Council of Chief State School Officers. (1992). Recommendations for

Improving and Assessment and Monitoring of Students with

161

Limited English Proficiency. Washington, DC: Council of Chief

State School Officers.

Creswell, J.W. (1998). Qualitative Inquiry and Research Design:

Choosing Among Five Traditions. Thousand Oaks, CA: Sage.

Davis, M. R. & Sacks, J. L. (2005). California, U.S. Department of

Education strike deal on NCLB rules [Electronic version].

Education Week.

Datnow, A., Borman, G.D., Stringfield, S., & Overman L.T. (2003).

Comprehensive School Reform in Culturally and Linguistically

Diverse Contexts: Implementation and Outcomes from a Four-Year

Study. Educational Evaluation & Policy Analysis, 25(2), 143-170.

D’Emilio, T. E. (2003). LEP Student Participation in Special Education:

Over or Under-Representation? Paper Presented at the CCSSO

Conference on Large-Scale Assessment, San Antonio, Texas

Denzin, H. (1970). The research act: A Theoretical Introduction to

Sociological Methods. New York: McGraw-Hill.

Dexter, L.A. (1970). Elite and Specialized Interviewing. Evanston, IL:

Northwestern University.

Edwards, V. (Ed.). (2003, Jan. 9). Quality Counts 2003: If I can’t learn

from you… (Education Week Special Report), 12(17). Bethesda,

MD: Editorial Projects in Education.

Erlandson, D., Harris, E., Skipper, B., & Allen, S. (1993). Doing

Naturalistic Inquiry. Newbury Park: Sage.

162

Erpenbach, W.J., Forte-Fast, E., & Potts, A. (2003). Statewide

educational accountability under NCLB (An Accountability

Systems and Reporting State Collaborative on Assessment and

Student Standards Paper). Washington, DC: Council of Chief

State School Officers.

Escamilla, K., Mahon, E., Riley-Bernal, H., & Rutledge, D. (2003). High-

Stakes Testing, Latinos, and English Language Learners: Lessons

from Colorado. Bilingual Research Journal, 27(1), 25-47.

FairTest (2000).

Flores, B.B., & Clark, E.R. (2003). Texas Voices Speak About High-

Stakes Testing: Preservice Teachers, Teachers, and Students,

Current Issues in Education (Vol.6).

Fowler, F. J. (1984). Survey Research Methods. Beverly Hills: Sage.

Fraenkel, J. & Wallen, N. (2003). How to Design and Evaluate Research

In Education (5th ed.). New York, NY: McGraw-Hill.

Freeman, Y.S., & Freeman, D., E. 2002). Closing the Achievement Gap:

How to Reach Limited-Formal-Schooling and Long-Term English

Learners. Portsmouth, NH: Heinemann.

Garcia, G. E., & Pearson, P. D. (1994). Assessment and diversity. In L.

Darling-Hammond (Ed.), Review of Research in Education (Vol. 20).

Washington, DC: American Educational Research Association.

Gewertz, C. (2005). Ed. Dept. allows Chicago to provide NCLB tutoring

[Electronic version]. Education Week.

163

Goldhaber, D., & Hannaway, J. (2001). Accountability with a kicker:

Observations on the Florida A+ Plan. Paper presented at the

annual meeting of the Association of Public Policy and

Management, Washington, DC.

Goldschmidt, P. and Wang, J. (1999) "When Can Schools Affect Dropout

Behavior: A Longitudinal Multilevel Analysis," American

Educational Research Journal, vol. 36, no. 4, pp.715-738, Winter.

Guilfoyle, C. (2006). “NCLB: Is There Life Beyond Testing?” Educational

Leadership, vol. 64, no. 3, pp. 8-13, November 2006.

Haladyna, T., Nolen, S., & Haas, N. (1991) Raising standardized

achievement test scores and the origins of test score pollution.

Educational Researcher, 20(5). 2-7

Hammersley, M. (1992). What’s Wrong with Ethnography? London:

Routledge.

Haney, W. (2000). The Myth of the Texas Miracle in Education, Education

Analysis Policy Archives (Vol.8).

Harlow, A., & Jones, A. (2003, July). Why students answer TIMSS

science test items the way they do. Paper presented at the annual

conference of the Australian Science Education Research

Association, Melbourne, Victoria, Australia.

Herman, J., & Golan, S. (n.d.). Effects of standardized testing on

teachers and learning (CSE Technical Report 334). Los Angeles:

164

National Center for Research on Evaluation, Standards, and

Student Testing.

Herszenhorn, D. (2003, July 23). Basic skills forcing cuts in art classes.

The New York Times, p. B1.

Heubert, J. (2000).High-Stakes Testing: Opportunities and Threats. In

Pines, M (ed.). The Continuing Challenge: Moving the Youth

Agenda Forward (Policy Issues Monograph 00-02, Sar Levitan

Center for Social Policy Studies. Baltimore, MD: Johns Hopkins

University Press.

Heubert, J.P. (2003). First, Do No Harm. Educational Leadership, 60(4),

26-30.

Heubert, J., & Hauser, R. (Eds.) (1999). High stakes testing for tracking,

promotion, and graduation. Washington, DC: National Academy

Press.

Hill, R.K., & DePascale, C. A. (2003). Reliability of no child left behind

accountability designs. Educational Measurement: Issues and

Practice, 22(3), 12-20.

Hinde, E.R. (2003). The Tyranny of the Test: Elementary Teacher’s

Conceptualizations of the Effects of State Standards and Mandated

Tests on Their Practice, Current Issues in Education. (Vol.6).

Hoffman, J. V., Assaf, L., & Paris, S. G. (2001). High Stakes Testing in

Reading: Today in Texas, Tomorrow? The Reading Teacher.

165

Holmes, E. (1911). What is and what might be: A study of education in

general and elementary in particular. London: Constable.

Hood, L. (2003). Immigrant Students, Urban High Schools: the Challenge

Continues. Revised 09/16/2003, from

http://www.carnegie.orge/immigrantstudents.pdf.

Horn, C. (2003). High-Stakes Testing and Students: Stopping or

Perpetuating a Cycle of Failure? Theory Into Practice, 42(1), 30-41.

Horn, C., Ramos, M., Blumer, I., & Madaus, G. (2000). Cut scores:

Results may vary. Chestnut Hill, MA: National Board on

Educational Testing and Public Policy, Boston College.

Isaac, S. & Michael, W. (1997). Handbook in Research and Evaluation

for Education and the Behavioral Sciences (3rd ed.) San Diego, CA:

EdiTs/Educational and Industrial Testing Services.

Johnson, D. (2001). Performance Pentagon: Five Strategies to Help All

Students Make the Grade. NASSP Bulletin, 85, 40-55.

Jones, M., Jones, B., Hardin B., Chapman, L., Yarbough, T., & Davis, M.

(1999). The impact of high-stakes testing on teachers and

students in North Carolina. Phi Delta Kappan, 81(3), 199-203.

Kaufman, P., Alt, M. N., & Chapman, C. D. (2001). Drop-Out Rates in the

United States: 2000 (No.NCES 2002-114). Washington, DC:

National Center on Educational Statistics.

Kinder, A. L. (2002). Survey of the states’ limited English proficient

students and available educational programs and services, 2000-

166

2001 summary report. Washington, DC: National Clearinghouse

for English Language Acquisition and Language instruction

Educational Programs.

Klein, S., Hamilton, L., McCaffrey, D., & Stecher, B. (2000). What do test

scores in Texas tell us? Education Policy Analysis Archives, 8(49).

[Online]. Available: http://epaa.asu.edu/epaa/v8n49

Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996a). Perceived effects

of the Kentucky Instructional results information system (KIRIS)

(MR-792-PCT/FF). Santa Monica, CA: Rand.

Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996b). Perceived

effects of the Maryland school performance assessment program

(CSE Technical Report 409). Los Angeles: National Center for

Research on Evaluation, Standards, and Student Testing.

Krathwohl, D. (1993). Methods of Educational and Social Science

Research: An Integrated Approach. New York: Longman.

Krueger, R.A., & Casey, M. A. (2000). Focus Groups: A Practical Guide for

Applied Research (3rd ed.). Thousand Oaks, CA: Sage Publications.

Lacelle-Peterson, M., & Rivera, C. (1994). Is it Real for all Kids? A

Framework for Equitable Assessment Policies for English Language

Learners. Harvard Educational Review, 64(1), 55-75.

Lachat, M. A. (1999). What Policy Makers and School Administrators

Need to Know About Assessment Reform for English Language

167

Learners. Northeast and Islands Regional Educational Laboratory

at Brown University. Providence.

Lane, S., & Stone, C. A. (2002). Strategies for Examining the

Consequences of Assessment and Accountability Programs.

Educational Measurement: Issues into Practice, 21(1), 23-30.

Lewis, A. (2002). A Horse Called NCLB. Phi Delta Kappan, 84(3),

179-180.

Linn, R. (1998). Assessments and Accountability (CSE Technical Report

490). Boulder, CO: CRESST/University of Colorado at Boulder.

Linn, R. L. (2000). Assessment and Accountability. Educational

Researcher, 29(2), 14-16.

Linn, R. L., Baker, E. L., & Herman, J. L. (2002). Minimum group size

for measuring adequate yearly progress: The CRESST line. Los

Angeles: University of California, Center for the Study of

Evaluation/National Center for Research on Evaluation,

Standards, and Student Testing.

Liu, K. & Thurlow, M. (1999). Limited English Proficient Students’

Participation and Performance on Statewide Assessments:

Minnesota Basic Standards Reading and Math, 1996-1998.

National Center on Educational Outcomes.

Madaus, G. (1988). The influence of testing on the curriculum. In L.

Tanner (Ed.), Critical issues in curriculum (pp. 83-121). Chicago:

University of Chicago Press.

168

Madaus, G. (1991, January). The effects of important tests on students:

Implications for a national examination or system of examinations.

Paper prepared for the American Educational Research Association

Invitational Conference on Accountability as a State Reform

Instrument, Washington, DC.

Madaus, G., & O’Dwyer, L. (1999). A Short History of Performance

Assessment. Lessons learned. Phi Delta Kappan, 80(9), 688-695.

Major, R.C., Fitzmaurice, S.M., Bunta, F., & Balasubramanian, C.

(2005). Testing the Effects of Regional, Ethnic, and International

Dialects of English on Listening Comprehension. Language

Learning 55 (1) 37-69.

Marshak, D. (2003). No Child Left Behind: A Foolish Race Into the

Past.Phi Delta Kappan, 85(3), 229-231.

Marshall, C. (1990). Goodness Criteria: Are They Objective or Judgment

Calls? In E. G. Guba (Ed.), The Paradigm Dialog (pp. 188-197).

Thousand Oaks, CA: Sage.

Maxwell, J. (2005). Qualitative Research Design. An Interactive Approach

(2nd ed.). Thousand Oaks, CA: Sage Publications.

McLaughlin, M.W., & Shepard, L.A. (1995). Improving Education

Through Standards-Based Reform. Stanford, CA: The National

Academy of Education.

McMillan, J., Myran, S., & Workman, D. (1999, April 19-23). The

Impact of Mandated Statewide Testing on Teachers’ Classroom

169

Assessment and Instructional Practices. Paper presented at the

annual meeting of the American Educational Research Association,

Montreal, Quebec, Canada.

McNeil, L.M. (2000). Contradictions of school reform: Educational costs of

standardized testing. New York: Routledge.

Mehrens, W. A. (1998). Consequences of assessment: what is the

evidence? Vice Presidential Address for Division D, annual meeting

of the American Educational Research Association, San Diego.

Menken, K. (2000).What Are the Critical Issues in Wide-Scale

Assessment of English Language Learners? Framing Effective

Practice, 23-27.

Merriam, S. B. (1998). Qualitative Research and Case Study Applications

in Education, San Francisco, CA: Jossey-Bass.

Miller & Fredericks (2000). Research on High-Stakes Tests and

Instruction. Retrieved October 9, 2005 from

http://vnweb.hwwilsonweb.com/hww/results/results_single_fullte

xt.jhtml.

Murnane, R. & Levy, F. (1996). Teaching the New Basic Skills. New York:

The Free Press.

Murnane, Richard J., John B. Willett, Yves Duhaldeborde, and John H.

Tyler. 2000. How important are the cognitive skills of teenagers in

predicting subsequent earnings? Journal of Policy Analysis and

Management 19 (4) 547-568.

170

Myers, K. (2003, July 16). A dream denied: Aspiring chef rethinks her

future as Falmouth school board bows to state pressure on MCAS.

Cape Cod Times. Available: www.capecodonline.com.

National Center for Education Statistics. (2002). Public school student,

staff, and graduate counts by state: School year 2000-01 (NCES

Publication 2002-348). Washington, DC: Author.

National Clearinghouse for Bilingual Education. (1997). High-Stakes

Assessment: A Research Agenda for English Language Learners.

Retrieved June 21, 1999, from

http://www.ncbe/gwu.edu/ncbepubs/reports/highstakes.htm#Te

sting.

National Commission of Excellence in Education. (1983). A Nation at

Risk: The Imperative for Educational Reform: a Report to the

Nation and the Secretary of Education, United States Department

of Education. Washington, D.C.: The Commission: [Supt. of Docs.,

U.S. G.P.O. distributor].

No Child Left behind Act of 2001, 20 U.S.C. 6301 et.seq. (2002).

O'Byrne, B. (2001). Needed: A Compass to Navigate the Multilingual

English Classroom. Journal of Adolescent & Adult Literacy 44 (5)

440-9.

Olson, L. (2005). Defying predictions, state trends prove mixed on

schools making NCLB targets [Electronic version]. Education

Week.

171

Orfield, G. & Kornhaber, M. L. (2001). Raising standards or raising

barriers? Washington, DC: The Century Foundation Press.

Patton, M. Q. (1990). Qualitative Evaluation and Research Methods (2nd

ed.). Thousand Oaks, CA: Sage Publications.

Pedulla, J., Abrams, L., Madaus, G., Russell, M., Ramos, M., & Miao, J

(2003). Perceived effects of state-mandated testing programs on

teaching and learning: Findings from a national survey of

teachers. Chestnut Hill, MA: National Board on Educational

Testing and Public Policy, Boston College.

Popham, J. W. (2003). The Seductive Allure of Data. Educational


Reyes, P. & Fletcher, C. (2003). Successful Migrant Students: The Case of

Mathematics. Journal of Curriculum and Supervision, 18(4), 306-

333.

Rex, L. A. & Nelson, M. C. (2004). How Teachers’ Professional Identities

Position High-Stakes Test Preparation in Their Classrooms.

Teachers College Record, 106(6), 1288-1331.

Rhoades, K., & Madaus, G. (2003). Errors in standardized tests: A

systemic problem. Chestnut Hill, MA: National Board on

Educational Testing and Public Policy, Boston College.

Rivera, C., & Stansfield, C. (1998). Leveling the Playing Field for English

Language Learners: Increasing the Participation in State and Local

Assessments Through Accommodations. In R. Brandt (Ed.),

172

Assessing Student Learning: New Rules, New Realities (pp. 65-92).

Arlington, VA: Educational Research Service.

Rivera, C., Stansfield, C., Scialdone, L., & Sharkey, M. (2000). An

analysis of state policies for the inclusion and accommodation of

ELLs in state assessment programs during 1998 – 1999 (Executive

Summary). Washington, DC: The George Washington University,

Center for Equity and Excellence in Education.

Rivera, C., & Vincent. (1997), High School Graduation Testing: Policies

and Practices in the Assessment of English Language Learners.

Educational Assessment, 4(4), 335-355.

Rose, L.C. & Gallup, A.M. (2003). The 35th Annual Phi Delta Kappa/

Gallop Poll of the Public’s Attitudes Toward the Public Schools. Phi

Delta Kappan, 85.1, 41-56.

Rosenbusch, M. H. (2005). The No Child Left Behind Act and Teaching

and Learning Languages in U.S. Schools. The Modern Language

Journal, 89(2), 250-261.

Russell, M., & Abrams, L. (in press). Instructional uses of computers for

writing: The impact of state testing programs. Teachers College

Record.

Sheldon K. and Biddle, B. (1998). Standards, accountability and school

reform: Perils and pitfalls. Teachers' College Record 100 (1): 164-

180.

Sirkin, R. M. (2006). Statistics for the Social Sciences. (3rd ed.). Thousand

173

Oaks, CA: Sage Publications.

Solano-Flores, G., & Trumbull, E. (2003). Examining Language in

Context: The Need for New Research and Practice Paradigms in the

Testing of English Language Learners. Educational Researcher,

32(2), 3-13.

Spellings, M. (2005). Decision Letter on Request to Amend Virginia

accountability plan. Retrieved November 5, 2005 from

http://www.ed.gov/print/admins/lead/account/letters/acva5.ht

ml.

Stake, R. (1999). The Goods on American Education. Phi Delta Kappan,

668-672.

Stecher, B., Barron, S., Chun, T., & Ross, K. (2000). The effects of the

Washington state education reform on Standards and Student

Testing.

Sum, A. (1999). Literacy in the Labor Force. Washington, DC: National

Center for Education Statistics.

Taylor, G., Shepard, L., & Kinner, F. (2003). A Survey of Teachers’

Perspectives on High-Stakes Testing in Colorado: What Gets

Taught, What Gets Lost (CSE Technical Report No. 588). Santa

Cruz, CA and Los Angeles, CA: Center for Research on Evaluation,

Diversity, and Excellence and CRESST.

Technical Digest, 2003 – 2004

174

Texas Assessments of Knowledge and Skills (TAKS), (2005). Data

Resources and Research. Texas Education Agency (TEA) from

http://www.tea.state.tx.us/data.html.

Thurlow, M. L., Nelson, J. R., Teelucksingh, E., & Ysseldyke, J. E. (2000).

Where’s Waldo? A Third Search for Students with Disabilities in

State Accountability Reports (Technical Report No.25).

Minneapolis, MN: National Center on Educational Outcomes.

Thompson, G. L. The Real Deal on Bilingual Education: Former

Language-Minority Students Discuss Effective and Ineffective

Instructional Practices. Educational Horizons, 78(20, 80-92.

U. S. Department of Education. (2002). Elementary and Secondary

Education. Title IX. General Provisions. Part A, Section 9101.

Definitions. [11] Core Academic Subjects. Washington, D.C.: U.S.

Dept. of Education.

U.S. Department of Education, Office of the Undersecretary, Planning

and Evaluation Service. (1999). Promising results, continuing

challenges: The final report of the national assessment of Title I.

Washington, DC: Author

U.S. Department of Education. (1995). The Improving America’s Schools

Act of 1994: Reauthorization of the Elementary and Secondary

Education Act. Washington, D.C.: U.S. Dept. of Education.

Van Dalen, D.B. (1979). Understanding Educational Research (4th ed.).

New York: McGraw-Hill.

175

Wall, M. E. (2004). A Case Study of Secondary School Efforts Toward

English Language Learner Success in a Standards-Based Reform

System.

Wang, M. & Koda, K. (2005). Commonalities and Differences in Word

Identification Skills among English Second Language Learners.

Language Learning, 55 (1) 71-98.

Weitzman, E.A. (2000). Software and Qualitative Research. In N.K.

Denizin & Y.S. Lincoln (Eds.), The Handbook of Qualitative

Research (2nd ed., pp. 803-820). Thousand Oaks, CA: Sage.

Whoriskey, P. (2006).Political backlash builds over high-stakes testing.

Public support wanes for tests seen as punitive. Washington Post

Company.

Winograd, M. (2002). Equity: A Prerequisite for Reform. Principal


Wright, W. E., (2006).A Catch-22 for Language Learners. Educational

Leadership. November 2006/Volume 64/Number 3.

Zehler, A. M., Fleischman, H. L., Hopstock, P. J., Pendzick, M. L., &

Stephenson, T. G. (2003). Descriptive Study of Services to LEP

Students and LEP Students with Disabilities. (No. 4 Special Topic

Report: Findings on Special Education LEP. students). Arlington,

Va.: Development Associates, Inc.

176

Zehler, A. M., Hopstock, P. J., Fleischman, H. L., & Greniuk, C. (1994).

An Examination of Assessment of Limited English Proficient

Students. Special Issues Analysis Center,Task Order Report.

Arlington, VA: Development Associates.

177

APPENDICES

178

APPENDIX A

IRB

179

PRAIRIE VIEW A&M UNIVERSITY

The Texas A&M University System

P. O. Box 4149

Prairie View, Texas 77446-4149

Office of the Vice President v. 936.857.4494

Research and Development f. 936. 857.2255

March 21, 2006

TO: Mr. Arthur L. Petterway, Principal Investigator

Doctoral Student, Education Leadership and Counseling

Dr. Ben DeSpain, EDLC – Faculty Advisor

FROM: Marcia C. Shelton, Compliance Officer, Regulatory Research

Institutional Review Board

SUBJECT: IRB Protocol Review – Protocol Status

Title: A Mixed Mthods Analysis of the Impact of High Stakes Testing on English Language

Learners in Major Urban High Schools in Texas Protocol Number: 200-103

Review Category: Full Review - (primary reviewer –L. Myers)

Approval Date: February 13, 2006

The approval determination was based on the following Code of Federal Regulations (CFR)

46.101(b) (2). Research involving the use of educational tests (cognitive, diagnostic,

aptitude, achievement), survey procedures, interview procedures or observation of public

behavior, unless: (i) information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and (ii) any disclosure of

the human subjects' responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects' financial standing,

employability, or reputation.

Remarks:

The Institutional Review Board – Human Subjects in Research, Prairie View A&M University

has reviewed and approved the above referenced protocol. Your study has been approved for

one year –February 13, 2006- February 12, 2007. As the principal investigator of this study, you

assume the following responsibilities:

Renewal: Your protocol must be re-approved each year in order to continue the research. You

must also complete the proper renewal forms in order to continue the study after the initial

approval period.

Adverse events: Any adverse events or reactions must be reported to the IRB immediately.

Amendments: Any changes to the protocol, such as procedures, consent/assent forms, addition of

subjects, or study design must be reported to and approved by the IRB.

Completion: When the study is complete, you must notify the IRB office and complete the

required forms.

180

Date: 14-Jan-2006 00:33:10 -0600

From: <[email protected]>

To: <[email protected]>

Subject: Exam Confirmation - Clinical Research Training

Registration Summary:

01/14/2006

Arthur L Petterway Ph.D

Prairie View A&M University

Educational Leadership & Counseling

P. O. Box 519

n/a

MSC

Prairie View TX 77446-0519

United States

Tel: (713)924-1622 Fax: (713)924-1619

E-mail: [email protected]

This e-mail is to verify that you successfully completed the NIH Clinical Research

Training course.

You answered 20 out of a total of 25 questions for a final grade of 80%.

If you are an NIH principal investigator, you have fulfilled the Training and Education

Standard issued by the NIH for conducting clinical research within the intramural

research program.

Please print this e-mail and retain for your records.

181

CITI Course in The Protection of Human Research Subjects

Sunday, February 19, 2006

CITI Course Completion Record for Arthur Petterway

To whom it may concern:

On 1/15/2006, Arthur Petterway (username=apetterw) completed all CITI Program requirements for the Basic CITI Course in The Protection of Human

Research Subjects.

Learner Institution: Texas A&M University

Learner Group: Group 2.

Learner Group Description: Social and Behavioral Research Investigators and Key Personnel

Contact Information:

Gender: Male

Which course do you plan to take?: Social & Behavioral Investigator Course Only

Role in human subjects research: Principal Investigator

Mailing Address:

5300 N. Braeswood Blvd

#247

Houston

Texas

77096

USA

Email: [email protected]

Office Phone: (713)924-1622

Home Phone: (713)498-8667

182

The Required Modules for Group 2. are: Date

completed

Introduction 01/15/06

History and Ethical Principles - SBR 01/15/06

Defining Research with Human Subjects - SBR 01/15/06

The Regulations and The Social and Behavioral Sciences -

SBR

01/15/06

Assessing Risk in Social and Behavioral Sciences - SBR 01/15/06

Informed Consent - SBR 01/15/06

Privacy and Confidentiality - SBR 01/15/06

Research with Prisoners - SBR 01/15/06

Research with Children - SBR 01/15/06

Research in Public Elementary and Secondary Schools - SBR 01/15/06

International Research - SBR 01/15/06

Internet Research - SBR 01/15/06

Conflicts of Interest in Research Involving Human Subjects 01/15/06

Texas A&M University 01/15/06

Additional optional modules completed:

Date

completed

For this Completion Report to be valid, the learner listed above must be

affiliated with a CITI participating institution. Falsified information and unauthorized use of the CITI course site is unethical, and may be considered scientific misconduct by your institution.

Paul Braunschweiger Ph.D. Professor, University of Miami Director Office of Research Education

CITI Course Coordinator

183

APPENDIX B

CONSENT FORM

184

CONSENT FORM

A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES


SCHOOLS IN TEXAS

I have been asked to participate in a research study. This study is

intended to explore what certified English as a Second Language (ESL)

teachers, non-certified ESL teachers who teach English Language

Learners (ELLs), campus administrators , and district ESL personnel

view as the impact that high stakes assessments have on ELLs, ESL

curriculum and instruction , and what they observe as occurring. I was

selected to be a possible participant because I am either a campus

administrator, a teacher handling ELLs or a district ESL coordinator of

the selected major urban high schools in Texas. A total of 118 people

have been asked to participate in this study. The purpose of this study is

to determine the views and opinions of campus administrators, teachers

handling ELLs, and district ESL coordinators regarding the impact of

statewide testing, specifically Texas Assessment of Knowledge and Skills

(TAKS), on the curriculum and instruction of this special group of

students.

If I agree to be in this study, I will be asked to answer an on-line

questionnaire. I may also form part of the focus group which will have

group interviews verifying and validating my views and opinions

regarding the open-ended questions. If I am a principal or a district ESL

coordinator my interview will be one-on-one. The proceedings will be

transcribed so that the researcher can go back to sections for better

understanding. I may volunteer another knowledgeable respondent in

order that the succeeding interviews will be more productive or

meaningful. This study will only take at least two interviews after the on-

line questionnaire has been answered. The risks associated with this

study are almost non-existent. The benefits of participation are my

contributions to the betterment of the ESL curriculum, instruction and

testing.

I will receive no compensation for my participation in this research

study. A simple ‘thank you’ note will suffice.

This study is confidential since the data will be dealt with only by the

researcher and transcripts will be kept in a safe storage. The records of

this study will be kept private. No identifiers linking me to the study will

be included in any sort of report that might be published.

Date _____________ Initial________________

Page 1 of 2

185

Research records will be stored securely and only the researcher, Arthur

L. Petterway, will have access to the records. For the transcripts, only

Arthur L. Petterway will also have access to the information contained

therein. My decision whether or not to participate will not affect my

current or future relations with Prairie View A & M University. If I decide

to participate, I am free to refuse to answer any of the questions that

may make me uncomfortable. I can withdraw at any time without my

relations with the University, job, benefits, etc., being affected. I can

contact Arthur L. Petterway at (713)748-8303 or [email protected]

and Dr. Robert Marshall, at (936)857-4127 [email protected] with

any questions about this study.

This research study has been reviewed by the Institutional Review Board-

Human Subjects in Research, Prairie View A & M University. For

research- related problems or questions regarding subjects’ rights, I can

contact the Institutional Review Board through Ms. Marcia C. Shelton,

Research Compliance Officer, Anderson Hall Room 311, PO Box 4149,

Prairie View, TX 77446, at 936.857.2541 and at [email protected].

I have read the above information. I have asked questions and have

received answers to my satisfaction. I have been given a copy of this

consent document for my records. By signing this document, I consent to

participate in the study.

Signature of the Subject: ______________________________ Date: __________

Signature of Investigator: ______________________________ Date: __________

Original – Researcher

Copy – Participant

Date _____________ Initial________________

Page 2 of 2

186

APPENDIX C

INTERVIEW QUESTIONS

187

INTERVIEW /FOCUS GROUP QUESTIONS

Please help us serve English Languages Learners (ELLs) more

effectively by taking a few moments to answer these interview

questions. Confidentiality will be maintained throughout this process, as

only I will have access to the data. Data and information will be kept in a

safe home vault for a period of seven years, after which time they will be

destroyed. Summarized data will be published in my dissertation. Your

participation is greatly appreciated, as you will be making a significant

contribution to the English as a Second Language (ESL) curriculum,

instruction and testing.

1. Describe your experience with English Language Learners.

2. In your estimation, how many English Language Learners

graduate each year from your school?

3. Do you think this number has increased or decreased in recent

years? Why?

4. Describe your school’s support system for former English

Language Learners.

5. What is your understanding of the Texas Comprehensive

Assessment Plan as it relates to ELLs?

6. Should the English Language Learner be held to a separate

standard of promotion or to the same standards as the regular

population? Why?

188

7. With respect to your school, how do the promotion standards

affect your English Language Learner population? What will it

take to facilitate the English Language Learners ’ success in

meeting graduation standards?

8. What concerns need to be addressed before administrating TAKS

to the English Language Learner? Who should be responsible in

resolving these concerns?

9. Why should English Language Learners be held/or not held liable

or accountable to the same standards or requirements?

10. What do you think are the intended consequences of the statewide

accountability test (TAKS), specifically in terms of ESL curriculum

and instruction?

11. Has the ESL program implemented in schools failed/or served its

purpose or the state’s standardized assessment?

12.How would you rate the success of the ESL program in regards to

standardized assessment?

13. To what extent do the consequences of a statewide assessment

affect the ELL student, educator and district?

14. In what ways, if any, has the curriculum for ESL students

changed as a result of their participation in the assessment?

15.What is your opinion of the cause of the low performing scores of

ELL students?

189

16.What do you need to do to adequately prepare these students for

success with standardized assessments and graduation

standards?

17. What are your recommendations for the future testing of the

English Language Learner?

18.Given that the criteria being used are high-stakes, what additional

supports are needed to ensure that ELLs will be able to meet

them?

19.How do alternative assessments (e.g., Spanish language exams)

compare to mainstream assessments?

20.When is the use of native language assessments appropriate?

21.How does the placement of accommodations impact comparability

with mainstream student performance?

22.Do wide-scale tests with the permitted accommodations fully

assess English Language Learners’ knowledge and abilities or does

the system need to be fully redesigned such that the needs of these

students are addressed in the development of the assessments?

23.Do you think other data collection methods, such as portfolios or

other performance assessments, would yield more accurate results

with regard to ELLs than traditional assessments?

24.What sorts of information is needed to make fair high-stakes

decisions about ELLs (e.g., grades, classroom performance, an

array of samples of student work, teacher recommendations)?

190

25.What would be the most beneficial system(s) of accountability to

ensure that these students are making progress in what they know

and can do in important content areas?

26.What supports are necessary to aid states and districts in their

alignment of assessments, standards, curricula, and instruction?

TAKS = Texas Assessment of Knowledge and Skills

191

APPENDIX D

ON-LINE/HARD COPY QUESTIONNAIRE

192

A MIXED METHODS ANALYSIS OF THE IMPACT OF

HIGH STAKES TESTING ON ENGLISH LANGUAGE

LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN

TEXAS

Please help us serve English Languages Learners (ELLs) more effectively by taking a few moments to fill out this questionnaire. The results will be returned to us automatically via the web. Confidentiality will be maintained throughout this process, as only I will have access to the data. A random numeric code will be electronically generated and assigned to each qualitative questionnaire. Data and information will be kept in a safe home vault for a period of seven years, after which time they will be destroyed. Summarized data will be published in my dissertation. Your participation is greatly appreciated, as you will be making a significant contribution to the English as a Second Language (ESL) curriculum, instruction and testing.

1.) GENDER

MALE

FEMALE

2.) AGE

21-30

31-40

41-50

51-60

61+

3.) CURRENT POSITION

Principal

Assistant Principal

193

ESL Certified Teacher

NON-ESL Certified Teacher (who teach ELLs)

District ESL Personnel

4.) HIGHEST DEGREE EARNED

Bachelors

Masters

Doctorate

5.) What certifications do you hold?

6.) YEARS OF EXPERIENCE IN PUBLIC EDUCATION

1-5

6-10

11-20

21+

7.) What are the anticipated results of the statewide testing,

specifically, English Language Arts (ELA) and Mathematics

portion of the Texas Assessment of Knowledge and Skills

(TAKS), on English Language Learners?

194

8.) What are the actual results of the statewide testing,

specifically, ELA and Mathematics portion of the Texas

Assessment of Knowledge and Skills (TAKS), on English

Language Learners?

9.) Why is the Texas Assessment of Knowledge and Skills

(TAKS) given as a statewide test?

10.) What are the intended consequences of this statewide

testing as it relates to English Language Learners?

11.) What has happened because of TAKS as it relates to

English Language Learners?

12.) What problems have occurred, if any, to the English

Language Learners because of TAKS?

195

13.) What changes have occurred as a result of statewide testing

as it relates to English Language Learners?

14.) What general recommendations would you suggest to

improve the overall performance of English Language Learners

on statewide testing ?

15.) Based on your recommendations above, which will be of

greatest value for ELLs success on statewide testing?

Thank you for taking the time to fill out this questionnaire. If you need to contact us - you can click on the following email [email protected]

Submit Form

Reset Form

-1

196

APPENDIX E

LETTER TO PARTICIPANTS

197

LETTER TO PARTICIPANTS

Arthur L. Petterway 05/12/2006

5300 N. Braeswood Blvd. # 247

Houston, Texas 77096

Dear Sir or Madam,

I am presently in the Ph.D. in Education Leadership program at

Prairie View A & M University. I am currently conducting a dissertation

research on high-stakes testing and English language learners. My

dissertation topic is “A Mixed Methods Analysis of the Impact of High-

Stakes Testing on English Language Learners in Major Urban High

Schools in Texas”. I realize how busy you are in meeting the challenges of

your work but I hope that you will take time to complete an on-line

qualitative questionnaire that I have prepared to gather research data.

You can access the qualitative questionnaire at

[email protected].

The questionnaire will include a major question on what are the

anticipated and observed consequences of the statewide testing,

specifically, Texas Assessments of Knowledge an Skills (TAKS), on

English as a Second Language (ESL) curriculum and instruction as

viewed by certified ESL teachers, non-certified ESL teachers who teach

English Language Learners (ELLs), school administrators, and district

ESL personnel.

Confidentiality will be maintained throughout this process, as only

I will have access to the data. A random numeric code will be

electronically generated and assigned to each qualitative questionnaire.

Data and information will be kept in a safe home vault for a period of

seven years, after which time they will be destroyed. Summarized data

will be published in my dissertation. Your participation is greatly

appreciated, as you will be making a significant contribution to the ESL

curriculum, instruction and testing.

Should you have any questions or comments, please feel free to

contact me at (713)748-8303 or [email protected]. You may

Contact my dissertation chair, Dr. Robert Marshall, at (936)857-4127 or

[email protected]. Once again, I appreciate your attention to this

matter and look forward to your favorable response.

Sincerely,

Arthur L. Petterway

Ph.D. student – Prairie View A & M University

198

APPENDIX F

REQUEST FOR EXTANT DATA FROM TEXAS EDUCATION AGENCY

(TEA)

199

Dear Mr. Rod Rowell,

As per our telephone conversation on July 3, 2006, I am submitting my request for extant data to TEA.

Requestor Name: Arthur L. Petterway

Company Name: PhD Doctoral Student at Prairie View A&M University

Address: 5300 N. Braeswood Blvd. #247

City/State/Zip: Houston, Texas 77096

Telephone: Office-(713)924-1600; Home-(713)748-8303; Cell-(832)693-2809

Fax Number: (713)924-1619

Requestor Email Address: [email protected]

Brief Summary of Request: I am currently working on my dissertation, A Mixed Methods Analysis of the

Impact of High Stakes Testing on English Language Learners in Major Urban High schools in Texas , at

Prairie View A&M University and need your help. This is a Public Information Request to please provide me

with the 2002-2003; 2003-2004; 2004-2005; 2005-2006 TAKS Summary Report-(ALL) for 10th grade

English Language Arts and Mathematics of all of the high schools in the following school districts:

015907 SAN ANTONIO ISD

015910 NORTH EAST ISD

015915 NORTHSIDE ISD

057905 DALLAS ISD

071902 EL PASO ISD

071905 YSLETA ISD

101912 HOUSTON ISD

220901 ARLINGTON ISD

220905 FORT WORTH ISD 227901 AUSTIN ISD.

Your immediate attention to this matter will be greatly appreciated. If you have any questions, please don't hesitate to let me know. Sincerely, Arthur L. Petterway Assistant Principal

HOUSTON INDEPENDENT SCHOOL DISTRICT

STEPHEN F. AUSTIN SENIOR HIGH SCHOOL 1700 DUMBLE HOUSTON,

TEXAS 77023-3195

PH: (713) 924-1600 FAX: (713) 924-1619

ARTHUR L.

PETTERWAY

Assistant Principal

Home of the Mighty Mustangs

200

From: Woli, Urbe [mailto:[email protected]] Sent: Tue 7/11/2006 3:23 PM To: Petterway, Arthur L Cc: PIR; Eaton, Jennifer; Woli, Urbe Subject: PIR # 6541 - Grade 10 Campus TAKS Summary Data

Mr. Arthur Petterway,

The 2003 – 2005 TAKS campus summary data you are requesting (see attached PDF document) are available for download at no cost from our website at http://www.tea.state.tx.us/student.assessment/reporting/taksagg/index.html. The 2006 TAKS campus summary data will be posted at the same location by the end of August, 2006. Due to the large size of the downloadable data files, you will need to use SAS or SPSS to process the data files.

Please let us know if you prefer that we generate the data for you. To enable us provide accurate data; please list all the statistics you need in the data. Contact me at (512) 463-9536 to discuss your data needs. Thank you. Urbe Woli Student Assessment Division Texas Education Agency Tel: (512) 463-9536

201

VITA

ARTHUR L. PETTERWAY

5300 N. Braeswood Blvd, #247

Houston, Texas 77096

EDUCATIONAL HISTORY

Prairie View A&M University, Prairie View, Texas, Ph. D.

in Education Leadership, Expected Graduation Date August, 2007

Prairie View A&M University, Prairie View, Texas, M.Ed.

in Education Administration, August, 1999

Dillard University, New Orleans, Louisiana B.A.

in Biology, June, 1969

CERTIFICATIONS

Administrator - Principal

Teacher - Elementary Mathematics (Grades 1-8)

Teacher - Elementary Self-Contained (Grades 1-8)

EMPLOYMENT HISTORY

2002 - 2007 Assistant Principal, Houston ISD

1996 - 2002 Math Department Chair, Houston ISD

1994 - 1996 Middle School Math Teacher, Houston ISD

1991 - 2006 Upward Bound Coordinator, UH/DT

1989 - 1991 President/CEO, Way Refining, Houston

1978 - 1989 Executive Vice President, Manufacturing,

Houston

1969 - 1978 Manufacturing Supervisor, General

Motors, Dayton, Ohio

A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR...

Education

Transcript of A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR...