NAEP The Nation’s Report Card: The Urban District

By: Katie UrbanczykChris Vanderhorst Russell Marzullo Jessica Hartle

http://youtube.com/v/eJTxtPJNZX8

Summary/NAEP

● 10-13 sections each including up to 10 multiple choice and a constructed response question.

● First implemented in 1964 (NAEP) ● Tests 4th and 8th graders● Acts as a “national” achievement marker

Summary

● TUDA uses the same test, but looks at specific groups and compares Urban to large cities and the nation. States and informs the public of Urban Education

● TUDA created in 2002● 18 schools participated from (2009-2011) 21

schools participated since

TUDA

● Trial Urban District Assessment ● Wanted to analyze progress of large

urban districts specifically● Assessments based off a national

standard and governed by National Assessment Governing Board

TUDA Reading Framework

● Based off scientific reading research in which reading is a dynamic cognitive process

● Understanding written texts● Developing and interpreting meaning● Using this meaning in the correct context (text, purpose,

etc.)● Two texts for assessment: Literary and informational● Cognitive targets: Locate and Recall, Integrate and

Interpret, and Critique and Evaluate.

Trial Urban District Assessment Writing

In 2007, 145,000 students in 45 states and Department of Defense schools participated in this measurement of writing skill.

2011 8 W4 1 ECR Medium Story about exploration of remote island

2011 8 W7 1 ECR Medium Article explaining change in thoughts or feelings

2011 8 W16

1 ECR Medium Letter persuading principal to require/not require community service

2011 12 W1 1 ECR Medium Story or personal narrative about real/imagined difficult choice

2011 12 W14

1 ECR Medium Essay about technology important to student

2011 12 W18

1 ECR Medium Letter persuading council to build/not build convenience store

2007 8 W5 1 ECR Medium William Carlos Williams poem about missing plums

2007 8 W14

1 ECR Medium Describe what a backpack is and how it functions

2007 8 W19

1 ECR Medium Letter to friend to convince them to try something new

Sample Writing Questions

Expectations for Writing

To Persuade, in order to change the readers point of view or affect the

reader’s action.To explain, in order to expand the

reader’s understanding.To convey experience, (real or

imagined) in order to communicate real or imagined experiences to others.

Basic - 120Proficient - 173

Eighth-grade students writing at the Proficient level should be able to develop responses that

clearly accomplish their communicative purposes. Their texts should be coherent and well structured, and they should include appropriate connections and transitions. Most of the ideas in the texts should be developed logically, coherently, and effectively. Supporting details and

examples should be relevant to the main ideas they support, and contribute to overall communicative effectiveness. Voice should be relevant to the tasks and support

communicative effectiveness. Texts should include a variety of simple, compound, and complex sentence types combined effectively. Words and phrases should be chosen

thoughtfully and used in ways that contribute to communicative effectiveness. Solid knowledge of spelling, grammar, usage, capitalization, and punctuation should be evident throughout the

texts. There may be some errors, but these errors should not impede meaning.

Advanced - 211

http://nces.ed.gov/nationsreportcard/writing/achieve.aspx#grade8

Participating DistrictsAlbuquerque Public SchoolsAtlanta Public SchoolsAustin Independent School DistrictBaltimore City Public SchoolsBoston Public SchoolsCharlotte-Mecklenburg SchoolsChicago Public SchoolsCleveland Metropolitan School DistrictDallas Independent School DistrictDetroit Public SchoolsDistrict of Columbia Public Schools

Fresno Unified School DistrictHillsborough County (FL) Public SchoolsHouston Independent School DistrictJefferson County Public Schools (Louisville, KY)Los Angeles Unified School DistrictMiami-Dade County Public SchoolsMilwaukee Public SchoolsNew York City Department of EducationSan Diego Unified School DistrictSchool District of Philadelphia

Measuring Scale

Below BasicBasicProficient Advanced

Accomodations

Many of the same accommodations are made for students with disabilities and English Language Learners as are provided for state tests. However, only the instructions can be read aloud.

Results/Findings

The results for each of the districts is made public and compares that district to large cities and the nation as a whole.

Cleveland Schools:http://nces.ed.gov/nationsreportcard/subject/publications/dst2013/pdf/2014467XV4.pdf

Results/Findings

● 6 of the 21 schools did better this year in math in at least one grade level

● 5 of the 21 schools did better this year in reading in at least one grade level

● Scores for 5 urban districts were higher than public large city counterparts in the 2011 reading assessment

● Teachers who had students read allowed more frequently scored lower than those who did not

http://www.edexcellence.net/

How does the info agree?

● Order of test questions may agree with Connectionism-could be reflective of law of readiness (way

questions were introduced)● The amount of testing done is minimal and may align

with Automatic Information-Processing Model-very few questions (focuses on attention and limited capacity)

● Aligns with bottom-up processing -it is about what info is on the page

How does the info disagree?

● Test questions go against associationism-not every student has background from the story

● Test does not account for Emergent Literacy Theory ideas (listening, talking, reading, etc. are interrelated)

-literacy development starts at birth and is ongoing-literacy rich home environments

● Family Literacy Theory will share similar characteristics of Emergent Literacy Theory (involvement of parents in school and at home)

Problems

● Although TUDA does compare and evaluate performance gains it does not effectively prescribe reasoning as to how to improve reading gains with specifics to urban students. Family Literacy Theory suggests that parents need to be active in the reading process.

● The Matthew Effect states that when a student knows they are doing poorly in a subject they will continue to do so. The students in these districts know they are in relation to the rest of the nation.

Strengths

● How the tests are split up● Short test taking time (broken up into 25

minutes)● Allows schools to determine performance

achievement● Tests and standards are national ● Helps fund schools ● Looks for trends across percentiles

Weaknesses ● Sample size of students is too small (350,000 per grade)● Too few districts participate● Ignores smaller urban schools and schools that don’t meet requirements● Large sample size coming from Title I schools● Scores across subjects are not easily comparable ● Does not account for background knowledge● Results are very limiting to determine any cause and effect relationships● Excluding of students that are SD and ELL in some situations● Accommodations are limited and go against state assessments (reading

aloud)● White-Black score gaps with hypocritical idea of Urban (50% minority)*● Does not align with what everyone is studying● Test given only every other year● Too few test questions are given

Recommendations in terms of literacy theories● Use generally applicable test questions that students

can draw conclusions from (different for certain areas)● Give teachers a topic for the test question (story topic)

-this will put students at a more even level for those without background knowledge-associationism

● Emphasize involvement of reading at home and involvement of parents for future generations to “close the gap”-emergent lit. theory and family lit. theory

● Account for social background differences*-make texts generally known across different social backgrounds.

Recommendations-Opinions● Stop publishing controversial data aligned with specific

areas and making unrealistic comparisons-Matthew Effects

● Be more accommodating for ELL students and students with disabilities

● Stay consistent in terms of what is being tested● Test more often or stop the test● Include more Urban Districts that are not categorized by

minorities● Stop comparing among ethnicities/demographics

Bibliography CPR. (2010). The Civil Rights Project. Retrieved from http://civilrightsproject.ucla.edu/Institute of Education Sciences. (2011). Reading 2011. The Nation’s Report Card. Retrieved from http://nces.ed.gov/nationsreportcard/pdf/dst2011/2012455.pdfInstitute of Education Sciences. (2014). Achievement Gaps. National Assessment of Educational Progress. Retrieved from http://nces.ed.gov/nationsreportcard/studies/gaps/The Nation’s Report Card. (2013). 2013 Mathematics and Reading: Trial Urban District Assessment. Retrieved from http://nationsreportcard.gov/reading_math_tuda_2013/#/Tracey, D. H. & Morrow, L. M. (2012). Lenses on Reading: An Introduction to Theories and Models.( 2nd ed.) New York: The Guilford Press.Ohio State Board of Education & The Kirwan Institute for the study of race and Ethnicity. (2011). Diversity Strategies for Successful Schools. Department of Education. Retrieved from http://education.ohio.gov/getattachment/State-Board/State-Board-Reports-and-Policies/Diversity-Strategies-Policy/Diversity-Strategies-Report-Final-Recommendations-9-12-11.pdf.aspx