Understanding Text Complexity: Grades 9-12

David Abel, Fellow for Curriculum and Assessment, ELA

NTI, December 10, 2014

Agenda

1. Introduction

2. What is text complexity and why is it important?

i.Quantitative Measuresii.Qualitative Measuresiii.Reader and Task

3. What are the characteristics of complex text?

4. Ways to determine if a text is sufficiently complex 5. Group examination of texts and determination of complexity

6. What to do about text complexity

What Makes a Text Complex?

“Although it was winter, the nearest ocean four hundred miles away, and the Tribal Weatherman asleep because of boredom, a hurricane dropped from the sky in 1976 and fell so hard on the Spokane Indian Reservation that it knocked Victor from bed and his latest nightmare.”

“Every Little Hurricane” from The Lone Ranger and Tonto Fistfight in Heaven by Sherman Alexie

What Makes a Text Complex?

In this example…•Long sentence(s) and complex syntax•Multiple and/or subtle themes and purposes•Lack of words, sentences or paragraphs that review or pull things together for the student•Unfamiliar settings, topics or events

What Makes a Text Complex?

“Society in every state is a blessing, but Government, even in its best state, is but a necessary evil; in its worst state an intolerable one: for when we suffer, or are exposed to the same miseries BY A GOVERNMENT, which we might expect in a country WITHOUT GOVERNMENT, our calamity is heightened by reflecting that we furnish the means by which we suffer.”

--Thomas Paine, Common Sense

What Makes a Text Complex?

“Call me Ishmael. Some years ago—never mind how long precisely—having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world. It is a way I have of driving off the spleen and regulating the circulation. Whenever I find myself growing grim about the mouth; whenever it is a damp, drizzly November in my soul; whenever I find myself involuntarily pausing before coffin warehouses, and bringing up the rear of every funeral I meet; and especially whenever my hypos get such an upper hand of me, that it requires a strong moral principle to prevent me from deliberately stepping into the street, and methodically knocking people's hats off—then, I account it high time to get to sea as soon as I can. This is my substitute for pistol and ball…”

--Herman Melville, Moby Dick.

CCLS Recommendations Regarding Text Complexity

• The CCR (College and Career Readiness) standards anchor the document and define general, cross-disciplinary literacy expectations that must be met for students to be prepared to enter college and workforce training programs ready to succeed.

• The K–12 grade-specific standards define end-of-year expectations and a cumulative progression designed to enable students to meet college and career readiness expectations no later than the end of high school.

• The CCR and high school (grades 9–12) standards work in tandem to define the college and career readiness line—the former providing broad standards, the latter providing additional specificity.

7

Why reading complex text is important

• Facility with complex text strongly predicts ability to an earn a “B” grade or better) in first year college courses. (ACT, 2006.)

• Students who come to school without a solid base of background knowledge need increasingly complex texts to build their schema about the world.

• The texts that students read in K-12 have become easier, but college texts have not (and instruction in College is significantly less scaffolded.)

• The texts that students read in K-12 are often not expository, but the texts that they read in college are mostly expository.

• What instruction students have had with complex expository texts has been largely superficial/strategy based (skimming, focusing on details, not forming a full picture/deep understanding.)

8

CCLS Reading Standards Literature Grades 11-12

Range of Reading and Level of Text Complexity RL.11-12.10

•By the end of grade 11, read and comprehend literature, including stories, dramas, and poems, in the grades 11-CCR text complexity band proficiently, with scaffolding as needed at the high end of the range.•By the end of grade 12, read and comprehend literature, including stories, dramas, and poems, at the high end of the grades 11-CCR text complexity band independently and proficiently.

CCLS Reading Standards Informational text Grades 11-12

Range of Reading and Level of Text Complexity RI.11-12.10

•By the end of grade 11, read and comprehend literary nonfiction in the grades 11–CCR text complexity band proficiently, with scaffolding as needed at the high end of the range.• By the end of grade 12, read and comprehend literary nonfiction at the high end of the grades 11–CCR text complexity band independently and proficiently.

CCLS Reading Standards for Informational Text Grades 11-12

Integration of Knowledge and Ideas

RI 11-12.7 Integrate and evaluate multiple sources of information presented in different media or formats (e.g., visually, quantitatively) as well as in words in order to address a question or solve a problem.

RI 11-12.8 Delineate and evaluate the reasoning in seminal U.S. texts, including the application of constitutional principles and use of legal reasoning (e.g., in U.S. Supreme Court majority opinions and dissents) and the premises, purposes, and arguments in works of public advocacy (e.g., The Federalist, presidential addresses)

RI 11-12.9 Analyze seventeenth-, eighteenth-, and nineteenth-century foundational U.S. documents of historical and literary significance (including The Declaration of Independence, the Preamble to the Constitution, the Bill of Rights, and Lincoln’s Second Inaugural Address) for their themes, purposes, and rhetorical features.

Determining text complexity

Explanation of the Text Complexity Factors

• Quantitative evaluation of the text Readability measures and other scores of text complexity

• Qualitative evaluation of the text Levels of meaning, structure, language conventionality and

clarity, and knowledge demands

• Matching reader to text and task Reader variables (such as motivation, knowledge, and

experiences) and task variables (such as purpose and the complexity generated by the task assigned and the questions posed)

Is there one way to determine text complexity?

No, determining text complexity is a complex act!

•Text complexity does not reside in the text, rather it is the product of the interaction between the text, the reader, and the purpose for reading. The What, the Who, the Why and the Where and When.

What—the characteristics of the text—qualitative/quantitative evaluation

Who—the characteristics of the reader (reading ability/age/experience/ motivation)

Why—the purpose for the reading (assignment/analysis of writers craft/ research/pleasure)

Where and When—the conditions of the reading—teacher/parent guided, group discussion/literary circle, independent (classroom/home/library/computer), instruction vs assessment, etc.

Text Complexity: Quantitative Measures

There is no single method for determining text complexity.

• The quantitative dimension of text complexity refers to those aspects that are difficult for a human reader to evaluate when examining a text.

• These factors are more efficiently measured by computer programs.

Text Complexity: Quantitative Measures

• Quantitative measures of text complexity generally measure measures of word difficulty (frequency, length) and sentence length.

• Some metrics add other features of words, sentence syntax, and text cohesion, creating a broader range of text and linguistic measures.

Free Quantitative Measures of Text Complexity

ATOS Analyzer from Renaissance Learning: http://www/renlearn.com/ar/overview/atos/

Degrees of Reading Power from Questar: http://www/questarai.com

The Lexile Framework from Metamatrix: http://www.lexile.com/analyzer

Coh-Metrix Easability Tool from University of Memphis: http://141.225.42.101/cohmetrixgates/Home

Reading Maturity from Pearson Knowledge Technologies: http://www.readingmaturity.com (Beta site)

SourceRater from Educators Testing Service: http://naeptba.ets.org/SourceRater3/ (Beta site)

Flesch-Kincaid (part of your Microsoft Word)

Text Complexity Grade Bands and Associated Ranges from Multiple Measures

Common Core Band ATOS

Degrees of ReadingPower

Flesch-Kincaid

TheLexile Framework

ReadingMaturity SourceRater

2nd-3rd 2.75 – 5.14 42 – 54 1.98 – 5.34 420 – 820 3.53 – 6.13 0.05 – 2.48

4th-5th 4.97 – 7.03 52 – 60 4.51 – 7.73 740 – 1010 5.42 – 7.92 0.84 – 5.75

6th-8th 7.00 – 9.98 57 – 67 6.51 – 10.34 925 – 1185 7.04 – 9.57 4.11 – 10.66

9th-10th 9.67 – 12.01 62 – 72 8.32 – 12.12 1050 – 1335 8.41 – 10.81 9.02 – 13.93

11th-CCR 11.20 – 14.10 67 – 74 10.34 – 14.2 1185 – 1385 9.57 – 12.00 12.30 – 14.50

Source: CCLS Appendix A

What are the Features of Complex Text?

• Subtle and/or frequent transitions

• Multiple and/or subtle themes and purposes

• Density of information

• Unfamiliar settings, topics or events

• Lack of repetition, overlap or similarity in words and sentences

• Complex sentences

• Uncommon vocabulary

• Lack of words, sentences or paragraphs that review or pull things together for the student

• Longer paragraphs

• Any text structure which is less narrative and/or mixes structures

19

Text Complexity: Qualitative Measures

Text Complexity: Qualitative Measures

Practice Evaluation of Text Complexity

In pairs or triads, read literature text #1 (Dickens) and discuss where it might fit in terms of the following qualitative criteria in the text complexity rubric•Meaning•Text Structure•Language Features•Knowledge Demands

THEN, using the quantitative measures provided, place it in a grade band for instruction and assessment….

Practice Evaluation of Text Complexity: DICKENS

 Criteria

Very Complex Moderately Complex Readily Accessible Notes

  

Meaning  

Multiple levels of meaning that may be difficult to identify, separate, and interpret; theme is implicit, subtle, or ambiguous and may be revealed over the entirety of the text.

  Multiple levels of meaning that are relatively easy to identify; theme is clear but may be conveyed with some subtlety.

  

One level of meaning: theme is obvious and revealed early in the text.

 

Moderately Complex

Multiple levels of meaning that are relatively easy to identify; theme is clear but may be conveyed with some subtlety.

NOTES: By beginning with Marley's death, the theme is arrived at in a non-linear manner and conveyed with some subtlety.

Practice Evaluation of Text Complexity: DICKENS

 Criteria

Very Complex Moderately Complex Readily Accessible Notes

  

Text Structure

 

Prose or poetry includes more intricate elements such as subplots, shifts in point-of-view, shifts in time or non-standard text structures.

  Prose includes two or more storylines or has a plot that is somewhat difficult to predict (e.g.: in the case of a non-linear plot); poetry has some implicit or unpredictable structural elements.

Prose or poetry is organized clearly and/or chronologically; the events in a prose work are easy to predict because the plot is linear; poetry has explicit and predictable structural elements.

 

Moderately Complex

Prose includes two or more storylines or has a plot that is somewhat difficult to predict (e.g.: in the case of a non-linear plot); poetry has some implicit or unpredictable structural elements.

NOTES:Again, by beginning with Marley's death, and including diversions about idioms, and given the stress on the death without revealing its import, the author includes some non-linear/discursive elements that add to the text’s structural complexity.

Practice Evaluation of Text Complexity: DICKENS

 Criteria

Very Complex Moderately Complex Readily Accessible Notes

  Language Features   

Language is generally complex with abstract, ironic, and/or figurative language, and regularly includes archaic, unfamiliar, and academic words; text uses a variety of sentence structures including complex sentences with subordinate phrases and clauses.

    

Language is often explicit and literal but includes academic, archaic, or other words with complex meaning (e.g.: figurative language); text uses a variety of sentence structures. 

  Language is explicit and literal, with mostly contemporary andfamiliar vocabulary; text uses mostly simple sentences.

 

Very Complex

Language is generally complex with abstract, ironic, and/or figurative language, and regularly includes archaic, unfamiliar, and academic words; text uses a variety of sentence structures including complex sentences with subordinate phrases and clauses.

NOTES:Text includes some ironic, figurative language and discussion of figurative language, archaic language and references, and a variety of sentence structures.

Practice Evaluation of Text Complexity: DICKENS

 Criteria

Very Complex Moderately Complex Readily

Accessible Notes

  

Knowledge Demands

  

The text explores complex sophisticated or abstract themes; text is dependent on allusions to other texts or cultural elements; allusions or references have context and require inference and evaluation.

   

The text explores several themes; text makes few references or allusions to other texts or cultural elements; the meaning of references or allusions may be partially explained in context.

   

The text explores a single theme; if there are any references or allusions, theyare fully explained in the text.

 

Very Complex Moderately Complex

The text explores complex sophisticated or abstract themes; text is dependent on allusions to other texts or cultural elements; allusions or references have context and require inference and evaluation.

The text explores several themes; text makes few references or allusions to other texts or cultural elements; the meaning of references or allusions may be partially explained in context.

NOTES:The text makes multiple allusions to other texts/cultural elements, including "the Country", "Hamlet," "Nature lived hard by," some of these references can be partially explained in context.

Practice Evaluation of Text Complexity: DICKENS

• OVERALL QUANTITATIVE COMPLEXITY RATINGS

• OVERALL QUALITATIVE COMPLEXITY RATING AND PLACEMENT

Very Complex/Moderately Complex

Appropriate for 9-10 instruction/ 11-12 assessment

Metrics Measures Grade BandLexile: 1020 6-8Flesch-Kincaid: 6.1 4-5Reading Maturity Metric:

8.6 6-8

Group Evaluation of Text Complexity

In pairs or triads, review informational text #1 on page 14 (Red Jacket ) and use the qualitative rubric to determine the ratings in each of the qualitative criteria:•Meaning•Text Structure•Language Features•Knowledge DemandsTHEN, using the quantitative measures provided, place it in a grade band for instruction and assessment….

NOTE: the criteria are different for informational text….

Practice Evaluation of Text Complexity: RED JACKET

 Criteria

Very Complex Moderately Complex Readily Accessible Notes

  

Meaning  

The text contains multiple purposes, and the primary purpose is subtle, intricate, and or abstract.

  The primary purpose of the text is not stated explicitly but is easy to infer based upon context or source; the text may include multiple perspectives.

 

The primary purpose of the text is clear, concrete, narrowly focused, and explicitly stated; the text has a singular perspective.

 

Purpose of text is focused but due in part to rhetorical style, it is not always indicated in an explicit manner.

  

Text Structure

 

Connections among an expanded range of ideas, processes, or events are often implicit, subtle, or ambiguous; organization exhibits some discipline-specific traits; any text features are essential to comprehension of content.

  Connections between some ideas, processes, or events are implicit or subtle; organization is generally evident and sequential; any text features help facilitate comprehension of content.

  

Connections between ideas, processes, and events are explicit and clear; organization is chronological, sequential, or easy to predict because it is linear; any text features help readers navigate content but are not essential to understanding content.

  Connections between ideas and events are subtle, though always framed within the primary purpose; organization is also framed around purpose but isn't strictly sequential. The rhetorical style adds to the complexity.

  

Language Features

   

Language is generally complex, with abstract, ironic, and/or figurative language, and archaic and academic vocabulary and domain-specific words that are not otherwise defined; text uses many complex sentences with subordinate phrases and clauses.

  Language is often explicit and literal but includes some academic, archaic, or other words with complex meaning; text uses some complex sentences with subordinate phrases or clauses. 

  

Language is explicit and literal, with mostly contemporary and familiar vocabulary; text uses mostly simple sentences.

  Some complex sentences with subordinate clauses, as well as some archaic and complex words, as well as the use of rhetorical devices such as questioning.

  

Knowledge Demands

  

The subject matter of the text relies on specialized, discipline-specific knowledge; the text makes many references or allusions to other texts or outside areas, allusions or references have no context and require inference.

  The subject matter of the text involves some discipline-specific knowledge; the text makes some references or allusions to other texts or outside ideas; the meaning of references or allusions may be partially explained in context.

  

The subject matter of the text relies on little or no discipline-specific knowledge; if there are any references or allusions, they are fully explained in the text.

  Some knowledge of the conflicts between Native Americans and colonists is helpful, though not essential to making meaning of the text.

Use of Graphics (optional)

Graphics are essential to understanding the text; they may clarify or expand information in the text and may require close reading and thoughtful analysis in relation to the text.

  Graphics are mainly supplementary to understanding of the text; they generally contain or reinforce information found in the text.

  Graphics are simple and may be unnecessary to understanding the text.

  N/A

Quantitative Analysis Briefly explain recommended placement        

Lexile 1130Flesch-Kincaid: 10.3Reading Maturity Metric RMM:

11.7

Final Placement 

Complexity Level : MC 

Practice Evaluation of Text Complexity: RED JACKET

• OVERALL QUANTITATIVE COMPLEXITY RATINGS

• OVERALL QUALITATIVE COMPLEXITY RATING AND PLACEMENT

Moderately Complex

Appropriate for 9-10 instruction/ 11-12 assessment

Metrics Measures Grade BandLexile: 1130 6-8, 9-10

Flesch-Kincaid: 10.3 6-8, 9-10

Reading Maturity Metric:

11.7 9-10

Practice Evaluation of Text Complexity

What about Literature text # 3 (Kafka) or #5 (Poe)?

What about informational text # 3 (Thoreau) or #4 (Sagan?)

How do these texts feature specific aspects of qualitative text complexity ?

For example: Kafka in terms of language features; Thoreau in terms of structure and/or meaning

Poe

“The margin of the river, and of the many dazzling rivulets that glided through devious ways into its channel, as well as the spaces that extended from the margins away down into the depths of the streams until they reached the bed of pebbles at the bottom, these spots, not less than the whole surface of the valley, from the river to the mountains that girdled it in, were carpeted all by a soft green grass, thick, short, perfectly even, and vanilla-perfumed, but so besprinkled throughout with the yellow buttercup, the white daisy, the purple violet, and the ruby-red asphodel, that its exceeding beauty spoke to our hearts in loud tones of the love and of the glory of God.”

WHAT NEXT?

• What are your key take-aways about complex text?

• What are the best ways to prepare students for reading complex text?

Lots of reading Students must build their academic vocabulary Students must build their knowledge about the world

Text Sets can be a good way to get at all three of these….

Importance of Knowledge

“It's true that knowledge gives students something to think about, but a reading of the research literature from cognitive science shows that knowledge does much more than just help students hone their thinking skills:It actually makes learning easier. Knowledge is not only cumulative, it grows exponentially. Those with a rich base of factual knowledge find it easier to learn more…”

--Daniel Willingham, “How Knowledge Helps,” American Educator

Importance of Vocabulary

“In four years, an average child in a professional family would accumulate experience with almost 45 million words, an average child in a working-class family 26 million words, and an average child in a welfare family 13 million words.”

“The Early Catastrophe: the 30 Million Word Gap by age 3”, Hart and Riley

Hart & Risley (1995) “The 30 Million Word Gap”

What do you think the gap will look like in 10 years?

What To Do About Vocabulary and Knowledge

“Building knowledge systematically in English language arts is like giving children various pieces of a puzzle in each grade that, over time, will form one big picture…”

The More Words You Know the More Words You Learn

• Most words are learned from reading (Nagy and Anderson 1985, Baumann & Kameenui 1991, Beck and McKeown 1991, Graves 1986)

• The more words you know the easier it is to learn new words (Stanovich 1986, Adams 2009)

The More You Knowledge You Have, the More You Can Learn

• Reading remains the most efficient way to grow knowledge (Adams 2009, Kintsch 1998)

• The growth of knowledge works the same as vocabulary; students from less affluent homes come to school with less knowledge. Thus each year students from more affluent homes acquire more knowledge.

Resources for Building Text Sets

• Book on “Teaching with Text Sets” http://bit.ly/Teaching_with_Text_Sets

• American Reading Company Webinar on Finding Texts at Different Lexile Levels: http://www.americanreading.com/

• Novel New York (NYS library databases, many searchable by lexile): http://novelnewyork.org/

• Light Sail: http://www.lightsailed.com/• Curriculet: http://www.curriculet.com• Starting in the spring full text sets will begin to be loaded up on

www.achievethecore.org• Nonfiction Text Sets from TC Reading and Writing Project

http://readingandwritingproject.com/resources/book-lists-classroom-libraries-and-text-sets-for-students/text-sets.html

Resources for Building Text Sets

• ELA SCASS Navigating Text Complexity http://www.ccsso.org/Navigating_Text_Complexity/

Showroom_Models.html

• ReadWorks http://www.readworks.org/

• Reading A-Z http://www.readinga-z.com/

• Webinar on Finding Texts at Different Lexile Levels: http://achievethecore.org/page/927/webinar-on-finding-texts-

hidden-resources-at-your-fingertips-detail-pg

• Nonfiction Text Sets from TC Reading and Writing Project http://readingandwritingproject.com/resources/book-lists-

classroom-libraries-and-text-sets-for-students/text-sets.html

Resources for Direct Vocabulary Instruction

• “Wordly Wise” published by Educators Publishing Service (EPS)

• Bringing Words to Life, Beck and McKeownwww.textproject.org

• Marzano Six Step Program http://www.ncresa.org/docs/PLC_Secondary/Six_Step_Process.pdf

• “Word Nerds” - Stenhouse Publishers• Frayer model (all over the internet) • Words Their Way

http://www.pearsonhighered.com/educator/series/Words-Their-Way-Series/10888.page

QUESTIONS?

THANK YOU

DAVID ABELDavid.Abel@nysed.gov