Depth of Knowledge and Cognitive Demand

QualityCore Professional Development

Day 1, 2–1

Day 1, 2–2

Essential Question

How can we know that our expectations of students are at a level of rigor that prepares them for the End-of-Course Assessment and for success in college or the workplace?

Day 1, 2–2

Day 1, 2–3

By the conclusion of this segment, you will be able to: Explain the differences between the cognitive

levels of the depth-of-knowledge (DOK) framework Analyze sample test items for depth of knowledge

and compare those items across levels to differentiate the level of thinking required

Use the QualityCore Test Blueprint to review the percent of items found at each depth-of-knowledge level and discuss the classroom assessment implications of that distribution

Learning Objectives

Day 1, 2–4

A framework for evaluating curriculum, standards, and assessments so they can be studied for alignment

Focuses on content and cognitive demand of test items, instructional strategies, and performance standards

What is Depth of Knowledge?

Day 1, 2–5

Items need to measure the content described in the ACT Course Standards,

And they need to measure them at the performance levels specified,

So that the intended curriculum (e.g., as specified in ACT Course Standards) aligns with enacted curriculum and assessed curriculum (e.g., end-of-course tests).

Alignment

Day 1, 2–6

An item measuring main idea may be at a low level of cognitive demand:

~ What happened in this story? Or it can be at a high level of

cognitive demand:~ What is the theme of this passage?

In English

Day 1, 2–7

First introduced in response to No Child Left Behind, where states were required to show that their standards, curriculum, and assessments were aligned with equivalent breadth and depth

Initially developed in collaboration with Council of Chief State School Officers (CCSSO)

Now widely used across the country

The Depth-of-Knowledge Framework

Day 1, 2–8

Level 1 measures Recall at a literal level. Level 2 measures a Skill or Concept at an

interpretive level. Level 3 measures Strategic Thinking at

an evaluative level. Level 4 measures Extended Reasoning.

Webb (2002)

QualityCore and DOK 4 Levels

Day 1, 2–9

Requires recall of information such as a fact, term, definition, or a simple procedure

Requires students to demonstrate a rote response or perform a simple procedure

DOK Level 1Recall: Literal

Day 1, 2–10

Find facts and details Identify figurative language Label simple parts of speech Recall elements and details of passage structure

(e.g., plot, sequence, setting, character) Use appropriate grammar, punctuation,

capitalization, and spelling

DOK Level 1: English

Day 1, 2–11

Who is Hamlet’s mother? What is the setting for the beginning

of the play Hamlet? What happens right after Laertes is

fatally wounded?

Sample Questions Level 1

Day 1, 2–12

Persuasive Language and Logic~ Locate important details and facts that

support ideas, arguments, or inferences in increasingly challenging text . . .

Writing~ Correctly spell commonly

misspelled/confused words

Sample Level 1 ACT Course Standards

Day 1, 2–13

What is another ACT Course Standard that is Level 1?

What kinds of questions could you ask in class discussion or on an exam to reflect the Level 1 standard you found?

Do You See Any Others?

Day 1, 2–14

15–25% of the questions on the English 9 End-of-Course Assessment are Level 1.

http://qc.vantage.com/qualitycore/do/log

English 9 Test Blueprint

Day 1, 2–15

Requires mental processing beyond recall or reproducing an answer

Students must make some decisions about how to approach a problem

Cognitive demands are more complex than

in Level 1

DOK Level 2 Skill/Concept: Interpretive

Day 1, 2–16

Use context clues to determine the meaning of a word

Predict a logical outcome based on information in the story

Summarize major events, ideas, and pertinent details in a text

Use organizational strategies to structure written work

DOK Level 2: English

Day 1, 2–17

What does the word vagabond mean in sentence 3 of the story?

What is likely to happen next, given the events described in the story?

What is the lesson of this fable? What does the narrator reveal about her

mother in the first paragraph? Write a paragraph summarizing

paragraphs 1–3 in the text.

Sample Questions Level 2

Day 1, 2–18

Apply knowledge of Greek, Latin, and Anglo-Saxon affixes, inflections, and roots to understand unfamiliar words . . .

Summarize and paraphrase information . . . identifying key ideas . . .

Distinguish between fact and opinion

Sample Level 2 ACT Course Standards

Day 1, 2–19

What is another ACT Course Standard that is Level 2?

What kinds of questions could you ask to reflect the standard?

Do You See Any Others?

Day 1, 2–20

50–70% of the questions on the English 9 End-of-Course Assessment are Level 2.

Test Blueprint for English 9

Day 1, 2–21

Requires planning, thinking, explaining, justifying, using evidence, conjecturing, and postulating

Cognitive demands are complex and abstract

DOK Level 3Strategic Thinking: Evaluative

Day 1, 2–22

Describe author’s purpose and how it affects the interpretation of text

Analyze and describe the characteristics of a passage

Infer across an entire passage Identify abstract themes Analyze and synthesize information Use voice appropriate to the purpose and

audience Edit writing to produce a logical progression

of ideas

DOK Level 3: English

Day 1, 2–23

How does Edgar Allen Poe create a feeling of suspense in his poem “The Raven”? How does this differ from the way Simenon creates suspense in his story about Inspector Maigret?

What makes John a true tragic figure in this passage?

What does the author do to make you believe Mary’s story?

Rewrite this story from the point of view of the brother.

Sample Questions Level 3

Day 1, 2–24

Explain the effects of the author’s life upon his or her work (e.g., Alexander Solzhenitsyn’s experience in the gulag as reflected in his novel One Day in the Life of Ivan Denisovich)

Critique the treatment and scope of ideas from multiple sources on the same topic . . .

Sample Level 3 ACT Course Standards

Day 1, 2–25

What is another ACT Course Standard that is Level 3?

What kinds of questions could you ask to reflect the standard?

Do You See Any Others?

Day 1, 2–26

15–25% of the questions on the English 9 End-of-Course Assessment are Level 3.

Test Blueprint for English 9

Day 1, 2–27

Requires complex reasoning, planning, developing, and thinking

Usually requires extended time Requires higher-order thinking

DOK Level 4 Extended Thinking

Day 1, 2–28

Students take information from at least one passage and are asked to apply this information to a new task. They may also be asked to develop hypotheses and perform complex analyses of the connections among texts. Some examples that represent but do not constitute all of Level 4 performance are:

Analyze and synthesize information from multiple sources.

Examine and explain alternative perspectives across a variety of sources.

Describe and illustrate how common themes are found across texts from different cultures.

Level 4 Reading

Day 1, 2–29

The standard at this level is a multiparagraph composition that demonstrates synthesis and analysis of complex ideas or themes. There is evidence of a deep awareness of purpose and audience. For example, informational papers include hypotheses and supporting evidence. Students are expected to create compositions that demonstrate a distinct voice and that stimulate the reader or listener to consider new perspectives on the addressed ideas and themes. An example that represents but does not constitute all of Level 4 performance is:

Write an analysis of two selections, identifying the common theme and generating a purpose that is appropriate for both.

Level 4 Writing

Day 1, 2–30

What kinds of assignments and standards are Level 4?

Level 4 Assignments

Day 1, 2–31

Find the sample test items in your Participant Handbook.

Fold over the page with the key for now. Take the test and indicate the depth-of-

knowledge level of each question. Check your answers against the key.

DOK and Assessment

Day 1, 2–32

What are three ways you can use the depth-of-knowledge framework to increase the rigor of your instruction, your assignments, and your assessments?

How Can You Use This Framework?