Literacy for Learning Maximizing Hawaii’s ASSESSMENT ITEM BANK Understanding the Item Bank: Beyond...

Literacy for Learning

Maximizing Hawaii’sMaximizing Hawaii’sASSESSMENT ITEM BANKASSESSMENT ITEM BANK

Understanding the Item Bank: Understanding the Item Bank: Beyond NavigationBeyond Navigation

SY 2010 - 2011SY 2010 - 2011


Learning Road TripLearning Road Trip

• Overview: Setting the Context

• Understanding the Item Bank: Beyond Navigation


“Since we initiated regular staff development sessions, we’ve turned the school around 360 degrees.”


Desired OutcomesDesired Outcomes

• understand how the Assessment Item Bank supports other DOE efforts

• understand the design and features of the Item Bank to maximize its use

• leave with ideas on how to use this resource in your classroom to meet your needs


Purposes of theData for School Improvement Assessment

Item Bank

• Support for teachers and principals by developing a common bank of robust and varied assessment items aligned to HCPS III

• Allows teachers to use benchmark aligned assessment items for diagnostic, formative, or summative purposes

• Provides a tool to help monitor and track student progress

• Determines prior knowledge and skills

• Identifies misconceptions, interests, or learning style preferences

• Provides information to assist planning and guide instructional decision making

• Ongoing, during the learning process


• At the end, to evaluate and communicate learning

• Informs future instruction as well as programmatic and overall curricular decisions

Assessment Continuum

Pre-Assessment

Formative

SummativeInvo

lves

stu

dent

s as

par

tner

s

MMann/SAS

1. Identify relevant content standards

2. Determine acceptable evidence and criteria

3. Determine learning experiences that will enable students to learn what they need to know and to do

4. Teach and collect evidence of student learning

5. Assess student work to inform instruction or use data to provide feedback

6. Evaluate student work and make judgment on learning results and communicate findings

Ret

each

, or

rep

eat

the

pro

cess

wit

h t

he

nex

t se

t o

f re

leva

nt

stan

dar

ds

Te

ach

er c

olla

bo

ratio

n th

rou

gho

ut t

he

pro

cess

.

Stu

de

nt in

volv

eme

nt th

rou

gho

ut th

e p

roce

ss.

Adapted from WestEd’s Learning from Assessment

Standards Based Implementation Model

RIG

OR

K

no

wle

dg

e T

axo

no

my

6

5

4

3

2

1

RELEVANCE Application Model

Knowledge/ Awareness

Comprehension

Application

Analysis

Synthesis

Evaluation

Rigor/Relevance Framework

1 2 3 4 5

Knowledge in one discipline

Apply in discipline

Apply across

disciplines

Apply to real-world predictable situations

Apply to real-world

unpredictable situations

A

C

B

D

Acquisition

Assimilation

Application

Adaptation

www.leadered.com/rigor.shtml


Literacy for LearningGuiding Principles

The Hawaii State Department of Education

will embrace and promote the following

guiding principles:

1. Assessment of and for learning drives

instruction

2. Evidence-based instructional practices

3. Aligned policies and resources at school, complex area and state levels

4. Instructional leadership and professional learning

5. Accountability

6. School, home and community partnerships


CSSS and RtI


Why an Item Bank?Why an Item Bank?


Support for Teachers & Principals

DSI: Hawaii Assessment Item Bank

• create

•administer

• score

monitor & track

student progress

adjust classroom

instructional decisions

improvement in student learning


FORMATIVE Assessments FOR Learning

– How can we use the assessment process & results to help students learn more?

SUMMATIVEAssessments OF Learning

– How much have students learned as of a particular point in time?

Different Assessment Purposes – Different Uses


Assessment for learning meets students’ information needs:

• Where am I going?• Where am I now?• How can I close the gap?


BenefitsBenefits

• free, readily accessible assessment items aligned to HCPS III • promotes common understanding and language of

curriculum goals and objectives• greater flexibility - - assess what is actually taught• customized assessments to meet own needs• save time when building a customized assessment• supports ongoing monitoring and tracking of student

progress in the classroom• aids in the collection of evidence for data-based instructional

decisions


Different Levels & Purposes of Use

Classroom Grade Level / Department

Between Elem, MS, HS

Complex/District Administration

Systemic Transparency for Data-Based Decision Making at All Levels

VALUEVALUEBeyondBeyond the Classroom the Classroom

State Administration

School Administration


Assessment Items

• Came from a variety of sources• Aligned by Hawaii DOE teachers• Address grades 3-8 & 10 benchmarks

in reading and math at this time• Plans for expansion• Cannot be sole source of data for

decision-making


Understanding the Understanding the Item Bank:Item Bank:

Beyond NavigationBeyond Navigation


Key Points Key Points When Using the Item BankWhen Using the Item Bank

#1 Check the Alignment or Taxonomic Match of an item

Why would this be important to know?


Marzano’s Taxonomic LevelsMarzano’s Taxonomic Levels

Marzano’s Taxonomic Levels of Understanding (Cognitive Domain)

Level 1: RETRIEVAL

Level 2: COMPREHENSION

Level 3: ANALYSIS

Level 4: UTILIZATION


Taxonomic LevelsTaxonomic Levels

6.10.3 Evaluate algebraic expressions

What is the value of the expression below when x = 4 and y = 2?

x 3 – 3y

Item is just asking students to solve the expression.

(Taxonomic Level 1)


Taxonomic LevelsTaxonomic Levels

Don’t just focus on the verb when thinking about which taxonomic level a benchmark addresses - - need to read and understand what the item is asking students to do.


Three levels of alignment to benchmarks:

– perfect alignment– acceptable alignment, but at a lower

taxonomic level– acceptable alignment, but at a higher

taxonomic level

How might this be useful in one’s classroom?


Alignments were made using best

professional judgment


Taxonomic Level = Perfect

Reading ExampleReading Example

READING Benchmark

Based on the play, which word best describes the Second Workman?

(A) hardworking

(B) untruthful

(C) helpful

(D) angry

LA 5.2.2

Make inferences and draw conclusions about grade-appropriate texts

Taxonomic Level =

perfect


Math ExampleMath Example

Taxonomic Level = Below

MA.6.11.1 – Analyze how data collection methods and sample size can affect the results of data sets

A poll is being taken at Baker Junior High School to determine whether to change the school mascot. Which of the following would be the best place to find a sample of students to interview that would be most representative of the entire student body?


Math ExampleMath ExampleMA.2.4.5 – Represent a given monetary amount using a combination of coins and bills

Which of the coins from Tony’s bank would he need to give to Marta so that they each have the same amount of money?

Taxonomic Level = Above


Some math benchmarks have two taxonomic levels

5.4.3 Use map scales to measure the distance between locations and make simple scale drawings

Taxonomic level = 1/2• Rationale: Some benchmarks have two

“cognitive demands”, i.e. “use” and “make”• The taxonomic levels correspond to the

different demands. Items may address only one part of a benchmark.


Reading Benchmarks

• that begin with “Use new grade-appropriate vocabulary” have a range of taxonomic levels 2-4

Because “use” may mean to symbolize it by drawing a picture, categorize it or compare it, or even evaluate how it was used.

Vocabulary use differs depending on the cognitive demand in the item or question.


Assessment Items Must Match the Instruction

in the Class

• Have students had the opportunity to learn the needed knowledge and skills addressed in the item?

• Is the level of cognitive demand appropriate?• Can the student scaffold to meet the

expectation?



#2 Check the breadth of each item in relation to the benchmark

Why would this be important to do?


Many of our benchmarks have multiple parts

One item will not address all the parts of the benchmark, however, it can provide useful

formative information about what a student knows and can do

MA 4.4.2 Select and apply appropriate customary and metric

units and tools to measure length, perimeter, and area for the degree of accuracy needed


ITEM: Square floor tiles will be put on the floor of a school

hallway. Each tile is 1 foot by 1 foot. The hallway is 85 feet long and 8 feet wide. About how many tiles will be needed to cover the floor of the hallway?

Which part of the benchmark is this item assessing?

Estimate and measure perimeter and area of common shapes and irregular shapes

(MA 3.4.6 -- TL1)


ITEM:

In 2010 a school had 125 students. Each year the number of students in the school increases by 50. Fill in the table to show the number of students expected for each year.

Extend, create, and generalize, growing and shrinking numeric and geometric patterns (including multiplication

patterns)

(MA 4.9.1 – TL3)

YEAR # of Students

2010 125

2011 ____

2012 ____

2013 ____

Which part of the benchmark is this item assessing? What do you think the taxonomic match is?


Explain how the author’s choice of language and use of literary elements contribute to the author’s purpose and the

effectiveness of the text (LA 4.3.3 -- TL3)

ITEM: Why does the author repeat the word shoot three times in

step 8?

(A) to highlight the importance of lots of practice(B) to show how professional photographers work(C) to suggest that the third picture will be the best(D) to encourage photographing a scene from three angle


ITEM: What is the main purpose of the information in the box titled

"A Matter of Fact"?(A) to give an example of a successful business that started small(B) to provide practical advice to people starting businesses(C) to show that restaurant businesses usually succeed(D) to explain how a family business is run


Draw and support conclusions about information or ideas in a text

(LA 6.2.3 – TL3)


Judgment about Proficiency

• Must be based on student performance on the whole

benchmark• Over time, check that items

selected address all parts of the benchmark



#3 Think about any prerequisite concepts or skills needed

before assessing the selectedbenchmarks

How might this help you create your assessment?


Some concepts and benchmarks (at a lower grade or on grade level) may be prerequisites for other benchmarks (based on content and/or taxonomic level)

Organize and represent data in more than one way (e.g., tallies, chart, tables, bar graphs, line plots, line graphs)

(MA 3.11.2 -- TL2)

Compare different representations of the same data and evaluate how well each representation shows important

aspects of the data

(MA 5.12.2 -- TL3)


Some concepts and benchmarks (at a lower grade or on grade level) may be prerequisites for other benchmarks (based on content and/or taxonomic level)

What other concepts and/or benchmarks would be prerequisites for this benchmark?

Explain how the author’s choice of language and use of literary elements contribute to the author’s purpose and

the effectiveness of the text

(LA 4.3.3 -- TL3)


Explain how the author’s choice of language and use of literary elements contribute to the author’s purpose and the

effectiveness of the text (LA 4.3.3 -- TL3)

ITEM: Why does the author repeat the word shoot three times in

step 8?

(A) to highlight the importance of lots of practice(B) to show how professional photographers work(C) to suggest that the third picture will be the best(D) to encourage photographing a scene from three angle


Grade 4 (Reading: focus non-fiction)• 3.1.6: Locate information in a variety of grade-appropriate sources• 3.2.3: Identify the main idea or problem and solution in a text• 4.1.1: Use new grade-appropriate vocabulary, including homophones and

homographs, learned through reading and word study, including root words, affixes, and word origins

• 4.2.1: Describe how common graphic structures (e.g., typeface, headings, illustrations) organize information in texts

• 4.2.2: Use organizational patterns (e.g., sequential, cause and effect) to access and understand information

• 4.2.4: Distinguish fact from opinion and cause from effect when reading informational texts

• 4.2.5: Summarize main points found in informational texts• 5.2.1: Use organizational patterns (e.g., compare and contrast,

proposition and support to access information)• 5.2.2 Make inferences and draw conclusions about grade-appropriate text

• 6.2.3 Draw and support conclusions about information or ideas in a text

* TIP: students should always give supporting evidence even though a benchmark doesn’t ask for it

Expanding the Search


Maximize the Item BankMaximize the Item Bank• use existing items as examples of how to write

items which align to benchmarks

• create additional items

• insert own passage or graphic (readings, math pictures, images, etc.)

• use features to involve the students.


Maximize the Item BankSAMPLE: Grade 4 Assessment



#4 Grade Range:Understand that some items willshow a grade range, i.e. grade 6-8 orgrade 9-12.


Reading Passages:

• Follow the NAEP model of using the same passage for different grade levels, keeping in mind what is age appropriate. Questions will differ depending on the grade level benchmark

• May address multiple and different grade level benchmarks


Grade Span

Reading Passages & ItemsReading Passages & Items


1. In the play, how does Mercury solve the First Workman's problem?(A) He finds the workman's axe for him.(B) He gives the workman money for food.(C) He helps the workman chop down trees.(D) He changes the workman’s axe into gold.

LA 3.1.6 Locate information in a variety of grade-appropriate sources

2. Based on the play, which word best describes the Second Workman?(A) hardworking(B) untruthful(C) helpful(D) angry

LA 5.2.2 Make inferences and draw conclusions about grade- appropriate texts


3. Which of the following shows that the play is a myth?(A) It has a main character.(B) One of the characters is a god.(C) It tells the exact words the characters speak.(D) Two of the characters argue with each other.

LA 2.3.3 Identify basic characteristics of familiar genres (e.g., stories, poems, textbook)

4. Read the sentence from lines 15-16 in the box below. Is this the axe you accidentally dropped into the deep pool?

How is the word dropped used in the sentence?(A) as a common noun(B) as a proper noun(C) as an adjective(D) as a verb

LA 3.1.4 Use hierarchies (e.g., specific to concrete, formal to informal) and categories (e.g., parts of speech, comparative and superlative forms, words with multiple meanings) to read increasingly complex words


High School Math

Math courses, such as Algebra I & II, Geometry, Calculus and Trigonometry may be offered at

different grade levels depending on the school’s master schedule. Items aligned to benchmarks in these courses will reflect a

grade 9-12 range.



#5 Maintain the intent of the item bank by how you use the items and what you do with the assessment data

Formative assessment to meet student needs• NOT just practice items for HSA• NOT just a replacement for “Quarterly Assessments”

• Determines prior knowledge and skills

• Identifies misconceptions, interests, or learning style preferences


• Ongoing, during the learning process


• At the end, to evaluate and communicate learning

• Informs future instruction as well as programmatic and overall curricular decisions

Assessment ContinuumAssessment Continuum

Pre-Assessment

(BEFORE)

Formative

(DURING)

Summative

(AFTER)

Invo

lves

stu

dent

s as

par

tner

s

PURPOSES:


“Innovations that include strengthening the practice of

formative assessment produce significant and often substantial

learning gains.”

--Black & Wiliam, 1998b, p. 140


Assessments can be. . .

“Quick” & “On the Run” to

“In-Depth” & “Deep”

• The breadth and depth of your assessment depends on your needs.


Clea

r Lea

rnin

g Ta

rget

Pre-

Asse

ssm

ent –

KW

L

Sum

mati

ve A

sses

smen

t

Lear

ning

Acti

vity

#1

– ex

it pa

ss

Lear

ning

Acti

vity

#2

– te

ache

r ob

serv

ation

Lear

ning

Acti

vity

#3

- qui

z

Lear

ning

Acti

vity

#4

– le

arni

ng lo

g

Lear

ning

Acti

vity

#5

– w

ritten

resp

onse

as

sess

men

t

Lear

ning

Acti

vity

#6

– w

hite

boa

rds

Formative Assessments are given throughout the learning process.

A Classroom View of the Assessment Continuum

1 23


Maximize the Item BankMaximize the Item Bank• use existing items as examples of how to write

items which align to benchmarks

• create additional items

• insert own passage or graphic (readings, math pictures, images, etc.)

• use features to involve the students.


Create ItemsCreate Items• may need to create items to address all parts of a

benchmark

Describe lines in the plane (i.e. parallel, perpendicular, intersecting) (MA 4.5.2 -- TL1)

ITEM:

Which of the following figures contains line segments that are perpendicular?


Describe lines in the plane (i.e. parallel, perpendicular, intersecting) (MA 4.5.2 -- L1)

Which of the following figures contains line segments that are parallel?

C. .

D. .

A. .

B. .

Classify different types of triangles and quadrilaterals according to their properties and identify the properties that define the classifications (MA 4.5.1 –TL3)

Which of the figures is a right triangle? Explain how you know.

Which of the figures is an equilateral triangle? Explain how you know.

Which of the figures are quadrilaterals? Explain how you know.


“Allow Student Comments”

Use Item Bank Features to Involve StudentsUse Item Bank Features to Involve Students


Involve Students in the Assessment Involve Students in the Assessment ProcessProcess

• Using the comment feature, students can:

explain their answer share thinking reflect on how they think they did give feedback to the teacher on items


Differentiated AssessmentDifferentiated Assessment

Selecting different items:

• based on lower/higher taxonomic levels and/or grade level benchmarks

• selecting reading items that address the same grade level benchmarks, but have easier or more difficult reading passages

• use item as is

• use item as is

• create an item

• use item as is

• create an item

• create an item

What is being compared in the lines above? (LA 4.3.4) (TL3) below (multiple choice)

What do these lines mean? (LA 4.3.4) (TL3) above (multiple choice)

What is the author of the poem trying to tell you about some people? (LA 2.3.1) (TL3) perfect (constructed response)

What is the theme of both poems? (LA 4.3.2) (TL3) perfect (could change multiple choice into a constructed response)

Explain whether or not the two poems share a similar theme.

Think about the people you’ve interacted with. Explain how you can relate (make a personal connection) to what the authors of these two poems are saying about people. Give at least 2 examples using details from the poems and details from your own experiences. (LA 5.3.7) (TL3) perfect


Creating an AssessmentCreating an Assessment• think about the purpose: pre-assessment, formative,

summative and how you will use this assessment to inform your instruction

• think about online or offline administration

• think about turn-around time to get scores and/or feedback to students

• remember: 3-8 pieces of evidence before drawing conclusions about proficiency; depends on depth and breadth of evidence

• assessments do not need to be in the form of a test


POINTS TO CONSIDERPOINTS TO CONSIDER

• if narrowly used, may lead to test prep mentality and feelings of too much testing

• may be viewed as time consuming

• not a cure all for assessment

• items may not match your reading or math programs


Use the Data

• For feedback• To inform instruction• To promote collaboration

NOT for punitive reasons

NOT for punitive reasons


Possible PLC ProcessPossible PLC Process

classroom scenario

customizing assessments choosing items creating items

differentiating assessments

your turn: creating an assessment to meet your needs debriefpractice


Practice Activity for Future Practice Activity for Future SessionsSessions

• think about upcoming benchmarks (learning targets) you want to assess

• determine purpose of assessment (pre-assessment, formative, summative)

• think about the design of the assessment (multiple choice and/or constructed response, # of items)

• select item(s) to add to your assessment


Practice

Activity for Understanding the Item Bank Design


Use the DSI Item Bank

• as a tool to help gather data so instructional decisions can be made to meet student needs.


Highly Recommended

• Take the first step and try the system out; explore the features by experimenting

• Introduce the system to the students so they can become involved in the process

• Collaborate with colleagues to ensure support and professional conversations

• Work with administration to ensure time, protocols, and clear expectations are communicated.


Let Students Show What They Know

Literacy for Learning Maximizing Hawaii’s ASSESSMENT ITEM BANK Understanding the Item Bank: Beyond...

Documents

Transcript of Literacy for Learning Maximizing Hawaii’s ASSESSMENT ITEM BANK Understanding the Item Bank: Beyond...