A Michigan Application of NAEP Tools

Paul Stemmer, Ph.D.

NAEP State Coordinator

Michigan Department of Education

stemmerp@mi.gov

2010 National Conference on Student Assessment

Detroit, Michigan

June 20, 2010

Concern for Michigan’s National

Performance/Competitiveness

Three Issues in the Mi-Lit

presentation

1. Michigan tends to be around the national

average on NAEP – how do we improve?

2. Minority gaps are a major area of concern

3. To what extent does the emphasis on

cognitive processes in reading effect

student performance?

Tools Used in Creation of Presentation

Subject: 2009 NAEP Reading

Tools:

Item Mapping Tool – Show average performance

NAEP Data Explorer (NDE) - trends

NDE - Analysis of Background Questions

Imaging Transfer – Snag It (commercial tool) to cut

and paste images

Addressing the National Average

Data Issue – Step 1.

Used Item Map Tool

http://www.nationsreportcard.gov/data_tools.asp

Addressing the National Average

Data Issue – Step 2.

Selected Item Maps Tool

Addressing the National Average

Data Issue – Step 2.

Selected

Addressing the National Average

Data Issue – Step 3.

Get results and use Snag It® to grab Box

and Whiskers

Addressing the National Average

Data Issue – Step 3.5

Snag It Capture Set-up

A Snagit Capture

Issue 2. Discussion of Trends -

NDE Generated Trend Map

Issue 2 Trends Analysis – Step 1.

Select Data Explorer

Issue 2 Trends Analysis – Step 2.

Use NDE to select National Public Average

and Comparison states

Issue 2 Trends Analysis – Step 3.

Select Variables

Issue 2 Trends Analysis – Step 4.

Build Report

Issue 2 Trends Analysis – Step 5.

Create Line Chart Option

Issue 2 Trends Analysis – Step 6.

Generate Chart and Snag It

MA

OH

MI

US

Demographic Gap Analysis

Issue 3: Explore Idea of

Cognitive Processes

Use NDE with

Teacher Practice Questions

from Questionnaire

Issue 3: Cognitive Process

Instruction

Use NDE Tool, select variables, search

content

Issue 3: Cognitive Process

Instruction

Edit Report to use Percentages

Issue 3: Cognitive Process

Instruction

NDE – Chart Options Column Chart

NAEP Tools

Pros

Can handle a huge number of questions

Relatively quickly get to data

Totally transparent, public has same access

Confident in maintaining confidentiality

Cons

Little slow (it’s a web tool!)

Interface takes getting used too

Significant wealth of data, more than I can process

Thank you!

Paul Stemmer, Ph.D.

NAEP State Coordinator

Michigan Department of Education

stemmerp@mi.gov

(517)241-2360

http:www.mi.gov/naep

2010 National Conference on Student Assessment

Detroit, Michigan, June 20, 2010