Language Arts Day!October 24, 2011

Language Arts teachers are invited to gather at these sessions to study NeSA-Reading and NeSA-Writing data and use this information to consider how to continue to improve student achievement. In addition to the data analysis and NeSA updates, we will study recent research, pedagogy, and strategies directly related to language arts instruction.

PurposesUnderstand NeSA protocol, resources, and

resultsStudy NeSA resultsUse data to inform decisions for improving

student achievementStudy explicit instructionShare experiences, tips, ideas

Getting Started IntroductionsNormsParking LotWikispace: http://esu6la.wikispaces.orgAgendaHandouts & CopiesSurvey

2011 NeSA TestingThink

What were the challenges of the NeSA-R or NeSA-W? How did (or might) these be overcome? What went well? Have you made any curricular adjustments?

Pair Someone with similar responsibilities

Share 3-5 minutes

Interaction Sequence

Ask all student the question.

Pause (3+ seconds).

Put students on-the-clock. “You have 2 minutes to share

your answer with your partner.”

Students share their thoughts with a partner.

Select student(s) to respond.

Conference with 1 or 2 pairs• Check student answers• Probe• Provide answers when missing

1. Purposeful Selection: Call on students you have visited.

2. Random Selection: Call on students so every student has an opportunity to be selected.

3. Volunteer Selection: Allow volunteer responses.

(Sharer, Anastasio, & Perry, 2007, p. 80-85)

NeSA Soup!DACCALDRCDRSAYPTOSPLDC4L

Understanding the NeSA-R & NeSA-W

http://www.education.ne.gov/assessment/NeSA_Presentations.htm

Nebraska schools should use NeSA data to . . .

Provide feedback to students, parents and the community

Inform instructional decisions.Inform curriculum development and revision.Measure program success and effectiveness.Promote accountability to meet state and federal requirements.

8

NeSA is . . . A criterion-referenced summative test.A measurement of the revised

Nebraska Reading Standards specific to vocabulary and comprehension.

A tool including 45 to 50 multiple-choice items.

A test administered to students online OR paper/pencil.

9

Tables of Specification

What are . . .

10

Performance Level

Descriptors

What are . . .

11

NeSA . . . Produces a raw score that converts to a scale

score of 0-200.

Allows for students to be classified into one of three categories: Below the Standards, Meets the Standards, Exceeds the Standards.

Provides comparability across Nebraska school buildings and districts.

12

~NeSA Terminology~

SCALE SCORE – a student’s transformed version of the raw score earned on NeSA

Performance Level Reading Scale-Score Range

Exceeds the Standards 135 -- 200Meets the Standards 85-134Below the Standards 84 and below

13

14

Cut score processes:

Contrasting Group Method – 400+ teachers

Bookmark Method – 100+ teachers State Board of Education Reviewed Examined results of both processes Examined NAEP and ACT results for Nebraska Made decisions within recommended range at

public meeting

How are performance levels determined?

~NeSA Terminology~

RAW SCORE – the number of items a student answers ‘right’ on NeSA-RContent

AreaPoints

PossiblePoints Earned

Student’s Scale Score

Reading 42 21 126Mathematics

42 21 127Raw

ScoreScale Score

Performance Level

25 200 Exceeds24 167 Exceeds23 148 Exceeds22 135 Exceeds21 126 Meets20 118 Meets19 111 Meets

on NeSA Reports

15

on Conversion Chart

~NeSA Terminology~

What is the difference between a raw score and a scale score?

What is a raw score?A raw score is the number of correct items. Raw scores have been typically used in classrooms as percentages: 18/20= 90% correct.

16

~NeSA Terminology~

What is a scale score?A scale score is a “transformation” of the number of items answered correctly to a score that can be more easily interpreted between tests and over time. The scale score maintains the rank order of students (i.e., a student who answers more items correctly gets a higher scale score). For NeSA, we selected 0-200 and will use it for all NeSA tests, including writing.

17

~NeSA Terminology~

Why convert raw scores to scale scores?Raw scores are converted to scale scores in order to compare scores from year to year. Raw scores should not be compared over time because items vary in difficulty level. Additionally, raw scores should not be compared across different content area tests. Scale scores add stability to data collected over time that raw scores do not provide.

18

~NeSA Terminology~

SCALE SCORE CONVERTED TO PERCENTILE RANK?

On score reports why is the . . .

The percentile rank was placed on the score reports because our Technical Advisory Committee felt that parents would want to know their child’s position in relation to other test takers.

A percentile rank of 84 means the child scored better than 84% of the students who took the test that year.

19

What does the Scale Score Look Like in

Action?Although the test items are comparable, they

are different.

2010 Reading Grade 8 2011 Reading Grade 8

Raw Score

Scale Score

LevelRaw

ScoreScale Score

Level

43 135 Exceeds 43 145 Exceeds42 128 Meets 42 139 Exceeds

      41 133 Meets           

32 85 Meets 32 96 Exceeds31 81 Below 31 92 Exceeds

      30 89 Exceeds      29 86 Meets      28 83 Below

Scale ScoreThink

What are the key points about scale scores that you would share with a parent who has questions about the NeSA-R?

Ink Write 2-3 points.

Link Find a partner; give one, and get one. Repeat.

Name That Concept!

AKA Talk a Mile a Minute and Password

NeSA Related TermsTable of Specifications (TOS)District Assessment Contact (DAC)Scale ScoreAdequate Yearly Progress (AYP)Percentile Rank

NeSA Related TermsCheck 4 Learning (C4L)Raw ScoreProficiency Level Descriptions (PLDs)Data Reporting System (DRS)Criterion-Referenced Assessment

NeSA Reports What can we learn from this report?

Do we have other data to support these results?

What are the implications of this report?

NeSA REPORTSIndividual Student ReportSchool Student RosterSchool Indicator SummarySchool Performance Level SummaryDistrict Reading Indicator SummaryDistrict Performance Level SummaryDistrict Report of School Performance

27

Individual Student Report 28

School Student Roster 29

School Indicator Summary 30

School Performance Level Summary31

District Reading Indicator Summary32

District Performance Level Summary

33

District Report of School Performance

34

Step 1:Define the

SituationWhat can we learn from each report?

What is the data telling us (strengths and concerns)?

Step 2:Establish

hypothesesWhy are we getting these results?

Step 3:Verify / Refute

HypothesesDo we have other data to support these results?

Step 4:Create the Action Plan

How can we use this NeSA data?

What is the goal? (How much change is expected and by when?)

What will be done to reach the goal(s), and how will progress toward goal(s) be measured?

Curriculum AlignmentExamine PLDs and Tables of Specification.

Are the tested indicators in our curriculum? -- Where?

When are they taught?How are they instructed?At what DOK (Depth of Knowledge) level?By whom?

Instructional EffectivenessExamine PLDs and Tables of Specification.

Do our students have opportunity to learn (and practice) the tested indicators?

Is our instruction efficient and effective?How are students performing on the indicators

on a day-to-day basis?Are we assessing them locally to find out what

they know and can do?

Test PreparationHave our students used practice tests?Are our students familiar with the testing tools?Are we familiar with appropriate

accommodations? Have we applied them?

2012 NeSA-R TestingGrades 3-8, 11Standardized, secure testing proceduresPaper and pencil or online (already submitted)Two independent sessionsUntimedCuts remain the same (0-84, 85-134, 135-200)

What can you do or not do? (pages 19-21 & 30-34 in SAA-8)

March 26 - May 4, 2012

2012 NeSA-W TestingGrade 4narrativetwo 40-minute sessions (timed)#2 pencilholistically scored in 2012 (analytically scored in 2013)same cut scores as previous years (new in 2013)

January 23 – February 10, 2012

2012 NeSA-W TestingGrades 8 & 11descriptive (8); persuasive (11)online test administrationone ~90-minute session (untimed; 2011 avg. = 45-65

min.)analytically scored (composite + 4 weighted domains)online dictionary and thesaurus; no spell-check6,000 character limit (approx. 3 pages)

January 23 – February 10, 2012

(SAA-8, p. 45-51)

2012 NeSA-W TestingGrades 8 & 11 (cont.)software update (wrap, spaces, dictionary)no “tab” (advise students to use 3-5 spaces) font size, spacing, margins do NOT affect scoringnew cut score set in April 2012composite score converted to scale of 0-70can print practice & operational tests

January 23 – February 10, 2012

(SAA-8, p. 45-51)

The way I see it… We have 3 years of data to

consider.We can do some things right

now… test procedures, format general test-taking skills motivation initial analysis, hypotheses,

instructional change curriculum alignment effective instruction

…and, we need to have a long-term, sustainable approach. analysis of trends hypotheses, instructional

change, study results, etc. (PDSA cycle)

diagnosis/intervention plan

2011 Statewide ResultsGrade Composit

e% Met

Avg. Scale Score

2010 AYP Goal

2011AYP Goal

3 70.95% 101 67% 78%

4 75.39% 104 67% 78%

5 70.01% 101 67% 78%

6 73.72% 101 67% 78%

2011 Statewide ResultsGrade Composit

e% Met

Avg. Scale Score

2009 AYP Goal

2010 AYP Goal

2011AYP Goal

7 73.87% 110 60% 67% 78%

8 71.43% 106 60% 70% 80%

11 67.32% 102 57% 68% 79%

Improving Adolescent Literacy…

http://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=

8

Explicit Instruction

http://explicitinstruction.org/?page_id=80

4 A’s Text Protocol

What assumptions does the author of the text hold?

What do you agree with in the text?What do you want to argue with in the text?To what parts of the text do you aspire?

Resource & Idea Sharing

What fabulous resources do you depend on for your professional practice?

Summary of Learning

When my administrator asks about today, I will say that I learned…

The most important / relevant thing I learned or was reminded of today is…

Future SessionsJanuary 31

June 18

See You Then!

Please complete the evaluation!Start at the esu6la wikispace on the Language Arts Days page (toward the bottom of today’s agenda).

Future Sessions:• January 31• June 18