1

Oregon Content Standards Oregon Content Standards Evaluation Project, Evaluation Project,

Contract Amendment Phase: Contract Amendment Phase: Preliminary FindingsPreliminary Findings

Dr. Stanley Rabinowitz

WestEd

November 6, 2007

2

Background: Overview of Standards Evaluation Project

One of several key projects funded by the Gates Foundation

Independent evaluation by WestEd, a non-profit educational research, development, and service agency

3

Oregon Content Standards and Assessment System Evaluation

WestEd was contracted to Review and evaluate the content

standards (all grades, all academic content areas)

Evaluate alignment among the state assessments and the content standards

Make recommendations for improvement of Oregon’s comprehensive standards and assessment system

4

Contract Amendment Phase

Extended scope of work to assist the ODE in implementing key recommendations that emerged from the comprehensive evaluation

Builds upon current state initiatives Time frame: work will extend from mid-

October to December 31, 2007

5

Contract Amendment Scope of Work

Three key steps:1. Analysis of the degree to which Oregon’s

existing content standards support the draft Essential Skills

2. Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills

3. Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts

6

Focus for Today: Step 1Support for Essential Skills in

Existing Standards

Based on draft 3.0 Essential Skills (ES) Evidence of degree to which current content

standards in ELA, mathematics, science, and social sciences support the ES

Backward mapping of support from CIM through benchmark grades 3, 5, and 8

Recommendations for improving/refining draft Essential Skills to ensure effective application across content areas

7

Step 1 analyses are grounded in and informed by Findings from the evaluation of Oregon’s

standards and assessments Responses to Oregon’s Critical Questions Review of research literature Trends in practices across other states WestEd staff expertise in the content areas and

in standards and assessment development, curriculum and instruction, measurement, and cognitive development

Input solicited from state stakeholder groups– Department of Education staff – Essential Skills Task Force

8

Draft 3.0 Essential Skills

1. Read and comprehend a variety of texts at different levels of difficulty.

2. Write clearly and accurately for a variety of purposes.

3. Listen actively, speak clearly, and present publicly.

4. Think critically.5. Apply math to solve problems in a variety of

settings.6. Use technology to learn, live, and work.7. Demonstrate civic and community engagement.8. Demonstrate global literacy.

9

Protocol for Step 1 Determine whether each CIM standard

supports a draft Essential Skill1. Which one (if any) of the eight ES does it primarily

support?2. What is the strength of that relationship?

• Strong- CIM standard supports a fundamental skill or concept as explicitly stated in the Essential Skill

• Partial- CIM standard supports the Essential Skill in a superficial way or at a lower complexity level

Backward map from the CIM standard to identify evidence of support for the ES in grades 3, 5, and 8

– Do standards in grades 3, 5, 8, and CIM support the Essential Skill in a logical and coherent sequence or progression?

10

Guidelines Used During Analyses

Analysts focused on the bold category-level statements that defined each Essential Skill.

Where provided, the numbered indicator statements were used to clarify the focus of each Essential Skill.

A CIM standard could support an Essential Skill without specifically addressing one of the indicator statements provided.

11

Preliminary Findings What percentage of the CIM content

standards in four content areas support the Essential Skills?

Number of CIM

Standards Analyzed

Number of Standards

Supporting an ES*

Percentage of Standards

Supporting an ES

ELA 105 105 100%

Mathematics 84 84 100%

Science 18 18 100%

Social Sciences 39 34 87%

total N = 246 241 98%

*Based on primary relationships (i.e., each standard’s support was recorded for primary link only).

12

More Preliminary Findings…

Overall, how was this support distributed across Essential Skills?

n = number of CIM standards, across four content areas, based on primary relationships

ES 1 ES 2 ES 3 ES 4 ES 5 ES 6 ES 7 ES 8

n=46 n=32 n=21 n=34 n=84 n=1 n=11 n=12

19% 13% 9% 14% 34% <1% 5% 5%

13

More Preliminary Findings… Within each content area, how do the CIM

standards support the Essential Skills?*

ES 1 ES 2 ES 3 ES 4 ES 5 ES 6 ES 7 ES 8

ELA(all standards

support an ES)

44% 30% 20% 5% 0 1% 0 0

Math(all standards

support an ES)

0 0 0 0 100% 0 0 0

Science(all standards

support an ES)

0 0 0 100% 0 0 0 0

Social Sciences

(34/39 standards

support an ES)

0 0 0 28% 0 0 28% 31%

*Based on primary relationships.

14

More Preliminary Findings… Of those CIM standards that support the

Essential Skills, what percentage was rated as providing strong support?

Number of CIM Standards

Supporting ES*

Number of Standards Providing

Strong Support

Percentage of Standards Providing

Strong Support

ELA 105 99 94%

Math 84 78 93%

Science 18 16 89%

Social Sciences 34 23 68%

total 241 216 90%

*Based on primary relationships.

15

More Preliminary Findings…

What percentage of CIM standards that support the ES show a coherent progression across grades 3, 5, 8, and CIM?

Number of CIM Standards Supporting

ES*

Number of Standards

Showing Coherent Progression

Percentage of Standards

Showing Coherent Progression

ELA 105 87 83%

Math 84 72 86%

Science 18 16 89%

Social Sciences

34 31 91%

total 241 206 85%

*Based on primary relationships.

16

Summary of Preliminary Findings

Overall, 98% of existing standards across the four content areas (ELA, math, science, and social sciences) primarily supported one of the draft Essential Skills.

Across content areas, the majority of relationships among supporting standards and the Essential Skills was strong.– ELA and math: >92% support was strong– Science: 89% support was strong– Social sciences: 68% support was strong

17

Summary of Preliminary Findings

The distribution of standards’ primary support across Essential Skills varied by content area.– Overall, less than 10% of standards’ primary support

was associated with ES 6, 7, and 8 combined.– Primary support in the ELA standards was balanced

across three ES (1, 2, 3).– 100% of math standards provided primary support for

ES 5.– 100% of science standards provided primary support

for ES 4.– Primary support in the social sciences standards was

balanced across three ES (4, 7, and 8). Across content areas, evidence of coherent progression

across grades ranged from 83-91% (85% overall).

18

Next Steps Analysis of the degree to which Oregon’s current

test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills

Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts

Formal report with findings and recommendations for improvement/ refinement emerging from the Amendment Phase

19

For more information about the standards and assessment evaluation:

www.ode.state.or.us/teachlearn/real/standards/Standards_Review.aspx

For more information about WestEd:

www.wested.org