JoAnne EreshSenior Associate, Achieve California Diploma ProjectOctober 26-28, 2009

An Analysis of the Early Assessment

 Program (EAP) English   

• Description of the EAP English Assessments

• Methodology

• Findings – Reading and Writing

• Findings – EAP Essay

• Recommendations

Session will cover:

2

• Grade 11 English-Language Arts CST item subset assesses content from the Grades 11-12 California Content Standards in reading and writing

• EAP Augmentation item subset assesses reading and writing content from the English Placement Test (EPT)

• EAP Essay assesses reading and writing

Description of the English EAP

3

Achieve Assessment to Standards (ATS) Protocol• Content Centrality• Performance Centrality• Source of Challenge• Level of Cognitive Demand• Balance• Level of Challenge

Forms Analyzed• a publicly released subset of 42 multiple choice reading and writing

items selected by CSU faculty from the English Language Arts CST• 15 reading and writing items that have been retired from the EPT and

selected by the faculty for augmentation • the EAP Essay and rubric that addresses direct assessment of student

writing• Not an analysis of the Grade 11 English Language Arts CST

Methodology

4

What Content Does the English EAP Cover in Reading?• Addresses essential content and skills in reading, ranging from word

analysis and the use of rhetorical devices to understanding the  structure of an argument. 

• Includes seven augmentation reading items—retired EPT items that assess analytical reading skills deemed essential by CSU faculty.

• Expects students to understand the meaning of words in the context of a  sentence as well as the relationship between two ideas - whether the ideas contrast, illustrate cause and effect, or contradict each other.

• Note: Though the augmentation items were not developed to specifically address the California Content Standards, Achieve reviewers mapped the items to the standards for the purposes of discussing full content coverage of the standards.

• Appendix C contains the full listing of standards that make up the content and performance expectations for the English EAP.

Findings – EAP Reading

5

Content CentralityAnalyzes the match between the content that is targeted by a 

test question and the content described in a standard.  (see also Table 2.1)

  

Findings –Reading EAP

6

Performance CentralityFocuses on the degree of the match between the type of

performance required by a test item and the performance required by the related standard (see also Table 2.2)

  

Findings –Reading EAP

7

READING DIFFICULTY.

Table 2.6: EAP Reading Passage Reading Difficulty Scores

Findings – Reading EAP

8

Level of Cognitive Demand• Focuses on the type and level of thinking and reasoning required by

students on individual items. • Level 1 (recall or locate) – item requires little beyond simple recall or

identification such as identify figurative language or basic facts in a reading passage.

• Level 2 (infer or integrate) – item demands a level of inference involving some comprehension and subsequent processing of texts, such as using context clues to identify unfamiliar words, predicting logical outcomes, or deciding appropriate verb tense by considering the meaning of a sentence.

• Level 3 (analyze or apply) – item requires analysis of the text and an understanding of how a passage works. Level 3 items include determining author’s purpose, understanding the use of symbols, or accomplishing a rhetorical purpose.

• Level 4 (critique or evaluate) – item requires evaluation of material and are often open-ended.

Findings – English EAP

9

Level of Cognitive Demand(see also Table 2.5)

Findings – Reading EAP

10

(see also Table 2.7)

Findings – INDIRECT WRITING

11

(see also Table 2.8)

Findings – INDIRECT WRITING

12

(see also Table 2.10)

Findings – INDIRECT WRITING

13

Content and Performance Centrality • 1. To what degree does the writing prompt reflect one of the

types of genres of writing identified in the standards?• 2. To what degree does the rubric reflect the kind of

performance described in the standards? Source of Challenge• Is the prompt accessible to all students? The prompt should

not require specific background knowledge or experience in order to address it.

• Is the task clear? If the assignment intends to prompt a persuasive essay, for example, that purpose should be clear to the writer.

• Are the basic criteria for assessment of the writing sample articulated for the writer?

Findings – EAP ESSAY

14

The Scoring Guide for the EAP Essay is clear, specific, and relevant to the standards  Six criteria  are applied by the scorers: • a. = response to the topic • b. = understanding and use of the passage • c. = quality and clarity of thought • d. = organization, development, and support • e. = syntax and command of language • f. = grammar, usage, and mechanics   

Findings – EAP ESSAY

15

• The State of California has direct writing assessments at grades 4 and 7.

• A next step question: How does the EAP Essay relate to other measures of college readiness?

• As is the case with many colleges and universities nationally, most often these placement tests take the form of a direct writing assessment that is developed and assessed at the local level or one of the commercially available assessments  such as the College Board’s WritePlacer or ACT’s e‐Write.

• Both the CSU and UC systems require an assessment of entering students’ literacy skills in order to find out whether or not they need remediation before they can succeed in college level work. Both the EAP and the AWPE are developed internally by faculty. While there are differences, there are strong similarities that could provide the opportunity for future alignment efforts. (Table 2.13)

Similarities between the EAP Essay and the UC Analytical Writing Placement Examination (AWPE)

16

Achieve found:

• The English EAP addresses essential college-ready content identified by CSU for entry-level English coursework.

• The English EAP includes rigorous reading passages.

• The EAP Essay assesses not only a student’s writing ability but also critical thinking.

Summary of Findings – English EAP

17

Achieve recommends that the EAP Test Committee consider:

• When assessing reading, the EAP Test Committee should ensure that questions on reading passages are as text dependent as possible in order to test students’ ability to derive meaning from the college level passages provided.

• Consider convening faculty representatives from the UC and the CSU higher education systems with Community Colleges to conduct a cross-sector analysis of the assessment of direct writing and identify opportunities for alignment.

• Consider a cross-sector communication outreach effort to educators, students, and communities regarding the expectations that students must meet in order to demonstrate readiness for entry-level college coursework. Clarify the importance of the both the CST and EAP for assessing this readiness and the opportunity the EAP provides for students entering CSU and the community colleges.

Summary of Recommendations – English EAP

18

JoAnne EreshSenior Associate, Achieve California Diploma ProjectOctober 26-28, 2009

An Analysis of the Early Assessment

 Program (EAP) English