Reproductions supplied by EDRS are the best that …DOCUMENT RESUME ED 464 893 SO 033 859 AUTHOR...

DOCUMENT RESUME

ED 464 893 SO 033 859

AUTHOR Lapp, Michael S.; Grigg, Wendy S.; Tay-Lim, Brenda S. H.TITLE The Nation's Report Card: U.S. History, 2001.INSTITUTION National Center for Education Statistics (ED), Washington,

DCREPORT NO NCES-2002-483PUB DATE 2002-05-00NOTE 179p.; Written in collaboration with Tatyana Petrovicheva

and Satwinder Thind. The work upon which this publication isbased was performed for the National Center for EducationStatistics by the Educational Testing Service.

AVAILABLE FROM ED Pubs, P.O. Box 1398, Jessup, MD 20794-1398. Tel:877-433-7827 (Toll Free) ; Tel: 877-576-7734 (Toll Free;TTY/TDD); Fax: 301-470-1244; e-mail: [email protected]; Website: http://www.ed.gov/pubs/edpubs.html. For full text:http://nces.ed.gov/nationsreportcard.

PUB TYPE Numerical/Quantitative Data (110) Reports Research(143)

EDRS PRICE MF01/PC08 Plus Postage.DESCRIPTORS *Academic Achievement; Comparative Analysis; Elementary

Secondary Education; *Grade 12; *Grade 4; *Grade 8; PublicSchools; School Surveys; *Student Evaluation; *United StatesHistory

IDENTIFIERS *National Assessment of Educational Progress

ABSTRACTThe National Assessment of Educational Progress (NAEP) is

the nation's only ongoing representative sample survey of student achievementin core subject areas. In 2001, NAEP conducted a national United Stateshistory assessment of fourth-, eighth-, and twelfth-grade students. Thisreport presents a summary of the results of the NAEP 2001 U.S. historyassessment for the nation. Results in 2001 are compared to results in 1994,the next most recent year in which the NAEP conducted a U.S. historyassessment and the only other assessment year in which the test questionswere based on the current framework. Students' performance on the assessmentis described in terms of average scores on a 0-500 scale and in terms of thepercentage of students attaining 3 achievement levels: (1) Basic; (2)

Proficient; and (3) Advanced. The report provides results for subgroups ofstudents at the three grade levels defined by various background andcontextual characteristics. Appended are an overview of the procedures usedin the NAEP assessment and data collection. (BT)

Reproductions supplied by EDRS are the best that can be madefrom the original document.

U S DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it

Minor changes have been made toImprove reproduction quality

0

0\00 071-VI)

°

di

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy

4.

L77-k-)11.,-

DepartmoM of EducationEdu lainarRese.arc "andt tnperoyeln.ent 2

Whe s The Ma Mon's Repon Cud?THE NATION'S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationallyrepresentative and continuing assessment of what America's students know and can do in various subject areas. Since1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, andother fields. By making objective information on student performance available to policymakers at the national, state,and local levels, NAEP is an integral part of our nation's evaluation of the condition and progress of education. Onlyinformation related to academic achievement is collected under this program. NAEP guarantees the privacy ofindividual students and their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Departmentof Education.The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP projectthrough competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is alsoresponsible for providing continuing reviews, including validation studies and solicitation of public comment, onNAEP's conduct and usefulness.

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelinesfor NAEP.The Board is responsible for selecting the subject areas to be assessed from among those included in theNational Education Goals; for setting appropriate student performance levels; for developing assessment objectives andtest specifications through a national consensus approach; for designing the assessment methodology; for developingguidelines for reporting and disseminating NAEP results; for developing standards and procedures for interstate,regional, and national comparisons; for determining the appropriateness of test items and ensuring they are free frombias; and for taking actions to improve the form and use of the National Assessment.

The RIgonal Assessmeht GoverMns &Dud

Mark D. Musick, ChairPresidentSouthern Regional Education BoardAtlanta, Georgia

Michael T. Nettles, Vice ChairProfessor of EducationUniversity of MichiganAnn Arbor, Michigan

Melanie A. CampbellFourth-Grade TeacherTopeka, Kansas

Honorable Wilmer S. CodyFormer Commissioner of EducationState of KentuckyFrankfort, Kentucky

Daniel A. DomenechSuperintendent of SchoolsFairfax County Public SchoolsFairfax,Virginia

Edward DonleyFormer ChairmanAir Products & Chemicals, Inc.Allentown, Pennsylvania

Thomas H. FisherDirectorStudent Assessment ServicesFlorida Department of EducationTallahassee, Florida

Edward H. HaertelProfessor, School of EducationStanford UniversityStanford, California

Juanita HaugenLocal School Board MemberPleasanton, California

Honorable Dirk KempthorneGovernor of IdahoBoise, Idaho

Honorable Nancy KoppState LegislatorAnnapolis, Maryland

Honorable Ronnie MusgroveGovernor of MississippiJackson, Mississippi

Roy M. Nageak, Sr.First Vice-ChairAlaska Board of Education and

Early DevelopmentBarrow, Alaska

Debra PaulsonEighth-Grade Mathematics TeacherEl Paso,Texas

Honorable Jo Ann PottorffState LegislatorWichita, Kansas

Diane RavitchResearch ProfessorNew York UniversityNew York, New York

Sister Lourdes Sheehan, R.S.M.Secretary for EducationUnited States Catholic ConferenceWashington, DC

John H. StevensExecutive DirectorTexas Business and Education

CoalitionAustin,Texas

3

Migdania D. VegaPrincipalCoral Way Elementary Bilingual

SchoolMiami, Florida

Deborah VoltzAssistant ProfessorDepartment of Special EducationUniversity of LouisvilleLouisville, Kentucky

Honorable Michael E. WardState Superintendent of Public

InstructionNorth Carolina Public SchoolsRaleigh, North Carolina

Marilyn A. WhirryTwelfth-Grade English TeacherManhattan Beach, California

Dennie Palmer WolfDirector, Annenberg InstituteBrown UniversityProvidence, Rhode Island

Grover J. Whitehurst (Ett-Officio)Assistant Secretary of EducationOffice of Educational Research and

ImprovementU.S. Department of EducationWashington, DC

Roy TrubyExecutive Director, NAGBWashington, DC

TheNational! Center for Education Statistics

ation'sU.S,

e rt J41 r

sthcry 2CM

Michael S. Lapp

Wendy S. Grigg

Brenda S.H. Tay-Lim

in collaboration with

Tatyana Petrovicheva

Satwinder Thind

May 2002

U.S. Department of EducationOffice of Educational Research and Improvement NCES 2002-483

U.S. Department of EducationRod PaigeSecretary

Office of Educational Research and ImprovementGrover J.WhitehurstAssistant Secretary

National Center for Education StatisticsGary W PhillipsDeputy Commissioner

May 2002

SUGGESTED CITATIONUS. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics.The Nation's Report Card: U.S. History 2001, NCES 2002-483, by M. S. Lapp,W. S. Grigg, & Brenda S.--H.Tay-Lim.Washington, DC: 2002.

FOR MORE INFORMATIONContent contact:Janis Brown202-502-7419

To obtain single copies of this report, limited number of copies available, or ordering information on other U.S.Department of Education products, call toll free 1-877-4ED-PUI3S (877-433-7827), or write:

Education Publications Center (ED Pubs)U.S. Department of EducationP.O. Box 1398Jessup, MD 20794-1398

TTY/TDD 1-877-576-7734FAX 301-470-1244

Online ordering via the Internet: http://wwwed.gov/pubs/edpubs.htmlCopies also are available in alternate formats upon request.This report also is available on theWorldWide Web: http://nces.ed.gov/nationsreportcard

The work upon which this publication is based was performed forthe National Center for Education Statistics by Educational Testing Service.

able of Contents

Executive Summary ix

Chapter 1

NAEP 2001 U.S. History Assessment 1

Introduction 1

Overview of the 2001 National Assessment of Educational Progress 2

The U.S. History Framework 2

U.S. History Assessment Instruments 5

Description of School and Student Samples 6

Reporting the Assessment Results 7

The Setting of Achievement Levels 8

Achievement-Level Descriptions for Each Grade 9

The Trial Status of Achievement Levels 13

Interpreting NAEP Results 15

Overview of the Remaining Report 16

Chapter 2

Average Scale Scores and Achievement-Level Results 17

Overview 17

Average Scale Score Results 18

Scale Scores by Percentile 19

Achievement-Level Results 20

Chapter 3

Average Scale Scores and Achievement-Level Results for Selected Subgroups 23

Gender 24

Race/Ethnicity 27

Region of the Country 32

Parents' Highest Level of Education 36

Type of School 40

Type of Location 45

Free/Reduced-Price School Lunch Program Eligibility 47

TABLE OF CONTENTS U.S. HISTORY REPORT CARD iii

Chapter 4

Becoming a More Inclusive National Assessment 49

Two Sets of 2001 NAEP U.S. History Results 50

Results for the Nation 53

National Results by Gender 55

National Results by Race/Ethnicity 57

Chapter 5

Sample Assessment Questions and Student Responses 59

Grade 4 Sample Assessment Question Results 60



Maps of Selected Item Descriptionson the NAEP U.S. History ScaleGrades 4, 8, and 12 84

Chapter 6

Classroom Contexts for Learning 89

Time Spent on Social Studies 89

State and Local Standards 91

Instructional Activities 92

Use of Technology 99

Appendix A

Overview of Procedures Used for the NAEP 2001 U.S. History Assessment 103

Appendix B

Data Appendix 133

Appendix C

Members of the NAEP U.S. History Standing Committee 165

Acknowledgments 166

Chapter 1: Figures and Tables

Figure 1.1: Descriptions of the four U.S. history themes 3

Table 1.1: Distribution of assessment time across historical themes,grades 4, 8, and 12: 2001 4

Table 1.2: Distribution of assessment time across historical periods,grades 4, 8, and 12: 2001 5

Figure 1.2: Policy definitions of the three NAEP achievement levels 8

Figure 1.3: Descriptions of NAEP U.S. history achievement levels for grade 4 10



iv TABLE OF CONTENTS U.S. HISTORY REPORT CARD7


Figure 2.1: Average U.S. history scale scores,grades 4, 8, and 12: 1994 and 2001 18

grades 4, 8, and 12: 1994 and 2001 19

U.S. history achievement levels, grades 4, 8, and 12: 1994 and 2001 21


Figure 3.1: Average U.S. history scale scores by gender,grades 4, 8, and 12: 1994 and 2001

Figure 3.2: Differences in average U.S. history scale scores by gender,grades 4, 8, and 12: 1994 and 2001

Figure 3.3: Percentage of students within and at or aboveU.S. history achievement levels by gender, grades 4, 8, and 12: 1994 and 2001

Figure 3.4: Average U.S. history scale scores by race/ethnicity,grades 4, 8, and 12: 1994 and 2001

Figure 3.5: Differences in average U.S. history scale scores by race/ethnicity,grades 4, 8, and 12: 1994 and 2001

Figure 3.6a: Percentage of students within and at or aboveU.S. history achievement levels by race/ethnicity, grade 4: 1994 and 2001

Figure 3.6b: Percentage of students within and at or aboveU.S. history achievement levels by race/ethnicity, grade 8: 1994 and 2001

Figure 3.6c: Percentage of students within and at or aboveU.S. history achievement levels by race/ethnicity, grade 12: 1994 and 2001

Figure 3.7: Average U.S. history scale scores by region of the country,grades 4, 8, and 12: 1994 and 2001

Figure 3.8a: Percentage of students within and at or aboveU.S. history achievement levels by region of the country, grade

Figure 3.8b: Percentage of students within and at or aboveU.S. history achievement levels by region of the country, grade

Figure 3.8c: Percentage of students within and at or aboveU.S. history achievement levels by region of the country, grade

Figure 3.9: Average U.S. history scale scores byparents' highest level of education, grades 8 and 12: 1994

Figure 2.2: U.S. history scale score percentiles,

Figure 2.3: Percentage of students within and at or above

4: 1994 and 2001

8: 1994 and 2001

12: 1994 and 2001 .

and 2001

Figure 3.10a: Percentage of students within and at or above U.S. historyachievement levels by parents' highest level of education, grade 8: 1994 and 2001 .

Figure 3.10b: Percentage of students within and at or above U.S. historyachievement levels by parents' highest level of education, grade 12: 1994 and 2001

Figure 3.11: Average U.S. history scale scores by type of school,grades 4, 8, and 12: 1994 and 2001

Figure 3.12a: Percentage of students within and at or aboveU.S. history achievement levels by type of school, grade 4: 1994 and 2001

Figure 3.12b: Percentage of students within and at or aboveU.S. history achievement levels by type of school, grade 8: 1994 and 2001

Figure 3.12c: Percentage of students within and at or aboveU.S. history achievement levels by type of school, grade 12: 1994 and 2001

eTABLE OF CONTENTS

24

25

26

27

28

29

30

31

32

33

34

35

37

38

39

40

42

43

44

. U.S. HISTORY REPORT CARD v

Table 3.1: Average U.S. history scale scores by type of location,grades 4, 8, and 12: 2001 45

Figure 3.13: Percentage of students within and at or above U.S. historyachievement levels by type of school location, grades 4, 8, and 12: 2001 46

Table 3.2: Average U.S. history scale scores by student eligibility forFree/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001 47

Figure 3.14: Percentage of students within and at or aboveU.S. history achievement levels by Free/Reduced-Price School Lunchprogram eligibility, grades 4, 8, and 12: 2001 48


Figure 4.1: The two sets of NAEP results based on a split-sample design 52

Table 4.1: National average U.S. history scale scores by type of results,grades 4, 8, and 12: 2001 54

Table 4.2: Percentage of students within and at or above U.S. historyachievement levels by type of results, grades 4, 8, and 12: 2001 55

Table 4.3: National average U.S. history scale scores by gender and type of results,grades 4, 8, and 12: 2001 55

Table 4.4: Percentage of students within and at or above U.S. historyachievement levels by gender and type of results, grades 4, 8, and 12: 2001 56

Table 4.5: National average U.S. history scale scores by race/ethnicityand type of results, grades 4, 8, and 12: 2001 57

Table 4.6: Percentage of students within and at or above U.S. history achievementlevels by race/ethnicity and type of results, grades 4, 8, and 12: 2001 58


Table 5.1: Grade 4 Sample Question 1 Results (Multiple-Choice) 60



Table 5.4: Grade 4 Sample Question 4 Results (Extended Constructed-Response) . 64



Table 5.7: Grade 8 Sample Question 7 Results (Short Constructed-Response) 70

Table 5.8: Grade 8 Sample Question 8 Results (Short Constructed-Response) 73





Figure 5.1: Grade 4 Item MapMap of selected item descriptions on theNational Assessment of Educational Progress U.S. history scale for grade 4 85



vi TABLE OF CONTENTS o U.S. HISTORY REPORT CARD a


Table 6.1: Percentage of students and average U.S. history scale scoresby teachers' reports on the amount of time spent on social studiesin a typical week at grade 4: 2001 90Table 6.2: Percentage of students and average U.S. history scale scoresby teachers' reports on the use of state/local standards in planninginstruction at grades 4 and 8: 2001 92Table 6.3: Percentage of students and average U.S. history scale scoresby teachers' reports on frequency of classroom activities atgrades 4 and 8: 1994 and 2001 94Table 6.4: Percentage of students and average U.S. history scale scoresby students' reports on frequency of classroom activities at grade 12:1994 and 2001 97

Table 6.5: Percentage of students and average U.S. history scale scoresby students' reports on computer use at grade 4: 2001 100Table 6.6: Percentage of students and average U.S. history scale scoresby students' reports on computer use at grades 8 and 12: 2001 101

1 0

TABLE OF CONTENTS . U.S. HISTORY REPORT CARD vii

xecutive Summary

The National Assessment of Educational Progress (NAEP) is

the nation's only ongoing representative sample survey of

student achievement in core subject areas. In 2001, NAEP

conducted a national U.S. history assessment of fourth-,

eighth-, and twelfth-grade students.

Authorized by Congress and administered by the National

Center for Education Statistics (NCES) in the U.S.

Department of Education, NAEP regularly reports to the

public on the educational progress of students in grades 4, 8,

and 12. This report presents the results of the NAEP 2001

U.S. history assessment for the nation. Results in 2001 are

compared to results in 1994, the next most recent year in

which NAEP conducted a U.S. history assessment and the

only other assessment year in which the test questions were

based on the current framework. Students' performance on

the assessment is described in terms of average scores on a

0-500 scale and in terms of the percentage of students

attaining three achievement levels: Basic, Proficient, and

Advanced. The achievement levels are performance standards

adopted by the National Assessment Governing Board

(NAGB) as part of its statutory responsibilities.They are

collective judgments of what students should know and be

able to do.

11EXECUTIVE SUMMARY

TheNation's

ReportCard

Major Findings at

Grades 4, 8,

and 12

Results for

Student

Subgroups

Becoming a

More Inclusive

NAEP

Classroom

Contexts for

Learning

U.S. HISTORY REPORT CARD ix

As provided by law, the Deputy Com-missioner of Education Statistics, uponreview of a congressionally mandatedevaluation of NAEP, has determined thatthe achievement levels are to be used on atrial basis and should be interpreted andused with caution. However, both theDeputy Commissioner and NAGB believethese performance standards are useful forunderstanding trends in student achieve-ment.They have been widely used bynational and state officials as a commonyardstick of academic performance.

In addition to providing average scoresand achievement-level performance in U.S.history for the nation's fourth-, eighth-,and twelfth-graders, this report providesresults for subgroups of students at thosegrade levels defined by various backgroundand contextual characteristics.

A summary of major findings from theNAEP 2001 U.S. history assessment ispresented on the following pages. Ininterpreting NAEP results, it should benoted that every test score has a standarderrora range of a few points plus orminus the scorethat includescomponents of sampling error andmeasurement error. Statistical tests thatfactor in these standard errors are used todetermine whether the differences betweenaverage scores are significant. Onlystatistically significant differences are citedin this report. Readers are also cautionedagainst making causal inferences based onNAEP results. Differences in performancebetween subgroups of students, forexample, reflect a variety of socioeconomicand educational factors.

112

EXECUTIVE SUMMARY U.S. HISTORY REPORT CARD

Major Findings atGrades 4, 8, and 121: Average U.S. history scores for fourth-

and eighth-graders were higher in2001 than in 1994, while the perfor-mance of twelfth-graders remainedrelatively stable.

O Score increases were evident among thelower-performing students at grade 4 (atthe 10th and 25th percentiles) and forboth lower- and higher-performingstudents at grade 8 (25th, 75th, and 90thpercentiles).

O Results of the 2001 U.S. history assess-ment show 18 percent of fourth-graders,17 percent of eighth-graders, and 11percent of twelfth-graders performing ator above the Proficient levelidentifiedby NAGB as the level at which allstudents should perform.

O At grade 4, the percentage of studentsperforming at or above Basic in 2001 washigher than in 1994. At grade 8, thepercentages of students performing at orabove Basic, at or above Proficient, and atAdvanced increased between 1994 and2001. At grade 12, however, the per-centages performing at or above eachlevel remained the same as in 1994.

Results for Student SubgroupsIn addition to overall results, NAEP reportson the performance of various subgroupsof students. Observed differences betweenstudent subgroups in NAEP U.S. historyperformance reflect a range of socioeco-nomic and educational factors notaddressed in this report or by NAEP

GenderO Any apparent differences in the average

scores of male and female students in2001 were not statistically significant atany of the three grades.

O At grade 4, both male and female stu-dents had higher average scores in 2001than in 1994. At grade 8, the averagescore of males increased between 1994and 2001, while the performance offemales remained stable.

Race/EthnicityIn 2001, the average scores of Whitestudents were higher than those of Black,Hispanic, and American Indian studentsat all three grades. Asian/Pacific Islanderstudents scored higher than Blackand Hispanic students across the gradesas well.

O At grade 4, both White students andBlack students had higher average scoresin 2001 than in 1994. At grade 8, onlyWhite students showed a gain since1994. At grade 12, only Hispanic stu-dents had higher average scores in 2001than in 1994.

O The 2001 results show a narrowing ofthe score point difference betweenWhite students and Black students atgrade 4, and between White students andHispanic students at grade 12.

Region of the CountryO Fourth- and eighth-grade students in the

Northeast, Southeast, and Central re-gions all had higher average scores thanstudents in the West. Fourth- andeighth-grade students in the Centralregion outperformed their peers in theSoutheast. There was no statisticallysignificant difference in the performanceof twelfth-graders from various regionsof the country.

O At grade 4, only the Northeast regionshowed a gain in the U.S. history averagescore since 1994. At grade 8, the onlyincrease occurred in the Southeastregion.

Parents' Highest Level of EducationO The 2001 results show a clear positive

relationship overall between parentaleducation level and the performance ofeighth- and twelfth-graders.

O At grade 8, the average score of studentswhose parents graduated from collegewas higher in 2001 than in 1994. Atgrade 12, there was an increase in theaverage score of students whose parentsdid not finish high school.

Type of School

O The 2001 results show that public schoolstudents at all three grades had loweraverage U.S. history scores than theirpeers attending nonpublic schools.

O Average scores among both fourth- andeighth-grade public school students werehigher in 2001 than in 1994.

Type of LocationO At grades 4 and 8, students attending

schools in rural and urban fringe loca-tions had higher average scores thanstudents in central city schools. At grade12, students attending schools in urbanfringe locations had higher scores thanstudents in both rural and central citylocations.

Eligibility for Free/Reduced-PriceSchool Lunch Programo At every grade, the average score of

students who were eligible for the Free/Reduced-Price School Lunch programwas lower than the average score ofstudents who were not eligible for theprogram (i.e., those not meeting thepoverty guidelines).

13EXECUTIVE SUMMARY U.S. HISTORY REPORT CARD xi

Becoming a More Inclusive NAEPIn the 2001 U.S. history assessment, theNAEP program used a split-sample design, sothat trends in students' history achievementcould be reported across assessment yearsand, at the same time, the program couldcontinue to examine the effects of includ-ing special-needs students assessed withaccommodations.While most of the resultsin this report include only the performanceof students assessed without accommoda-tions, the report also presents an overviewof a second set of results that include theperformance of special-needs students whorequired and were provided accommoda-tions during the assessment administration.

0 At grade 8, the average score whenaccommodations were permitted waslower than the average score whenaccommodations were not permitted.However, there were no statisticallysignificant differences between averagescores in the accommodations-permittedresults and the accommodations-not-permitted results at grades 4 and 12.

Classroom Contexts for LearningNAEP collects information about thecontexts for student learning by adminis-tering questionnaires to assessed students,their teachers, and their school administra-tors. Using the student as the unit ofanalysis, NAEP examines the relationshipbetween selected contextual variables drawnfrom these questionnaires and students'average scores on the U.S. history assessment.

Time Spent on Social Studies0 In 2001, fourth-graders whose teachers

reported spending more than 180minutes on social studies instruction in atypical week had higher average scoresthan those whose teachers reportedspending less time.

14xii EXECUTIVE SUMMARY U.S. HISTORY REPORT CARD

State and Local StandardsO About two-thirds of the fourth- and

eighth-graders assessed had teachers whoreported that they used state or localstandards to a large extent in planningsocial studies instruction. There were nostatistically significant differences instudents' performance at either grade 4or grade 8 based on the extent to whichteachers reported using such standardsin planning instruction.

Instructional ActivitiesO A large majority of fourth-graders had

teachers who reported having them readmaterial from a textbook on a daily orweekly basis. Reading from a textbookdaily was associated with higher averagescores than was doing so on a weekly ormonthly basis.

O Eighth-graders whose teachers reportedusing primary historical documents suchas letters, diaries, or essays written byhistorical figures, on a weekly basis hadhigher average scores than those whoseteachers did so less frequently.

O Twelfth-graders who reported neverreading extra material, such as biogra-phies or historical stories, scored lower,on average, than those who reporteddoing so a few times a year or moreoften.

Use of TechnologyO A strong positive association was evident

between using computers for conduct-ing research and for writing reports andperformance at grades 8 and 12.

O Students in grades 4, 8, and 12 whoreported daily general use of computersat school for social studies or history hadlower average scores than those whoreported less frequent general use. Itshould be noted that relatively fewstudents reported using a computer forhistory or social studies.

(--- I NAEP 2001 U.S. History Assessment

IntroductionKnowledge of United States history is an important

component of effective citizenship. Having a thorough grasp

of our country's struggles and achievements better enables

young people to make informed and intelligent decisions

about contemporary issues. Nourishing the curiosity

children exhibit about major events, customs and

institutions, and the families and individuals that

comprise United States history, creates a valuable

resource for our nation's future.

Efforts to improve the rigor and quality of

history education have been an important element of

the standards-based educational reform movement of

recent years. This emphasis on high expectations has

been demonstrated by the rapid increase in the

number of states that have, within the last 10 years,

established content standards for history or social

studies; between 1995 and 2000 the number of states

with such standards increased from 20 to 46.' The

renewed interest in history instruction underscores

the need for accurate information about what

students know and can do in U.S. history.

What is the NAEP

U.S. history

assessment?

How does the

NAEP U.S. history

assessment

measure and

report student

progress?

I Council of Chief State School Officers. (2000). Key state education policies on K-12education: 2000. (table 13, p. 23).Washington, DC: Author.

1 5CHAPTER 1

ChapterContents

Overview

U.S. History

Framework

U.S. History

Assessment

School and

Student Samples

Reporting

Results

NAEP

Achievement

Levels

InterpretingNAEP Results

This Report

U.S. HISTORY REPORT CARD 1

Overview of the 2001National Assess ent ofEducafional ProgressFor over 30 years, the National Assessmentof Educational Progress (NAEP) has beenauthorized by Congress to collect, analyze,and report reliable and valid informationabout what American students know andcan do in core subject areas. NAEP assessesthe performance of public and nonpublicschool students in grades 4, 8, and 12. In2001, student performance in U.S. historyand geography was assessed at all threegrades.This report deals only with theresults of the U.S. history assessment.

All NAEP assessments are based oncontent frameworks developed through anational consensus process. The NAEP2001 U.S. history assessment was thesecond administration of an assessmentbased on the NAEP US. History Framework,which was originally developed for the1994 assessment.2 In both 1994 and 2001,assessments based on the framework wereadministered to national samples of fourth-,eighth-, and twelfth-graders.

This report describes the results of the2001 U.S. history assessment at grades 4, 8,and 12 and compares results in 2001 tothose in 1994. Comparisons across assess-ment years are possible because the assess-ments were developed under the same

basic framework and share a common setof U.S. history questions. In addition, thepopulations of students were sampled andassessed using comparable procedures.

The U.S. History FrameworkAlthough U.S. history was assessed byNAEP in 1986 and 1988, a rigorous newNAEP U.S. History Framework wasdeveloped for the 1994 assessment. Thenew framework provided the operationalspecifications for both the 1994 and 2001assessments. The development of theframework was managed by the Council ofChief State School Officers (CCSSO)under the direction of the National Assess-ment Governing Board (NAGB). Approxi-mately 50 professional historians, educators,administrators, and other interested indi-viduals worked to achieve consensus on thegeneral goals as well as the specific lan-guage of the framework. In addition,several hundred educational experts andinterested members of the public contrib-uted to the process, either by participatingin public hearings or by reviewing drafts.The framework document produced bythis consensus process called for the assess-ment of a broad range of outcomes. Itrepresented an ambitious vision both ofwhat students should know and be able todo in U.S. history, and of the ways in whichthose competencies should be tested.

2 National Assessment Governing Board. (1993). US. history framework for the 1994 National Assessment of EducationalProgress.Washington, DC:Author.

s2 CHAPTER 1 U.S. HISTORY REPORT CARD

The framework is organized aroundthree concepts or dimensions: majorthemes of U.S. history, chronologicalperiods, and ways of knowing and thinkingabout U.S. history with the four themesproviding the core organizing structure ofthe framework. The themes were intendedto ensure that all major branches of

historical study were covered and thatemphasis on various areas was balanced.The themes are also used to define thesubscales that make up the NAEP U.S.history composite scale. (See appendix Afor more information on how the scale wasconstructed.) Figure 1.1 provides descrip-tions of each theme.

Change and Continuity in American Democracy: Ideas, Institutions, Practices, and Controversies

This theme concerns the development of American political democracy from colonial times to

the present. It covers political events that shaped American democracy, such as the American

Revolution, the Civil War, the fight for civil rights, as well as the core ideas and principles that

underlie our institutions. This theme covers students' knowledge of the founding of the nation,

the writing of the Constitution, and other fundamental components of the nation's political

history. At the same time, it calls for evaluating students' understanding of the role that

major political ideas and conflicts have played at different points in our history.

The Gathering and Interactions of Peoples, Cultures, and Ideas

This theme is broadly defined because it covers a vast component of U.S. history: the

interactions among the peoples and cultures of many countries, racial and ethnic groups, and

religious traditions that have contributed to the development of American society. This theme

covers immigration, cultural developments, patterns of social organization, and changing

roles of men and women.

Economic and Technological Changes and Their Relation to Society, Ideas, and the Environment

This theme focuses on the economic history of the nation and its development from a rural,

agricultural society to an urban, industrialized superpower. It also covers the roles of

geography and of developments in science and technology in bringing about socio-economic

change.

The Changing Role of America in the World

This theme calls for coverage of the many factorspolitical ideas, economic interests, public

opinionthat have shaped American foreign policy. It also addresses specific interactions

between the United States and other nations and domestic consequences of developments in

foreign policy.

SOURCE: National Assessment Governing Board. U.S. History Framework for the 1994 and 2001 National Assessment of Educational Progress.

CHAPTER 1 U.S. HISTORY REPORT CARD 3

Eight periods provide chronologicalstructure for the many issues included inthe four themes. These periods focusattention on several major eras of U.S.history. They overlap at some pointsbecause they were conceived to ensurethorough coverage of major trends andevents. The historical periods are not usedas subscales, but rather were used in theassessment development process to ensureappropriate chronological coverage. Theperiods are as follows:

o Three Worlds and TheirMeeting in the Americas(Beginnings to 1607)

0 Colonization, Settlement,and Communities(1607 to 1763)

0 The Revolution and the New Nation(1763 to 1815)

O Expansion and Reform(1801 to 1861)

O Crisis of the Union:Civil War and Reconstruction(1850 to 1877)

O The Development of Modern America(1865 to 1920)

O Modern America and the World Wars(1914 to 1945)

O Contemporary America(1945 to Present)

The percentages of assessment timeallotted to each theme and period de-scribed in the framework are presented intables 1.1 and 1.2, respectively. It should benoted that these percentages vary some-what from the targeted distribution. (Seeappendix A,Table A.1, for a comparison ofthe actual and targeted distributions.)

Distribution of assessment time across historical themes, grades 4, 8, and 12: 2001

Change and The Gathering Economic and The Changing

Continuity in and Interactions Technological Role of

American Democracy:

Ideas, Institutions,

Practices, and

of Peoples, Cultures,

and Ideas

Changes and Their

Relation to Society,

Ideas, and the

America

in the World

Controversies Environment

4 CHAPTER 1

32%

32%

26%

32°/0

25%

22%

NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 2001 U.S. History Assessment.

U.S. HISTORY REPORT CARD

Distribution of assessment time

Beginnings

to 1607

1607 to

1763

Grade 4 13% 14°/0

Grade 8 3% 70/3

Grade 12 1% 80/0

across

1763

to 1801

1815 to

1861

11% l 4 O/0

historical periods, grades 4, 8, and 12: 2001

9 %

11%

1850 1914

to 1865 to 1945

1877 to 1945 to1920 Present

90/0 13% 7 % 10 %

8 % 18% 14% 10 %

7% 11 % 32 % 20%

No

period*

9%

12%

1%

NOTE: Percentages may not add to 100 due to rounding.

" A number of questions (especially at grades 4 and 8) assessed general historical skills and could not be appropriately classified in any given period.

/ This high percentage is largely a function of the 50-minute theme block, which focuses on the World War II homefront. If this block were excluded fromcalculations, questions covering this period would make up 22% of the assessment at grade 12.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 U.S. History Assessment.

The framework also considers ways ofknowing and thinking about U.S. history.These are divided into two general cogni-tive domains that were used as a guide inexercise development. The two domainsand their definitions are as follows:

0 Historical Knowledge and PerspectiveThis domain includes knowing and

understanding people, events, concepts,themes, movements, contexts, and historicalsources; sequencing events; recognizingmultiple perspectives and seeing an era ormovement through the eyes of differentgroups; and developing a generalconceptualization of U.S. history

0 Historical Analysis and Interpretation

This domain includes explaining issues,identifying historical patterns; establishingcause-and-effect relationships; finding valuestatements; establishing significance; apply-ing historical knowledge; weighing evi-dence to draw sound conclusions; making

defensible generalizations; and renderinginsightful accounts of the past.

U.S. History AssessmentinstrumentsAs the only federally authorized ongoingassessment of U.S. history achievement, theNAEP assessment must reflect the spirit ofthe framework as well as the specificationsprovided by it. In order to achieve thosegoals, the assessment development processinvolves stages of review by teachers andteacher educators, state officials, and mea-surement experts. All components of theassessment are evaluated for curricularrelevance, developmental appropriateness,and fairness. Final approval of NAEP testquestions is given by the National Assess-ment Governing Board. A list of the U.S.history development committee membersfor the 2001 assessment is provided inappendix C.

19 CHAPTER 1 U.S. HISTORY REPORT CARD 5

The 2001 U.S. history assessment book-lets at grades 4, 8, and 12 generally con-tained three or four sections: a set ofgeneral background questions, a set ofsubject-specific background questionsdealing largely with the student's use oftechnology, and one or two sets, or"blocks," of cognitive questions assessingknowledge and skills in U.S. history. Atgrades 8 and 12, students were given eithertwo 25-minute blocks or one 50-minuteblock. At grade 4, however, only 25-minute blocks were used.

At grade 4 a total of six sections, or"blocks", of cognitive questions were given,while at grades 8 and 12 nine blocks wereadministered. In addition to the cognitivequestions, each assessment booklet alsoincluded a set of background questions thatasked students to give information abouttheir school practices, such as the frequencywith which they use computers at schoolfor social studies, do research projects usinga CD or the Internet, and write reports.The assessment time for each grade was 50minutes plus the 10-15 minutes needed tocomplete the background questions.

Each block of cognitive questionsconsisted of both multiple-choice and"constructed-response" questions. ("Con-structed-response" is the term used todescribe test questions in which studentswrite a response, as distinct from multiple-choice questions, in which students choosean answer from one of several options.)Typically, a block will contain about 16-18questions, but there is considerable varia-tion depending on the balance betweenmultiple-choice and constructed-responsequestions. Overall, more than 50 percent of

6 CHAPTER 1 U.S. HISTORY REPORT CARD

student assessment time was devoted to thelatter question type. Two types of con-structed-response questions were used:

El short constructed-response questionsthat required students to provide an-swers, usually in response to a text orvisual stimulus, in one or two sentences;and

o extended constructed-response questionsthat required students to provide answersof a paragraph or more in length

The 50-minute blocks administered atgrades 8 and 12 included questions focus-ing on a particular theme, and includedextended constructed-response questionsrequiring students to synthesize elementsfrom various primary sources. The totalnumber of test questions used in grades 4,8, and 12 were 94, 145, and 154, respec-tively. Each student answered only a smallportion of the total number of questions.Additional information about the design ofthe 2001 U.S. history assessment is pre-sented in appendix A.

Description of Schooland Student SamplesThe NAEP 2001 U.S. history assessmentincluded representative samples of bothpublic and nonpublic schools. Approxi-mately 7,000 fourth-graders, 11,000eighth-graders, and 11,000 twelfth-graderswere assessed. The number of schools inthe reporting sample were 365 at gradefour, 369 at grade eight, and 374 at gradetwelve. Each selected school that partici-pated in the assessment and each studentassessed represent a portion of the popula-tion of interest. For additional informationon sample sizes and participation rates, seeappendix A.

This report contains two different sets ofnational results based on two reportingsamples that differed in terms of whetheror not accommodations were made avail-able to special-needs students. The nationalresults presented in chapters 2,3,5, and 6of this report are based on a nationallyrepresentative sample that included special-needs students only if they could be as-sessed meaningfully without accommoda-tions. These results can be compared tothose from 1994, because accommodationswere also not made available in that assess-rnent year. Chapter 4 presents a second setof national results from 2001 for a repre-sentative sample that includes the perfor-mance of students who required and wereprovided with accommodations (e.g.,bilingual dictionary, extended time, smallgroup testing). No comparison of theseresults to those from 1994 can be madebecause of the inclusion of these accom-modated special-needs students.

In the sample that did not permit ac-commodations, 7 percent of fourth-graders,8 percent of eighth-graders, and 4 percentof twelfth-graders were excluded from theU.S. history assessment in 2001. School stafffamiliar with these students made thedeterinination that these students could notbe assessed meaningfully without accom-modations, because of their disability and/or limited English proficiency. In 1994,5percent at both the fourth and eighthgrades, and 3 percent at the twelfth gradewere excluded. Additional informationregarding exclusion rates is also provided inappendix A.

Reporting theAssessment Resu itsStudent performance on the NAEP U.S.history assessment is presented in two ways:as average scores on the NAEP U.S. historyscale, and in terms of the percentage ofstudents attaining NAEP U.S. historyachievement levels. The average scalescores are a measure of students' perfor-mance on the assessment. The achieve-ment level results indicate the degree towhich student performance meets expecta-tions of what they should know and beable to do.

Average scale score results are presentedon the NAEP U.S. history composite scale,which ranges from 0-500. Students' re-sponses on the NAEP 2001 U.S. historyassessment were analyzed to determine thepercentages of students that respondedcorrectly to each multiple-choice questionand the percentages of students that re-sponded at each score level for the con-structed-response questions. Scales thatsummarize results for each of the fourthemes described earlier were created. Thecomposite scale is a weighted average ofthe separate subscales for the four themes.The weight for each theme corresponds tothe theme's relative importance in theNAEP U.S. history framework.A fulldescription of NAEP scale procedures canbe found in the forthcoming NAEP 2001Technical Report.

Achievement-level results are presentedin terms of U.S. history achievement levelsas authorized by the NAEP legislation andadopted by the National AssessmentGoverning Board.' For each grade tested,

3 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.107-110 (H.R. 1).

National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297,20, U.S.C. 1211.

21CHAPTER 1 U.S. HISTORY REPORT CARD 7

NAGB has adopted three achievementlevels: Basic, Proficient, and Advanced. For

reporting purposes, the achievement-levelcut scores are placed on the U.S. historyscale, resulting in four ranges: below Basic,Basic, Proficient, and Advanced.

The Setting ofAchievement LevelsThe 1988 NAEP legislation that createdthe National Assessment Governing Boarddirected the Board to identify "appropriateachievement goals...for each subject area"that NAEP measures.4 The 2001 NAEPreauthorization reaffirmed many of theBoard's statutory responsibilities, includingdeveloping "appropriate student achieve-ment levels for each grade or age in eachsubject area to be tested ... "5 In order tofollow this directive and achieve the man-date of the 1988 statute to "improve theform and use of NAEP results," NAGBundertook the development of studentperformance standards called "achievementlevels." Since 1990 the Board has adoptedachievement levels in mathematics, reading,U.S. history, geography, science, writing,and civics.

The Board defined three levels for eachgrade: Basic, Proficient, and Advanced. TheBasic level denotes partial mastery of theknowledge and skills that are fundamentalfor proficient work at a given grade. TheProficient level represents solid academicperformance. Students reaching this leveldemonstrate competency over challengingsubject matter. The Advanced level pre-sumes mastery of both the Basic andProficient levels. Figure 1.2 presents thepolicy definitions of the achievement levelsthat apply across all grades and subjectareas. The policy definitions guided thedevelopment of the U.S. history achieve-ment levels, as well as the achievementlevels established in all other subject areas.Adopting three levels of achievement foreach grade signals the importance oflooking at more than one standard ofperformance. The Board believes, however,that all students should reach the Proficientlevel: the Basic level is not the desired goal,but rather represents partial mastery that isa step toward Proficient.

L:4 Figure 1.2

4: Achievement Levels

1,192 KEI? achievement Ucelb

Basic This level denotes partial mastery of prerequisite knowledge and skills that arefundamental for proficient work at each grade.

Proficient This level represents solid academic performance for each grade assessed. Studentsreaching this level have demonstrated competency over challenging subject matter,including subject-matter knowledge, application of such knowledge to real-worldsituations, and analytical skills appropriate to the subject matter.

Advanced

8 CHAPTER 1

This level signifies superior performance.


4 National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297, 20, U.S.C. 1211.


U.S. HISTORY REPORT CARD22

The achievement levels in this reportwere adopted by the Board based on astandard-setting process designed andconducted under a contract with ACT, Inc.To develop these levels, ACT convened across section of educators and interestedcitizens from across the nation and askedthem to judge what students should knowand be able to do relative to a body ofcontent reflected in the NAEP frameworkfor U.S. history. This achievement-level-setting process was reviewed by a variety ofindividuals including policymakers, repre-sentatives of professional organizations,teachers, parents, and other members of thegeneral public. Prior to adopting theselevels of student achievement, NAGBengaged a large number of persons tocomment on the recommended levels andto review the results.

The results of the achievement-level-setting process, after NAGB's approval,became a set of achievement-level descrip-tions and a set of achievement-level cutpoints on the 0-500 NAEP U.S. historyscale. The cut points are the scores that

define the boundaries between below Basic,Basic, Proficient, and Advanced performance

at grades 4, 8, and 12. The Board estab-lished these U.S. history achievement levelsbased upon the U.S. history content frame-work.

Achievement-Levelescriptions for Each Grade

Specific definitions of the Basic, Proficient,and Advanced U.S. history achievementlevels for grades 4, 8, and 12 are presentedin figures 1.3 through 1.5. As noted previ-ously, the achievement levels are cumula-tive. Therefore, students performing at theProficient level also display the competenciesassociated with the Basic level, and studentsat the Advanced level also demonstrate theskills and knowledge associated with boththe Basic and the Proficient levels. For eachachievement level listed in figures 1.3through 1.5, the scale score that corre-sponds to the beginning of that level isshown in parentheses. For example, infigure 1.3 the scale score of 243 corre-sponds to the beginning of the grade 4Proficient level of achievement.

23

CHAPTER1 U.S. HISTORY REPORT CARD 9

Basic Fourth-grade students performing at the Basic level should be able to identify and describe a

(195) few of the most familiar people, places, events, ideas, and documents in American history.

They should be able to explain the reasons for celebrating most national holidays, have some

familiarity with the geography of their own state and the United States, and be able to express

in writing a few ideas about a familiar theme in American history.

Proficient I Fourth-grade students performing at the Proficient level should be able to identify, describe

(243) and comment on the significance of many historical people, places, ideas, events, and

documents. They should interpret information from a variety of sources, including texts, maps,

pictures, and timelines. They should be able to construct a simple timeline from data. These

students should recognize the role of invention and technological change in history. They

should also recognize the ways in which geographic and environmental factors have influ-

enced life and work.

Advanced Fourth-grade students performing at the Advanced level should have a beginning

(276) understanding of the relationship between people, places, ideas, events, and documents.

They should know where to look for information, including reference books, maps, local

museums, interviews with family and neighbors, and other sources. They should be able to

use historical themes to organize and interpret historical topics, and to incorporate insights

from beyond the classroom into their understanding of history. These students should

understand and explain the role of invention and technological change in history. They should

also understand and explain the ways in which geographic and environmental factors have

influenced life and work.


2 4


Basic(252)

Proficient(294)

Advanced(327)

Eighth-grade students performing at the Basic level should be able to identify and place in

context a range of historical people, places, events, ideas, and documents. They should be

able to distinguish between primary and secondary sources. They should have a beginning

understanding of the diversity of the American people and the ways in which people from a

wide variety of national and cultural heritages have become part of a single nation. Eighth-

grade students at the Basic level should also have a beginning understanding of the

fundamental political ideas and institutions of American life and their historical origins. Theyshould be able to explain the significance of some major historical events.

Eighth-grade students performing at the Proficient level should be able to explain the

significance of people, places, events, ideas, and documents, and to recognize the connection

between people and events within historical contexts. They should understand and be able to

explain the opportunities, perspectives and challenges associated with a diverse cultural

population. They should incorporate geographic, technological, and other considerations in

their understanding of events and should have knowledge of significant political ideas and

institutions. They should be able to communicate ideas about historical themes while citing

evidence from primary and secondary sources to support their conclusions.

Eighth-grade students performing at the Advanced level should recognize significant themes

and movements in history and begin to understand particular events in light of these themes

and movements. They should have an awareness of continuity and change over time and be

able to draw relevant analogies between past events and present-day situations. They should

be able to frame questions about historical topics and use multiple sources to develop

historical generalizations and interpretations. They should be able to explain the importance

of historical themes, including some awareness of their political, social, and economicdimensions.


25


Figure,

AchiOemen

Level

Descriptions

Basic(294)

Proficient(325)

Advanced(355)

ce KO? Tgo history achievement tb. grade

Twelfth-grade students performing at the Basic level should be able to identify the

significance of many people, places, events, dates, ideas, and documents in U.S. history. They

should also recognize the importance of unity and diversity in the social and cultural history

of the United States, and an awareness of American's changing relationships with the rest of

the world. They should have a sense of continuity and change in history and be able to relate

relevant experience from the past to their understanding of contemporary issues. They should

recognize that history is subject to interpretation and should understand the role of evidence

in making an historical argument.

Twelfth-grade students performing at the Proficient level should understand particular

people, places, events, ideas, and documents in historical context, with some awareness of

the political, economic, geographic, social, religious, technological, and ideological factors

that shape historical settings. They should be able to communicate reasoned interpretations

of past events, using historical evidence effectively to support their positions. Their written

arguments should reflect some in-depth grasp of issues and refer to both primary and

secondary sources.

Twelfth-grade students achieving at the Advanced level should demonstrate a comprehensive

understanding of events and sources of U.S. history. Recognizing that history is subject to

interpretation, they should be able to evaluate historical claims critically in light of the

evidence. They should understand that important issues and themes have been addressed

differently at different times and that America's political, social, and cultural traditions have

changed over time. They should be able to write well-reasoned arguments on complex

historical topics and draw upon a wide range of sources to inform their conclusions.



26

The Trial Status ofAchievement LevelsThe 2001 NAEP reauthorization lawrequires that the achievement levels beused on a trial basis until the Commis-sioner of Education Statistics determinesthat the achievement levels are "reasonable,valid, and informative to the public."6 Untilthat determination is made, the law re-quires the Commissioner arid the Board tostate clearly the trial status of the achieve-ment levels in all NAEP reports.

In 1993, the first of several congression-ally mandated evaluations of the achieve-ment level setting process concluded thatthe procedures used to set the achievementlevels were flawed and that the percentageof students at or above any particularachievement level cutpoint may be under-estimated.' Others have critiqued theseevaluations, asserting that the weight of theempirical evidence does not support suchconclusions.8

In response to the evaluations andcritiques, NAGB conducted an additionalstudy of the 1992 reading achievement

levels before deciding to use those readingachievement levels for reporting 1994NAEP results.9When reviewing the find-ings of this study, the National Academy ofEducation (NAE) Panel expressed concernabout what it saw as a "confirmatory bias"in the study and about the inability of thisstudy to "address the panel's perception thatthe levels had been set too high."° In 1997,the NAE Panel summarized its concernswith interpreting NAEP results based onthe achievement levels as follows:

First, the potential instability of the levels

may inteere with the accurate portrayal oftrends. Second, the perception that few American

students are attaining the htgher standards we

have set for them may deflect attention to the

wrong aspects of education reform.The public has

indicated its interest in benchmarking against

international standards, yet it is noteworthy that

when American students pey-ormed very well on

a 1991 international reading assessment, these

results were discounted because they were

contradicted by poor peyOrmance against the

possibly flawed NAEP reading achievement

levels in the following year''


7 United States General Accounting Office. (1993). Education achievement standards: NAGB's approach yields misleadinginterpretations. U.S. General Accounting Office Report to Congressional Requestors.Washington, DC:Author.

National Academy of Education. (1993). Setting performance standards for achievement:A report of the National Acadentyof Education Panel on thc evaluations of the NAEPTrial State Assessment:An evaluation qf the 1992 achievement levels.Stanford, CA: Author.

8 Cizek, G. (1993). Reactions to National Academy of Education report. Washington, DC: National Assessment GoverningBoard.

Kane, M. (1993). Continents on the NAE evaluation of the NAGB achievement levels. Washington, DC: NationalAssessment Governing Board.

9 American College Testing. (1995). NAEP reading revisited:An evaluation of the 1992 achievement level descriptions.Washington, DC: National Assessment Governing Board.

10 National Academy of Education. (1996). Reading achievement levels. In Quality and utility:The 1994 Trial StateAssessment in reading.The fourth report of the National Academy of Education Panel on the evaluation of the NAEP TrialState Assessment. Stanford, CA:Author.

National Academy of Education. (1997). Assessment in transition:Monitoring the nation's educational progress (p. 99).Mountain View, CA: Author.

11

2 7CHAPTER 1 U.S. HISTORY REPORT CARD 13

The National Center for EducationStatistics and the National AssessmentGoverning Board have sought and con-tinue to seek new and better ways to setperformance standards on NAEP.12 Forexample, NCES and NAGB jointly spon-sored a national conference on standardsetting in large-scale assessments, whichexplored many issues related to standardsetting.' Although new directions werepresented and discussed, a proven alterna-tive to the current process has not yet beenidentified.The Deputy Commissioner ofEducation Statistics and the Board con-tinue to call on the research community toassist in finding ways to improve standardsetting for reporting NAEP results.

The most recent congressionally man-dated evaluation conducted by the Na-tional Academy of Sciences (NAS) reliedon prior studies of achievement levels,rather than carrying out new evaluations,on the grounds that the process has notchanged substantially since the initialproblems were identified. Instead, the NASPanel studied the development of the 1996science achievement levels.The NAS Panelbasically concurred with earlier congres-sionally mandated studies.The Panelconcluded that "NAEP's current achieve-

14 CHAPTER 1

ment level setting procedures remainfundamentally flawed.The judgment tasksare difficult and confusing; raters' judg-ments of different item types are internallyinconsistent; appropriate validity evidencefor the cut scores is lacking; and the processhas produced unreasonable results."4

The NAS Panel accepted the continuinguse of achievement levels in reportingNAEP results on a developmental basis,until such time as better procedures can bedeveloped. Specifically, the NAS Panelconcluded that "....tracking changes in thepercentages of students performing at orabove those cut scores (or, in fact, anyselected cut scores) can be of use in de-scribing changes in student performanceover time."15

The National Assessment GoverningBoard urges all who are concerned aboutstudent performance levels to recognizethat the use of these achievement levels is adeveloping process and is subject to variousinterpretations.The Board and the DeputyCommissioner believe that the achieve-ment levels are useful for reporting trendsin the educational achievement of studentsin the United States.'6 In fact, achievementlevel results have been used in reports bythe President of the United States, the

12 Reckase, Mark, D. (2000). The evolution of the NAEP achievement levels setting process:A summary of the research anddevelopment efforts conducted by ACT Iowa City, IA:ACT, Inc.

13 National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedings of the jointconference on standard setting for large-scale assessments of the National Assessment Governing Board (NAGB) and theNational Center for Education Statistics (NCES).Washington, DC: Government Printing Office.

14 Pellegrino, J.W., Jones, L.R., & Mitchell, K.J. (Eds.). (1998). Grading the nation's report card: evaluating NAEP andtransforming the assessment of educational progress. Committee on the Evaluation of National Assessments of Educa-tional Progress, National Research Council. (p.182). Washington, DC: National Academy Press.

15 Ibid., page 176.

16 Forsyth, Robert A. (2000).A description of the standard-setting procedures used by three standardized testpublishers. In Student performance standards on the National Assessment of Educational Progress:Affirmations andimprovements.Washington, DC: National Assessment Governing Board.

Nellhaus, JetTrey M. (2000). States with NAEP-like performance standards. In Student performance standards on theNational Assessment of Educational Progress:Affirmations and improvements.Washington, DC: National AssessmentGoverning Board.


Secretary of Education, state governors,legislators, and members of Congress.Government leaders in the nation and inmore than 40 states use these results intheir annual reports.

However, based on the congressionallymandated evaluations so far, the DeputyCommissioner agrees with the NationalAcademy's recommendation that cautionneeds to be exercised in the use of thecurrent achievement levels.Therefore, theDeputy Commissioner concludes thatthese achievement levels should continueto be used on a trial basis and shouldcontinue to be interpreted with caution.

Interpreting NAEP ResultsThe average scores and percentages pre-sented in this report are estimates becausethey are based on samples of students ratherthan on entire populations. Moreover, thecollection of questions used at each gradelevel is but a sample of the many questionsthat could have been asked to assess studentknowledge of the framework content. Assuch, the results are subject to a measure ofuncertainty, reflected in the standard errorof the estimates. The standard errors forthe estimated scale scores and percentagesin this report are provided in appendix B.

The differences between scale scores andbetween percentages discussed in thefollowing chapters take into account thestandard errors associated with the esti-mates. Comparisons are based on statisticaltests that consider both the magnitude ofthe difference between the group averagescores or percentages and the standarderrors of those statistics. Throughout thisreport, differences between scores orbetween percentages are pointed out onlywhen they are significant from a statisticalperspective.All differences reported aresignificant at the 0.05 level with appropri-ate adjustments for multiple comparisons.The term significant is not intended toimply a judgment about the absolutemagnitude or the educational relevance ofthe differences. It is intended to identifystatistically dependable population differ-ences to help inform dialogue amongpolicymakers, educators, and the public.

Readers are cautioned against interpret-ing NAEP results in a causal sense. Infer-ences related to student subgroup perfor-mance or to the effectiveness of public andnonpublic schools, for example, should takeinto consideration the many socioeco-nomic and educational factors that mayalso impact on performance in U.S. history.

2g


OveMew of theRemaining ReportThe results in chapters 2 and 3 of thisreport are based on the set of data with noaccommodations offered to students.Findings are presented for the nation andfor all the major reporting subgroupsincluded in all NAEP report cards. Com-parisons with results from the 1994 assess-ment are noted where the data permit.

NAEP has sought to assess samples thatare as inclusive as possible. Nevertheless,there has always been some exclusion ofstudents with disabilities (SD) and limitedEnglish proficient (LEP) students whocould not be assessed meaningfully withoutaccommodations. Local school officialshave made decisions about exclusion inaccordance with explicit criteria providedby the NAEP program. In order to expandthe proportion of students who can beassessed meaningfully, the NAEP programbegan in recent assessments to explore theuse of accommodations with special-needsstudents. Chapter 4 presents an overview ofa second set of resultsthose that includestudents who were provided accommoda-tions during the test administration. Byincluding these results in the nation's U.S.history report card, the NAEP programcontinues a phased transition toward amore inclusive reporting sample. Future


30

assessment results will be based solely on astudent and school sample in which ac-commodations are permitted.

Chapter 5 provides sample assessmentquestions and student responses from the2001 assessment.Also presented in chapter5 are item maps that position selectedquestion descriptions along the NAEP U.S.history scale where they are likely to beanswered successfully by students.Thedescriptions used on these item maps focuson the U.S. history skill or knowledgeneeded to answer the question. Chapter 6examines contexts for learning U.S. historyin terms of classroom practices and studentvariables. The data presented in bothchapters 5 and 6 are based on the set ofresults that did not include accommodatedspecial-needs students.

This report also contains appendices thatsupport or augment the results presented.Appendix A contains an overview of theNAEP U.S. history framework and specifi-cations, information on the nationalsample, and a more detailed description ofthe major reporting subgroups featured inchapters 2 and 3. Appendix B contains thefull data with standard errors for all tablesand figures in this report. Appendix Ccontains a list of the NAEP U.S. historycoinmittee members.

verage Scale Scchlevement-Lev

res ndesults

Overview

This chapter presents results for the nation from the NAEP

2001 U.S. history assessment at grades 4, 8, and 12. Student

performance is described in two ways: one, by average scores

on the NAEP U.S. history scale, which ranges

from 0 to 500; and two, in terms of the percentages of

students who attained each of the three U.S. history

achievement levels: Basic, Proficient, and Advanced.

Results of the NAEP 2001 U.S. history assessment

are compared with results from the previous

assessment, which took place in 1994. This

comparison is possible because the assessments share

a common set of tasks based on the current U.S.

history framework and because the population of

students in both years was sampled and assessed using

comparable procedures. The results presented in this

chapter are based on a representative sample of

students assessed under conditions that did not

permit accommodations for special-needs students.

These were the same conditions under which the

1994 history assessment was administered, thus

making it possible to report trends in student performance

across the assessment years. A second set of results, reflecting

part of a phased transition toward a more inclusive reporting

sample in which accommodations were permitted for

special-needs students, is presented in chapter 4.

Are the nation's

fourth-, eighth-,and twelfth-

graders making

progress in

U.S. history?

31

MI

Overview

Average Scale

Score Results

Scale Scores by

Percentile

Achieviment-Level Results


Average Scale Score ResultsThe results of the NAEP 2001 U.S. historyassessment show improvement in studentperformance over the 1994 results at grades4 and 8, but do not show a statistically

significant change at grade 12. Figure 2.1presents the average U.S. history scalescores for the nation for fourth-, eighth-,and twelfth-graders attending both publicand nonpublic schools in 1994 and 2001.

National Scale Sco

1Results

500

300

290

280

270

260 259

250

240

230

220

210

200

1994 2001

286

0

205

287 Grade 12

262* Grade 8

209* Grade 4

* Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 U.S. History Assessments.


32

Scale Scores by PercentheOne way to view students' scale scoreperformance on the U.S. history assessmentis by looking at the scale scores attained bystudents across the performancedistribution. The percentile indicates thepercentage of students whose scores fellbelow a particular average score. The

advantage of looking at the data this way isthat it reveals changes in performance forboth lower- and higher-performingstudents. Figure 2.2 presents the U.S.history scale scores for grades 4, 8, and 12at the 10th, 25th, 50th, 75th, and 90thpercentiles for both the 1994 and the 2001assessments.

500

260

250

240

230

220

210 210

200

190

180 180

170

160

150

140r

Grade 4

1994 2001

253

234

0

147

500

255 90th

235 75th

212 50th

186* 25th

158* 10th

Grade 8

1994 2001

310

300 299

290

280

270

260

250

240 239

230

220

210

200

190r

500O _

V

0

330

320

310 309

300

290 288

280

270

260

250

220 10th 240

230

220

Grade 12

1994 2001

326

265

243

7'4

'cr)0_

327 90th

309 75th

288 50th

266 25th

246 10th



3 3

CHAPTER 2 U.S. HISTORY REPORT CARO 19

At grade 4, the scale scores at the lowerpercentiles (10th and 25th) were higher in2001 than in 1994. At the other percentilesfor grade 4, apparent changes since 1994were not statistically significant. Increases inaverage scores at grade 8 were evidentamong students at both the lower percen-tile (25th) and the upper percentiles (75thand 90th).At the 10th and 50th percentiles,however, apparent changes since 1994 werenot statistically significant.There were nostatistically significant changes in averagescores when viewed across the score distri-bution at grade 12.

Achievement-Level ResultsThe achievement levels are performancestandards adopted by the National Assess-ment Governing Board, based on thecollective judgements of experts about


3 4

what students should be expected to knowand be able to do in terms of the NAEPU.S. history framework. A discussion ofthe trial status of achievement levels is inchapter 1.

Achievement-level results for each gradeare presented in figure 2.3. Results arepresented in two ways: 1) the percentage ofstudents within each achievement-levelrange, and 2) the percentage of students ator above the Basic and at or above theProficient levels. In reading figure 2.3, it isnecessary to keep in mind that the percent-ages at or above specific achievement levelsare cumulative. For example, includedamong the percentage of students at orabove the Basic level are also those whohave achieved the Proficient and Advancedlevels of performance.

Natidnal Achievement

eifet**44

Advanced

Proficient

Basic

Below

Basic

2%

Grade 4

2%

1994 2001

Advanced

At or above ProficientProficient

At or above

Basic

Basic

Below

Basic

Grade 12

Advanced 1% 1%Proficient

Basic

Below

Basic

1%

Grade 8

20/0*

1994

1994 2001

At or above

Proficient

At or above

Basic

2001

At or above

Proficient

At or above

Basic

* Significantly different from 1994.

NOTE: Percentages within each U.S. history achievement level may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 U.S. History Assessments.

3 5


In the 2001 U.S. history assessment, 18percent of fourth-graders, 17 percent ofeighth-graders, and 11 percent of twelfth-graders performed at or above the Proficientlevelidentified by NAGB as the level atwhich all students should perform. Stu-dents' attainment of the achievement levelsacross years generally reflects the changes inscale score results described in the previoussection: improvement at some levels atgrade 4, general improvement at grade 8,and no statistically significant change atgrade 12.


3 6

At grade 4, the percentage of studentsperforming at or above Basic in 2001 washigher than in 1994. At grade 8, there wasimprovement across the levels compared tothe previous assessment, with the percent-ages of students performing at or aboveBasic, at or above Proficient, and at Advanced

increasing from 1994 to 2001. At grade 12,however, there was no statistically signifi-cant change in the percentages of studentsperforming at each level. As in 1994, onlya small percentage of students at each gradeperformed at the Advanced level, with 2percent at grades 4 and 8, and 1 percent atgrade 12 attaining that level. Fifty-sevenpercent of twelfth-graders were below theBasic achievement level in 2001.

Average Scale Scores and

Achievement-Level Resultsfor Selected Subgroups

In addition to reporting on the performance of all students,

NAEP also provides results for various subgroups of students

at each grade. Examining subgroup results provides insight,

not only into how these groups of students performed in

comparison to one another, but also into how each group

has progressed over time. The information presented in this

chapter serves as a valuable indicator of the progress of

subgroups of the students across the nation.

Results for the NAEP 2001 U.S. history. -

assessment are presented by gender, race/ethnicity,

region of the country, parents' highest level ofAre selected education, type of school, type of location, andsubgroups of

eligibility for the Free/Reduced-Price School Lunchstudents makingprogress in U.S. program. For all subgroups except twotype ofhistory? location and free/reduced-price school lunch

eligibilityresults are available from 1994, and are

presented here for comparison with results from 2001.

All differences reported in this chapter between

demographic subgroups for the 2001 assessment and

between the 2001 and the 1994 results are based on

statistical tests that consider both the magnitude of the

difference between the group average scores or percentages

and the standard errors of those statistics. Differences

between groups and between assessment years are discussed

only if they have been determined to be statistically

significant. Furthermore, the reader should bear in mind that

3 7

CHAPTER 3

Gender

Race/Ethnicity

Region of the

Country

Parents'

Education

Type of School

Type of Location

Eligibility forFree/Reduced-

Price School

Lunch Program


differences in performance among subgroupsof students most likely reflect a range ofsocioeconomic and educational factors notaddressed in this report or by NAEP.

GenderFigure 3.1 presents the 1994 and 2001average U.S. history scale scores by gender.Patterns in performance among male andfemale students varied somewhat by grade.At grade 4 both male and female studentshad higher average scores in 2001 than in

1994, while at grade 8 only males showed again in 2001 over 1994. The apparent gainfor female eighth-graders was not statisti-cally significant. At grade 12, there was nostatistically significant change in the perfor-mance of male and female students fromone assessment to the next. At all threegrades in 2001, there was no statisticallysignificant difference between the perfor-mance of males and females.

Figure 3.1

National Scale Score

...Results by Gender

500

Grade 4

1994 2001

250

240

230

220

210

200

190

0

206203

0 Male

209*

MI Female

500

280

270

260

250

240

230

220

Grade 8

1994 2001

0 Male Female

50011994

Grade 12

2001

290

280

288 c22 m85

288286

la

270

260

250

240

230

0 Male 1111 Female

* Significantly different from 1994.NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scale scores, when rounded, were the

same for male and female students at grade 4 in 2001 and grade 8 in 1994.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of



38

The performance of subgroups on theU.S. history assessment can also be com-pared by determining whether a differenceor "gap" exists between groups' averagescores and, if it does, whether that gapincreases or decreases between assessment

years.As stated previously, no significantdifference between male or female averagescores was observed in 2001. Figure 3.2also shows that there was no statisticallysignificant change between 1994 and 2001in these small and nonsignificant gender gaps.

Figu

Nattona ca

bi feu cesft4i4.

Grade 4

2001 1

1994 2*

MaleFemale

Grade 8

2001 0 2

1994 lb*

f

-10 0 10 20 30 40 -10 0 10 20 30 40Score Differences Score Differences

Grade 12

2001 0 2

1994 63

-10 0 lb 20 30 40Score Differences

St Difference is between -0.5 and 0.5.

NOTE: Score differences are calculated based on differences between unrounded average scale scores.



The percentages of male and femalestudents at or above the U.S. historyachievement levels and within eachachievement level range are presented infigure 3.3.Achievement-level results formales and females at both grades 4 and 12showed no statistically significant increasesor decreases since 1994.At grade 8, how-ever, the percentages of male students at orabove Basic, at or above Proficient, and atAdvanced were all higher in 2001 than in

1994, while any apparent changes in thepercentages of females at or above any ofthe achievement levels were not statisticallysignificant. A comparison of the percent-ages of male and female students at orabove the Basic and Proficient levels in 2001shows no difference at grade 4, but doesshow a higher percentage of males thanfemales at or above Proficient at grade 8, anda higher percentage of males than femalesat or above Basic at grade 12.

3 9


Grade 4

Grade 8

Grade 12

Advanced

Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

Male

2%

1994

10/0

2%

19%At or above

Proficient

At or above

Basic

Advanced

Proficient

Basic

Below

Basic

17°/0

66%

34%

2001

Male

2%*

1994

10/0

2001

Male

10/0

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient

Basic

Below

Basic

1994

1%

Female

2%

16% I

65%

2001

Female

1%

11°/0

680/0

1994

Advanced 1%Proficient (9%)

Basic

Below

Basic

2001

Female

1994

(90/0)

1%

Ator above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

2001

At or above

Proficient

At or above

Basic

* Significantly different from 1994.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due torounding.



4 026 CHAPTER 3 U.S. HISTORY REPORT CARD .1 .

Race/EthnicityStudents participating in the U.S. historyassessment were asked to indicate which ofthe following racial/ethnic subgroups bestdescribed themWhite, Black, Hispanic,Asian/Pacific Islander, or American Indian(including Alaska Native). Figure 3.4presents average scale scores for students bythese subgroups at grades 4, 8, and 12.

At grade 4, both White students andBlack students had higher average scores in2001 than in 1994. At grade 8,Whitestudents showed a gain since 1994, and atgrade 12 Hispanic students had higheraverage scores in 2001, compared to 1994.No other changes were statisticallysignificant.

In 2001, differences in performance byracial/ethnic subgroup continue to be

evident at all three grades. On average,White students had higher scores thanBlack, Hispanic, and American Indianstudents at all three grades.Asian/PacificIslander students had higher average scoresthan Black and Hispanic students at allthree grades.White fourth-grade studentshad higher average scores than Asian/Pacific Islander fourth-graders. Thesedifferences should be interpreted withcaution. The average score of a selectedsubgroup does not represent the entirerange of performance within that group.Furthermore, differences between groupsof students cannot be attributed solely togroup identification. A complex array ofeducational and social factors interacts toaffect average student performance.

Figure 3

National Scale Score .

Results by Raee/Ethnicity

Average

mil MI)history Ream Cy ffmcotalhOU grades 40,EgaTia 994

5001

230

220

210

200

190

180

170

Grade 4

1994

215

205

190

180171

2001

220*213

197

188*186

Grade 8

1994 2001

280

270

260

250

240

230

220

01 0

White E Hispanic

0 Asian/Pacific Islander 0 Black

American Indian

White Hispanic


A American Indian

500

300

290 292

285280 I 279

270 261

260 265

250

240

0

Grade 12

1994 2001

295292

277274*269

White 51 Hispanic


A American Indian


NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scale scores, when rounded, were thesame for Hispanic and Black students at grade 8 in 2001.



Scale score differences between Whitestudents and Black students and betweenWhite students and Hispanic students arepresented in figure 3.5. Results from the2001 U.S. history assessment reflect a

narrowing of the score gap between Whitestudents and Black students at grade 4, andbetween White students and Hispanicstudents at grade 12.

P, Figure 3.5

National Scale ScoreDifferences by Race/Ethnicity .

Differences average history @k02scores tffo2MNOAgio grades

WhiteBlack

2001

1994

Grade 4

38

2001

1994

Grade 8

28 2001

1994

Grade 12

24.31*

2128

i I f I f10 0 10 20 30 40 0 10 20 30 40 10 0 10 20 30 40Score Differences Score Differences Score Differences

WhiteHispanic

Grade 4 Grade 8 Grade 12

2001 33 2001 2001 19*.28

1994 35 1994 199424 26

10 0 10 20 30 40 10 0 10 20 30 40Score Differences

10 0 10 20 30 40Score Differences Score Differences

* Significantly different from 1994.NOTE: Score differences are calculated based on differences between unrounded average scale scores.



Achievement-level results for the racial/ethnic subgroups are presented in figure3.6a, b, and c.While there have been someoverall achievement gains since 1994 atgrades 4 and 8, not all groups show im-provement. At grade 4, both White stu-dents and Black students had higher per-centages at or above Basic in 2001 com-pared to 1994. At grade 8, however, onlyWhite students showed an increase in thepercentages at or above Proficient and at

28 CHAPTER 3 U.S. HISTORY REPORT CARD 4 2

Advanced. At grade 12, none of the appar-ent changes between 1994 and 2001 in thepercentages of students at or above any ofthe history achievement levels were statisti-cally significant.

Comparing the subgroups' performancein 2001 shows higher percentages ofWhite students and Asian/Pacific Islanderstudents at or above the Basic and Proficientlevels than Black and Hispanic students atall three grades.

Advanced

Proficient

Basic

Below

Basic

White30/0

1994

Advanced

Proficient (6%)

Basic

Below

Basic

10/0

2001

Hispanic

At or above

Proficient

At or above

Basic

1%6% (6%) 7%

1994 2001

American Indian

Advanced # 4%Proficient (9%) (8%) 12%

Basic

Below

Basic

1994

53%

2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient (4%)

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

1994 2001

Asian/Pacific Islander

4cY0 3%

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

* Significantly different from 1994.# Percentage is between 0.0 and 0.5.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.



4 3 CHAPTER 3 U.S. HISTORY REPORT CARD 29

Figure 1

tional Achievementl.tivel Results by Race/.Ethnicity. .

Percentage students

1o52% gradeccfifolo EJ4 cfs

5 II 994gffiiI]above history achievement MIA

Advanced

Proficient

Basic

Below

Basic

Advanced

Proficient (5%)

Basic

Below

Basic

White1% 2%*

1994 2001

At or above

Proficient

At or above

Basic

1994 2001

American IndianAdvanced 0% co/ 1%

profoeni (5%) s' ic) (7%)

Basic

Below

Basic

42%

1994

58c/0

2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient (4%)

Basic

Below

Basic

4% At or above

Proficient

1994 2001

Asian/Pacific IslanderAdvanced 2% 2%

Proficient

Basic

Below

Basic

1994 2001

At or above

Basic

At or above

Proficient

At or above

Basic





Fi g re,3,6,4,44ttlxNational AchAvemenr 4,d *.toLevel Result TRacettilDICITy.

Advanced

Proficient

Basic

Below

Basic

AdvancedProficient (4%)

Basic

Below

Basic

1 70

White

10/0

1994 2001

Hispanic

4% (5%)

1994

At or above

Proficient

At or above

Basic

2001

American Indian

Pro f cAirefivirigealh) 0% 50/0 (1%) 0% 1%

Basic

Below

Basic

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient (2%)

Basic

Below

Basic

Black

1994 2001

Asian/Pacific IslanderAdvanced 2%

Proficient

Basic

Below

Basic

50/0

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

it Percentage is between 0.0 and 0.5.

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.



Region of the CountryNAEP assessments traditionally provideresults for four regions of the country:Northeast, Southeast, Central, and West.Appendix A (see page 129) contains adescription of the states and other jurisdic-tions that make up each region.

Scale score results by region are pre-sented in figure 3.7. Although overall gainsin student performance were observed atgrades 4 and 8, not all regions showedincreases. At grade 4, only the Northeastregion showed a gain in the U.S. historyaverage score since 1994, while at grade 8,the only increase occurred in the Southeastregion. None of the other apparentchanges between 1994 and 2001 in re-

gional average scores were statisticallysignificant. Regional results at grade 12were consistent with the overall nationalresults that did not show a change instudents' performance from 1994 to 2001.

Some differences in performance be-tween regions of the country were evidentin 2001. At both grades 4 and 8, studentsin the Northeast, Southeast, and Centralregions all had higher average scores thanstudents in the West, and students in theCentral region outperformed their peers inthe Southeast on average. At grade 12,none of the apparent differences in averagescores among the four regions were statisti-cally significant.

,National Scale, Spore.:,Reiulti tiy Regiori Of

,., the Country

32 CHAPTER 3

5001

250

240

230

220

210

200

Grade 4

1994

212204202201

20015001

280

270

260

Grade 8

1994

266

256

211 250 I251

215*208 240

200230

190 I 220

0 Northeast

0 Central

0 Southeast

0 West

2001

0 Northeast

0 Central

269267

261*255

CM Southeast

0 West

5001

290

280

270

260

250

240

230

0

Grade 12

1994 2001

289288286

282

0 Northeast

0 Central

289

286284

0 Southeast

0 West

* Significantly different from 1994.NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scale scores, when rounded, were the

same for Northeast and Central regions at grade 8 in 1994 and at grade 12 in 2001.



U.S. HISTORY REPORT CARD4 6

Achievement-level results for the fourregions are displayed in figure 3.8a, b, and cby grade. The only gains occurred in theSoutheast region at grade 8, where thepercentage both at or above Basic and at orabove Proficient increased in 2001 comparedto 1994.

As with the scale score results by region,some differences between regions in thepercentages of students at or above thedifferent achievement levels were evidentin 2001. A higher percentage of fourth-grade students in the Northeast and Cen-

tral regions were at or above both the Basicand Proficient levels than in the West, and ahigher percentage of fourth-graders in theCentral region were at or above Basic thanin the Southeast. There were also higherpercentages of eighth-grade students in theNortheast and Central regions at or aboveBasic and Proficient than in the West. Inaddition, the percentage of eighth-gradersat or above Basic was higher in the North-east and Central region than in the South-east, and higher in the Southeast than inthe West.

Ii1ghievenitht

,IERCARO;i0,#010.,S4

Advanced

Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

3%Northeast

3%

1994

3%

2001

Central3%

1994 2001

240/0

75%

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

2%

130/0

46%

Southeast

20/0

15% I

610/0

14°/0 16%IAt or above

Proficient

660/0

34%

1994 2001

10/0

Westi%

1994 2001

At or above

Basic

At or above

Proficient

At or above

Basic




figtire 3811p%

National Achievemen4Level Results byVfegioii of the Coubtr

Percentageregion

students

cg lo@

dila geountry, grade 5 fi 994

above

Effl 1CIfihistory achievement

Advanced 1%

Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

Northeast2%

1994

10/0

2001

Central2%

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient (8%)

Basic

Below

Basic

Advanced

Proficient (10)%

Basic

Below

Basic

Southeast

2°/090/0

51%

49%

1994

1%

2001

West1

1 60/0*

62%*

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic





4 8

National Achievemep*Level Results by :u''Rgion bfLtte.p.otintly

Advanced

Proficient

Basic

Below

Basic

Northeast1% 2%

130/0

1994


Basic

Below

Basic

110/0 13%

45%

2001

Central10/0

(10%)

1994

At or above

Proficient

At or above

Basic

2001

At or above

Proficient

At or above

Basic

Advanced

Proficient (8%)

Basic

Below

Basic

Southeast1%

(9%)

1994


Basic

Below

Basic

2001

West

10/0

(10%)

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

It Percentage is between 0.0 and 0.5.

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 U.S. History Assessments.

4 9


Parents' Highest Levelof EducationEighth- and twelfth-grade students whoparticipated in the NAEP U.S. historyassessment were asked to indicate thehighest level of education completed byeach parent. Four levels of education wereidentified: did not finish high school,graduated from high school, some educa-tion after high school, and graduated fromcollege. Students could also choose theresponse,"I don't know." For this analysis,the highest education level reported foreither parent was used. Data were notcollected at grade 4 because in previousNAEP assessments fourth-graders' re-sponses about their parents' education werehighly variable and contained a largepercentage of "I don't know" responses.

The scale score results for all levels ofstudent-reported parent education level arepresented in figure 3.9. In 2001, almostone-half of the eighth- and twelfth-grade

50


students reported at least one parent hadgraduated from college (47 and 46 percent,respectively), whereas only a small percent-age reported that their parents had notgraduated high school (7 percent at bothgrades). Additional information on thepercentage of students reporting parents'highest level of education is available inappendix B.

At grade 8, students who reported that atleast one parent graduated from collegehad higher average scores in 2001 than in1994. At grade 12, there was an increase inthe average scores of students who reportedthat neither parent finished high school.The 2001 results indicate that, overall, therewas a clear positive relationship betweenparent education level and the performanceof both eighth- and twelfth-graders on theU.S. history assessment. At both grades,the higher the parental education levelreported, the higher the average scoreattained.

5001

290

280

270 210

260

250 251

240 2412

23038

Grade 8

1994

264

0

2001

275*

265

253

244*243

0 Graduated college

13 Some education after high school

3 Graduated high school

O Less than high school

A Unknown

Grade 12

1994 2001

300

290

280

270

260

250

240

0 Graduated college

OSome education after high school

Graduated high school

0 Less than high school

A Unknown

* Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 U.S. History Assessments.

Achievement-level results across years bylevel of parental education are presented infigure 3.10a and b.

There were higher percentages ofeighth-graders at or above Proficient and atAdvanced in 2001 among students whoreported that at least one parent graduatedfrom college. None of the other changes inachievement level results by level of paren-tal education were statistically significant.

As with the average scale score results,the 2001 achievement-level results showhigher percentages of eighth- and twelfth-grade students at or above both the Basicand Proficient levels among students whoseparents graduated from college than amongthose who reported parents having lowerlevels of education.

51CHAPTER 3 0 U.S. HISTORY REPORT CARD 31

ffigike11111a,

,,..ttatiyonvaqhievehipo

rr

Less than high schoolAdvanced 0I

Proficient (3%,3% g 3%(3%)

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

1994 2001

Some education after high school1%

14%

1994

Advanced

Proficient (3%)

38 CHAPTER 3

Basic

Below

Basic

2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

1994 2001

At or above

Proficient

At or above

Basic

Graduated high schoolAdvanced #

Proficient (6%) 7% (7%) 7%

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

1994 2001

Graduated college

1% 3%*

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

* Significantly different from 1994.ft Percentage is between 0.0 and 0.5.




U.S. HISTORY REPORT CARO

5 2

Advanced

Proficient (1 %)

Basic

Below

Basic

Less than high school

1994 2001

Some education after high schoolAdvanced 1% 1%

Proficient (7%) (8%) 8%

Basic

Below

Basic


Basic

Below

Basic

1994 2001

Unknown

# 1%(3%)

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Graduated high schoolAdvanced #

Proficient (4%) 4 A, (4%) 4%

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

zcze,26%

YR,

1994 2001

Graduated college

1994 2001

180/0

58%

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

# Percentage is between 0.0 and 0.5.





Type of SchoolThe schools that participate in the NAEPassessment are classified as either public ornonpublic. A further distinction is thenmade within the nonpublic classificationbetween nonpublic schools that are Catho-lic and other nonpublic schools. In 2001, asin previous NAEP assessments, fourth-,eighth, and twelfth-grade students attend-ing various types of nonpublic schools hadhigher average scores than did their peersattending public schools. Readers are,however, cautioned against making assump-tions about the comparative quality ofinstruction in public and nonpublicschools. Socioeconomic and sociologicalfactors that may affect student performanceshould be considered when interpretingthese results.

Average U.S. history scale scores by typeof school are presented in figure 3.11 andshow higher average scores in 2001 than in1994 among fourth- and eighth-gradestudents attending public schools. Fourth-graders attending Catholic nonpublicschools also had higher average scores in2001 than in 1994. None of the apparentchanges seen at grade 12 were statisticallysignificant.

Comparisons of scale score resultsbetween the types of schools in 2001 showstudents at Catholic nonpublic schoolsoutperforming public school students at allthree grades, while the performance ofstudents in other nonpublic schools washigher than that of public school studentsat grades 4 and 8 only.

igure.3.11

tf; National Scale ScoreResult's by Type ofSchool':

history gs0g scores Di9rG school grades (1,6wiM

40 CHAPTER 3

Grade 4

5001

250

240

230

220 I 222

210

200

190

1994

0

203

2001


226

207*

5 4

1994 2001

250

240

230

220

210

200

190

0 Public

0 Nonpublic

0 Nonpublic: Catholic

ill Nonpublic: Other

See footnotes at end of table. r>

igure i.

a tonal Scale ScoreIts bTypeofjJ

c ool (continjiecl)

Grade 8

Grade 12

5001

290

280

270

260

250

240

230

1994 2001

278

257

5001

310

300 I 299

1994

2902

28084

270

260

250

1994500

290

279 280 I 219

2270

17

260

250

240

230

260*

2001

298

286

500

310

300 299298

2001

280

278

1994 2001

290

280

270

260

250

302

293

0 Public

CI Nonpublic

0 Nonpublic: Catholic

MI Nonpublic: Other




Achievement-level results by school typeare presented in figure 3.12a, b, and c bygrade. At grade 4, there was a higherpercentage of Catholic nonpublic schoolstudents at or above Proficient in 2001 thanin 1994. At grade 8, there were higherpercentages of public school students at orabove Proficient in 2001 than in 1994.

Comparisons of 2001 achievement-levelresults between types of schools showhigher percentages of nonpublic schoolstudents at or above the Basic and Proficientlevels than public school students at allthree grades. There was also a.higherpercentage of eighth-grade students innonpublic schools at the Advanced levelthan in public schools.

National Achievement:level Resulte by Typ,e,Of Schoor,

Advanced

Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

2 %

Public2 %

1994 2001

170/0

65 %

Nonpublic: Catholic

2 °h 50/0

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced 3 %

Proficient

Basic

Below

Basic

Nonpublic

4%

1994

Advanced 5 %

Proficient

Basic

Below

Basic

2001

Nonpublic: Other

3%

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

* Significantly different from 1994.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.



5 6


National

3.12b

Aohievement-

dgA03

Percentage

cMGclucg students

school,aTOSERIECOT5i1 994 5a1 ffla

above

gradehistory achievement

PublicAdvanced 1% 1%Proficient

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

1994 2001

Nonpublic: Catholic

2% 3%

26%29%

85%

1994 2001

Advanced 2%At or above

Proficient Proficient

At or aboveBasic

Basic

At or above

Proficient

At or above

Basic

Below

Basic

Nonpublic

3%

1994

Advanced 2%

Proficient

Basic

Below

Basic

2001

Nonpublic: Other4%

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic




5 7


Advanced

Proficient (9%)

Basic

Below

Basic

Advanced

Proficient

Basic

Below

Basic

1%

Public1%

10% (9%) 11%

1994 2001

Nonpublic: Catholic1% 2%

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Advanced

Proficient

Basic

Below

Basic

1 0/0

Nonpublic

2%

1994 2001

Nonpublic: OtherAdvanced 2 % 1%

Proficient

Basic

Below

Basic

1994 2001

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

NOTE: Percentages within each U.S. histoiy achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due torounding.




58

Type of LocationThe schools from which NAEP draws itssamples of students are classified accordingto their type of location. Based on CensusBureau definitions of metropolitan statisti-cal areas, including population size anddensity the three mutually exclusive cat-egories are: central city, rural/small town,and urban fringe/large town. Because ofnew methods used by NCES to identifythe type of location assigned to each schoolin the Common Core of Data, schoolswere not classified in exactly the same wayin 2001 as in 1994. Therefore, comparisonsbetween the two assessment years are not

possible, and only the data for the 2001assessment are reported. More informationon the definitions of the 2001 assessmentclassifications of location type is given inappendix A.

The performance of students by type ofschool location is shown in table 3.1. Atgrades 4 and 8, students attending schoolsin rural and urban fringe locations hadhigher average scores than students incentral city schools. At grade 12, studentsattending schools in urban fringe locationshad higher scores than students in bothrural and central city locations.

Average U.S. history scale scores by type of location, grades 4, 8, and 12: 2001

Grade 4

Central city

19

Urban fringe/large town

211

Rural/small town

215

Grade 8257 265 263

Grade 12283 292 284


Figure 3.13 presents achievement-levelresults by type of school location. Com-parisons of achievement-level resultsbetween locations show higher percentagesof fourth- and eighth-grade students at orabove Basic in rural and urban fringelocations than in central city locations.There was also a higher percentage of

fourth-graders at or above the Basic level inrural schools than in urban fringe schools.At grade 12, the percentages of students ator above the Basic and Proficient levels werehigher in schools located in urban fringeareas than those in both rural and centralcity locations.


3.13 Percentage

biYUG

cfstudentsschool

dilMEedglocation, grades

above 111 history

400e2w1M AuIJ

achievement 0g9A

Advanced 2%

Proficient 13% 15% I

Basic

55%

Below

Basic (NM

Central city

Advanced 2%

Proficient

Basic

Below

Basic

Central city

Advanced

Proficient (9%)

Basic

Below

Basic

10/0

Central city

Grade 430/0


Grade 810/0

20/0

Rural/small town

130/0

At or above

Proficient

At or above

Basic

15%IAt or above

Proficient

66%

Urban fringe/ Rural/large town small town

Grade 122%


(70/0)

Rural/small town

At or aboveBasic

At or aboveProficient

At or aboveBasic

# Percentage is between 0.0 and 0.5.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.




Free/Reduced-Price SchoolLunch Program EliglityFunded by the U.S. Department of Agri-culture (USDA) as part of the NationalSchool Lunch Program, the Free/Re-duced-Price School Lunch program isdesigned to assure that children at or nearthe poverty line receive nourishing meals.Eligibility guidelines for the lunch programare based on the Federal income povertyguidelines and are stated by householdsize.' NAEP first began collecting data onstudent eligibility for this program in 1996;therefore cross-year comparisons back to1994 are not possible. Table 3.2 presentsthe 2001 U. S. history scale score results bystudents' eligibility for the program. At

every grade, the average scale scores forstudents who were not eligible for theFree/Reduced-Price School Lunch pro-gram (i.e., those above the poverty guide-lines) were higher than the scores for thestudents who are eligible for the program.Since information on eligibility is notavailable for a substantial percentage of thestudents at each grade, table 3.2 also dis-plays the scale score averages for this thirdgroup of students. (Some schools do notoffer free/reduced-price lunches. Studentsfrom these schools are counted in theInformation Not Available category.) Thisgroup also had higher scale scores at everygrade than the students eligible for the Free/Reduced-Price School Lunch program.

able LT National &D324 Score Results Free/Reduced-Price School Program eligibility

Average U.S. history scale scores by student eligibility for Free/Reduced-Price School Lunchprogram, grades 4, 8, and 12: 2001

Grade 4

Eligible Not eligible Info not available

220 217189

Grade 8

245 I 269 268

Grade 12

271 289 295

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics. National Assessment ofEducational Progress (NAEP), 2001 U.S. History Assessment.

The pattern for achievement-level resultsis displayed in figure 3.14 and parallels thatseen in the scale scores. There were higherpercentages of fourth-, eighth, and twelfth-

grade students at or above Basic and Profi-cient who were not eligible for the programthan those who were eligible for it.

I U.S. General Services Administration. (1999). Catalopc offederal domestic assistance.Washington, DC: ExecutiveOffice of the President, Office of Management and Budget.


FigUleNaticNaCtileemliiIgeplaWr t-RedutelltWicOYMlitel

P tOair a,11,(ctligi bill ity

Percentage studentsreenteduced-Price

dab) eiR cff

ME&School

above

programtustory

eligibUittoThtmook MI@ 47

0, Eal ENgrades

Advanced 1%Proficient (6%) 6%

Basic

Below

Basic


Basic

Below

Basic

Advanced

Proficient (3%)

Basic

Below

Basic

Eligible

Eligible

Eligible

Grade 4

3%

Not eligible

Grade 8

2%

20%

Not eligible

(10%)

Grade 1210/0

Not eligible

3%

nfo not

available

3%

Info not

available

Info not

available

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

At or above

Proficient

At or above

Basic

Percentage is between 0.0 and 0.5.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.




@hapter

Sec Mg a More MdusiveNatbnal Assess eraIn its efforts to assess a representative sample of all students

in the nation, NAEP consistently has striven to include

special-needs studentsthose with disabilities (SD) or

limited English proficient students (LEP). A certain

percentage of such students, however, has always been

excluded because they could not be assessed meaningfully

without accommodations. Schools that participate in NAEP

have been asked to use specific criteria in making

decisions to exclude certain students who have been

classified as having a disability under the Individuals

with Disabilities Education Act (IDEA), based upon

their Individualized Education Programs (IEP) and

Section 504 of the Rehabilitation Act of 1973.

Similarly, schools have been permitted to exclude

some students they identify as being limited English

proficient)

In order to increase the inclusiveness of

NAEP's samples, and in an attempt to remain

consistent with state- and district-level testing

policies that increasingly offer accommodations to

special-needs students, NAEP began to explore the

use of accommodations in the 1996 and 1998 assessments. A

split-sample design was used to identify a portion of schools

that were permitted to provide accommodations to their

special-needs students who required them, and a portion of

schools in which accommodations were not offered (the

How would

the NAEP results

differ ifaccommodations

were permitted

for special-needs

students?

1 See appendix A for a description of specific criteria provided to assist them in makingexclusion decisions.

6 3

Two Sets of 2001

NAEP U.S. History

Results

Results for the

Nation

National Results

by Gender

National Results

by Race/Ethnicity

CHAPTER 4 U.S. HISTORY REPORT CARO 49

standard administration procedure prior to1996). The split-sample design made itpossible to study the effects on NAEPresults of including special-needs studentswho required and were provided accom-modations, while at the same time, obtain-ing results that were comparable to thosefrom previous assessments. Based on re-search conducted and published since thattime, it was determined that NAEP couldbegin a transition to reporting results thatincluded the performance of accommo-dated special-needs students.2 It is antici-pated that in the near future, NAEP willonly report results based on this moreinclusive sample.

Two Sets of 2001 NAEPU.S. History ResultsThis report is the first to display twodifferent sets of NAEP U.S. history resultsbased on the split-sample design: 1) thosethat reflect the performance of regular andspecial-needs students when accommoda-tions were not permitted, and 2) those thatreflect the performance of regular andspecial-needs studentsboth those whowere accommodated and those who couldbe tested without accommodationswhenaccommodations were permitted. It shouldbe noted that accommodated studentsmake up a small proportion of the total

50 CHAPTER 4

weighted number of students assessed (seetable A.6, page 115 in appendix A fordetails). Making accommodations availablemay change the overall assessment results insubtle and different ways. For example,when accommodations are permitted, theremay be some occurrences of students beingaccommodated who might have taken thetest under standard conditions if accommo-dations were not permitted. This couldlead to an overall increase in the averageassessment results if accommodations wereto increase special-needs students' perfor-mance. Conversely, when accommodationsare permitted, special-needs students whocould not have been tested without ac-commodations could be included in thesample. Assuming that these are generallylower-performing students, their inclusionin the sampleeven with accommoda-tionscould result in an overall loweraverage score.

The two sets of results presented in thischapter were obtained by administering theassessment to a nationally representativesample of students and schools. In onesample, no accommodations were permit-ted; all students were assessed under thesame conditions that were the basis forreporting results from the 1994 NAEP U.S.history assessment. In another part of the

2 Olson, J. F. & Goldstein, A. A. (1997). The inclusion of students with disabilities and limited-English-prqicient students inlarge-scale assessments:A suntmary of recent progress. (NCES Publication No. 97-482). Washington, DC: NationalCenter for Education Statistics.

Mazzeo, J., Carlson, J. E.,Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of special needs students inNAEP: A report on 1996 research activities. (NCES Publication No. 2000-473). Washington, DC: National Centerfor Education Statistics.


schools sampled, accommodations werepermitted for students with disabilities andlimited English proficient students whonormally receive accommodations in theirdistrict or state assessment programs. Mostaccommodations that schools routinelyprovide for their own testing programswere permitted. The permitted accommo-dations included, but were not limited tothe following:

O one-on-one testing,

O bilingual dictionary,

O large print book,

O small-group testing,

O extended time,

O oral reading of questions, and

O use of an aide for transcribing responses.

(See appendix A, table A.7, page 117, forgreater detail on the numbers and percent-ages of students accommodated by accom-modation type in the 2001 assessment.)

Figure 4.1 provides a visual representa-tion of how the two sets of results werebased on the two samples in 2001. In-cluded in both sets of results (accommoda-tions not permitted and accommodationspermitted) are those students from bothsamples of schools who were not identifiedas either SD or LEP. In addition, the firstset of results (accommodations not permit-ted) includes SD and LEP students fromthe sample of schools where accommoda-tions were not permitted (see middleportion of figure 4.1). This is the set of

results that allows for trend comparisonsback to 1994 and are presented in theother chapters of this report.

The second set of results, accommodationspermitted (see bottom portion of figure4.1), includes SD and LEP students fromthe sample of schools where accommoda-tions were permitted. This is the set ofresults that form the new, more inclusivebaseline for future reporting of trendcomparisons for the NAEP U.S. historyassessment.

In the NAEP 2001 sample whereaccommodations were not permitted,16 percent of fourth-graders, 16 percent ofeighth-graders, and 11 percent of twelfth-graders, were identified by their schools ashaving special needs (i.e., either as studentswith disabilities or limited English profi-cient students). In the other sample whereaccommodations were offered, 18 percentof fourth-graders, 17 percent of eighth-graders, and 10 percent of twelfth-graderswere identified as having special needs. Inthe sample where accommodations werenot permitted, between 45 and 51 percentof the special-needs students at each of thethree grade levels (between 4 and 8 percentof all studentssee appendix A, table A.5,page 114) were excluded from NAEPtesting by their schools. In the samplewhere accommodations were offered,between 23 and 33 percent of the special-needs students were excluded from theassessment (between 2 and 3 percent of the


The two sets of NAEP results based on a split-sample design

Sample with no Sample with Split-sample designaccommodations permitted accommodations permitted The national sample was split. In part of the

schools, accommodations were not permittedfor students with disabilities (SD) and limitedEnglish proficient (LEP) students. In the other

Non-SD/LEP Non-SD/LEP schools, accommodations were permitted forstudents students SD and LEP students who routinely received

them in their school assessments.

SD/LEP SD/LEP

students students

Sample with no Sample with

accommodations permitted accommodations permitted

&010on-SD/LOP

st dents

SD/LEP

students

Sample with no Sample with

accommodations permitted accommodations permitted

Non-SD

stude s

SD/LEP

students

SDILOP

students

6 652 CHAPTER 4 U.S. HISTORY REPORT CARD

Accommodations-not-permitted results

The accommodations-not-permitted resultsinclude the performance of students from bothsamples who were not classified as SD or LEPand the performance of SD and LEP studentsfrom the sample in which no accommodationswere permitted.

Accommodations-permitted results

The accommodations-permitted results alsoinclude the performance of students from bothsamples who were not classified as SD or LEP;however, the SD and LEP students whoseperformance is included in this set ofresults were from the sample in whichaccommodations were permitted. Sincestudents who required testing accommodationscould be assessed and represented in theoverall results, it was anticipated that theseresults would include more special-needsstudents and reflect a more inclusive sample.

total sample). Thus, offering accommoda-tions would appear to lead to greaterinclusion of special-needs students.

Chapters 2, 3, 5, and 6 of this report arebased on the first set of results (no accom-modations permitted). This chapter pre-sents an overview of the second set ofresultsresults that include students whowere provided accommodations during theassessment administration. Overall resultsare provided for the nation and for studentsubgroups by gender and by race/ethnicityThese results are discussed in terms ofstatistically significant differences betweenthe two sets of results and differencesbetween subgroups of students within eachset of results. Throughout this chapter, theassessment results that include SD and LEPstudents for whom accommodations werenot permitted will be referred to as the

results.The set of results that includes SD and LEPstudents for whom accommodations werepermitted will be referred to as the "ac-commodations-permitted" results.

esults for the NationAccommodations Not Permitted andAccommodations Permitted

Table 4.1 displays the average U.S. historyscale scores for the nation in 2001 for twosets of results: 1) accommodations notpermitted, and 2) accommodations permit-ted. There were no significant differencesin the average scores between the two setsof results at grades 4 and 12. At grade 8,however, the average score when accom-modations were permitted was lower thanthe average score when accommodationswere not permitted.

As noted in the introduction to thischapter, NAEP has always sought to in-clude special-needs students proportionalto their representation in the U.S. popula-tion. Offering accommodations tends toreduce exclusion rates for special-needsstudents and therefore allows NAEP tooffer a fairer and more accurate picture ofthe status of American education. Becausespecial-needs students are typically classi-fied as eligible for special educational

6 7

CHAPTER 4 e U.S. HISTORY REPORT CARD 53

National average U.S. history scale scores by type of results, grades 4, 8, and 12: 2001

Accommodations not permitted Accommodations permitted

Grade 4 209 208

Grade 8 262 260 t

Grade 12 287 287

t Significantly different from the sample where accommodations were not permitted.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of


services after having shown some difficultyin the regular learning environment, somemay assume that including the perfor-mance of these students would tend tolower the overall results. This assumptionappears to have been justified only in theobserved difference between the two setsof grade 8 U. S. history results in 2001,where the accommodations-permittedresults, which included slightly morespecial-needs students because of theavailability of accommodations, were lowerthan the accommodations-not-permittedresults. It is important to examine thepercentages of students attaining the NAEPachievement levels, however, to see if there

68


were higher percentages at the lowerachievement level (i.e., Basic), when stu-dents were assessed with accommodations.

Table 4.2 shows the percentages ofstudents attaining each of the achievementlevels. The percentages are similar acrossthe two sets of results for grades 4 arid 12;apparent differences between theaccommodations-not-permitted and theaccommodations-permitted results werenot significantly different. At grade 8,however, the percentage of students belowBasic was higher when accommodationswere permitted than when they were notpermitted.

Percentage of students within and at or above U.S. history achievement levels by type of results,grades 4, 8, and 1 2: 2001

Grade 4

Below Basic

Accommodations were not permitted 33

Accommodations were permitted 34

Grade 8

Accommodations were not permitted 36Accommodations were permitted 38 t

Grade 12



At or above

Proficient

At or above

BasicAt 8asic At Proficient At Advanced

49 16 2 67 18

48 16 2 66 18

48 15 2 64 17

46 t 14 1 62 t 16

32 10 1 43 11

32 10 1 43 11

t Significantly different from the sample where accommodations were not permitted.

NOTE: Percentages within each U.S. histoiy achievement-level range may not add to 100, or to the enact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of


National Results by GenderAccommodations Not Permitted andAccommodations Permitted

The average U.S. history scale scores bygender for both sets of results in 2001 areprovided in table 4.3. Both male andfemale students at grade 8 had higher U.S.history scores when accommodations were

not permitted than when accommodationswere permitted.

At all three grades, the average scores formale students were not significantly differ-ent from that of female students regardlessof whether or not accommodations werepermitted.

National average U.S. history scale scores by gender and type of results, grades 4, 8, and 1 2: 2001

Grade 4

Male Female

Accommodations were not permitted 209 209Accommodations were permitted 207 209

Grade 8

Accommodations were not permitted 264 261Accommodations were permitted ; 261 t 260

Grade 12

Accommodations were not permitted ; 288 286Accommodations were permitted 288 286

t significantly different from the sample where accommodations were not permitted.SOURCE: U.S. Department of Education, Otfice of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 U.S. History Assessment.

IPCHAPTER 4 U.S. HISTORY REPORT CARD 55

The percentages of male and femalestudents attaining the Basic, Proficient, andAdvanced levels are provided in table 4.4.Comparing the two sets of results in 2001,there were no statistically significant differ-ences in the percentages of male or female

students attaining each of the achievementlevels at grades 4 or 12. At grade 8, how-ever, a higher percentage of male studentswere below Basic when accommodationswere permitted than when they were not.

Percentage of students within and at or above U.S. history achievement levels by gender and typeof results, grades 4, 8, and 1 2: 2001

Below basic At Basic

34 47

Grade 4

Male

Accommodations were not permitted

Accommodations were permitted

Female



35 46

Grade 8

Male



Female

Accommodations were not permitted 37 48Accommodations were permitted 39 47

51

50

35 4738 t 45

Grade 12Male

Accommodations were not permitted 55 33

Accommodations were permitted 55 32

Female



59 31

60 31

At or above

Basic

At or above

ProficientAt Proficient At Advanced

17 2 66 19

16 2 65 19

15 2 68 17

15 2 67 17

17 2 65 18

16 2 62 t 17

14 1 63 15

13 1 61 14

11 1 45 12

11 2 45 12

9 1 41 10

9 1 40 10

t Significantly different from the sample where accommodations were not permitted.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.




National Results byRace/EthnicityAccommodations Not Permitted andAccommodations Permitted

NAEP assessments across academic subjectshave typically reported large score differ-ences according to race and ethnic groupmembership. If students with disabilities orlimited English proficient students areover-represented in a particular racial orethnic group, that group's assessment scoresmay decrease. Table 4.5 provides theaverage U. S. history scale scores for each ofthe race/ethnicity categories for the twosets of results in 2001. At grade 8, bothWhite students and Black students hadhigher average scores when accommoda-tions were not permitted than whenaccommodations were permitted. Therewere no statistically significant differences

observed between the average scores whenaccommodations were not permitted andwhen accommodations were permitted forany of the race/ethnicity categories atgrades 4 and 12.

As noted in chapter 3, a pattern ofperformance differences by race/ethnicitycan be seen in the accommodations-not-permitted results in 2001. Both White andAsian/Pacific Islander students at all threegrades scored higher than Black andHispanic students. The same pattern can beobserved in the accommodations-permit-ted results. However, while White studentsoutperformed their Asian/Pacific Islanderpeers at grade 4 when accommodationswere not permitted, the difference was notstatistically significant when accommoda-tions were permitted.

National average U.S. history scale scores by race/ethnicity and type of results, grades 4, 8, and 12:2001

White

Grade 4

Black Hispanic

Asian/Pacific

IslanderAmerican

Indian

Accommodations were not permitted 220 188 186 213 197Accommodations were permitted I 218 186 187 214 197

Grade 8

Accommodations were not permitted 1 271 243 243 267 249Accommodations were permitted 269 t 240 t 240 265 248

Grade 12

Accommodations were not permitted 292 269 274 295 277Accommodations were permitted 292 268 271 294 274




The percentages of students in eachrace/ethnicity category who attained theBasic, Proficient, and Advanced levels are

provided in table 4.6. No significant differ-ences were found at any of the three grades

between the accommodations-not-permit-ted results and the accommodations-permitted results for the percentages ofstudents attaining each of the achievementlevels in 2001.

71CHAPTER 4 o U.S. HISTORY REPORT CARD 51

Percentage of students within and at or above U.S. history achievement levels by race/ethnicityand type of results, grades 4, 8, and 12: 2001

Grade 4

White



Black



Hispanic



Asian/Pacific IslanderAccommodations were not permitted


American Indian



Below Basic At Basic At Proficient At Advanced

At or above

Basic

At or above

Proficient

21 55 21 3 79 24

23

56

58

58

58

29

26

53 21 3 77 24il

38 1 5 # 44 6

36 5 # 42 5

3536

53

54

47 41

44

Grade 8

White



Black



Hispanic





American Indian



Grade 12

White



Black



Hispanic





American Indian



25

27

62

65

6063

32

34

50

54

51

51

80

80

74

74

47

48

66

68

44

61

1 42 7

6 4 42 6

16 3 71 19

17 3 74 20

8 4 53 12

9 3 56 12

53

52

3431

3432

19 2 75 21

19 2 73 20

4

4

38

35

4

4

5 4 40 5

4 37 4

48 H 18

47 17

2

2

68 20

66 19

42 7 1 50 8

38 7 1 46 8

36 12

36 12

18 1 3

17 3

21

21

5

5

1 49 13

1 49 13

20 3

20 3

26 5

26 5

# Percentage is between 0.0 and 0.5.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.


Educational Progress (NAEP), 2001 U.S. Flistory Assessment.


-

Sa pBe fssessment uestionsa d Student Respo ses

This chapter presents sample questions from the 2001

NAEP U.S. history assessment and examples of student

responses to those questions. Four sample questions at each

grade level are provided, including multiple-choice, short

constructed-response, and extended constructed-response

questions. Each sample question is classified according to a

historical theme or historical period, as described in the U.S.

history framework. Actual student responses to the

sample constructed-response questions have been

reproduced from test booklets to illustrate answers

representative of the indicated scoring categories.

The tables accompanying each sample question

present two types of performance data: the overall

percentage of students who answered successfully and

the percentage of students within a specific score

range on the NAEP U.S. history scale who answered

successfully. The score ranges presented are those

that correspond to the three achievement-level

intervalsBasic, Proficient, and Advancedas well as

the score range that falls below Basic.

Materials from

the 2001

U.S. history

assessment

7 3

Sample

Questions

Student

Responses


Grade 4 Sample AssessmentQuestion ResultsAssessment questions at grade 4, as at theother grade levels, included both con-structed-response and multiple-choice

k6iig Sample Question

formats. Many, like the examples below,used visual or textual stimuli. Questionstended to concern topics that are typicallyaddressed in fourth-grade social studiesclasses.

In pioneer schools, feathers like this were most often used for

0 measuringsewing

0 writing0 playing a game

Economic and Technological Changesand Their Relation to Society, Ideas, andthe Environment

Expansion and Reform(1801 to 1861)

able Sample Question Results Mlultiple-Ohoiee

Overall percentage correct and percentages correct within each achievement level range: 2001

60 CHAPTER 5

Overall percentagecorrect

93

ere ntage w-GOdaffn1gt5:0-5xgrililecda

Below Bask Basic Proficient Advanced

194 and below* 195-242* 243-275* 276 and above*

84 96 99 ***

*NAEP U.S. history composite scale range.***Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.

U.S. HISTORY REPORT CARD `(/ 4

I W TftFOR U.S.ARMY

NEAREST RECRUITING STATION

The poster shown above is trying to attract recruits by appealing to their

homesickness

religious beliefs

0 patriotism0 need for money

The Changing Role of America in the World Modern America and the World Wars(1914 to 1945)

116b52 Sample Question Results Mlultiple-ehoice


Overall percentage

correct

45

Below Basic194 and below*

25

Basic195-242*

46

Proficient243-275*

74

Advanced276 and above*

***



What was a major cause of the Civil War?

0 People in the North and in the South had different religions.

0 People in the North and in the South disagreed over slavery.

0 People in the North wanted control of the country when theyfound out that gold had been discovered in the South.

0 People in the South wanted control of the country when theyfound out that oil had been discovered in the North.

Change and Continuity in American Democracy: Crisis of the Union: Civil War and ReconstructionIdeas, Institutions, Practices, and Controversies (1850 to 1877)

Sample Question Results 0Multiple-Ghoiee


62 CHAPTER 5

-1 IOverall percentage Below Basic 11 Basic Proficient Advanced

correct 194 and below* H 195-242* 243-275* 276 and above*

57 34 59 86 ***

*NAEP U.S. history compos.te scale range.***Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.


7 6

cz:17' Inuit

, .-----,- ---------,

s

,

Sioux %...s,

, Cliippewa

-

,Nez Perce \ ,; ---------- ,

,,

L), ,

, Shoshone \ Cimyenn \Pomo'',

; .21-: ------- ---,, PMute ------ \ Arapaho

--: quiwnee-- Navajo \

Hopi \ ,...----`- Cherokee c,

Apache \ \ CreekYuma

Mohawk

Dela are

Seminole

Gulf ofcz.zz,

Mexico V%

Choose an American Indian group from the map, and circle its name directlyon the map.

On the chart below, list one way this American Indian group got food,shelter, and clothing in the period before Europeans came to theAmericas. Then list one way your family gets food, shelter, and clothing.

American Indians in the PeriodYour FamilyBefore Europeans Came

1. Food: 1. Food:

2. 2.Shelter: Shelter:

3. 3.Clothing: Clothing:

Give one reason why the American Indian group long ago and your familytoday differ in the ways they get their food, shelter, or clothing.

The Gathering and Interactions of Peoples,Cultures, and Ideas

Three Worlds and Their Meeting in the Americas(Beginnings to 1607)

77

CHAPTER5 U.S. HISTORY REPORT CARD 63

Responses to this question were scored according to a four-level rubric asInappropriate, Partial, Essential, or Complete.

Sample Question Results (Mended Constructed-Response

Overall percentage "Essential" or better and percentages "Essential" or better within each achievementlevel range: 2001

64 CHAPTER 5

Pwat-to, augiftar Clf MOTgelkogimit-VAInialTimig .

Overall percentage

"Essential" or better

42


13

Basic195-242*

48

Proficient243-275*

76

Advanced

276 and above*

***



Responses scored "Complete" circled an American Indian group and correctly listed waysthat the group chosen got food, shelter, and clothing, and ways that the student's own familygets food, shelter, and clothing. They then gave one appropriate reason for differencesbetween the way the American Indian group obtained those necessities and the way in whichmodern families obtain them.

Sample "Complete" Response:

Inuit4'1;1,4

HudsonBay

\ ..-------; ---------, Sioux,

s Ciiiirewas.

Nez Perce ....\s

,

Shoshone \ Cheyenne ., Dela aret'

roan Paiute ----- 1% Arapaho !

Hopi % -Cherokee

Apache \ \ Creek

Seminole

Gulf ofMexico sr%

(Cr

American Indians in the PeriodBefore Europeans Came

1. Food:1-11 e ci,v4 e A

-60d.

1. Food:

2. Shelter: 'IA-ousL,Ines 2. Shelter:

macke A irees3. Clothing: -The9 qsea. AfNe. 3. Clothing:

ski h ctvitmck5

Your Family

(:) +0

sup ex trnafkth,

We_ \otAticX koLAses.

WQ loam -crorn

MR%-Give one reason why the American Indian group long ago and your familytoday differ in the ways they get their food, shelter, or clothing.

WQ- dAer be_c cluse. a Ions 'rime01/40\ o wer-enlk Qnu Stor-P, or- rec,1

?eopk CO we, Inavee

Th


Responses scored "Essential" circled an American Indian group and correctly listed two waysthat the group chosen got food, shelter, or clothing, and two ways that modern families obtainthem. They did not give an appropriate reason for these differences.

Sample "Essential" Response:

66 CHAPTER 5

Inuit

Cree

-------- -----Sioux ss,

Chsippewa' Nez Perce \

/ - ---------- \Shoshone \ Cheyenne

Porno i, \ Arapaho% ......--- Navajo \ Shawnee

r-----.- -CherOkeeHorn c...

Apache \ \ Creek

- ---------------

Mohawk'-./Dela are

Serninole

Gulf ofoct,

Mexico eb,


I. Food: Thel

2. Shelter: useA04v -610 Kole

3. Clothing: 4j /Y) aA pa(s

Your Family

I. Food: Glrocefi,\

2. Shelter: -140USeS

3. Clothing: Dep a C4 rnent

SAuCe_5Give one reason why the American Indian group long ago and your familytoday differ in the ways they get their food, shelter, or clothing.

()le Li\ s +ou cal


Responses scored "Partial" circled an American Indian group and were able to identifycorrectly one way the group chosen got food, shelter, or clothing and one way modern familiesget them.

Sample "Partial" Response:

,ex17 Inuit

--------------- Cree---------

Sioux \ ,

ChIpcNez Pace 2),

----------, Shoshone \ Chem.me

Pomm Aran. ho; \ ; ShawneePaiuteNavajo \ , ---------

Hopi ---- Cherokee s

Apache \ CreekYuma

11

Dela arc

Seminole

Gulf ofMexico ,1060


I. Food:

2. Shelter:

3. Clothing:

rt12,1.0,Citto-pel _

..1714:141

I. Food:

2. Shelter:

Your Family

3. Clothing:

aXe40-

rnctke.Give one reason why the American Indian group long ago and your familytoday differ in the ways they get their food, shelter, or clothing.

/cerY&aPc

-6(%-d)c,

sit


Grade 8 Sample ssessmentQuestion ResultsIn addition to assessing content appropriateto an eighth-grade U.S. history course,

assessment questions for eighth gradeassessed a range of history skills, such as thetext interpretation skills measured in samplequestion 8.

Why was Roger Williams forced to leave the Massachusetts Bay Colony?

0 He claimed that the Puritan government had no right to controlreligious beliefs.

He was more loyal to the King of Spain than to the Englishmonarchy.

0 He refused to do his share of the farming and other work.

0 He wanted to lead a war against the American Indians.

Change and Continuity in American Democracy: Colonization, Settlement, and CommunitiesIdeas, Institutions, Practices, and Controversies (1607 to 1763)

VetOo LT Sample Question Results


68 CHAPTER 5

Pe eente. -o eotao te e ent4km9

dilitront rvals


52

Below Basic II

251 and below* 11

34 H

Basic252-293*

56

Proficient294-326*

79


***



What was the most significant factor that led the American colonists to formthe First Continental Congress in 1774 ?

cp Religious conflict inside the colonies

The desire of the colonists to write a Constitution to replace theArticles of Confederation

0 Colonial frustration with laws passed by the British Parliament

0 The desire of the colonists to stop the war between Britain andthe colonies

Change and Continuity in American Democracy: The Revolution and the New NationIdeas, Institutions, Practices, and Controversies (1763 to 1815)

if Fag L1,5 Sample Question Results



39


29

Basic252-293*

39

11

1: 294-326*

62


***

*NAEP U.S. history compos.te scale range.**"Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.

8 3

CHAP1ER 5 U.S. HISIORY REPORT CARD 69

Why was the invention of the steel plow important in United Stateshistory?

Economk and Technological Changes and Their The Development of Modern AmericaRelation to Society, Ideas, and the Environment (1805 to 1920)

Responses to this question were scored according to a three-level rubric asInappropriate, Partial, or Appropriate

VC@ Sample Question Results (Short Oontrueted-Response

Overall percentage "Appropriate" and percentages "Appropriate" within each achievement levelrange: 2001

70 CHAPTER 5

Overall percentage"ApproPriate"

30

Pe on a a op tiV) MolDMkgweikaehie e en -le el

Below Basic Basic251 and below* ' 252-293*

9 34

Proficient Advanced294-326* 327 and above*

64 ***



4

This "Appropriate" response indicated that the steel plow increased efficiency in agriculturalproduction.

Sample "Appropriate" Response:


ae-A ftJ Wats- 4YortterjctS\-eA I onler) vJor\CaA ;14.Apsi ana cotM

co.trm \AgAiar

This "Partial" response indicated in a general way that the steel plow made farming easier; itwas correct but was not specific about the steel plow's impact.



vi\aVvL__-FoKana

CHAPTER 5 o U.S. HISTORY REPORT CARD 11

Sample Question

In America, it was no disgrace to work at a trade. Workmen andcapitalists were equal. The employer addressed the employee as you,not familiarly as thou. The cobbler and the teacher had the same title,"mister," and all the children, boys and girls, Jews and Gentiles, wentto school!

Polish immigrant, 1910

Using the quotation above and your knowledge of history, explain in yourown words two important aspects of life in the United States that seemedgood to this immigrant.

1)

2)

What do you think was the most important difference this man sawbetween Poland and the United States?


8 6


The Development of Modern America(1865 to 1920)

Responses to this question were scored according to a three-level rubric asInappropriate, Partial, or Appropriate

If& LO Sample Question Results (Short Gonstruoted-Response

Overall percentage "Appropriate" and percentages "Appropriate" within each achievement levelrancre- 2001

eree age Ap ropria eao erne ClaTE4

610terva s

Overall percentage Below Basic Basic I Proficient Advanced"Appropriate" 251 and below* 252-293* 294-326* 327 and above*

33 11 38 64 ***

*NAEP U.S. history composite scale range.*"*Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.

Responses scored "Appropriate" correctly identified two positive aspects of American life asperceived by the immigrant quoted and explained an important distinction between theUnited States and Poland in the eyes of that immigrant.

Sample "Appropriate" Response:

Using the quotation above and your knowledge of history, explain in yourown words two important aspects of life in the United States that seemedgood to this im igr nt.


/

f-/) Aee4

8 7


Responses scored "Partial" correctly identified two positive aspects of American life asperceived by the immigrant quoted but did not explain an important distinction between theUnited States and Poland in the eyes of the immigrant.


14 CHAPTER 5

Using the quotation above and your knowledge of history, explain in yourown words two important aspects of life in the United States that seemedgood to this immigrant.

1)

...e.do..,


eaU.S. HISTORY REPORT CARD

Grade 12 Sampie AssessmentQuestion esuitsQuestions at the twelfth-grade levelassessed U.S. history knowledge and skillsat a more sophisticated level than those atgrades 4 and 8.This greater complexity is

presented in the questions included belowboth in the degree of detail required andthe extent to which students must under-stand and analyze historical issues fromvarious perspectives, as is evident in theconstructed-response questions.

The Progressive movement of 1890-1920 is best described as

0 a broad-based reform movement that tried to reduce the abusesthat had come with modernization and industrialization

0 a loose coalition of groups primarily dedicated to passing a consti-tutional amendment prohibiting the consumption of alcohol

0 an anti-tariff movement led by a federation of business ownersand manufacturers who wanted to promote trade abroad

0 a grass-roots movement that attempted to gather support for theestablishment of an international organization such as the Leagueof Nations

Change and Continuity in American Democracy: The Development of Modern AmericaIdeas, Institutions, Practices, and Controversies (1865 to 1920)

Vag Sample Question Results



36

Below Basic

293 and below* I

23

Basic294-324*

47

Proficient325-354*

73

Advanced

355 and above*

***

*NAEP U.S. history compos.te scale range.***Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.


The phrase "Harlem Renaissance" refers to

0 African American political gains during the Reconstructionperiod

0 African American achievements in art, literature, and music inthe 1920's

0 a religious revival in the African American community thatswept the nation in the 1950's

© a series of urban renewal projects that were part of the GreatSociety program of the 1960's


Modern America and the World Wars(1914 to 1945)

ZUG MT Sample Question 111D Results 0Multiple-Ohoice


Percentage WI&achieveme t-9A4CotGRAig

`Overail percentage-correct

68

Below Basic I

*

293 and below* !

54

Basic294-324*

82

1'1

Proficient325-354*

95


***

*NAEP U.S. history composle scale range.***Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.

9 0


Article I, Section 2, United States Constitution

"[The population of the states] shall be determined by adding to thewhole number of free persons . . . three-fifths of all other persons."

An important debate led to the writing of this section of theConstitution. Identify the issue being debated.

Describe the northern position in this debate and explain whymany northerners took it.

Describe the southern position in this debate and explain whymany southerners took it.

Change and Continuity in American Democracy: The Revolution and the New NationIdeas, Institutions, Practices, and Controversies (1763 to 1815)

91


Responses to this question were scored according to a four-level rubric asInappropriate, Partial, Essential, or Complete

TIM:a Sample Question Results Oktended Construoted-Response


Overall percentage

"Essential" or better

21

FNAGofr,i* T.,wOraoh evemen

be ier dflik

Below Basic ii Basic Proficient Advanced

293 and below* I ; 294-324* 325-354* 355 and above*

I

4 i; 30 74 ***


Responses scored "Complete" identified and explained the debate between the North and theSouth over the counting of slaves for purposes of representation, making explicit the relation-ship between counting the slaves and representation in Congress.



0-% e A/z-x-/s zeiez

,e4-4 r7144,1-757;^


ASA4 AA6 114.0 "4t) /LI in 46.2-1 ca4sdi- dot.

ir)/d/rit 1/iee Yetil e;ti, 74. Afg.ctit E-404


Lty,-4-ZA Itta-AA 4-6 ertkkru_ Lt .4er-t-i-L

J218 CHAPTER 5 U.S. HISTORY REPORT CARD

Responses scored "Essential" identified the debate without explicitly noting the desire of thesouthern states to maximize their voting power in the House of Representatives.



air

(Am tN J-%


-rko- /Y16'')--t-A/v\ 1)0-1-4.

U)-t-t/NtN-t -4"21:1-Ve-c) t.:±* Jdp42.614.


-txtzA;PhaA"\. LL.Jcie, e_owni\frIND- .,-04,jad

/4-42_,Dp1 ZL.

9 3


Responses scored "Partial" identified the issue but did not make the northern and southernpositions clear.



4,444,01. daatcZ wct4c2A4.44,t,e) of "-tia to, 4.tiet holt


P"Ar°4-"


-r4 &a-

94


Grade Sample Question

"In spite of the obvious advantages held by the North, the Southwas able to fight for four years and to achieve some real militarysuccesses. So while the North held most of the cards, the South hadone or two aces up its sleeves."

Identify two of the "aces" (significant advantages) that theSouth had in the Civil War. Explain how these advantages helped theSouth.

Change and Continuity in American Democracy: Crisis of the Union: Civil War and ReconstructionIdeas, Institutions, Practices, and Controversies (1850 to 1877)

CHAPTER 5 o U.S. HISTORY REPORT CARD 81

Responses to this question were scored according to a four-level rubric asInappropriate, Partial, Essential, or Complete

Vet& LH Sample Question Results (Mended Construeted-Response


een geai eve

nba CC' b ,te r unto)f4U14 VW&

Overall percentage"Essential" or better

39

Below Basic Basic

293 and below 294-324*

17

Proficient Advanced

325-354* 355 and above*

62 88 .**

"NAEP U.S. history compos.te scale range."**Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, NationalCenter for Education Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. HistoryAssessment.

Responses scored "Complete" identified two advantages held by the South and explainedhow each advantage identified aided the southern war effort.



7,4 e _.5-06z/h cl/la" ce:L09/e.

ae OtiO e ,-C/eett'sMe9 e",&"' 74:11),Ity lAi-/-Aoir he) ni e . rAep eJ ,7(/).,/ k a

-/D AA( 'e/1 a sh1rij c se-votee- 05,:r91-a4fe_1'Z2 rP4e ,,0 / f ovA

at21-716 rA, AA? 4,0...r 497&ye..--*/).,-a. kFtes. z

afteer7i, AeZ

O / fAe 44.7'M AAvi-7) 4 d 'Or7eye)


Responses scored "Essential" identified one advantage the South had and explained how thisadvantage aided the southern war effort (or the responses identified two advantages but didnot fully explain how both advantages aided the southern war effort).



rno -ich.Q. CA/U.(1)6T WA5 pu on souifromLand.. bigk -Oksz. --tuo ak.05u1 t th a t'r ()( IfteW

--taraln tb4tRir-than -thQ,}4'04frurn 5DI39Lorict 9c1Li*Jun ms (bus 7.tha Ahsz_

oxv-Ifks_ nor-thmAr-c uArl, nor

Responses scored "Partial" identified one advantage the South had, but did not sufficientlyexplain how this advantage aided the southern war effort.



Ak% roll _Malta.. 016. 11.10.1. 411 d in. .

it-vINa -4.1 ci.kik tar

97


Maps of Selected ItemDescriptions on theNAEP U.S. History ScaleGrades 4, 8, and 12The U.S. history performance of fourth-,eighth-, and twelfth-graders can be illus-trated by maps that position item descrip-tions along the NAEP U.S. history scalewhere items are likely to be answeredsuccessfully by students.' The descriptionsused on the maps focus on the U.S. historyknowledge or skill needed to answer thequestion. For multiple-choice questions,the description indicates the knowledge orskill demonstrated by selection of thecorrect option; for constructed-responsequestions, the description takes into ac-count the knowledge or skill specified bythe different levels of scoring criteria forthat question. Five of the questions de-scribed on the item maps are includedamong the sample questions in the preced-ing section. Each of these sample questionsis identified as such on the item map.

Figures 5.1 through 5.3 are item mapsfor grades 4, 8, and 12, respectively. Themap location for each question identifieswhere that question was answered success-fully by at least 65 percent of the studentsfor constructed-response questions and 74percent of the students for four-optionmultiple-choice questions. For each ques-tion indicated on the map, students whoseaverage score fell at or above the scalepoint had a higher probability of success-fully answering the question, and studentswhose average score was below that scalepoint had a lower probability of success-fully answering the question.

84 CHAPTER 5

As an example of how to interpret theitem maps, consider the multiple-choicequestion in figure 5.1 that maps at scorepoint 245. As the description indicates,fourth-graders were required to "identifyhow railroad affected Chicago." As this wasa four-option multiple-choice question,students with an average score at or above245 had at least a 74 percent probability ofanswering the question correctly. Studentswith an average score below 245 had lessthan a 74 percent probability of doing so.This does not mean that all students withan average score of 245 or above alwaysanswered the question correctly, or that allstudents scoring below 245 always an-swered the question incorrectly. Rather, theitem map indicates higher or lower prob-ability of answering the question success-fully depending on students' overall U.S.history ability as measured by the NAEPscale.

The three U.S. history achievementlevels for a specific grade are indicated onthe item map for that grade. It is importantto note that, although the same 0-500 U.S.history scale is used at each grade, theachievement levels are grade specific andeach achievement level begins at a differentscore point at each grade.

l Details on the procedures used to develop item maps are provided in appendix A.


[2311 o3 Asi4[6v

This at) escriab9 hitx.k0 ijMaiogagx"3 tii4Attel

Aftai.v,Tft 5.ffT

KLUirift tt* w3cit

.urenifJftlJ1jGT a "My gTrog;4*

quo ,to

310

NAEP U S Histov Scae

3.0.0.....300- Identify branch of government responsible for passing laws

295 Recognize immigrant quotation

29u 291 Describe colonial-era hardships

288 Describe ways American Indians helped Pilgrims

284 Recognize excerpt from Declaration of Independence

Advanced 28u 282 Explain importance of one of the Founding Fathers

VI@

210-270 Use map to identify continent from which slaves came

267 Identi6/ purposes of key 18th-century document

260 263 Compare an American Indian group's way of life with that of families todaySample Question 4

258 Recognize that theme of photographs is racial prejudice

250 252 Describe American Indian use of animal resources

Identify factor affecting American Revolution

245 Identify how railroad affected Chicago

-240 240 Identify purpose of 19th-century means of transportation

235 Recognize name for anti-segregation laws

230-230 Identify reason for major piece of progressive legislation

Proficient 247

Basic

220

210

224 Identify reason for location of 19th-century industry

218 Identify government spending pattern from graph

216 Identify region for colonial economic activity

209 Recognize cultural significance of totem poles

................... .......... ° ° °192 Use historical photograph to make inference about suffragists' goal

Identify status of the author of 19th-century letter

180

0

184 Describe way in which American Indians interacted with colonists

NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question.

* Each grade 4 U. S. history question in the 2001 assessment was mapped onto the NAEP 0-500 U. S. history scale. The position ofthe question on the scale represents the scale scoreattained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-option

multiple-choice question. Only selected questions are presented. Scale score ranges for U. S. history achievement levels are referenced on the map. For constructed-response questions, the

question description represents students' performance at the scoring criteria level being mapped.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP),2001 U. S. History Assessment.


A400il(0@wiltOtoi

AvavailwilkogiLAB41iJ

(MD(45a)A7

9cupW? D

to a

s4,-*

ro

Wifg Btti)

(1010,V0tirMTAIG6 I dal?mx`4114jE

[Wm(Waite&

des i e

div du

rd n ifie.xir gg mike&

(ix ababil

aDWAV91117v

am:Alit:4G

En()%AA)

340

NAEP U S Wstonf ScaDe

.3aAdvanced 3 332 Use pie charts to identify and interpret changes in colonial population330 Recognize which of a group of sources is secondary

Proficient&ge3

Basic

3 322 Identi6/ African American leaders319- Identi4, political factor involved in adoption of Constitution317 Use map to identify and explain regional economic distinctions in particular era

31_0312 Interpret and put in historical context a Revolutionary-era image

308 Interpret message of political cartoon about attitudes toward immigrants

3305 Identi4, impetus for a program of government-sponsored reform302 Explain why steel plow was important in U.S. historySample Question 7

295 Recognize Mesoamerican group powerful at time of Columbus.290. . . . . . . . . . . . . . . . . . . . . .................288 Describe effects on West of the expansion of the railroad system

280 283 Interpret major Civil War-era speech

270 272 Identify reason for Mormon migration to Utah-269 'Explain a cause of Civil War, with reference to northern and southern views

2.60 261 Recognize and explain importance of particular technological changes

258 Interpret a quotation from an immigrant to the United StatesSample Question 8

50Identifyunion strategy against factory owners2:_

248 Explain one way in which the railroads affected particular U.S. region246 Interpret the meaning of an important Civil War-era speech243 Categorize group of events as belonging to civil rights movement

230

220

210

238 Identify importance of major event in civil rights movement

227 Use photograph to infer purpose of late-19th-century photographer

216 Identify major U.S. political leader of Revolutionary and early national periods


* Each grade 8 U. S. history question in the 2001 assessment was mapped onto the NAEP 0-500 U. S. history scale. The position of the question on the scale represents the scale score

attained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-optionmultiple-choice question. Only selected questions are presented. Scale score ranges for U. S. history achievement levels are referenced on the map. For constructed-response questions, the

question description represents students' performance at the scoring criteria level being mapped.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress INAEP),

2001 U. S.History Assessment.


Ithrp Map

-Eto ck 41141ant4(Iwo- t ens waQ

etiona ,S3Dwi; mi0

yablx(14t-*gy

eser'bes

Mass eitgaktqw

T. klaWyumOto4

111:; MIGGEWIROgi)

ggeWlabikaiR.91Vie

ident f e

studen

prob bilIty

es ion.

3/0

NAEP U.S. Histovy Scale

366 Explain basic features of Constitutional three-fifths compromise--Sample Question 11r. 365 Recognize significance of Gulf of Tonkin Resolution

Advancedaui358 List a factor that led to US involvement in late-20th-century war356. Identi6i major,point of Bryark"C'ross,,of,Gplcf" speech

aso348 Interpret a political cartoon about 1960's foreign policy

345 Describe conditions leading to 20th-century reform movement

34u 341 Identi6i conditions experienced by U.S. soldiers in World War I

334 Describe two ways war affected society, use supporting evidence

Proficient .3 3 v 330 Use cartoon to identify a slogan of Theodore Roosevelt's foreign policyn 332 Recognize Bay of Pigs invasion of Cuba as subject of political cartoon

328 Describe and explain religious institution's involvement in post-World War II reform

324 identify changes in 0 Cheidee sOcreti Snd ideritify their Co'ns`eqtrehtloiced ierVval** *** 0

.32.0316 Identify book by Upton Sinclair about conditions in meat-packing industry313 Analyze passenger lists to infer contrasts between southern and New England colonies

3 .31111. 0_Eldxepnlatlifnyd rnifaf jeorrenccaeussebeor WfrohmiteinadnednAtumreedricsaervn alnndtitaonsalattvietuldaebsortoinwavridrglianniad ownership

307 Infer from 1950's job survey changing social attitudes toward women306 Recognize key assumption of 20th-century government reform program

I1 304 Use photograph to identify theme of 20th-century women's protest

.2.8.0---279-Identi6i meaning of 1787 quotation about superiority of small republics

275 Use photograph to identify dates of 20th-century demographic shift

27 0 272 Use photograph to identify a major event marking end of Cold War

295Recognize achievements of the Harlem RenaissanceSample Question 10

284 Identi4, a cause of 19th-century urban population explosion

2 6 0

249-Identify an effect of post-Civil War constitutional amendment


* Each grade 12 U. S. history question in the 2001 assessment was mapped onto the NAEP 0-500 U. S. history scale. The position of the question on the scale represents the scale scoreattained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-optionmultiple-choice question. Only selected questions are presented. Scale score ranges for U. S. history achievement levels are referenced on the map. For constructed-response questions, thequestion description represents students' performance at the scoring criteria level being mapped.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP),2001 U. S. History Assessment.


CDassroom Contexts for Learning

This chapter presents information about practices that take

place in and around the school that may affect the teaching

and learning of U.S. historythe amount of time studentsspend in social studies classes, adherence to state and local

standards for social studies education, classroom practices,

and the use of technology The information in this chapter

is based on teachers' and students' responses to background

questions administered as part of the NAEP 2001

U.S. history assessment. The percentage of students

and average scale scores are presented for each

contextual variable reported in order to examine the

relationship between students' home and school

experiences and their performance on the assessment

In interpreting these data, readers are reminded that

the relationship between contextual variables and

student performance is not necessarily causal. There

are many factors that may play a role in a student's

performance on NAEP

Time Spent on Social StudiesThe NAEP US. History Framework that served

as the blueprint for the 1994 and 2001 assessments

recognized that most fourth-graders do not have a formal

class in U.S. history (Attention was paid in the framework to

ensuring coverage of material that is likely to be addressed in

fourth-grade classrooms, such as state history.) The majority

of fourth-grade teachers, however, did report instructing

their students regularly in social studies. More than one-half

of fourth-graders had teachers who reported spending

How much time

do fourth-grade

teachers devote

to social studies

instruction?

How do classroom

activities and

computer use

relate to student

achievement?

102

lime Spent on

Social Studies

State and Local

Standards

lnstruclionalActivities

Use of

Technology


between 61 and 180 minutes per week onsocial studies instruction in 2001. Aboutone-fifth of the students had teachers whoreported spending more than 180 minutesper week on social studies, and 14 percenthad teachers who reported spending 60minutes or less.

The results presented in table 6.1 show agenerally positive relationship between theamount of time teachers reported spending

on social studies instruction and fourth-graders' performance. Students whoseteachers reported spending 60 minutes orless per week had lower scores than thosewhose teachers reported spending morethan 60 minutes per week. Fourth-graderswhose teachers spent more than 180minutes per week on social studies had thehighest average scores.

Table 6.1

Percentage of students and averageU.S. history scale scores by teachers' reportson the amount of time spent on social studiesin a typical week at grade 4: 2001

Less than 30 minutes

30 to 60 minutes

61 to 120 minutes

Grade

2

191

12

195

Time Spent on

,Fourth-Grade Social

Studie

Students

teachers

spending

reported

121 to 180 minutes

More than 180 minutes

31

211

scored

studies

highest.

The percentage of students is listed first with the corresponding average scale score presented below.





1 03

State and Lon 11 StandardsWith the rise of the movement for stan-dards-based reform in K-12 education, anincreasing number of states have adoptedstandards for history or social studies.'

In 2001, teachers of fourth- and eighth-grade students were asked about the extentto which they used state or local standardsin planning their history or social studiesinstruction. Table 6.2 presents the percent-age of students and their average scoresbased on teachers' responses to this ques-tion. Only a small percentage of studentshad teachers who reported that there wereno state or local standards that applied toteaching social studies (3 percent at grade

4, and 1 percent at grade 8). About two-thirds of the students had teachers whoreported that standards were used to a largeextent in planning instruction (63 percentat grade 4, and 69 percent at grade 8).There were, however, no statistically signifi-cant differences in students' performance ateither grade 4 or grade 8, based onwhether or not there were standards or onthe extent to which teachers reportedusing standards for planning social studiesinstruction. Because state and local stan-dards are diverse and are used in variousways, readers should interpret this data withcaution.

I Council of Chief State School Officers (2000). Key state education policies on K-12 education (table 13, p. 23).Washington DC:Author


Table 6.2

Percentage of students andaverage U.S. history scale scoresby teachers' reports on the use ofstate/local standards in planninginstruction at grades 4 and 8: 2001

Grade 4

Not at all

Small extent

Moderate extent

Large extent

Grades

Use,ofatate/Local

Standards in

planning Instruct Kin

2

212

9

210

23

206

No standards for teaching social studies

Grade 8

Not at all

Small extent

Moderate extent

Large extent

No standards for teaching social studies

224

2

274

7

264

21

266

firiffl lidefourth-grade

Ecoi eighth-

students

teachers

state/localdards

planning

instruction.

1

276





instructional ActivitiesFourth- and eighth-grade teachers whosestudents participated in the NA_EP U.S.history assessments in 1994 and 2001 wereasked a series of questions about the fre-quency with which they engaged theirclasses in certain types of instructionalactivities when teaching social studies orU.S. history. The results, presented in table6.3, vary somewhat by grade level and

1 0 592 CHAPTER 6 U.S. HISTORY REPORT CARD

show that most students had teachers whoreported having them read from a textbookat least once or twice a week (88 percent atgrade 4, and 90 percent at grade 8). In2001, fourth-graders whose teachers askedthem to read from a textbook on a dailybasis had higher average scores than stu-dents whose teachers did so on a weekly ormonthly basis. Although only a smallpercentage of fourth-graders had teachers

who reported never or hardly ever askingthem to read from a textbook, the averagescores for these students were not signifi-cantly different from those whose teachersdid so more frequently. In contrast, eighth-graders whose teachers reported never orhardly ever having them read from atextbook had higher scores than their peerswhose teachers engaged in this activitydaily, weekly, or monthly. Here again, thisinvolved a small percentage of students.

A large majority of fourth- and eighth-graders were in classes where teachersreported asking students to read extramaterials that were not in the regulartextbook, such as biographies or historicalfiction, on at least an occasional basis(weekly or monthly). There were, however,no differences in students' performance ateither grade, related to the frequency withwhich teachers reported having them readthese extra materials.

Teachers were also asked about thefrequency of their use of primary historicaldocuments, letters, diaries, or essays writtenby historical figures.While the use of suchtexts as part of history or social studiesinstruction was not related to studentperformance at grade 4, there was apositive relationship associated withweekly use of these materials at grade 8.Eighth-graders whose teachers reportedusing primary sources on a weekly basishad higher average scores than thosewhose teachers reported doing so monthlyor never.

A question that asked teachers about thefrequency with which they engaged theirstudents in writing reports revealed differ-ent performance patterns at grades 4 and 8.The average scores of fourth-graders whoseteachers asked them to write reportsweekly were lower than the scores ofstudents whose teachers did so less fre-quently. There was no relationship be-tween the performance of eighth-gradersand the frequency of writing reports.

On the whole there has been littlechange between 1994 and 2001 in thepercentage of students whose teachersreported various amounts of time spent inparticular instructional activities. Therewere some exceptions to this pattern ofstability, however. At grade 4, the percent-age of students with teachers who reportedreading extra material once or twice aweek increased from 33 percent in 1994 to44 percent in 2001, while the percentagewith teachers reporting doing so once ortwice a month decreased. The percentageof fourth-graders with teachers who usedhistorical documents once or twice a monthalso increased, while the percentage withteachers who never or hardly ever usedthem decreased. At grade 8, the percentageof students whose teachers reported havingthem read extra material every day increasedfrom 3 percent in 1994 to 7 percent in2001. The percentage of eighth-graderswhose teachers never used primary histori-cal documents decreased from 1994 to 2001.


Table 6.3

Percentage of students and averageU.S. history scale scores by teachers' reportson frequency of classroom activities atgrades 4 and 8: 1994 and 2001

Grade

1994 2001

Reading material from a textbook

Almost every day 43

207

Once or twice a week

Frequency ofA,t441:ekk

Fourtht-and

'Elghth,-Grade,

ErWroonACtis `t*

Calip

204

reading

teMbook

associated

Once or twice a month

204

higher

reading

Never or hardly ever

Reading extra material not in the regular textbook

Almost every day




Using primary historical documents

Almost every day

5 5

204 209

Cuca

cd01

iKuq@

co @mg*LE&monthly

6 9

208 210

33 44 *

205 211

46 35 *204 208

15 12

208 208

Once or twice a week 8

201

Once or twice a month 29

208

Never or hardly ever 62

205

11

207

39 *212

48 *208

Writing a report

Almost every day ***1

*** 194

Once or twice a week 6 6

188 198

Once or twice a month 63 59

207 210

Never or hardly ever 31 34

205 210

1 7


See footnotes at end of table. r>

Table 6.3 (continued)

Percentage of students and averageU.S. history scale scores by teachers' reportson frequency of classroo activities atgrades 4 and 8: 1994 and 2001

Grade

1994 2001


Almost every day




45 45

259 264

42 45

259 262

8 7

266 262

5 3

265 275

Reading extra material not in the regular textbook

Almost every day




3 7 *254 265

32 37

258 261

47 44

263 264

17 13

258 264


Almost every day 2 4

268 264


260

Once or twice a month 55 54

261 262

Never or hardly ever 23 16

258 259

Writing a report

Almost every day




1

*** 255

4 7

256 266

66 66

261 263

30 27

259 263

W9GGet ow

primary documents

associated

highersOD

gom@D91frequent 0B21.

The percentage of students is listed first with the corresponding average scale score presented below.*Significantly different from 1994.*** Sample size is insufficient to permit a reliable estimate.# Percentage is between 0.0 and 0.5. 108NOTE: Percentages may not add to 100 due to rounding.




Table 6.4 presents the data for questionsasked of twelfth-grade students that weresimilar to those asked of fourth- andeighth-grade teachers. A large majority ofstudents reported reading from a textbookon a daily or weekly basis and had higheraverage scores than those who did so onlya few times a year or never. Students whoreported that they never read extra mate-rial, such as biographies or historical stories,not in the regular textbook had loweraverage scores than students who did so afew times a year or more. However, therewas no statistically significant difference inthe performance of students who read suchextra material as frequently as every daycompared to others who used it to a morelimited extent, including those who didso as infrequently as a few times a year.Both twelfth-graders who reported neverusing letters, diaries, or essays written by

1 9

96 CHAPTER 6 e U.S. HISTORY REPORT CARD

historical people and those who reporteddoing so on a daily basis had lower averagescores than students who reported engag-ing in this activity on a more moderatebasisweekly, monthly, or yearly. A similarpattern was evident in the results presentedfor writing reports.

A general pattern noticeable between1994 and 2001 was that an increasingpercentage of twelfth-graders reportedregularly engaging in the instructionalactivities that they were asked about. Thepercentage of students who reported dailyreading fiom a textbook increased from40 percent in 1994 to 44 percent in 2001.There were also higher percentages oftwelfth-graders who reported using histori-cal documentation and writing reportsdaily, weekly, and monthly in 2001 thanin 1994.

Table 6.4

Percentage of students and averageU.S. history scale scores by students'reports on frequency of classroomactivities at grade 12: 1994 and 2001

Grade e-

I 1

Frequency,of.

Twelfth-Grade

Classtroom Activities

Read material from a textbook

About every day



A few times a year

Never

Read extra material not in the regular textbook

About every day



40 44 *

289 290

40 38

289 289

9 8

284 283

6 6

278 276

6 5 *268 270

9 10

288 290

30 31

289 291 Students

24 25

291 290

A few times a year

Never

18 17

288 289

18 7

274

reported

reading

material

lowest.

scored

See footnotes at end of table. E>


Table ued)

Percentage of students and averageU.S. history scale scores by students'reports on frequency of classroomactivib e 12: 1994 and 2001

Grade

Ficerjuency sit

Clisrotim ACtnnties:

Use letters, diaries, or essays written by historical people

About every day 4 5 *

280 280


292


291

A few times a year 26

291

Never 32

8 Moderate

historical

documents

associated

higher

difo

25

279 279

Write a report

About every day 2 3 *

267 271

Once or twice a week 9 14 *

279 288

Once or twice a month 35 41 *

287 290

A few times a year 40 34 *

293 290

Never 14 8 *278 271


*Significantly different from 1994.NOTE: Percentages may not add to 100 due to rounding.



In98 CHAPTER 6 U.S. HISTORY REPORT CARO

Use of Techno8ogyComputer use in history and social studieseducation is an area of rapidly growinginterest on the part of history educators.'Fourth-, eighth-, and twelfth-grade studentswho participated in the 2001 assessmentwere asked a series of questions related totheir use of computers for history andsocial studies both at school and at home.The results presented in tables 6.5 and 6.6show the relationship between students'performance and their responses to threequestions about their use of computers.

Relatively few students reported using acomputer for social studies or history. Onlyabout one-quarter of the fourth-gradestudents reported using computers atschool for social studies at least once everyfew weeks. This proportion increased toabout one-third of the students at grade 8,and remained at about one-third amongstudents at grade 12.

The data presented in table 6.5 for grade4 and table 6.6 for grades 8 and 12, showthat the reported frequency related togeneral usage of computers at school for

history or social studies appeared to have anegative association with students' perfor-mance at all three grades. Students ingrades 4, 8, and 12 who reported daily useof computers at school for social studies allhad lower average scores than those whoreported less frequent use. There was not,however, a consistently negative associationbetween computer use and students'performance. A positive association withperformance was evident for using com-puters specifically for conducting researchand for writing reports at grades 8 and 12.At grades 8 and 12, students who used aCD or the Internet for research projects orused the computer to write reports morefrequently had higher average scores thantheir peers who did so less frequently. Thispattern did not hold true at grade 4,however, where there was no significantdifference between the performance ofthose students who reported using a CD orthe Internet for research projects or towrite reports and those who reported thatthey did not.

2 Martorella, P. H. (Ed.). (1997). Interactive technologies and the social studies: Emerging.issues and applications. Albany, NY:State University of New York Press.

Li 2CHAPTER 6 U.S. HISTORY REPORT CARD 99

Table 6.5

Percentage of students and averageU.S. history scale scores by students' reportson computer use at grade 4: 2003.

Use computers at school for social studies

Every day

Grade

frequent

general

Two or three times a week 5

186

Once a week 7

197

computer

school

associated

Owee @Km

Once every few weeks 10

212

Never or hardly ever 74

214

Do research projects using a CD or the Internet

Yes

Use computer to write reports

Yes

46

211

54

208

52

209

48

210





1 3


Table 6.6

Percentage of students and averageU.S. history scale scores by students' reportson computer use at grades 8 and 12: 2001

Grade

Use computers at school for social studies

Every day

Two or three times a week

Once a week

Once every few weeks


1

239

5

252

9

261

21

268

64

263


Not at all

Small extent

Moderate extent

26

253

31

262

26

267

6Large extent

Write reports on the computer

Not at all

Small extent

Moderate extent

Large extent

18

253

30

260

29

266

23

270

Eighthand

Wcol Galauftwaprojects

MiE)

@OCUTA

computer

research

associated

higher

CHAPTER 6

See footnotes at end of table.>


Table 6.6 (continued)

Percentage of students and averageU.S. history scale scores by students' reports Gradeon computer use at grades 8 and 12: 2001

EighthàidTWegirrn

Use computers at school for studying history

Every day


Once a week



Haven't studied history this year


Not at all

Small extent

Moderate extent

Large extent

2

265

6

277

7

280

16

291

42

289

27

289

23

274

33

286

29

294

15

300


Not at all 14

271

Small extent 27

281

Moderate extent 33

290

Large extent

DOITG Galaa?03ff

Wek

computer

writing reports

associated

higher

The percentage of students is listed first with the corresponding average scale score presented below.NOTE: Percentages may not add to 100 due to rounding.



ppen Ix AOv rview if Proc dures Ustd for theN EP 20 1 U. S. Hsttrv Assessment

This appendix provides an overview of the NAEP 2001

history assessment's primary componentsframework,

development, administration, scoring, and analysis. A more

extensive review of the procedures and methods used in the

history assessment will be included in the forthcoming

NAEP 2001Technical Report.

The NAEP 2001 History AssessmentThe National Assessment Governing Board (NAGB),

created by Congress in 1988, is responsible for

formulating policy for NAEP. The NAGB is

specifically charged with developing assessment

objectives and test specifications through a national

consensus approach. That consensus approach results

in the development of an assessment framework.

The design of the NAEP 2001 U.S. history

assessment followed the guidelines provided in the

framework developed for the 1994 assessment.'

The framework underlying both the NAEP

1994 and 2001 assessments reflects current consensus

among educators and researchers about the study of

U.S. history. Developing this framework and the

specifications that guided development of the assessment

involved the critical input of hundreds of individuals across

the country, including representatives of national education

Technical aspects ofthe NAEP 2001

history assessment

I National Assessment Governing Board (1994). US. history framework for the 1994 NationalAssessment of Educational Progress. \Washington, DC:Author.

The Assessment

The Samples

Students withDisabilities (SD)

and

Limited English

Proficient (LEP)Students

Data Collection

Data Analysis

NAEP Reporting

Groups

Cautions inInterpretations

APPEnu A U.S. HISTORY REPORT CARD 103

organizations, teachers, parents, policy-makers, business leaders, and the interestedgeneral public. This consensus process wasmanaged by the Council of Chief StateSchool Officers for NAGB.

The assessment framework specified notonly the particular aspects of U.S. historyto be measured (see chapter 1 for adescription of these aspects), but also thepercentage of assessment questions thatshould be devoted to each. The targetpercentage distributions of historicalthemes, as specified in the framework,along with the actual percentage distribu-tions in the 1994 and 2001 assessments, arepresented in table A.1. Notice that thesepercentages shift from grade 4 to grade 12to reflect the shift in curricular emphasis asstudents move from the fourth to thetwelfth grade. For example, the emphasison "the changing role of America in theworld" grows at each successive grade level.

1 7

104 APPENDIX A U.S. HISTORY REPORT CARD

It should also be noted that the actualcontent of the assessment has varied some-what in both 1994 and 2001 from thetargeted distribution. At grades 8 and 12,these variances are in part explained by theuse of"theme blocks," which focus onparticular historical themes and allowstudents to respond to a range of primarysources more extensively than would bepossible in blocks that include the fullspectrum of themes. In addition, it isevident from table A.1 that some varianceexists between the actual distribution ofquestions among the themes in 1994 andthe actual distribution in 2001. Thesevariances exist because seven new blockswere introduced in 2001 to replace blocksthat were released to the public.Within thenew blocks, the distribution of itemsdiffered somewhat from the distributionwithin the blocks that they replaced.

Target and actual percentage distribution of questions by historical theme, grades 4, 8, and 12:1994 and 2001

ba39 01E112 C3 tabActual Actual Actual Actual Actual Actual

Historical Themes Target 1994 2001 Target 1994 2001 Target 1994 2001

Change and Continuity inAmerican Democracy:

Ideas, Institutions,Practices, and

Controversies 25 , 24',

25 30 28 30 25 29 28

The Gathering andInteractions of Peoples,

Cultures, and Ideas 35 32 32 30 30 32 25 23 26

Economic andTechnological Changes

and Their Relation toSociety, Ideas, and the

Environment 25 25 32 20 23 25 25 26 22

The Changing Role ofAmerica in the World 15 19 12 20 19 13 25 21 25


Educational Progress (NAEP), 1994 and 2001 U. S. History Assessments.

The Assessment DesignEach student who participated in the U.S.history assessment received a bookletcontaining three or four sections: a set ofgeneral background questions, a set ofsubject-specific background questionsdealing largely with the student's use oftechnology, and one or two sets, or"blocks,"of cognitive questions assessing knowledgeand skills in U.S. history as outlined in theframework. At grade 4, only 25-minuteblocks were used.At grades 8 and 12,students were given either two 25-minuteblocks or one 50-minute block. The 50-minute blocks administered at grades 8 and12 (one at each grade) included extendedconstructed-response questions requiring

students to synthesize elements fromvarious primary sources.

At grade 4 a total of six blocks of cogni-tive questions were given, while at grades 8and 12 nine blocks were administered.'Some of the blocks at each grade level(three at grade 4, six at grade 8, and six atgrade 12) were carried forward from the1994 assessment to the 2001 assessment toallow for the measurement of change acrosstime. Each block consisted of both mul-tiple-choice and constructed-responsequestions. Short constructed-responsequestions required a few sentences for ananswer, while extended constructed-response questions generally required aparagraph or more. It was expected that

2 These blocks were distributed across the student booklets in a Balanced Incomplete Block (BIB) design that isdescribed later in this section.

1 2A4ENDIX A U.S. HISTORY REPORT CARD 105

students could adequately answer the shortconstructed-response questions in abouttwo to three minutes and the extendedconstructed-response questions in aboutfive minutes.

The data in table A.2 display the numberof questions by type and by grade level forthe 1994 and 2001 assessments. Some ofthese questions were used at more than onegrade level; thus, the sum of the questionsthat appear at each grade level is greaterthan the total number of unique questions.The total number of questions at eachgrade level was slightly smaller in 2001than in 1994 because in 2001 there were

slightly fewer extended constructed-response questions at grades 4 and 12, andfewer multiple-choice questions at grade 8.This decrease in the total number ofquestions simply reflects the fact that thenew blocks that replaced blocks released tothe public from the 1994 assessmentcontained slightly fewer questions. Itshould be noted that these variations acrossyears do not affect the ability of NAEP toreport changes in students' performanceacross years since the estimated changes arebased on the presence of blocks that werecommon to both assessment years.

VEM Distribution Questions Question

Distribution of questions administered by question type, grades 4, 8, and 1 2: 1 994 and 2001

tab(3 *Eib C3 tal)121994 2001 1994 2001 1994 2001

Multiple-choice 63 63 102 99 103 106

Short constructed-response 26 28 37 39 33 35

Extended constructed-

response 6 3 12 7 19 13

Total 95 94 151 145 155 154



1 9


The assessment design allowed formaximum coverage of U.S. history contentat grades 4, 8, and 12, while minimizingthe time burden for any one student. Thiswas accomplished through the use ofmatrix sampling of questions, in whichrepresentative samples of students tookvarious portions of the entire pool ofassessment questions.The aggregate resultsacross the entire assessment allowed forbroad reporting of the U.S. history abilitiesfor the targeted population.

In addition to matrix sampling, theassessment design utilized a procedure fordistributing booklets that controlled forposition and context effects. Studentsreceived different blocks of questions intheir booklets according to a procedurecalled "Balanced Incomplete Block (BIB)spiraling." This procedure assigns blocks ofquestions so that every block appears in thefirst or second position within a booklet anequal number of times. Every block ofquestions is paired with every other block,with the exception of the 50-minutetheme block, which appears on its ownwithout another block of cognitive ques-tions. The spiraling aspect of this proce-dure cycles the booklets for administration,so that typically only a few students in anyassessment session receive the same booklet.This design allows for some balancing ofthe impact of context and fatigue effects tobe measured and reported, but makesallowance for the difficulties of administer-ing the 50-minute blocks.'

In addition to the student assessmentbooklets, three other instruments provideddata relating to the assessment: a teacherquestionnaire, a school questionnaire, and aStudents with Disabilities/Limited-EnglishProficiency (SD and/or LEP) question-naire. The teacher questionnaire wasadministered to the history or social studiesteachers of fourth- and eighth-gradestudents participating in the assessment.The questionnaire consisted of threesections and took approximately 20 min-utes to complete. The first section focusedon the teacher's general background andexperience; the second section on com-puter resources available in the school; andthe third section on classroom informationabout social studies instruction.

The school characteristics and policyquestionnaire was given to the principal orother administrator in each participatingschool and took about 20 minutes tocomplete. The questions asked aboutschool policies, programs, facilities, and thedemographic composition and backgroundof the student body.

The SD and/or LEP student question-naire was completed by a school staff mem-ber knowledgeable about those studentswho were selected to participate in theassessment and who were identified as:1) having an Individualized EducationProgram (IEP) or equivalent program(for reasons other than being gifted andtalented) or 2) being limited English

3 For further details on the booklet design, see the forthcoming NAEP 2001 Technical Report.

120APPENDIX A U.S. HISTORY REPORT CARD 101

proficient (LEP). A questionnaire wascompleted for each SD and/or LEP stu-dent sampled regardless of whether thestudent participated in the assessment. Eachquestionnaire took approximately 3 min-utes to complete and asked about thestudent and the special programs in whichhe or she participated.

National SampleThe national results presented in this reportare based on a nationally representativeprobability sample of fourth-, eighth-, andtwelfth-grade students. The sample waschosen using a multistage design thatinvolved sampling students from selectedschools within selected geographic areasacross the country. The sample design hadthe following stages:

1) selection of geographic areas (a county,group of counties, or metropolitanstatistical area);

2) selection of schools (public andnonpublic) within the selected areas; and

3) selection of students within selectedschools.

Each selected school that participated inthe assessment and each student assessedrepresents a portion of the population ofinterest. Sampling weights are needed tomake valid inferences between the studentsamples and the respective populationsfrom which they were drawn. Samplingweights account for disproportionaterepresentation due to the oversampling ofstudents who attend schools with highconcentrations of Black and/or Hispanicstudents and students who attend nonpublic

schools. Among other uses, samplingweights also account for lower samplingrates for very small schools and are used toadjust for school and student nonresponse.4

Unlike the 1994 national assessment, aspecial feature of the 2001 national assess-ment was the collection of data fromsamples of students where assessmentaccommodations for special-needs studentswere not permitted and from samples ofstudents where accommodations forspecial-needs students were permitted.NAEP inclusion rules were applied, andaccommodations were offered only when astudent had an Individualized EducationProgram (IEP) because of a disability and/or was identified as being a limited Englishproficient student (LEP); all other studentswere asked to participate in the assessmentunder standard conditions.

Table A.3 shows the number of studentsincluded in the national samples for theNAEP 1994 and 2001 history assessmentsat each grade level. For the 2001 assess-ment, the table includes the number ofstudents in the sample where accommoda-tions were not permitted and the numberof students in the sample where acconmio-dations were permitted. The table showsthat the same non-SD and/or LEP studentswere included in both samples in 2001;only the SD and/or LEP students differedbetween the two samples. The 1994 designdiffered somewhat in that the SD and/orLEP students were assessed in standardconditions and accommodations were notpermitted.

4 Additional details regarding the design and structure of the national and state samples will be included in theforthcoming NAEP 2001Technical Report. In addition, the reader may consult the NAEP 2000Technical Report for adiscussion of sampling procedures that are mostly common to all NAEP assessments.


National student sample size by type of results, grades 4, 8, and 12: 1994 and 2001

Grade 4

Non SD/LEP students assessed

SD/LEP students assessed

1994

Accommodationsnot permitted

sample

5,067

Accommodations

not permittedsample

2001

6,446

Accommodationspermittedsample

without accommodations 432 581 504


with accommodations NA NA 359

Total students assessed 5,499 7,027 7,309

Grade 8

Non SD/LEP students assessed 8,227 10,321






Grade 12

Non SD/LEP students assessed 7,427 10,658






SD = Students with Disabilities.

LEP = Limited English Proficient students.

NA = Not applicable. No accommodations were permitted in this sample.




Table A.4 provides a summary of thenational school and student participationrates for the U.S. history assessment sampleswhere accommodations were not per-mitted and where accommodations werepermitted. Participation rates are presentedfor public and nonpublic schools, individu-ally and combined. The first rate is theweighted percentage of schools participat-ing in the assessment before substitution ofdemographically similar schools.' This rateis based only on the number of schools thatwere initially selected for the assessment.The numerator of this rate is the sum ofthe number of students represented by eachinitially selected school that participated inthe assessment. The denominator is thesum of the number of students representedby each of the initially selected schools thathad eligible students enrolled.

The second school participation rate isthe weighted participation rate after substi-tution. The numerator of this rate is thesum of the number of students representedby each of the participating schools,whether originally selected or selected as asubstitute for a school that chose not toparticipate. The denominator is the sameas that for the weighted participation ratefor the initial sample. Because of thecommon denominators, the weighted

participation rate after substitution is atleast as great as the weighted participationrate before substitution.

Also presented in table A.4 are weightedstudent participation rates. The numeratorof this rate is the sum across all studentsassessed (in either an initial session or amakeup session) of the number of studentsthat each represents. The denominator ofthis rate is the sum across all eligiblesampled students in participating schools ofthe number of students that each repre-sents. The overall participation rates takeinto account the weighted percentage ofschool participation before or after substi-tution and the weighted percentage ofstudent participation after makeup sessions.

For the grade 12 national sample, whereschool and student response rates did notmeet NCES standards, an extensive analysiswas conducted that examined, amongother factors, the potential for nonresponsebias at both the school and student level.No evidence of any significant potential foreither school or student nonresponse biaswas found. Results of these analyses, as wellas nonresponse bias analyses for the grade 4and grade 8 national samples, will beincluded in the forthcoming NAEP 2001Technical Report.

5 The initial base sampling weights were used in weighting the percentages of participating schools and students.Anattempt was made to preselect (before field processes began) a maximum of two substitute schools for each sampledpublic school (one in-district and one out-of-district) and each sampled Catholic school, and one for each samplednonpublic school (other than Catholic).To minimize bias, a substitute school resembled the original selection asmuch as possible on affiliation, estimated number of grade-eligible students, and minority composition.


National school and student participation rates for public schools, nonpublic schools, and publicand nonpublic schools combined, grades 4, 8, and 12: 2001

Weighted school participation

Grade 4

Percentage

before

substitution

Percentage

after

substitution

Total

number

of schools

Public 83 88 276

Nonpublic 83 91 89

Combined 83 88 365

Grade 8

Public 79 87 259

Nonpublic 84 88 110

Combined 79 87 369

Grade 12

Public 73 80 311

Nonpublic 67 77 63

Combined 72 80 374

Samples where accommodations

were not permitted

Student participation Overall participation rate

Weighted Total

percentage number of

student students Before After

participation assessed substitution substitution

96 5,978

97 1,049

96 7,027

93 9,694

96 1,561

93 11,255

77 10,051

90 1,265

78 11,316

80 84

81 88

80 85

73 81

81 84

74 81

56 62

61 70

56 62

Samples where accommodations

were permitted

Student participation Overall participation rate

Weighted Total

percentage number ot

student students Before After

participation assessed substitution substitution

96 6,266 80 84

97 1,043 81 88

96 7,309 80 85

93 10,180 73 81

96 1,582 80 84

93 11,762 74 81

76 10,220 56 61

90 1,257 61 70

77 11,477 56 62


Educational Progres (NAEP), 2001 U. S. History Assessment.

Students with Disabilities (SD)and/or Limited English Proficient(LEP) StudentsIt is NAEP's intent to assess all selectedstudents from the target population.Therefore, every effort is made to ensurethat all selected students who are capable ofparticipating in the assessment are assessed.Some students sampled for participation inNAEP can be excluded from the sampleaccording to carefully defined criteria.These criteria were revised in 1996 tocommunicate more clearly a presumptionof inclusion except under special circum-stances. According to these criteria, stu-

dents with Individualized EducationPrograms (IEPs) were to be included in theNAEP assessment except in the followingcases:

1) The school's IEP team determined thatthe student could not participate, OR,

2) The student's cognitive functioning wasso severely impaired that she or he couldnot participate, OR,

3) The student's IEP required that the studenthad to be tested with an accommodationor adaptation and that the student couldnot demonstrate his or her knowledgewithout that accommodation.'

6 As described in the following section, a second sample in the 2001 national assessments was assessed that includedstudents who required and were provided with accommodations.

124 APPENDIX A U.S. HISTORY REPORT CARD 111

All LEP students receiving academicinstruction in English for three years ormore were to be included in the assess-ment. Those LEP students receivinginstruction in English for fewer than threeyears were to be included unless schoolstaff judged them to be incapable of par-ticipating in the assessment in English.

Participation of SD and/or LEPStudents in the NAEP SamplesTesting all sampled students is the best wayfor NAEP to ensure that the statisticsgenerated by the assessment are as repre-sentative as possible of the performance ofthe entire national population and thepopulations of participating jurisdictions.However, all groups of students includecertain proportions that cannot be tested inlarge-scale assessments (such as studentswho have profound mental disabilities), orwho can only be tested through the use of((accommodations" such as extra time, one-on-one administration, or use of magnify-ing equipment.

Some students with disabilities and someLEP students cannot show on a test whatthey know and can do unless they areprovided accomrnodations.When suchaccommodations are not allowed, studentsrequiring such adjustments are oftenexcluded from large-scale assessments suchas NAEP. This phenomenon has becomemore common in the last decade andgained momentum with the passage of theIndividuals with Disabilities Education Act(IDEA), which led schools and states toidentify increasing proportions of studentsas needing accommodations on assessments

to best show what they know and can do.'Furthermore, Section 504 of the Rehabili-tation Act of 1973 requires that, whenstudents with disabilities are tested, schoolsmust provide them with appropriateaccommodations so that the test resultsaccurately reflect what the students knowand are able to do.' In addition, as theproportion of limited English proficientstudents in the population has increased,some states have started offering accommo-dations, such as translated versions ofassessments or the use of bilingual dictio-naries as part of assessments.

Before 1996, NAEP did not allow anytesting under nonstandard conditions(i.e., accommodations were not permitted).At that time, NAEP samples were able toinclude almost all sampled students in"standard" assessment sessions. However, asthe influence of IDEA grew more wide-spread, the failure to provide accommoda-tions led to increasing levels of exclusion inthe assessment. Such increases posed twothreats to the program: 1) they threatenedthe stability of trend lines (because exclud-ing more students in one year than thenext might lead to apparent rather than realgains), and 2) they made NAEP samplesless than optimally representative of targetpopulations.

NAEP reacted to this challenge byadopting a multipart strategy It becameclear that, to ensure that NAEP sampleswere as inclusive as possible, the programhad to move toward allowing the sameassessment accommodations that wereafforded students in state and district

7 Office of Special Education Programs (1997). Nineteenth annual report to Congress on the implementation of theindividuals with disabilities education act. Washington, DC: U. S. Department of Education.

8 Section 504 of the Rehabilitation Act of 1973 is a civil rights law designed to prohibit discrimination on the basis ofdisability in programs and activities, including education, that receive federal financial assistance.


testing programs. However, allowingaccommodations represents a change intesting conditions that may affect measure-ment of changes over time. Therefore,beginning with the 1996 national assess-ments and the 1998 state assessments,NAEP has assessed a series of parallelsamples of students. In one set of samples,testing accommodations were not permit-ted; this has allowed NAEP to maintain themeasurement of achievement trends. Inaddition to the samples where accommo-dations were not permitted, parallel samplesin which accommodations were permittedwere also assessed. By having two overlap-ping samples and two sets of related datapoints, NAEP could meet two core pro-gram goals.9 First, data trends could bemaintained. Second, parallel trend linescould be set in ways that ensure that infuture years the program will be able to usethe most inclusive practices possible andmirror the procedures used by most stateand district assessments. Beginning in 2002,NAEP will use only the more inclusivesamples in which assessment accommoda-tions are permitted.

In U.S. history, national data from 1994and 2001 are reported for the sample inwhich accommodations were not per-mitted. National data for the secondsample, in which accommodations werepermitted, are reported at all grades for2001 only.

In order to make it possible to evaluatethe impact of increasing exclusion rates,data on exclusion in both assessment years

are included in this appendix. Since theexclusion rates may affect average scalescores, readers should consider the magni-tude of exclusion rate changes wheninterpreting score changes.

Percentages of students with disabilities(SD) and/or limited English proficient(LEP) students for the national samplewhere accommodations were not permit-ted are presented in table A.5. The data inthis table include the percentages of stu-dents identified as SD and/or LEP, thepercentage of students excluded, and thepercentage of assessed SD and/or LEPstudents. Percentages of these students inthe national sample where accommoda-tions were permitted are presented in tableA.6. The data in this table include thepercentages of students identified as SDand/or LEP, the percentage of studentsexcluded, the percentage of assessed SD and/or LEP students, the percentage assessedwithout accommodations, and the percentageassessed with accommodations.

In the 2001 accommodations-not-permitted national sample, 7 percent ofstudents at grade 4, 8 percent of students atgrade 8, and 4 percent of students at grade12 were excluded from the assessment.The comparable percentages in the 2001accommodations-permitted nationalsample were 3 percent at grades 4 and 8,and 2 percent at grade 12, respectively.This comparison would suggest thatallowing accommodations did help todecrease the percentage of studentsexcluded from the assessment.

9 The two samples are described as "overlapping" because in 2001 the same group of non-SD and/or LEP studentswere included in both samples.


Percentage of students identified as SD and/or LEP where accommodations were not permitted,grades 4, 8, and 12: 1994 and 2001

Grade 4

SD and/or LEP students

Number of

studentssampled

1994

Weighted

percentageof students

2001

Number of

studentssampled

Weighted

percentage

of students

Identified 1,457 13 1,059 16

Excluded 1,025 5 478 7

Assessed 432 8 581 9

SD students only

Identified 961 10 582 10

Excluded 685 4 346 5


LEP students only




Grade 8


Identified 1,818 11 1,727 16

Excluded 1,278 5 809 8


SD students only

Identified 1,358 8 1,197 12



LEP students only


Excluded 323 187 1

Assessed 163 418 3

Grade 12


Identified 1,339 8 1,336 11


Assessed 391 658 6

SD students only

Identified 1,013 6 913 8



LEP students only


Excluded 184 145 1



SD = Students with Disabilities.LEP = Limited English Proficient students.NOTE: Within each grade level, the combined SD/11P portion of the table is not a sum of the separate SD and LEP portions because some students wereidentified as both SD and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.Within each portion of the table, percentages may not sum properly due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 U. S. History Assessments.


Percentage of students identified as SD and/or LEP where accommodations were permitted,grades 4, 8, and 12: 2001

Grade 4

SD and/or LEP students IdentifiedExcluded

Assessed

Assessed without accommodationsAssessed with accommodations

Number of

students sampled

1,126

263863

504

359

Weighted percentage

of students

18

3

14

6

8

SD students only Identified 645 13

Excluded 142 2

Assessed 503 11

Assessed without accommodations 180 3

Assessed with accommodations 323 7

LEP students only Identified 584 6

Excluded 154 1

Assessed 430 4Assessed without accommodations 336 3


Grade 8

SD and/or LEP students Identified 1,916 17

Excluded 484 3

Assessed 1,432 13



SD students only Identified 1,308 13

Excluded 312 2

Assessed 996 10




Excluded 201 1

Assessed 514 3


Assessed with accommodations 69 #

Grade 12SD and/or LEP students Identified 1,216 10

Excluded 397 2

Assessed 819 7



SD students only Identified 834 8Excluded 327 2

Assessed 507 5




Excluded 82 #Assessed 335 2


Assessed with accommodations 34 #


SD = Students with Disabilities.LEP = Limited English Proficient students.

NOTE: Within each grade level, the combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students wereidentified as both SD and LEP. Such students would be counted separately in the bottom portions but counted only once in the top portion.Within each portion of the table, percentages may not sum properly due to rounding.



Investigating the Effects of ExclusionRates on Assessment ResultsAs indicated by the data in the previoussection, exclusion rates have tended toincrease across assessment years in thesamples that did not permit accommoda-tions. In considering the effects of exclu-sion rates on assessment results, at least onemajor issue becomes evident. If exclusionrates vary substantially across assessmentyears, then the ability to report trends(i.e., compare results between years) maybe threatened by the fact that the resultsfrom different years are based on differentproportions of the population.

NCES has funded research into ways inwhich excluded students might be in-cluded in the estimation of scores for totalpopulations and has also commissionedstudies of the impact of assessment accom-modations on overall scores. Severalstatistical adjustment approaches for esti-mating full populations (including estimatesfor excluded students) have been proposed,but none has yet been judged ready foroperational use. Regarding the impact ofassessment accommodations on overall

scores, ETS has conducted differential itemfunctioning (DIF) studies of items assessedwith accommodations in the 1996 assess-ment.'° In these studies, ETS researchersfound little evidence that accommodationschanged the functioning of test questions.

Types of Accommodations PermittedTable A.7 displays the number and thepercentages of SD and/or LEP studentsassessed with the variety of availableaccommodations. It should be noted thatstudents assessed with accommodationstypically received some combination ofaccommodations. The numbers andpercentages presented in the table reflectonly the primary accommodation pro-vided. For example, students assessed insmall groups (as compared to standardNAEP sessions of about 30 students)usually received extended time. In one-on-one administrations, students often receivedassistance in recording answers and wereafforded extra time. Extended time wasconsidered the primary accommodationonly when it was the sole accommodationprovided.

10 For information on D1F studies of items assessed with accommodations in the 1996 mathematics and scienceassessments, see Mazzeo, J. M., Carlson, J. E.,Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of specialneeds students in NAEP;A report on 1996 NAEP research activities.Washington, DC: National Center for EducationStatistics.

116 APPENDIX A U.S. HISTORY REPORT CARD1 9

Percentage of students identified as SD and/or LEP by type of accommodation whereaccommodations were permitted, grades 4, 8, and 12: 2001


Tual Om& ais:321

Numberof studentssampled

Weightedpercentageof students

Numberof students

sampled


Numberof studentssampled


Bilingual dictionary 38 0.391 10 0.05 23 0.11

Large-print book 2 0.04 2 0.01 3 0.02Extended time 30 0.38 104 1.09 86 0.72

Read aloud 26 0.65 15 0.11 13 0.15Small group 238 5.80 416 4.81 124 1.55One-on-one 19 0.38 7 0.10 0 0.00

Scribe/computer 5 0.17 2 0.08 2 0.02Other 1 0.02 1 13 0.23 2 0.02

SD students only

Bilingual dictionary 2 0.02 1 0.01 1 0.00Large-print book 2 0.04 2 0.01 3 0.02

Extended time 30 0.38 104 1.09 86 0.72Read aloud 26 0.65 15 0.11 13 0.15

Small group 238 5.80 i 416 4.81 124 1.55One-on-one 19 0.38 7 0.10 0 0.00

Scribe/computer 5 0.17 2 0.08 2 0.02Other 1 0.02 11 0.18 2 0.02

LEP students only

Bilingual dictionary 38 0.39 10 0.05 23 0.11Large-print book 0 0.00 0 0.00 0 0.00

Extended time 15 0.15 L 25 0.16 8 0.04Read aloud 6 0.05 1 0.01 0 0.00

Small group 30 0.25 31 0.19 3 0.02One-on-one 4 0.04 0 0.00 0 0.00

Scribe/computer 1 0 0.00 0 0.00 0 0.00Other 1

I

I 0.02 2 0.05 0 0.00

SD = Students with Disabilities. LEP = Limited English Proficient students.NOTE: The combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students were identified as both SDand LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 U. S. History Assessment.

130

APPENDIX A U.S. HISTORY REPORT CARD 111

Data Collection and ScoringThe 2001 U.S. history assessment wasconducted from January through March2001, with some makeup sessions in earlyApril. As with all NAEP assessments, datacollection for the 2001 assessment wasconducted by a trained field staff.This wasaccomplished by staff from Westat, Inc.

Materials from the 2001 assessmentwere shipped to NCS Pearson, wheretrained staff evaluated the responses to theconstructed-response questions usingscoring rubrics or guides prepared byETS. Each constructed-response questionhad a unique scoring rubric that definedthe criteria used to evaluate students'responses. The extended constructed-response questions were evaluated withfour-level rubrics, and almost all of theshort constructed-response questionswere rated according to three-levelrubrics that permitted partial credit.Other short constructed-response questionswere scored as either acceptable orunacceptable.

For the 2001 U.S. history assessment,approximately 315,000 constructedresponses were scored. This numberincludes rescoring to monitor inter-raterreliability. The within-year average per-centage of agreement for the 2001 nationalreliability sample was 93 percent at grade 4,91 percent at grade 8, and 88 percent atgrade 12.

Data Analysis and ORT ScalingSubsequent to the professional scoring, allinformation was transcribed to the NAEPdatabase at ETS. Each processing activitywas conducted with rigorous qualitycontrol. After the assessment informationwas compiled in the database, the data wereweighted according to the populationstructure. The weighting for the nationalsample reflected the probability of selectionfor each student as a result of the samplingdesign, adjusted for nonresponse. Throughpost-stratification, the weighting assuredthat the representation of certain subpopu-lations corresponded to figures from theU.S. Census and the Current PopulationSurvey."

Analyses were then conducted to deter-mine the percentages of students who gavevarious responses to each cognitive andbackground question. In determining thesepercentages for the cognitive questions, adistinction was made between inissingresponses at the end of a block (i.e., missingresponses subsequent to the last questionthe student answered) and missingresponses prior to the last observedresponse. Missing responses before the lastobserved response were considered inten-tional omissions. In analysis, omittedresponses to multiple-choice items werescored as fractionally correct.'2For con-structed-response items, omitted responseswere placed into the lowest score category.

11 These procedures are described more fully in the "Weighting andVariance Estimation" section later in this docu-ment. For additional information about the use of weighting procedures, see the forthcoming NAEP 2001TechnicalReport. In addition, the reader may consult the NAEP 2000 Technical Report for a discussion of weighting proceduresthat are common to all NAEP assessments.

12 Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence ErlbaumAssociates.


Missing responses at the end of the blockwere considered "not reached" andtreated as if the questions had not beenpresented to the student. In calculatingresponse percentages for each question,only students classified as having beenpresented the question were included inthe denominator of the statistic.

It is standard NAEP practice to treat allnonrespondents to the last question in ablock as if they had not reached thequestion. For multiple-choice and shortconstructed-response questions, thispractice produces a reasonable pattern ofresults in that the proportion reachingthe last question is not dramaticallysmaller than the proportion reaching thenext-to-last question. However, forhistory blocks that ended with extendedconstructed-response questions, thestandard practice would result inextremely large drops in the proportionof students attempting the final question.Therefore, for blocks ending with anextended constructed-response question,students who answered the next-to-lastquestion but did not respond to theextended constructed-response questionwere classified as having intentionallyomitted the last question.

Item Response Theory (IRT) was usedto estimate average history scale scores forthe nation and for various subgroups ofinterest within the nation. IRT models theprobability of answering a question in acertain way as a mathematical function ofproficiency or skill. The main purpose ofIRT analysis is to provide a conmion scaleon which performance can be comparedacross groups such as those defined bycharacteristics, including gender and race/ethnicity.

In producing the U.S. history scales,three distinct IRT models were used.Multiple-choice questions were scaledusing the three-parameter logistic (3PL)model; short constructed-response ques-tions rated as acceptable or unacceptablewere scaled using the two-parameterlogistic (2PL) model; and short con-structed-response questions rated accordingto a three-level rubric, as well as extendedconstructed-response questions rated on afour-level rubric, were scaled using aGeneralized Partial-Credit (GPC)Developed by ETS and first used in 1992,the GPC model permits the scaling ofquestions scored according to multipointrating schemes. The model takes fulladvantage of the information available

13 Muraki, E. (1992).A generalized partial credit model: Application of an EM algorithm.Applied PsychologicalMeasurement,(16)2, 159-176.


from each of the student response catego-ries used for these more complex con-structed-response questions.'4

The U.S. history scale is composed ofthree types of questions: multiple-choice,short constructed-response (scored eitherdichotomously or allowing for partialcredit), and extended constructed-response(scored according to a partial-credit model).One question about the U.S. history scalesconcerns the amount of informationcontributed by each type of question.Unfortunately, this question has no simpleanswer for the NAEP U.S. history assess-ment, due to the procedures used to formthe composite history scale. The informa-tion provided by a given question is deter-mined by the IRT model used to scale thequestion. It is a function of the itemparameters and varies by level of U.S.history proficiency:5 Thus, the answer tothe query "How much information do thedifferent types of questions provide?" willdiffer for each level of U.S. history perfor-mance.When considering the compositeU.S. history scale, the answer is even morecomplicated. The U.S. history data arescaled separately by the four themes(change and continuity in Americandemocracy: ideas, institutions, practices, andcontroversies; the gathering and inter-actions of peoples, cultures, and ideas;economic and technological changes andtheir relation to society, ideas, and theenvironment; and the changing role of

America in the world), resulting in fourseparate subscales at each grade. Thecomposite scale is a weighted combinationof these subscales. IRT informationfunctions are only strictly comparablewhen the item parameters are estimatedtogether. Because the composite scale isbased on four separate estimation runs,there is no direct way to compare theinformation provided by the questions onthe composite scale.

Because of the BIB-spiraling design usedby NAEP, students do not receive enoughquestions about a specific topic to providereliable information about individualperformance. (For more information onBIB-spiraling, see "The Assessment Design"section presented earlier in this appendix.)Traditional test scores for individual stu-dents, even those based on IRT, would leadto misleading estimates of populationcharacteristics, such as subgroup means andpercentages of students at or above a certainscale-score level. Consequently, NAEPconstructs sets of plausible values designedto represent the distribution of perfor-mance in the population. A plausible valuefor an individual is not a scale score for thatindividual, but may be regarded as a repre-sentative value from the distribution ofpotential scale scores for all students in thepopulation with similar characteristics andidentical patterns of item response. Statisticsdescribing performance on the NAEP U.S.history scale are based on the plausible

14 More detailed information regarding the IRT analyses used in NAEP assessments will be provided in the forth-coming NAEP 2001 Technical Report. In addition, the reader may consult the NAEP 2000 Technical Report for adiscussion of analysis procedures that are common to all NAEP assessments.

15 Donoghue, J. R. (1994). An empirical examination of the IRT information of polytomously scored reading itemsunder the generalized partial credit model. Journal of Educational Measurement, (31)4, 295-311.

133120 APPENDIX A U.S. HISTORY REPORT CARO

values. Under the assumptions of the scalingmodels, these population estimates will beconsistent, in the sense that the estimatesapproach the model-based populationvalues as the sample size increases, whichwould not be the case for populationestimates obtained by aggregating optimalestimates of individual performance.''

item Mapping ProceduresThe U.S. history performance of fourth-,eighth-, and twelfth-graders can be illus-trated by "item maps," which positionquestion or "item" descriptions along theNAEP U.S. history scale at each grade.Each question shown is placed at the pointon the scale where questions are likely tobe answered successfully by students. Thedescriptions used on these maps focus onthe U.S. history knowledge or skill neededto answer the question. For multiple-choice questions, the description indicatesthe knowledge or skill demonstrated byselection of the correct option; for con-structed-response questions, the descrip-tion takes into account the knowledge orskill specified by the different levels ofscoring criteria for that question.

To map questions to particular points onthe NAEP U.S. history scale, a responseprobability convention was adopted thatwould divide those who had a higherprobability of success from those who hada lower probability. Establishing a responseprobability convention has an impact onthe mapping of the test questions onto theU.S. history scale. A lower boundaryconvention maps the history questions at

lower points along the scale, and a higherboundary convention maps the samequestions at higher points on the scale.The underlying distribution of U.S. historyskills in the population does not change,but the choice of a response probabilityconvention does have an impact on theproportion of the student population that isreported as "able to do" the questions onthe U.S. history scales.

There is no obvious choice of a pointalong the probability scale that is clearlysuperior to any other point. If the conven-tion were set with a boundary at 50 per-cent, those above the boundary would bemore likely to get a question right than getit wrong, while those below the boundarywould be more likely to get the questionwrong than right. Although this conven-tion has some intuitive appeal, it wasrejected on the grounds that having a50/50 chance of getting the question rightshows an insufficient degree of mastery. Ifthe convention were set with a boundary at80 percent, students above the criterionwould have a high probability of successwith a question. However, many studentsbelow this criterion show some level ofU.S. history ability that would be ignoredby such a stringent criterion. In particular,those in the range between 50 and 80percent correct would be more likely toget the question right than wrong, yetwould not be in the group described as"able to do" the question.

In a compromise between the 50 per-cent and the 80 percent conventions,

16 For theoretical and empirical justification of the procedures employed, see Mislevy, R. J. (1988). Randomization-basedinferences about latent variables from complex samples. Psychometrika, (56)2, 177-196.

For computational details, see the forthcoming NAEP 2001Technical Report.


NAEP has adopted two related responseprobability conventions: 65 percent forconstructed-response questions (whereguessing is not a factor) and 74 percent formultiple-choice questions with fourresponse options (to correct for thepossibility of answering correctly by guess-ing). These probability conventions wereestablished, in part, based on an intuitivejudgment that they would provide the bestpicture of students' U.S. history skills.

Some additional support for the dualconventions adopted by NAEP was pro-vided by Huynh." He examined the IRTinformation provided by items, accordingto the IRT model used in scaling NAEPquestions. ("Information" is used here in atechnical sense. See the forthcomingNAEP 2001 Technical Report for details.)Following Bock, Huynh decomposed theitem information into that provided by acorrect response [P(q) I(q)] and that pro-vided by an incorrect response [(1 P(q))I(q)].'8 Huynh showed that the iteminformation provided by a correct responseto a constructed-response item is maxi-mized at the point along the U.S. historyscale at which the probability of a correctresponse is 0.65 (for multiple-choice items,the information provided by a correctresponse is maximized at the point atwhich the probability of getting the itemcorrect is 0.74). It should be noted, how-ever, that maximizing the item informationI(q), rather than the information providedby a correct response [P(q) I(q)], wouldimply an item mapping criterion closer to50 percent.

Results are presented in terms of thecomposite U.S. history scale. However, theU.S. history assessment was scaled sepa-rately for the four themes in history atgrades 4, 8, and 12. The composite scale isa weighted combination of the foursubscales for the four themes in U.S.history. To obtain item map information, aprocedure developed by Donoghue wasused.' This method models the relation-ship between the item response functionfor the subscale and the subscale structureto derive the relationship between the itemscore and the composite scale (i.e., an itemresponse function for the composite scale).This item response function is then used toderive the probability used in the mapping.

Weighting andVariance EstimationA multistage sampling design was used toselect the students who were assessed.The properties of a sample selectedthrough such a design could be verydifferent from those of a simple randomsample, in which every student in thetarget population has an equal chance ofselection and in which the observationsfrom different sampled students can beconsidered to be statistically independentof one another. Therefore, the propertiesof the sample for the data collection designwere taken into account during the analysisof the assessment data.

One way that the properties of thesample design were addressed was by usingsampling weights to account for the factthat the probabilities of selection were not

17 Huynh, H. (1994, October). Some technical aspects of standard setting. Paper presented at the Joint Conference onStandard Setting for Large-Scale Assessrnent,Washington, DC.

18 Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in timo or more latentcategories. Psychometrika, 37, 29-51.

19 Donoghue, J. R. (1997, March). Item mapping to a weighted composite scale. Paper presented at the annual meeting ofthe American Educational Research Association, Chicago, IL.

122 APPENDIX A U.S. HISTORY REPORT CARD 13 5

identical for all students. All populationand subpopulation characteristics based onthe assessment data were estimated usingsampling weights. These weights includedadjustments for school and studentnonresponse.

Not only must appropriate estimates ofpopulation characteristics be derived, butappropriate measures of the degree ofuncertainty must be obtained for thosestatistics. Two components of uncertaintyare accounted for in the variability ofstatistics based on student ability: 1) theuncertainty due to sampling only a rela-tively small number of students, and 2) theuncertainty due to sampling only a rela-tively small number of cognitive questions.The first component accounts for thevariability associated with the estimatedpercentages of students who had certainbackground characteristics or whoanswered a certain cognitive questioncorrectly.

Because NAEP uses multistage samplingprocedures, conventional formulas forestimating sampling variability that assumesimple random sampling are inappropriate.NAEP uses a jackknife replication proce-dure to estimate standard errors. Thejackknife standard error provides a reason-able measure of uncertainty for any studentinformation that can be observed withouterror. However, because each studenttypically responds to only a few questionswithin any theme of history, the scale scorefor any single student would be imprecise.

In this case, plausible values methodologycan be used to describe the performance ofgroups and subgroups of students, but theunderlying imprecision involved in thisstep adds another component of variabilityto statistics based on NAEP scale scores.2°

Typically, when the standard error isbased on a small number of students orwhen the group of students is enrolled in asmall number of schools, the amount ofuncertainty associated with the estimationof standard errors may be quite large.Estimates of standard errors subject to alarge degree of uncertainty are followed bythe "!" symbol to indicate that the natureof the sample does not allow accuratedetermination of the variability of thestatistic. In such cases, the standard errorsand any confidence intervals or significancetests involving these standard errorsshould be interpreted cautiously. Addi-tional details concerning procedures foridentifying such standard errors are dis-cussed in the forthcoming NAEP 2001Technical Report.

Drawing Inferencesfrom the ResultsThe reported statistics are estimates and aretherefore subject to a measure of uncer-tainty. There are two sources of suchuncertainty. First, NAEP uses a sample ofstudents rather than testing all students.Second, all assessments have some amountof uncertainty related to the fact that theycannot ask all questions that might be

20 For further details, see Johnson, E. G. & Rust, K. F (1992). Population inferences and variance estimation for NAEPdata. Journal of Educational Statistics, (17)2, 175-190.

I 3 aArrENDIX A U.S. HISTORY REPORT CARD 123

asked in a content area. The magnitude ofthis uncertainty is reflected in the standarderror of each of the estimates.When thepercentages or average scale scores ofcertain groups are compared, the standarderror should be taken into account, andobserved similarities or differences shouldnot be relied on solely. Therefore, thecomparisons are based on statistical teststhat consider the standard errors of thosestatistics and the magnitude of the differ-ence among the averages or percentages.

Using confidence intervals based on thestandard errors provides a way to take intoaccount the uncertainty associated withsample estimates and to make inferencesabout the population averages and percent-ages in a manner that reflects that uncer-tainty. An estimated sample average scalescore plus or minus 1.96 standard errorsapproximates a 95 percent confidenceinterval for the corresponding populationquantity. This statement means that onecan conclude with approximately a 95percent level of confidence that the averageperformance of the entire population ofinterest (e.g., all fourth-grade students inpublic and nonpublic schools) is withinplus or minus 1.96 standard errors of thesample average.

As an example, suppose that the averageU.S. history scale score of the students in a

137


particular group was 256 with a standarderror of 1.2. An approximate 95 percentconfidence interval for the populationquantity would be as follows:

Average ± 1.96 standard errors256 ± 1.96 x 1.2

256 ± 2.35(253.65, 258.35)

Thus, one can conclude with a 95percent level of confidence that the averagescale score for the entire population ofstudents in that group is between 253.65and 258.35. It should be noted that thisexample, and the examples in the followingsections, are illustrative. More preciseestimates carried out to one or moredecimal places are used in the actualanalyses.

Similar confidence intervals can beconstructed for percentages, if the percent-ages are not extremely large or extremelysmall. Extreme percentages should beinterpreted with caution. Adding orsubtracting the standard errors associatedwith extreme percentages could cause theconfidence interval to exceed 100 percentor go below 0 percent, resulting in num-bers that are not meaningful. The forth-coming NAEP 2001 Technical Report willcontain a more complete discussion ofextreme percentages.

naiyzing Group Differences inAverages and PercentagesStatistical tests determine whether theevidence, based on the data from thegroups in the sample, is strong enough toconclude that the averages or percentagesare actually different for those groups inthe population. If the evidence is strong(i.e., the difference is statistically signifi-cant), the report describes the groupaverages or percentages as being different(e.g., one group performed higher than orlower than another group), regardless ofwhether the sample averages or percentagesappear to be approximately the same.

The reader is cautioned to rely on theresults of the statistical tests rather than onthe apparent magnitude of the differencebetween sample averages or percentageswhen determining whether the sampledifferences are likely to represent actualdifferences among the groups in thepopulation.

To determine whether a real differenceexists between the average scale scores (orpercentages of a certain attribute) for twogroups in the population, one needs toobtain an estimate of the degree of uncer-tainty associated with the differencebetween the averages (or percentages) ofthese groups for the sample. This estimateof the degree of uncertainty, called the"standard error of the difference" betweenthe groups, is obtained by taking the squareof each group's standard error, summing

the squared standard errors, and taking thesquare root of that sum.

Standard Error of the Difference =

SEA_B = -V(SEA' + SEB')

Similar to how the standard error for anindividual group average or percentage isused, the standard error of the differencecan be used to help determine whetherdifferences among groups in the populationare real. The difference between theaverages or percentages of the two groupsplus or minus two standard errors of thedifference represents an approximate 95percent confidence interval. If the resultinginterval includes zero, there is insufficientevidence to claim a real difference betweenthe groups in the population. If the intervaldoes not contain zero, the differencebetween the groups is statistically signifi-cant (different) at the 0.05 level.

As an example of comparing groups,consider the problem of deterininingwhether the average U.S. history scale scoreof group A is higher than that of group B.Suppose that the sample estimates of theaverage scale scores and standard errorswere as follows:

Group

Average

Scale Score Standard Error

A 218 0.9

B 216 1.1

138


The difference between the estimates ofthe average scale scores of groups A and Bis two points (218 216). The standarderror of this difference is

-V(0.92 + 1.12) = 1.4

Thus, an approximate 95 percent confi-dence interval for this difference is plus orminus two standard errors of the difference

2 ± 1.96 x 1.42 ± 2.74

(-0.74, 4.74)

The value zero is within the confidenceinterval; therefore, there is insufficientevidence to claim that group A outper-formed group B.

Conducting Multiple TestsThe procedures in the previous section andthe certainty ascribed to intervals (e.g., a 95percent confidence interval) are based onstatistical theory that assumes that only oneconfidence interval or test of statisticalsignificance is being performed. However,many different groups are being compared(i.e., multiple sets of confidence intervalsare being analyzed). In sets of confidenceintervals, statistical theory indicates that thecertainty associated with the entire set ofintervals is less than that attributable toeach individual comparison from the set.To hold the significance level for the set ofcomparisons at a particular level (e.g., 0.05),adjustments (called "multiple comparisonprocedures"21) must be made to the meth-ods described in the previous section. One

such procedure, the False Discovery Rate(FDR) procedure,22 was used to control thecertainty level.

Unlike the other multiple comparisonprocedures (e.g., the Bonferroni procedure)that control the familywise error rate (i.e.,the probability of making even one falserejection in the set of comparisons), theFDR procedure controls the expectedproportion of falsely rejected hypotheses.Furthermore, farnilywise procedures areconsidered conservative for large families ofcomparisons.' Therefore, the FDR proce-dure is more suitable for multiple compari-sons in NAEP than other procedures. Adetailed description of the FDR procedureappears in the forthcoming NAEP 2001Technical Report.

To illustrate how the FDR procedure isused, consider the comparisons of currentand previous years' average U.S. historyscale scores for the five groups presented intable A.8. Note that the difference inaverage scale scores and the standard errorof the difference are calculated in a waycomparable with that of the example in theprevious section. The test statistic shown isthe difference in average scale scoresdivided by the standard error of thedifference.

The difference in average scale scoresand its standard error can be used to findan approximate 95 percent confidenceinterval as in the example in the previoussection or they can be used to identify a

21 Miller, R. G. (1966). Simultaneous statistical inference. NewYork:Wiley.

22 Benjamini,Y. & Hochberg,Y. (1995). Controlling the false discovery rate:A practical and powerful approach tomultiple testing. Journal of the Royal Statistical Society, Series B, No. 1., pp 289-300.

23 Williarns,V. S. L., Jones, L.V, & Tukey, J. W. (1999). Controlling error in multiple comparisons with examples from state-to-state differences in educational achievement. Journal of Educational and Behavioral Statistics, 24(1), 42-69.

139126 APPENDIX A o U.S. HISTORY REPORT CARD

Example of FDR comparisons of average scale scores for different groups of students

Previous year Current year Previous year and current year

Average

scale scoreStandard

errorAverage

scale scoreStandard

errorDifferencein averages

Standard

error ofdifference

Test

statisticPercent

confidence*

Group 1 224 1.3 226 1.0 2.08 1.62 1.29 20

Group 2 187 1.7 193 1.7 6.31 2.36 2.68 1

Group 3 191 2.6 197 1.7 6.63 3.08 2.15 4

Group 4 229 4.4 232 4.6 3.24 6.35 .51 62

Group 5 201 3.4 196 4.7 5.51 5.81 .95 35

*The percent confidence is 2(1F(x)) where F(x) is the cumulative distribution of the t-distribution with the degrees of freedom adjusted to reflect thecomplexities of the sample design.

confidence percentage. In the example inthe previous section, because an approxi-mate 95 percent confidence interval wasdesired, the number 1.96 was used tomultiply the standard error of the differ-ence to create the approximate confidenceinterval. In the current example, the confi-dence interval for the test statistics isidentified from statistical tables. Instead ofchecking to see if zero is within the 95percent confidence interval about themean, the significance level from thestatistical tables can be directly compared to100-95 = 5 percent.

If the comparison of average scale scoresacross two years were made for only one ofthe five groups, there would be a significantdifference between the average scale scoresfor the two years if the significance levelwere less than 5 percent. However, becausewe are interested in the difference inaverage scale scores across the two years for

all five of the groups, comparing each ofthe significance levels to 5 percent is notadequate. Groups of students defined byshared characteristics, such as race/ethnicitygroups, are treated as sets or families whenmaking comparisons. However, compari-sons of average scale scores for each pair ofyears were treated separately. So the stepsdescribed in this example would be repli-cated for the comparison of other currentand previous year average scale scores.

To use the FDR procedure to take intoaccount that all comparisons are of interestto us, the percents of confidence in theexample are ordered from largest to small-est: 62, 35, 20, 4, and I. In the FDR proce-dure, 62 percent confidence for the Group4 comparison would be compared to 5percent, 35 percent for the Group 5comparison would be compared to0.05*(5-1)/5 = 0.04*100 = 4 percent,'20 percent for the Group 1 comparison

24 The level of confidence times the number of comparisons minus one divided by the number of comparisons is0.05*(5-1)/5 = 0.04*100 = 4 percent.


would be compared to 0.05*(5-2)/5 =0.03*100 = 3 percent, 4 percent for theGroup 3 comparison would be comparedto 0.05*(5-3)/5 = 0.02*100 = 2 percent,and 1 percent for the Group 2 comparison(actually slightly smaller than 1 prior torounding) would be compared to0.05*(5-4)/5 = 0.01*100 = 1 percent.The last of these comparisons is the onlyone for which the percent confidence issmaller than the FDR procedure value.The difference in the current year andprevious years' average scale scores for theGroup 2 students is significant; for all of theother groups, average scale scores forcurrent and previous year are not signifi-cantly different from one another. Inpractice, a very small number of counter-intuitive results occur when using the FDRprocedures to examine between-yeardifferences in subgroup results by jurisdic-tion. In those cases, results were not in-cluded in this report. NCES is continuingto evaluate the use of FDR and multiple-comparison procedures for future reporting.

NAEP Reporting GroupsResults are provided for groups of studentsdefined by shared characteristicsregionof the country, gender, race or ethnicity,school's type of location, eligibility for thefree/reduced-price school lunch program,and type of school. Based on participationrate criteria, results are reported for sub-populations only when sufficient numbersof students and adequate school representa-tion are present. The minimum require-ment is at least 62 students in a particular

subgroup from at least five primary sam-pling units (PSUs).25 However, the data forall students, regardless of whether theirsubgroup was reported separately, wereincluded in computing overall results.Definitions of the subpopulations arepresented below.

RegionResults in NAEP are reported for fourregions of the nation: Northeast, Southeast,Central, and West. Figure A.1 shows howstates are subdivided into these NAEPregions. All 50 states and the District ofColumbia are listed. Other jurisdictions,including territories and the two Depart-ment of Defense Educational Activitiesjurisdictions are not assigned to any region.

GenderResults are reported separately for malesand females.

Race/EthnicityThe race/ethnicity variable is derived fromtwo questions asked of students and fromschool records, and it is used for race/ethnicity subgroup comparisons. Twoquestions from the set of general studentbackground questions were used to deter-mine race/ethnicity:

If you are Hispanic, what is your Hispanicbackground?

I am not Hispanic

Mexican, Mexican American, or Chicano

U Puerto Rican

U Cuban

1:1 Other Spanish or Hispanic background

25 For the national assessment, a PSU is a selected geographic region (a county, group of counties, or metropolitanstatistical area). Further details about the procedure for determining minimum sample size appear in the NAEP 2000Technical Report and the forthcoming NAEP 2001 Technical Report.

l el 1128 APPENDIX X U.S. HISTORY REPORT CARD

Northeast Southeast Central West

Connecticut Alabama Illinois AlaskaDelaware Arkansas Indiana Arizona

District of Columbia Florida Iowa CaliforniaMaine Georgia Kansas Colorado

Maryland Kentucky Michigan HawaiiMassachusetts Louisiana Minnesota Idaho

New Hampshire Mississippi Missouri MontanaNew Jersey North Carolina Nebraska NevadaNew York South Carolina North Dakota New Mexico

Pennsylvania Tennessee Ohio OklahomaRhode Island *Virginia South Dakota Oregon

Vermont West Virginia Wisconsin Texas

*Virginia Utah

Washington

Wyoming

*The part of Virginia that is included in the Northeast region is the Washington, DC metropolitan area; the remainder of the state is included in the Southeastregion.

Students who responded to this questionby filling in the second, third, fourth, or fifthoval were considered Hispanic. For studentswho filled in the first oval, did not respondto the question, or provided informationthat was illegible or could not be classified,responses to the following question wereexamined to determine their race/ethnicity:

Which best describes you?

E.:11 White (not Hispanic)

Black (not Hispanic)

Hispanic ("Hispanic" means someonewho is Mexican, Mexican American,Chicano, Puerto Rican, Cuban, or otherSpanish or Hispanic background.)

El Asian or Pacific Islander ("Asian orPacific Islander" means someone who isfrom a Chinese, Japanese,Korean, Filipino,Vietnamese, Asian American or someother Asian or Pacific Islander back-ground.)

ri American Indian or Alaskan Native("American Indian or Alaskan Native"means someone who is from one of theAmerican Indian tribes or one of theoriginal people of Alaska.)

U Other (specify)

Students' race/ethnicity was then assignedon the basis of their responses. For studentswho filled in the sixth oval ("Other"),provided illegible information or informa-tion that could not be classified, or did notrespond at all, race/ethnicity was assignedas determined by school records.

Race/ethnicity could not be determinedfor students who did not respond to eitherof the demographic questions and whoseschools did not provide information aboutrace/ethnicity.

Also, some students indicated that theywere from a Hispanic background(e.g., Puerto Rican or Cuban) and that a

1. 4QPENDIX A U.S. HISTORY REPORT CARD 129

racial/ethnic category other than Hispanicbest described them. These students wereclassified as Hispanic based on the rulesdescribed above.

Type of LocationResults from the 2001 assessment arereported for students attending schools inthree mutually exclusive location types:central city, urban fringe/large town, andrural/small town:

Central City:This category includes centralcities of all Standard Metropolitan Statisti-cal Areas (SMSA) as defined by the Officeof Management and Budget. Central Cityis a geographical term and is not synony-mous with "inner city."

Urban Fringe/LargeTown: The urban fringecategory includes all densely settled placesand areas within SMSA's that are classifiedas urban by the Bureau of the Census, butwhich do not qualify as Central City. ALarge Town is defined as a place outside aSMSA with a population greater than orequal to 25,000.

Rural/SmallTown: Rural includes all placesand areas with populations of less than2,500 that are classified as rural by theBureau of the Census. A Small Town isdefined as a place outside a SMSA with apopulation of less than 25,000, but greaterthan or equal to 2,500.

Results for each type of location are notcompared across years. This was due tonew methods used by NCES to identifythe type of location assigned to each schoolin the Common Core of Data (CCD).The new methods were put into place by

NCES in order to improve the quality ofthe assignments and they take into accountmore information about the exact physicallocation of the school.The variable wasrevised in NAEP beginning with the 2000assessments.

Eligibility for the Free/Reduced-PriceSchool Lunch ProgramBased on available school records, studentswere classified as either currently eligiblefor the free/reduced-price school lunchcomponent of the Department of Agri-culture's National School Lunch Programor not eligible. Eligibility for the programis determined by students' family incomein relation to the federally establishedpoverty level. Free lunch qualification is setat 130 percent of the poverty level, andreduced-price lunch qualification is set at170 percent of the poverty level. Theclassification applies only to the school yearwhen the assessment was administered(i.e., the 2000-2001 school year) and is notbased on eligibility in previous years. Ifschool records were not available, thestudent was classified as "Information notavailable." If the school did not participatein the program, all students in that schoolwere classified as "Information notavailable."

Type of SchoolResults are reported by the type of schoolthat the student attendspublic or non-public. Nonpublic schools include Catholicand other private schools." Because theyare funded by federal authorities, not state/local governments, Bureau of Indian Affairs(BIA) schools and Department of Defense

26 Through a pilot study, more detailed breakdowns of nonpublic school results are available on the NAEP Web Site(http://nces.ed.gov/nationsreportcard/history/results/index.asp).


Domestic Dependent Elementary andSecondary Schools (DDESS) are notincluded in either the public or nonpubliccategories; they are included in the overallnational results.

Grade 12 Participation RatesNAEP has been described as a "low-stakes"assessment. That is, students receive noindividual scores, and their NAEP perfor-mance has no effect on their grades, pro-motions, or graduation. There has beencontinued concern that this lack of conse-quences affects participation rates of stu-dents and schools, as well as the motivationof students to perform well on NAEP. Ofparticular concern has been the perfor-mance of twelfth-graders, who typicallyhave lower student participation rates thanfourth- and eighth-graders, and who aremore likely to omit responses compared tothe younger cohorts.

In NAEP, there has been a consistentpattern of lower participation rates forolder students. In the 2001 NAEP assess-ments, for example, the student partici-pation rates were 96 percent and 93 per-cent at grades 4 and 8, respectively. Atgrade 12, however, the participation ratewas 78 percent. School participation rates(the percentage of sampled schools thatparticipated in the assessment) have alsotypically decreased with grade level. Againciting the 2001 assessments, the schoolparticipation rate was 88 percent for thefourth grade, 87 percent for the eighthgrade, and 80 percent for the twelfth grade.

The effect of participation rates onstudent performance, however, is unclear.Students may choose not to participate inNAEP for many reasons, such as desire toattend regular classes so as not to miss

important instruction or conflict withother school-based activities. Similarly,there are a variety of reasons for whichvarious schools do not participate. Thesampling weights and nonresponse adjust-ments, described earlier in this document,provide an approximate statistical adjust-ment for nonparticipation. However, theeffect of some school and student non-participation may have some undeterminedeffect on results.

More research is needed to delineate thefactors that contribute to nonparticipationand lack of motivation. To that end, NCESis currently investigating how varioustypes of incentives can be effectively usedto increase participation in NAEP. Onereport that examines the impact of mon-etary incentives on student effort andperformance is available on the NCES WebSite at http://nces.ed.gov/pubsearch/.Enter NCES#: 2001024.

Cautions in InterpretafionsAs described earlier, the NAEP U.S. historyscale makes it possible to examine relation-ships between students' performance andvarious background factors measured byNAEP. However, a relationship that existsbetween achievement and another variabledoes not reveal its underlying cause, whichmay be influenced by a number of othervariables. Similarly, the assessments do notcapture the influence of unmeasuredvariables. The results are most useful whenthey are considered in combination withother knowledge about the student popu-lation and the educational system, such astrends in instruction, changes in the school-age population, and societal demands andexpectations.

14 4


Appendixat ApirrdxLi

This appendix contains complete data for all the tables and

figures presented in this report, including average scores,

achievement-level results, and percentages of students. In

addition, standard errors appear in parentheses next to each

scale score and percentage.The comparisons presented in

this report are based on statistical tests that consider the

magnitude of the difference between group averages

or percentages and the standard errors of those

statistics. Because NAEP scores and percentages are

based on samples rather than the entire population(s),

the results are subject to a measure of uncertainty

reflected in the standard errors of the estimates. It can

be said with 95 percent certainty that for each

population of interest, the value for the whole

population is within plus or minus two standard

errors of the estimate for the sample.As with the

figures and tables in the chapters, significant

differences between results of previous assessments

and the 2001 assessment are highlighted.

Focus

Complete da

for all tables

and figures.

14 5

APPENDIX B

Average Scores

Achievement-

Level Results

Percentages of

Students

Standard Errors


Average U.S. history scale scores, grades 4, 8, and 12: 1994 and 2001

1994

2001

Grade 4

205 (1.0)

209 (1.0) *

Grade 8

259 (0.6)

Grade 12

286 (0.8)

262 (0.8) * 287 (1.0)

Standard errors of the estimated scale scores appear in parentheses.



[322 g Dab te Figure National Performance Distribution

National U.S. history scale score percentiles, grades 4, 8, and 12: 1994 and 2001

10th 25th

Grade 4 1994 147 (2.1) 180 (1.5)

2001 158 (1.6) * 186 (1.9) *

Grade 8 1994 217 (1.1) 239 (0.9)

2001 220 (1.3) 241 (0.9) *

Grade 12 1994 243 (1.2) 265 (1.2)

2001 246 (1.1) 266 (0.9)

50th

210 (0.9)

212 (1.3)

261 (1.1)

264 (0.8). 1

e.

288 (0.8)

288 (1.0P I

75th 90th

234 (1.2) 253 (1.4)

235 (0.9) 255 (1.4)

282 (0.7) 299 (0.6)

285 (0.7) 303 (1.1) * ,

309 (0.9) 326 (1.0)

309 (1.2) . 327 (1.5)




146

134 APPENDIX B U.S. HISTORY REPORT CARD

ifMCGOA Data .acc Figure National Aohievement4gg Results

Percentage of students within and at or above U.S. history achievement levels, grades 4, 8, and 12:1994 and 2001

At or above

Proficient

At or above

BasicBelow Basic At Basic At Proficient At Advanced

Grade 4 1994 36 (1.1) 47 (0.9) 15 (0.9) 2 (0.3) 64 (1.1) 17 (1.0)

2001 33 (1.1)* 49 (1.1) 16 (0.9) 2 (0.5) 67 (1.1) * 18 (1.0)

1

Grade 8 1994 I 39 (0,9) 48 (0.8) 13 (0.6) 1 (0.1) 61 (0.9) 14 (0.6)

2001 L 36 (0.9) * 48 (0.9) i 15 (0.8) * 2 (0.3) * 64 (0.9) * 17 (0.8) *

Grade 12 1994 57 (1.1) 32 (0.9) 10 (0.6) 1 (0.2) 43 (1.1) 11 (0.7)

2001 57 (1.2) 32 (0.9) 10 (0,6) 1 (0.4) 43 (1.2) 11 (0.9)

Standard errors of the estimated percentages appear in parentheses.




147

APPENDIX B U.S. HISTORY REPORT CARD 135

Percentage of students and average U.S. history scale scores by gender, grades 4, 8, and 12:1994 and 2001

Grade 4 1994

2001

Male

50 (0.8)

203 (1.5)

50 (0.8)

209 (1.1) *

Grade 8 1994

2001

50 (0.5)

259 (0.8)

49 (0.6)

264 (0.9) *

Grade 12 1994 50 (0,8)

288 (0.8)

2001 49 (0,6)288 (1.3)

Female

50 (0.8)

206 (1.1)

50 (0.8)

209 (1.2) *

50 (0.5)

259 (0.7)

51 (0.6)

261 (0.9)

50 (0.8)

285 (0.9)

51 (0.6)

286 (0.9)


Standard errors of the estimated percentages and scale scores appear in parentheses.

" Significantly different from 1994.NOTE: Percentages may not add to 100 due to rounding.



Differences in average U.S. history scale scores by gender, grades 4, 8, and 1 2: 1 994 and 2001

Male-Female

Grade 4 1994 2 (1.8)

2001 1 (1.6)

Grade 8 1994 # (1.0)

2001 2 (1 2)

Grade 12 1994 3 (1.2)

2001 2 (1.5)

Standard errors of the estimated difference in scale scores appear in parentheses.Score differences are calculated based on differences between unrounded average scale scores.

4 Difference is between 0.5 and 0.5.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 200111.5. History Assessments.


a

ITEM 8.6: Data Figure National Achievement-Level Results Gender

Percentage of students within and at or above U.S. history achievement levels by gender,grades 4, 8, and 12: 1994 and 2001

41:

At or above

Proficient

At or above

Basiclow Basic At Basic 1, At Proficient I At Advanced

Grade 4 Male 1994 38 (1.6) 44 (1.5) 16 (1.3) 2(0.4) 62 (1.6) 18 (1.4)2001 34 (1.3) 47 (1.3) 17 (1.1) 2(0.7) 66 (1.3) 19 (1.2)

Female 1994 35 (1.4) 50 (1.5) 14 (1.0) 2 (0.4) 65 (1.4) 16 (1.1)2001 32 (1,4) 51 (1.6) 15 (1.1) 2 (0.4) 68 (1.4) 17 (1.1)

Grade 8 Male 1994 39 (1.0) 47 (0.9) 1 14 (0,9) 1 (0.3) 61 (1.0) 15 (0.8)2001 35 (1.1) 47 (1,3) 17 (1.0) 2 (0.3)*

165 (1.1) * 18 (1.0) *

Female 1994 39 (1.3) 49 (1.2) 12 (0..8) 1 (0.1) 61 (1.3) 13 (0.8)2001 37 (1.2) 48 (1.0) 14 (0.8) 1 (0.4) 63 (1.2) 15 (0.8)

Grade 12 Male 1994 55 (1,2) 34 (0.9) 11 (0.7) 1 (0.3) 45 (1.2) 12 (0.7)2001 55 (1.6) 33 (1.1) 11 (0.8) 1 (0.5) 45 (1.6) 12 (1.1)

Female 1994 60 (1.4) 31 (1.3) 9 (0.7) 1 (0.2) 40 (1.4) 9 (0.8)2001 59 (1.3) 31 (1.2) 9 (0.7) 1 (0.3) 41 (1.3) 10 (0.9)

Standard errors of the estimated percentages appear in parentheses.* Significantly different from 1994.

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due torounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of


149


Percentage of students and average U.S. history scale scores by race/ethnicity, grades 4, 8, and 12:1994 and 2001

Grade 4

Asian/ American

White Black Hispanic Pacific Islander Indianr--

1994 69 (0.3) 15 (0.1)

215 (1.2) 177 (1.6)

2001 65 (0.4) 14 (0.2)

220 (1.1) * 188 (1.8) *

11 (0.2)

180 (2.7)

16 (0.3)

186 (2.5)

3 (0.3)

205 (3.9)

3 (0.2)

213 (2.7)

2 (0.3)

190 (6,1)

2 (0.2)

197 (6.9)

Grade 8 1994

2001

69 (0.2)

267 (0.8)

67 (0.4)

271 (0.8) *

15 (0.1)

239 (1.4)

14 (0.2)

243 (1.8)

11 (0.1)

243 (1.3)

14 (0.2)

243 (1.5)

3 (0.4)

263 (4.0)

4 (0.2)

267 (3.4)

1 (0.3)

246 (3.7) !

1 (0.2)

249 (4.5)

Grade 12 1994 74 (0.4)

292 (0.8)

2001 70 (0.4)

292 (1.0)

12 (0.2)

265 (1.5)

13 (0.2)

269 (1.5)

7(0.3)

267 (1.6)

12 (0.2)

274 (1.7) *

4 (0.2)

285 (3.4)

5 (0.2)

295 (4.6)

1 (0,2)

279 (4.0) !

1 (0.2)

277 (5.5) !



* Significantly different from 1994.! The nature of the sample does not allow accurate determination of the variability of the statistic.NOTE: Percentage may not add to 100 due to rounding.



Differences in average US. history scale scores by race/ethnicity, grades 4, 8, and 12: 1994 and 2001

White-Black White-Hispanic

Grade 4 1994 38 (2.0) 35 (3.0)

2001 31 (2.1)* 33 (2.7)

Grade 8 1994 28 (1.6) 24 (1.6)

2001 28 (2.0) 28 (1.7)

Grade 12 1994 27 (1.7) 26 (1.8)

2001 24 (1.8) 19 (2.0) *

Standard errors of the estimated difference in scale scores appear in parentheses.

*Significantly different from 1994.Score differences are calculated based on differences between unrounded average scale scores.



138 APPENDIX B U.S. HISTORY REPORT CARD 150

Percentage of students within and at or above U.S. history achievement levels by race/ethnicity,grades 4, 8, and 12: 1994 and 2001

Grade 4 White 1994

2001

Black 1994

2001

Hispanic 1994

2001

Asian/Pacific Islander 1994

2001

American Indian 1994

2001

Grade 8 White 1994

2001

Black 1994

2001

Hispanic 1994

2001


2001


2001

Grade 12 White 1994

2001

Black 1994

2001

Hispanic 1994

2001


2001


2001

above

-7At or above

Basic

1 At or

1 ProficientBelow Basic At Basic At Proficient At Advanced

26 (1.1) 52 (1.0) 19 (1.2) 3 (0.5) 74 (1.1) 22 (1.4)

21 (1.3)* 55 (1.4) 21 (1.4) 3 (0.7) 79 (1.3) * 24 (1.4)

64 (1.8) 32 (2.0) 4 (1.0) # (***) 36 (1.8) 4 (1.0)56 (2.1) * 38 (1.9) 5 (0.9) # (0.3) 44 (2.1) * 6 (1.0)

59 (3.6) 35 (3.3) 6 (1.2) 1 (***) 41 (3.6) 6 (1.2)58 (3.0) 35 (2.6) 6 (1.1) 1 (0.3) 42 (3.0) 7 (1.1)

38 (3.4) 42 (4.6) 17 (4.2) 4 (1.9) 62 (3.4) 20 (3.8)29 (3.8) 53 (4.6) 16 (2.7) 3 (1.9) 71 (3.8) 19 (3.2)

49 (6.9) 41 (7.8) 9 (2.4) # (***) 51 (6.9) 9 (2.7)47 (6.4) 41 (6.0) 8 (3.0) ' 4 (***) 53 (6.4) 12 (4.6)

29 (1.1) 54 (0.9) 16 (0.8) 1 (0.2) 71 (1.1) 17 (0.8)

25 (1.0) 53 (1.1) 19 (1.1) 2 (0.4) * 75 (1.0) 21 (1.1) *

67 (2.3) 29 (2.2) 4 (0.6) # (0.1) 33 (2.3) 4 (0.6)62 (2.4) 34 (2.1) 4 (0.8) # (***) 38 (2.4) 4 (0.8)

59 (2.3) 36 (2.2) 5 (0.8) # (***) 41 (2.3) 5 (0.7)60 (1.7) 34 (1.5) 5 (0.7) # (0.2) 40 (1.7) 5 (0.7)

35 (5.5) 46 (3.7) 17 (2.7) 2 (0.9) 65 (5.5) 19 (3.0)32 (3.8) 48 (3.0) 18 (3.2) 2 (0.8) 68 (3.8) 20 (3.6)

58 (5.8) ! 37 (5.9) ! 5 (2.6) ! 0 (***) ! 42 (5.8) ! 5 (2.6) !

50 (7.1) 42 (6.7) 7 (3.2) 1 (***) 50 (7.1) 8 (3.5)

50 (1.2) 37 (0.9) 12 (0.8) 1 (0.2) 50 (1.2) 13 (0.8)51 (1.4) 36 (1.1) 12 (0.8) 1 (0.4) 49 (1.4) 13 (1.0)

83 (1.6) 15 (1.3) 2 (0.8) # (***) 17 (1.6) 2 (0.8)80 (1.5) 18 (1.4) 3 (0.6) # (***) 20 (1.5) 3 (0.6)

78 (2.1) 18 (2.1) 4 (0.8) # (***) 22 (2.1) 4 (0.7)74 (2.4) 21 (1.8) 5 (1.0) # (***) 26 (2.4) 5 (1.1)

57 (4.8) 29 (3.0) 12 (3.5) 2 (0.9) 43 (4.8) 13 (3.7)

47 (5.1) 31 (2.4) 17 (4.1) 5 (2.3) 53 (5.1) 21 (6.0)

70 (7.6) ! 25 (7.3) ! 5 (2.3) ! 0 (***) ! 30 (7.6) ! 5 (2.3) !

66 (7.2) ! 33 (7.4) ! 1 (***) ! 0 (***) ! 34 (7.2) ! 1 (***) !


* Significantly different from 1994.# Percentage is between 0.0 and 0.5.

! The nature of the sample does not allow accurate determination of the variability of the statistic.

(***) Standard error estimates cannot be accurately determined.

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of

Educational Progress (NAEP), 1994 and 2001 History Assessments.

151APPENDIX B U.S. HISTORY REPORT CARD 139

Percentage of students and average U.S. history scale scores by region of the country,grades 4, 8, and 12: 1994 and 2001

Northeast Southeast Central West-

Grade 4 1994 22 (0.7) 23 (1.0) 25 (0.8) 30 (0.6)204 (2.4) 201 (1.9) 212 (2.6) 202 (2.1)

2001 21 (0.8) 24 (1.3) 24 (0.4) 31 (1.4)215 (2.5) * 208 (2.6) 217 (2.0) 200 (2.3)

Grade 8 1994 20 (0.8)

266 (1.7)

2001 20 (0.9)

269 (1.9)

25 (0.9)

251 (1.3)

23 (1.1)

261 (2.0) *

24 (0.6)

266 (1.3)

25 (0.5)

267 (1.7)

31 (0.8)

256 (1.1)

32 (1.3)

255 (1.3)

Grade 12 1994 20 (0.5) 23 (0.8) 27 (0.7)289 (1.9) 282 (1.4) 288 (1.4)

2001 21 (0.9)

289 (3.4)

22 (1.2)

284 (1.7)26 (0.6)

289 (1.4)

30 (0.7)

286 (1.6)

31 (1.5)

286 (1.6)


Standard errors of the estimated percentages and scale scores appear in parentheses.* Significantly different from 1994.NOTE: Percentage may not add to 100 due to rounding.

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress INAEP), 1994 and 2001 U.S. History Assessments.

152


VELlb Data Figure National Achievement-Level Results Region gJag gauntry

Percentage of students within and at or above U.S. history achievement levels by region of thecountry, grades 4, 8, and 12: 1994 and 2001

At or above

Proficient

At or above

BasicBelow Basic At Basic' At Proficient At Advanced

Grade 4 Northeast 1994 37 (2,4) 45 (2.4) 16 (1.9) 3 (0.7) 63 (2.4) 18 (2.1)

2001 27 (3A) . 50 (2.9) 20 (2.4) 3(1.1) 73 (3.1) 23 (2.9)

Southeast 1994 "39 (2.3) 46 (2.4) 13 (1.4) 2 (0.5) 61 (2.3) 15 (1.5)

2001 34' (23) 50 (1.8) 14 (1.5) 2 (1.2) 66 (2.7) 16 (2.2)

Central 1994 29 (2.8) 51 (2.6) 17 (2.0) 3 (1.0) 71 (2.8) 20 (2.6)

2001 25 (2.3) 52 (2.6) 20 (2.1) 3 (1.1) 75 (2.3) 24 (2.4)

West 1994 39 (2.5) 45 (2.4) 14 (1.9) 1 (0.8) 61 (2.5) 16 (1.8)

2001 41 (2.5) 46 (1.9) 12 (1.1) 1 (0.4) 59 (2.5) 13 (1.2)

Grade 8 Northeast 1994 31 (2.0) 50 (1.2) 18 (1.6) 1 (0.3) 69 (2.0) 19 (1.7)

2001 28 (2,2) 50 (1.7) 20 (2.0) 2 (0.8) 72 (2.2) 22 (2.1)

Southeast 1994 49 (1.9) 42 (1,4) 8 (0.8) # (0.2) 51 (1.9) 9 (0.8)

2001 38 (2.3)* 46 (1.5) 14 (1.1) 2 (0.4) 62 (2.3) * 16 (1.3) *

Central 1994 31 (2;3) 52 (2.0) 16 (1.3) 1 (0.4) 69 (2.3) 17 (1.2)

2001 29 (22) 53 (1.5) 17 (1.5) 2 (0.5) 71 (2.2) 19 (1.5)

West 1994 42 (1.3) - 47 (IA) 10 (1.1) 1 (0.2) 58 (1.3) 11 (1.2)

2001 45 (1.7) 43 (1.6) 11 (1.1) 1 (0.2) 55 (1.7) 12 (1.3)

Grade 12 Northeast 1994 54(2.4) 33 (1.7)' 12 (1.3) 1 (0.4) 46 (2.4) 13 (1.5)

2001 :5513.8) 31 (L9) 11 (1.7) 2 (***) 45 (3.8) 13 (3.2)

Southeast 1994 63 (L9) 29 (1.6) 8 (1.3) # (0.3) 37 (1.9) 8 (1.4)

2001 61(2.3) 29 (1.4) 9 (1.2) 1 (0.3) 39 (2.3) 10 (1.3)

Central 1994 55 (2.0) 34 (1.3r 10 (1.1) 1 (0.5) 45 (2.0) 11 (1.2)

2001 54 (2.2) 35 (2.0) 10 (1.2) 1 (0.4) 46 (2.2) 11 (1.3)

West 1994 5712.3) 33 (2.0) 9 (1.2) 1 (0.2) 43 (2.3) 10 (1.2)

2001 ,58 (2.2) 31 (1.5) 10 (1.2) 1 (0.4) 42 (2.2) 11 (1.5)


* Significantly different from 1994.# Percentage is between 0.0 and 0.5.

(***) Standard error estimates cannot be accurately determined.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.


Educational Progress (NAEP), 1994 and 2001 History Assessments.

153


Percentage of students and average U.S. history scale scores by parents' highest level of education,grades 8 and 12: 1994 and 2001

Some education

Less than Graduated after Graduated

high school high school high school college Unknown

Grade 8 1994 7 (0.4) 23 (0.8) 19 (0.5) 42 (1.0) 9 (0.4)

241 (1.3) 251 (0.8) 264 (0.8) 270 (0.8) 238 (1.4)

2001 7 (0.5) 18 (0.7) 19 (0.5) 47 (1.2) 9 (0.4)

243 (2.3) 253 (1.1) 265 (1.0) 275 (0.8) * 244 (1.5) *

Grade 12 1994 7 (0.4) 20 (0.7) 25 (0.7) 45 (1.0) 3 (0.2)

263 (1.4) 276 (1.1) 287 (1.2) ; 296 (0.9) 256 (2,7)

2001 7 (0.4) 19 (0.6) 25 (0.7) 46 (1.2) 3 (0.2)

269 (1.5) * 274 (1.0) 286 (0.8) 298 (1.3) 262 (2.4)



*Significantly different from 1994.NOTE: Percentage may not add to 100 due to rounding.

SOURCE:11.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of


154


Percentage of students within and at or above U.S. history achievement levelsby parents' highest level of education, grades 8 and 12: 1994 and 2001

Grade 8

Less than high school 1994

2001

Graduated high school 1994

2001

Some education after high school1994

2001

Graduated college 1994

2001

Unknown 1994

2001

Grade 12

Less than high school 1994

2001

Graduated high school 1994

2001

Some education after high school1994

2001

Graduated college 1994

2001

Unknown 1994

2001

At or above

Basic

At or above

ProficientBelow Basic At Basic-At Proficient At Advanced]

63 (2.2) 34 (2.0) 3 (0.7) 0 (***) 37 (2.2) 3 (0.7)

59 (3.3) 38 (3.2) 3 (1.7) # (***) 41 (3.3) 3 (1.8)

50 (1.4) 44 (1.5) 6 (0.9) # (0.1) 50 (1.4) 7 (0.9)

48 (1.7) 44 (1.6) 7 (1.0) # (***) 52 (1.7) 7 (1.0)

32 (1.3) 54 (1.5) 13 (1.2) # (0.2) 68 (1.3) 14 (1.1)

30 (1.3) 56 (1.6) 14 (1.3) 1 (0.3) 70 (1.3) 14 (1.3)

26 (1.2) 53 (1.2) ! 2011.1) ! 1 (0.3) 1 74 (1.2) 22 (1.1)

22 (1.0) 52 (0.9) ! 24 (1.1) 3 (0.5) * ' 78 (1.0) 27 (1.1)*

64 (2.2) 33 (1.8) 3 (1.1) it (***) 36 (2.2) 3 (1.1)

59 (2.6) 37 (3.0) 4 (1.1) # (***) 41 (2.6) 4 (1.2)

85 (1.9) 14 (1.8) 1 (0.7) # (*") 15 (1.9) 1 (0.6)

80 (2.1) 18 (1.9) 2 W.7) # (***) 20 (2.1) 2 (0.7)

71 (1.5) 24 (1.3) ' 4 (0.8) 4 ("1 29 (1.5) 4 (0.8)

74 (1.3) 22 (1.3) 1 4 (0.8) 4 (0.1) 26 (1.3) 4 (0.8)

58 (1.9) 34 (2.0) 7 (1.1) 1 (0.3) 42 (1.9) 8 (1.1)

61 (1.3) 31 (1.2) 8 (0.7) 1 (0.2) , 39 (1.3) 8 (0.7)

44 (1.3) 39 (1.1) 15 (1.0) 1 (0.4) 56 (1.3) 17 (1.0)

42 (1.5) 40 (1.3) ' 16 (1.0) 2 (0.8) 58 (1.5) 18 (1.5)

88 (3.5) 11 (3.2) 1 (***) it (***) 12 (3.5) 1 (***)

83 (3.3) 14 (2.9) 3 (1.5) # (***) 17 (3.3) 3 (1.5)

Standard errors of the estimated percentages appear in parentheses.* Significantly different from 1994.


(*"*) Standard error estimates cannot be accurately determined.

NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due torounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 U.S. History Assessments.

155


Percentage of students and average U.S. history scale scores by type of school, grades 4, 8, and 12:1994 and 2001

Public Nonpublic Nonpublic: Catholic

Grade 4 1994 90 (0.8) 10 (0.8)

203 (1.2) 222 (1.9)

2001 88 (1.1)

207 (1.2) *

12 (1.1)

226 (1.9)

6 (0.7)

221 (2.5)

6 (0.7)

229 (2.5) *

Grade 8 1994 90 (0.9)

257 (0.7)

2001 90 (0.9)

260 (0.8) *

10 (0.9)

278 (1.1)

10 (0.9)

279 (2.4)

6 (0.6)

279 (1.5)

5 (0.6)

280 (1.8)

Grade 12 1994 89 (1.1) 11 (1.1)

284 (0.8) 299 (1.3)

2001 93 (0.8)

286 (1.1)

7 (0.8)

298 (2.0)

6 (0.9)

298 (2.2)

4 (0.6)

302 (2.4)

Nonpublic: Other

4 (0.5)

224 (3.1)

5 (0.9)

223 (2.8)

4 (0.6)

277 (2.1)

5 (0.7)

278 (4.7)

5 (0.6)

299 (2.2)

3 (0.5)

293 (3.1)



* Significantly different from 1994.NOTE: Percentages may not add to 100 due to rounding.



156


TIEM LIM $528te Figure National Achievement-Level Results ilvg School

Percentage of students within and at or above U.S. history achievement levels by type of school,grades 4, 8, and 12: 1994 and 2001

1

At or above ,

Basic

At or above ,

ProficientBelow Biiic At Basic At Proficient At Advanced

Grade 4 Public 1994 38 (1.2) 46 (1.0) 14 (1.0) 2 (0.3) 62 (1.2) 16 (1.1)

2001 35 (1.3) 48 (1.3) 15 (1.0) 2 (0.5) 65 (1.3) 17 (1.1)

Nonpublic 1994 18 (2;.1) 55 0.,7) l 23 (1.8) 3 (0.7) 82 (2.1) 26 (1.9)

2001 15 (2.1) 55 (2.8) ' 26 (2.4) 4 (1.2) 85 (2.1) 30 (2.7)

Nonpublic: Catholic 1994 19 (2.6) 56 (2.0) 22 (2.2) 2 (0.7) 81 (2.6) 24 (2.3)2001 14 (2,1) 51 (2.7) 29 (2.2) 5 (1.7) 86 (2.1) 35 (2.9) *

Nonpublic: Other 1994 17 (4.0) 54 (3.6) 25 (3.5) 5 (1.4) 83 (4.0) 29 (3.9)2001 16 (4 0) 59 (6.2) 22 (4.4) 3 (1.3) 1 84 (4.0) 25 (4.9)

Grade 8 Public 1994 41 (1.0) 47 (0.8) 11 (0.6) 1 (0.1) 59 (1.0) 12 (0.6)

2001 38 (1.0) 47 (111) 14 (0.8) 1 (0.3) 62 (1.0) 15 (0.8) *

Nonpublic 1994 16 (1,2) 57 (1.8) 25 (1.7) 2 (0.6) 84 (1.2) 28 (1.8)2001 16 (3.2) 53 (2.5) 28 (2.1) 3 (0.9) 84 (3.2) 31 (2.6)

Nonpublic: Catholic 1994 15 0.7) 57 (2.1) 26 (2.1) 2 (0.8) 85 (1.7) 29 (2.3)2001 15 (2.0) 55 (1:,.7) 28 (2.1) 3(0.8) 85 (2.0) 31 (2.4)

Nonpublic: Other 1994 17 (2.8) 57 (31) 24 (3.0) 2 (0.8) 83 (2.8) 26 (3.4)2001 18 (6.2) 51 (4.7) , 27 (3.9) 4 (1.6) 82 (6.2) 31 (4.9)

Grade 12 Public 1994 59 (1.2) 31 (0.9) 9 (0.7) 1 (0.2) 41 (1.2) 10 (0.7)2001 58 (1.3) 31 (0.9) 9 (0.7) 1 (0.4) 42 (1.3) 11 (1.0)

Nonpublic 1994 41 (2.2) 41 (1.7) 17 (1.3) 1 (0.4) 59 (2.2) 18 (1.3)2001 41 (2.8) 42 (2.0) 15 (2.0) 2 (0.4) 59 (2.8) 17 (2.1)

Nonpublic: Catholic 1994 43 (3,8) 40 (2.7) 17 (1.9) 1 (0.4) 57 (3.8) 18 (2.0)2001 36 (3,3) 44 (2.2) 18 (2.4) 2(0.7) 64 (3.3) 20 (2.7)

Nonpublic: Other 1994 39 (2,9) 42 (2.1) 17 (2.5) 2 (0.8) 61 (2.9) 19 (2.7)

2001 47 (4.7) 39 (3.7) 12 (2.8) 1 (0.6) 53 (4.7) 13 (2.9)



NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due torounding.


157


Percentage of students and average U.S. history scale scores by type of location, grades 4, 8,and 12: 2001

Central city Urban fringe/large town Rural/small town

Grade 4 27 (1.6) 45 (2.0) 28 (2.8)

199 (2.4) 211 (2.1) 215 (1.7)

Grade 8 27 (2.0) 45 (2.7) 28 (2.5)

257 (1.8) 265 (1.4) 263 (1.4)

Grade 12 25 (1.9) 40 (2.8) 35 (2.6)

283 (1.7) 292 (2.0) 284 (1.0)

The percentage of students is listed first with he corresponding average scale score presented below.





Percentage of students within and at or above U.S. history achievement levelsby type of location, grades 4, 8, and 12: 2001

Grade 4

Grade 8

Grade 12

Central city


Rural/small town

Central city


Rural/small town

Central city


Rural/small town

At or above

Basic

At or above

ProficientBelow Basic At Basic At Proficient At Advanced

45 (2.5) 41 (2.0) 13 (1.8) 2 (0.5) 55 (2.5) 15 (2.0)

31 (2.1) 49 (1.8) 17 (1.6) 3 (0.9) 69 (2.1) 20 (1.8)

23 (2.3) 58 (1.9) 17 (1.8) 2 (0.8) 77 (2.3) 19 (2.2)

43 (2.1) 42 (1.4) 13 (1.2) 2 (0.4) 57 (2.1) 15 (1.3)

33 (1.8) 49 (1.2) 17 (1.0) 1 (0.4) 67 (1.8) 18 (1.1)

34 (1.9) 51 (1.8) 13 (1.9) 2 (0.6) 66 (1.9) 15 (1.8)

61 (2.2) 29 (1.7) 9 (0.8) 1 (0.2) 39 (2.2) 10 (0.9)

52 (2.3) 33 (1.4) 13 (1.4) 2 (0.9) 48 (2.3) 15 (2.0)

60 (1.8) ,._32 (1.6) 7 (0.7) ft (0.2) 40 (1.8) 7 (0.7)


ft Percentage is between 0.0 and 0.5.




148 APPENDIX B U.S. HISTORY REPORT CARD158

Percentage of students and average U.S. history scale scores by student eligibility forFree/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001

Eligible Hot eligible Info not available

Grade 4 33 (1.4) 48 (2.1) 19 (2.4)

189 (1.6) 220 (1.4) 217 (2.8)

Grade 8 25 (1.1) 54 (2.1) 21 (2.2)

245 (1.2) 269 (0.9) 268 (2.0)

Grade 12 16 (0.9) 64 (2.2) 21 (2.5)

271 (1.3) 289 (1.2) 295 (2.0)

The percentage of students is listed first with the corresponding average scale sco e presented below.





Percentage of students within and at or above U.S. history achievement levels by student eligibilityfor the Free/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001

At or above

BasicBelow Basic At Basic At Proficient At Advanced

Grade 4 Eligible 53 (47), 40 (1.5) 6 (0.7) 1 (0.2) 47 (1.7)

Not eligible 21 (1.7) 55 (2.0) 21 (1.4) 3 (0.8) 79 (1.7)

Info not available 25 (2.8) 51 (2.5) 21 (2,4) 3 (1.1) 75 (2.8)

Grade 8 Eligible 59 (L4) 35 (1.4) 5 (0.7) # (0.2) 41 (1.4)

Not eligible 27 (1.2) 53 (1.3) 18 (1.2) 2 (0.3) 73 (1.2)

Info not available 30 (2.4) 48 (1.5) 19 (1.7) 3 (0.6) 70 (2.4)

Grade 12 Eligible 77 (1.8) 19 (1.7) 3 (0.7) # 23 (1.8)

Not eligible 55 (1.5) 34 (1.1) 10 (0.7) 1 (0.6) 45 (1.5)

Info not available 47 (2.9) 36 (2.3) 16 (1.9) 2(0.6) 53 (2.9)

At or above

LProficient

6 (0.8)

25 (1.6)

24 (2.9)

6 (0.7)

20 (1.2)

22 (2.1)

3 (0.7)

11 (1.1)

17 (2.3)

Standard errors of the estimated percentages appear in parentheses.# Percentage is between 0.0 and 0.5.

(*"*) Standard error estimates cannot be accurately determined.




National average U.S. history scale scores by type of results, grades 4, 8, and 12: 2001

Accommodations not permitted Accommodations permitted

Grade 4 209 (1.0) 208 (0.9)

Grade 8 262 (0.8) 260 (0.8) t

Grade 12 287 (1.0) 287 (0.9)




Percentage of students within and at or above U.S. history achievement levels by type of results,grades 4, 8, and 12: 2001

Grade 4

At or above

Proficient

At or above

BasicBelow Bask` At Basic At Proficient At Advanced

Accommodations weie not permitted 33 (1.1) 49 (1.1) 16 (0.9) 2 (0.5) 67 (1.1) 18 (1.0)

Accommodations were permitted 34 (1.2) 48 (1.1) 16 (0.9) 2 (0.3) 66 (1.2) 18 (0.9)

Grade 8

Accommodations were not permitted 36 (0.9) 48 (0.9) 15 (0.8) 2 (0.3) 64 (0.9) 17 (0.8)

Accommodations were permitted 38 (1.0)1 46 (0.9) t 14 (0.7) 1 (0.2) 62 (1.0) t 16 (0.7)

Grade.12




t Significantly different from the sample where accommodations were not permitted.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages af or above achievement levels, due to rounding.


1 0148 APPENDIX B U.S. HISTORY REPORT CARD

National average U.S. history scale scores by gender and type of results, grades 4, 8, and 12: 2001

Female

Grade 4

Male

Accommodations were not permitted 209 (1.1)Accommodations were permitted 207 (11)

Grade 8

Accommodations were not permitted 264 (0.9)Accommodations were permitted 261 (0.9)1

Grade 12

Accommodations were not permitted 288 (1.3)Accommodations were permitted 288 (1.1)

209 (1.2)

209 (1.2)

261 (0.9)

260 (0.9) t

286 (0.9)

286 (0.9)




VE110® Oas Data it/ %1109 613 Gomparison ce Rai National Achievement-Level Results Gender

Percentage of students within and at or above U.S. history achievement levels by gender andtype of results, grades 4, 8, and 12: 2001

Grade 4

Male



Female



Grade 8

Male



Female



Grade 12

Male



Female



At or above

Basic

At or above

ProficientBelow Basic At Basic,

' At Proficient At Advanced

3435

(1.3)(1.3)

47

46

(1,3)

(1.3)17

1

16

(1.1)

(1.3)2

2

(0.7)(0.5)

66

65

(1.3)

(1.3)

19

19

(1.2)

(1.3)

32 (1.4) 51 (1.6) 15 (1.1) 2 (0.4) 68 (1.4) 17 (1.1)33 (1.4) 50 (1.5) 15 (1.3) 2 (0.4) 67 (1.4) 17 (1.2)

35 (1.1) 47 (1.3) 1 17 (1.0) 2 (0.3) 65 (1.1) 18 (1.0)38 (1.1) t 45 (1,0) 16 (0.9) 2 (0.3) 62 (1.1) t 17 (0.9)

37 (1.2) 48 (1,0) 14 (0,8) 1 (0.4) 63 (1.2) 15 (0.8)39 (1,2) 47 (1.2) 13 (19) 1 (0.2) 61 (1.2) 14 (0.9)

55 (1.6) 33 (1.1) 11 (0.8) 1 (0.5) 45 (1.6) 12 (1.1)55 (1.6) 32 (1.3) 11 (0.9) 2 (0.4) 45 (1.6) 12 (1.1)

59 (1.3) 31 (1.2) 9 (0.7) 1 (0.3) 41 (1.3) 10 (0.9)60 (1.2) 31 (1.1) 9 (0.8) 1 (0.3) 40 (1.2) 10 (0.9)



NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of



National average U.S. history scale scores by race/ethnicity and type of results, grades 4, 8, and 12:2001

Grade 4

White Black Hispanic

Asian/Pacific

Islander

American

Indian

Accommodations were not permitted 220 (1.1) 188 (1.8) 186 (2.5) 213 (2.7) 197 (6.9)

Accommodations were permitted 218 (1.2) 186 (2.0) 187 (2.0) 214 (3.3) 197 (5.1)

Grade 8

Accommodations were not permitted 271 (0.8) 243 (1.8) 243 (1.5) 267 (3.4) 249 (4.5)

Accommodations were permitted 269 (0.9) t 240 (1.8) t 240 (1.8) 265 (2.6) 248 (4.4)

Grade 12

Accommodations were not permitted 292 (1.0) 269 (1.5) 274 (1.7) 295 (4.6) 277 (5.5) !

Accommodations were permitted 292 (0.9) 268 (1.4) 271 (1.8) 294 (5.5) 274 (5.5) !

Standard errors of the estimated scale scores appear in parentheses.t Significantly different from the sample where accommodations were not permitted.

! The nature of the sample does not allow accurate determination of the variability of the statistic.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of


162


VELlh [122* Data UT& 4115 Domparison WIT® UcIU National Achievement-001 Results kee/Ethnieity

Percentage of students within and at or above U.S. history achievement levels by race/ethnicity andtype of results, grades 4, 8, and 12: 2001

Grade 4

White

At or above

Basic

I At or above 1

[ Proficient 1Below Basic At Basic At Proficient At Advanced



Black

Accommodations were not permitted 56 (2.1) 38 (1.9) 5 (0.9) # (0.3) 44 (2.1) 6 (1.0)

Accommodations were permitted 58 (2.3) 36 (2,0) 5 (1.0) # (0.3) 42 (2.3) 5 (0.9)

Hispanic . .

Accommodations were not permitted 58 (3.0) ' 35 (2.6) 6 (1.1) 1 (0.3) 42 (3.0) 7 (1.1)

Accommodations were permitted 58 (2.0) 36 (1.8) 6 (0.7) # (0.3) 42 (2.0) 6 (0.8)

Asian/Pacific IslanderAccommodations were not permitted 29 (3.8) 53 (4.6) 16 (2.7) j 3 (1.9) 71 (3.8) 19 (3.2)

Accommodations were permitted 26 (4.0) 54 (4.2) 17 (2.6) 3 (.91 74 (4.0) 20 (3.2)

American IndianAccommodations were not permitted 47 (6,4) 41 (6.0) 8 (3.0) 4 (***) 53 (6.4) 12 (4.6)

Accommodations were permitted 44 (5.4) 44 (4.7) 9 (3.0)3 (***) 56 (5.4) 12 (4.2)

Grade 8

White



BlackAccommodations were not permitted 62 (2.4)

,

34 (2.1) 4 (0.8) # (**) 38 (2.4) 4 (0.8)

Accommodations were permitted 65 (2.1) , 31 (L7) 4 (0.6) # (***) 35 (2.1) 4 (0.7)

Hispanic

Accommodations were not permitted 60(1,7) 34 (1.5) 5 (0.7) # (0.2) 40 (1.7) 5 (0.7)

Accommodations were permitted 63 (2.2) - , -32 (L9) 4 (0.7) # (0.2) 37 (2.2) 4 (0.6)

Asian/Pacific IslanderAccommodations were not permitted 32 (3.8) '48 (3.0) 18 (3.2) 2 (0.8) 68 (3.8) 20 (3.6)

Accommodations were permitted 34 (3,1) 47 (3.1) 17 (3.4) , 2 (1.0) 66 (3.1) 19 (3.2)

American IndianAccommodations were not permitted 50 (7.1) 42 (6.7) 7 (3.2), 1 (***) 50 (7.1) 8 (3.5)

Accommodations were permitted 54 (6,0) 38 (5.4) 7 (2,2) 1 (***) 46 (6.0) 8 (2.4)

Grade 12

,

..

White . .. , .

Accommodations were not permitted 51 (1.4) . 36 (1.1)- 12 (0.8)' 1 (0.4) 49 (1.4) 13 (1.0)

Accommodations were permitted 51 (1,4) 36 (1.0) 12 (0.9)' 1 (0.3) 49 (1.4) 13 (1.0)

BlackAccommodations were not permitted 80 (1.5) 18 (1A) 3 (0,5) # (*") 20 (1.5) 3 (0.6)

Accommodations were permitted 80 (1.6) '17' (1.4) 3 (0.5) # (.**) 20 (1.6) 3 (0.5)

Hispanic

Accommodations were not permitted 74 (2.4) 21 (1.8) 5 (1.0)# (...) 26 (2.4) 5 (1.1)

Accommodations were permitted 74 (2.1) 21 (1.5) 5 (0.9) # (0.2) 26 (2.1) 5 (1.0)

Asian/Pacific IslanderAccommodations were not permitted 47 (5.1) 31(2.4) 17 (4:1) 5 (2.3) 53 (5.1) 21 (6.0)

Accommodations were permitted 48 (5.8) 31 (2,4) 16 (4.8) 5 (2.7) 52 (5.8) 21 (6.6)

American IndianAccommodations were not permitted 66 (7.2) ! 33 (7.4) ! 1 (***) 0 (***) 34 (7.2) ! 1 (***)

Accommodations were permitted 68 (7.8) ! 31 (8.0) 1 1 (***) 0 (***) 32 (7.8) ! 1 (***)


# Percentage is between 0.0 and 0.5.(***) Standard error estimates cannot be accurately determined.! The nature of the sample does not allow accurate determination of the variability of the statistic.NOTE: Percentages within each U.S. history achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.

SOURCE: U.S. Department of Education, Otfice of Educational Research and Improvement, National Center for Education Statistics, National Assessment of



1i02 [3A CeN f i1U TA Grade Sample Question Results (iMultiple-Dhoice

Overall percentage correct and percentages correct within each achievement-level range: 2001

Overall percentage Below Basic Basic Proficient Advancedcorrect 194 and below* 195-242* 243-275* 276 and above*

93 (0.7) 84 (1.8) 96 (0.8) 99 (***) (***)


*NAEP U.S. history composite scale range.

) Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education

Statistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History Assessment.

1MIk [322/8 121535 ite 1I11 @,2 Grade Sample Question Results 0Multiple-Dhoice



45 (1.3):


25 (2.0)

Basic195-242*

46 (2.2)

Proficient243-275*

74 (3.3)

AdvancedI 276 and above*

***(***)



***(***) Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education


able Data VELit TA Grade Sample Question Results (iMultiple-Ghoice



57 (1.5)

eroentage eo redaohievement-level

dalterva

Below Basic Basic Proficient Advanced

194 and below 195-242* Ii. 243-215* 276 and above*

86 (2.2) ***(***)



"*"(***) Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education


164APPENDIX 8 U.S. HISTORY REPORT CARD

1102 ONe Data flw Tel OA Grade Sample Question Results Otended Gonstructed-Response

Overall percentage "Essential" or better and percentages "Essential" or better within each achievement-

level range: 2001


42 (CZ

Below Basic Basic194 and below* 195-242*

13 (1.8) 48 (2.3)

Proficient Advanced243-275* 276 and above*

76 (3.2) *** (***)


*NAEP U.S. history composite scale range.***("**) Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education


irmo2 8.30: Ei i171Iff1k taig 0 Sample Question Results (iMultiple-Ohoice



52 (1.9)

Below Baiic251 and below

34 (2.3)

Basic252-293*

56 (2.4)

Proficient294-326*

79 (2.7)


*** (***)


*NAEP U.S. history composite scale range.**"(***) Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center thr Education


1MG DAS Data ITECOg Grade Sample Question Results 0Multiple-Bhoice

Overall percentage correct and percentages correct within each achievernent-level range: 2001

Overall percentagecorrpct,

39 (1.2)

Percentage corr c cziEreo

vOtrzgactl0=MAI 410-09elk


29 (1 3)' 39 (1.7)

Proficient i. Advanced

294-326* I 327 and above*

62 (4.0) **. (.**)


*NAEP U.S. history composite scale range.***(*"") Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education



Vag [la Data tc LW Grade Sample Question Results (Short Construeted-Response

Overall percentage "Appropriate" and percentages "Appropriate" within each achievement-levelrange: 2001

Overall percentage

"Appropriate"

30 (1.5)

Pereentag propriateaehi ement level

daftin ervals

Below Basic

251 and below*

9 (1.9)

Basic252-293*

34 (2.2)

Proficient294-326*

64 (4.3)

Advanced

327 and above*

***(***)



) Sample size insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education


VELN DiSh Data 407Ung Grade Sample Question Results (Short Construeted-Response

Overall percentage "Appropriate" and percentages "Appropriate" within each achievement-levelrange: 2001

Overall percentage , Below Basic Basic Proficient Advanced

"Appropriate 251 and below 252-293* 294-326* 327 and above*

33 (1.1) 11 (1.5) 38 (2.0) 64 (4.9) *** (***)


"NAEP U.S. history composite scale range.

***(***) Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education



APPENDIX B

Pe reenta coffeeaehi vement4GM

dabtervals


36 (1.3)


23 (1.6)

Basic294-324*

47 (2.3)

Proficient Advanced

325-354* 355 and above*

***(***)73 (4.8)



***(*"") Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education



1q36


Overall Percentagicorrect

68 (1.3)


Proficient325-354*

54 (1.7) L 82 (2.1) H 95 (1 8)

Advanced

355 and above*

(...)



*"*(***) Sample size insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education


eonstructed-Response

Overall percentage "Essential" or better and percentages "Essential" or better within each achievement-level range: 2001


21(1.5)

Percenta e Essenialachievement- evel

bette zVBWrimk 1


4 (0.8)

Basic294-324*

30 (2.6)

Proficient325-354*

74 (3.8)

i.


*** (***)



"""("*") Sample size insufficient to permit a reliable estimate (see appendix A).

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 U.S. History Assessment.

Nib CE1N INTL& LE Grade Sample Question fl Results *Mended Constructed-Response

Overall percentage "Essential" or better and percentages "Essential" or better within each achievement-level range: 2001


39 (1.7)

Percentage -ssential"

Eehramt-oadgGea

betterintervals

Below Basic Basic293 and balmy* 294-324*

17 (1.6) 62 (3.1)

Proficient325-354* ' 355 and above*

Advanced

88 (3.5)

Standard errors of the estimated percentages appear in parentheses.*NAEP U.S. history composite scale range.

*""(*"") Sample size insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education



Percentage of students and average U.S. history scale scores by teachers' reports on the amount oftime spent on social studies in a typical week at grade 4: 2001

Less than 30 minutes

2001

2

191

(0.6)

(7.8) !

30 to 60 minutes 12 (1.1)

195 (2.5)

61 to 120 minutes 37 (2.1)

210 (1.6)

121 to 180 minutes 31 (2.5)

211 (2.2)

More than 180 minutes 19 (2.1)

218 (2.3)

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.

! The nature of the sample does not allow accurate determination of the variability of the statistic.NOTE: Percentages may not add to 100 due to rounding.



168


1:11% B.39: Data flaTEL09132koce State/Looal Standards Planning Instruaion

Percentage of students and average U.S. history scale scores by teachers' reports on the use of state/local standards in planning instruction at grades 4 and 8: 2001

Grade 4

Not at all

2001

2

212

(0.4)

(5.3) !

Small extent 9 (1.1)

210 (2.5)

Moderate extent 23 (1.8)

206 (2.6)

Large extent 63 (2.3)

210 (1.4)

No standards for teaching 3 (1.2)social studies 224 (6.4) !

Grade 8

Not at all 2 (0.6)

274 (4.4) !


264 (3.6) !


266 (1.9)


262 (1.1)

No standards for teaching 1 (0.4)social studies 276 (6.4) !



! The nature of the sample does not allow accurate determination of the variability of the statistic.NOTE: Percentages may not add to 100 due to rounding.


169


Itet 0402 Date 4/x4lb (3A Frequency ciJ ourth- Mghth-Grade Classroom Activities

Percentage of students and average U.S. history scale scores by teachers' reports on frequency ofclassroom activities at grades 4 and 8: 1994 and 2001

Grade 4


Almost every day

1994

43 (2.6)

207 (2.2)

Once or twice a week 44 (2.6)

204 (1.3)

Once or twice a month 8 (1.4)

204 (4.6)

Never or hardly ever 5 (0.9)

204 (5.0)

Reading extra material not

in the regular textbook

Almost every day 6 (1.2)

208 (4.4) !

2001

41 (2.2)

214 (1.4)

47 (2.2)

207 (1.8)

7 (1.3)

202 (3.4)

5 (1.3)

209 (6.7) !

9 (1.2)

210 (3.4)

lfilllo VI l11,. la 14-.1...1 - ,--/205 (1.9)

,/211 (1.5)

Once or twice a month 46 (2.5) 35 (2.4)*204 (1.9) 208 (1.8)

Never or hardly ever 15 (13) 12 (1.5)

208 (3.2) 208 (2.8)


Almost every day 1 (0.4)I

1 (0.4)*** (***) *** (***)

Once or twice a week 8 (1.2) 11 (1.3)

201 (4.5) 207 (2.6)

Once or twice a month 29 (1.9) 39 (2.2)*208 (2.1) 212 (2.0)

Never or hardly ever 62 (2.0) 48 (1.9)*205 (1.3) 208 (1.6)

Writing a report

Almost every day *** (***) 1 (0.4)*** (***) 194 (8.0) !

Once or twice a week 6 (1.2)1

6 (1.0)

188 (4.2) 198 (3.4)

Once or twice a month 63 (2.3) 1 59 (2.1)

207 (1.4) 210 (1.4)

Never or hardly ever 31 (2.1) 1 34 (2.0)

205 (2.1) 210 (1.8)

1 7 0



veto@ 8.40: Dta Oa' VELOg requency 'ourth- Bighth-Drade Classroom Activities continued0fQ

Percentage of students and average U.S. history scale scores by teachers' reports on frequency ofclassroom activities at grades 4 and 8: 1994 and 2001

1994 2001

Grade 8


Almost every day 45

259

(3.6)

(1.3)

45 (2.6)

264 (1.3)


259 (1.5) 262 (1.1)

Once or twice a month 8 (1.4) 7 (1.4)

266 (2.1) 262 (3.1)

Never or hardly ever 5 (1.8) 3 (0.6)

265 (3.6) ! 275 (4.0) !

Reading extra material not


Almost every day 3 (0.6) 7 (1.2) *254 (4.5) 265 (3.1)


258 (1.5) 261 (1.4)


263 (1.2) 264 (1.2)


258 (2.5) 264 (2.5)


Almost every day 2 (0.7) 4 (0.6)

268 (9.4) ! 264 (4.0)


260 (2.0) 267 (1.6)


261 (1.1) 262 (1.0)

Never or hardly ever 23 (1.9) 16 (2.2) *258 (1.6) 259 (1.9)

Writing a report

Almost every day # (0.2) 1 (0.3)*** (***) 255 (4.0) !


256 (4.0) ! 266 (3.6) !


261 (0.9) 263 (0.9)


259 (1.6) 263 (1.7)



! The nature of the sample does not allow accurate determination of the variability of the statistic.

*Significantly different from 1994."**(***) Sample size is insufficient to permit a reliable estimate.If Percentage is between 0.0 and 0.5.

NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of



Vdt DAS Data gar VECle) M Frequency Twelfth-Grade Gla&sroom Activities

Percentage of students and average U.S. history scale scores by students' reports on frequency ofclassroom activities at grade 12: 1994 and 2001

Read material from a textbook

About every day

1994

40 (0.9)

2001

44 (1.1) *

289 (0.8) 290 (1.0)


289

(0.9)

(0.9)

38

289

(0.9)

(1.2)


284

(0.5)

(1.9)

8

283

(0.4)

(2.4)

A few times a year 6

278

(0.4)

(1.8)

6

276

(0.3)

(1.9)

Never 6 (0.5) 5 (0.3) *268 (1.8) 270 (2.6)

Read extra material not


About every day 9 (0.4) 10 (0.4)

288 (1.7) 290 (1.5)


289 (1.2) 291 (1.2)


291 (1.1) 290 (1.4)

A few times a year 18 (0.6) 17 (0.5)

288 (1.0) 289 (1.2)

Never 18 (0.7) 17 (0.5)

274 (1.3) 276 (1.1)

PPENDIX B U.S. HISTORY REPORT CARD

1 ? 2


Tut BA 8 Data Vainb M Vrequency Twelfth-Grade Dlassroom Activities continuedi)

Percentage of students and average U.S. history scale scores by students' reports on frequency ofclassroom activities at grade 12: 1994 and 2001

1994 2001

Use letters, diaries, or essays

written by historical people

About every day 4 (0.2)

280 (3.2)

5 (0.3) *

280 (2.1)

Once or twice a week 14 (0.5) 18 (0.6) *292 (1.5) 290 (1.4)

Once or twice a month 24 (0.5) 26 (0.6) *291 (1.2) 291 (1.1)

A few times a year 26 (0.5) 26 (0.5)291 (0.9) 292 (1.2)

Never 32 (0.7) 25 (0.6)*279 (0.9) i 279 (0.9)

Write a report

About every day 2 (0.2) 3 (0.2) *267 (3.6)

I271 (3.0)

Once or twice a week 9 (0.4) 14 (0.6) *279 (1.9) 288 (1.4)

Once or twice a month 35 (0.7) 41 (0.8) *287 (0.9) 290 (1.0)

A few times a year 40 (0.8) 34 (1.1)*293 (0.9) 290 (1.1)

Never 14 (0.8) 8 (0.4) *278 (1.4) 271 (1.7)



*Significantly different from 1994.NOTE: Percentages may not add to 100 due to rounding.


173


Percentage of students and average U.S. history scale scores by students' reports on computer use atgrade 4: 2001

Use computers at school

for social studies

Every day

2001

3

167

(0.2)

(3.5)

Two or three times a week 5 (0.4)186 (2.5)

Once a week 7 (0.5)

197 (3.9)

Once every few weeks 10 (0.5)

212 (2.4)


214 (0.9)

Do research projects using

a CD or the Internet

Yes 46 (1.1)

211 (1.4)

No 54 (1.1)

208 (1.1)

Use computer to write reports

Yes 52 (1.2)

209 (1.4)

No 48 (1.2)

210 (0.9)






r/4162 APPENDIX B U.S. HISTORY REPORT CARD

Percentage of students and average U.S. history scale scores by students' reports on computer useat grades 8 and 12: 2001

2001

Grade 8


for social studies

Every day


Once a week



1 (0.2)

239 (3.7)

5 (0.4)

252 (2.9)

9 (0.6)

261 (1.7)

21 (0.8)

268 (0.9)

64 (1.2)

263 (0.9)

Do research projects usinga CD or the Internet

Not at all

Small extent

Moderate extent

Large extent

26 (1.0)

253 (1.1)

31 (0.7)

262 (0.8)

26 (0.7)267 (1.0)

16 (0.7)

272 (1.1)


Not at all

Small extent

Moderate extent

Large extent

18 (0.9)

253 (1.3)

30 (0.7)

260 (0.9)

29 (0.7)

266 (0.9)

23 (0.8)270 (1.1)

1 5APPENDIX B

See footnotes at end of table. (>


Percentage of students and average U.S. history scale scores by students' reports on computer useat grades 8 and 12: 2001

2001

Grade 12


for studying history

Every day 2

265

(0.1)

(4.0)

Two or three times a week 6 (0.3)

277 (2.1)

Once a week 7 (0.5)

280 (1.6)

Once every few weeks 16 (0.7)

291 (1.5)


289 (1.1)

Haven't studied history this year 27 (1.3)

289 (1.1)

Do research projects using

a CD or the Internet

Not at all 23 (0.7)

274 (1.0)


286 (1.0)


294 (1.2)


300 (1.7)


Not at all 14 (0.5)

271 (1.1)


281 (1.1)


290 (1.0)


300 (1.4)






1 6


Appendix CMembers tf theN E U.S. Histiiy Standi

Carol Ruth BerkinGraduate CenterCUNYNew York, NY

Paul CohenEast Brunswick Public SchoolsEast Brunswick, NJ

Richard del RioMuirlands Middle SchoolLa Jolla, CA

Anthony EdmondsBall State UniversityHistory Dept.Muncie, IN

Sara ForteBowne-Munro Elementary SchoolEast Brunswick, NJ

Dennis GreggGeorge Washington High SchoolSan Francisco, CA

Linda LevstikUniversity of KentuckyLexington, KY

Tedd LevyNathan Hale Middle SchoolNorwalk, CT

g Committee

Louisa MoffittMarist SchoolAtlanta, GA

Lori MortonWoodley Hills Elementary SchoolAlexandria,VA

Diane PalmerBelmont, MA

John PatrickSocial Studies Development CenterIndiana UniversityBloomington, IN

Joann PrewittDelaware State Dept. of EducationD over, DE

Bruce VanSledrightCollege of EducationUniversity of MarylandCollege Park, MD

Allan WinklerDepartment of HistoryMiami UniversityOxford, OH

APPENDIX C U.S. HISTORY REPORT CARD 165

cknowledgmentsThis report is the culmination of the efforts of many individuals who contributed their consider-able knowledge, experience, and creativity to the NAEP 2001 U.S. history assessment.The assess-ment was a collaborative effort among staff from the National Center for Education Statistics(NCES), the National Assessment Governing Board (NAGB), Educational Testing Service (ETS),Westat, and NCS Pearson. Most importantly, NAEP is grateful to the students and school staffwhose participation made the assessment possible.

The NAEP 2001 U.S. history assessment was funded through NCES, in the Office of Educa-tional Research and Improvement of the U.S. Department of Education.The Deputy Commis-sioner of Education Statistics, Gary W. Phillips, and the NCES staffJanis Brown, Peggy Carr,Arnold Goldstein, Steven Gorman, Carol Johnson, and Andrew Kolstadworked closely andcollegially with the authors to produce this report.

The NAEP project at ETS is directed by Stephen Lazer and John Mazzeo, with assistance fromJohn Barone. Sampling and data collection activities were conducted by Westat under the directionof Rene Slobasky, Nancy Caldwell, Keith Rust, and Dianne Walsh. Printing, distribution, scoring,and processing activities were conducted by NCS Pearson under the direction of Brad Thayer,William Buckles, Mathilde Kennel, Linda Reynolds, and Connie Smith.

Test development activities were conducted by ETS under the direction of Madeline Goodmanand Michael Lapp.

The complex statistical and psychometric activities necessary to report results for the NAEP2001 U.S. history assessment were directed by Catherine Hombo and by Brenda Siok-Hoon Tay-Lim, and were advised by Nancy Allen.The analyses that produced results for the data for whichstudent accommodations were permitted were supervised by Brenda Tay-Lim with assistance fromHui Deng.

The extensive data processing and computer programming activities underlying the statisticaland psychometric analyses conducted at ETS are under the direction of David Freund, EdwardKulick, Bruce Kaplan, and Steven Isham. Data analyses presented in this report were managed byTatyana Petrovicheva with assistance from Satwinder Thind,VincentWeng, Norma Norris,AlfredRogers, and Mei-Jang Lin.The complex database work for this assessment was managed byKatherine Pashley with assistance from Gerry Kokolis.

The design and production of this report was overseen by Loretta Casalaina. Joseph Kolodey andRick Hasney contributed invaluable design and production expertise to the effort.Wendy Griggcoordinated the documentation and data checking procedures with assistance from Janice Goodis,Andrea Bergen, and Alice Kass. Shari Santapau coordinated the editorial and proofreading proce-dures with assistance from Valerie Mukuna and Julie McGuire.The Web version of this report wascoordinated by Pat O'Reilly with assistance from Rick Hasney.

Many thanks are due to the numerous reviewers, both internal and external to NCES and ETS.The comments and critical feedback of the following reviewers are reflected in the final version ofthis report: Janis Brown, Jay Campbell, James Carlson, Mary Crovo,Young Chun, DouglasCochrane, Mary Crovo,Arnold Goldstein, Steven Gorman, Ellen Harkavy, Dana Kelly,AndrewKolstad, Carolyn Lee,Ted Levy, John Patrick, Marianne Perie, Michael Petrilli, Patrick Rooney,John Sietsema, Rod Schwartz,AllenVanneman, and Emily Wurtz.

166 ACKNOWLEDGMENTS U.S. HISTORY REPORT CARD'/8

United StatesDepartment of EducationED Pubs

.8242-B Sandy CourtJessup, MD 20794-1398

Official Business OnlyPenalty for Private Use, $300

17 9

Postage and Fees PaidU.S. Department of Education

Permit No. G-17

Parcel

U.S. Department of EducationOffice of Educational Research and Imptovement (OERI)

Naffs:mai Library of Education (NLE)

Educational Resources information Center(ERIC)

NOTICE

Reproduction Basis

IC

This document is covered by a signed "Reproduction Release(Blanket)" form (on file within the ERIC system), encompassing all

or classes of documents from its source organization and, therefore,does not require a "Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, maybe reproduced by ERIC without a signedReproduction Release form(either "Specific Document" or "Blanket").

EFF-089 (3/2000)

Reproductions supplied by EDRS are the best that …DOCUMENT RESUME ED 464 893 SO 033 859 AUTHOR...

Documents

Transcript of Reproductions supplied by EDRS are the best that …DOCUMENT RESUME ED 464 893 SO 033 859 AUTHOR...