The Nation's Report Card: Geography 2001

Geography 2001The Nation’s Report Card

National Center for Educat ion Stat is t ics

U.S. Department of EducationOffice of Educational Research and Improvement NCES 2002-484

What is The Nation’s Report Card?THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationallyrepresentative and continuing assessment of what America’s students know and can do in various subject areas. Since1969, assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, andother fields. By making objective information on student performance available to policymakers at the national, state,and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Onlyinformation related to academic achievement is collected under this program. NAEP guarantees the privacy ofindividual students and their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Departmentof Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP projectthrough competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is alsoresponsible for providing continuing reviews, including validation studies and solicitation of public comment, onNAEP’s conduct and usefulness.

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelinesfor NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in theNational Education Goals; for setting appropriate student performance levels; for developing assessment objectives andtest specifications through a national consensus approach; for designing the assessment methodology; for developingguidelines for reporting and disseminating NAEP results; for developing standards and procedures for interstate,regional, and national comparisons; for determining the appropriateness of test items and ensuring they are free frombias; and for taking actions to improve the form and use of the National Assessment.

The National Assessment Governing BoardMark D. Musick, ChairPresidentSouthern Regional Education BoardAtlanta, Georgia

Michael T. Nettles, Vice ChairProfessor of EducationUniversity of MichiganAnn Arbor, Michigan

Melanie A. CampbellFourth-Grade TeacherTopeka, Kansas

Honorable Wilmer S. CodyFormer Commissioner of EducationState of KentuckyFrankfort, Kentucky

Daniel A. DomenechSuperintendent of SchoolsFairfax County Public SchoolsFairfax, Virginia

Edward DonleyFormer ChairmanAir Products & Chemicals, Inc.Allentown, Pennsylvania

Thomas H. FisherDirectorStudent Assessment ServicesFlorida Department of EducationTallahassee, Florida

Edward H. HaertelProfessor, School of EducationStanford UniversityStanford, California

Juanita HaugenLocal School Board MemberPleasanton, California

Honorable Dirk KempthorneGovernor of IdahoBoise, Idaho

Honorable Nancy KoppState LegislatorAnnapolis, Maryland

Honorable Ronnie MusgroveGovernor of MississippiJackson, Mississippi

Roy M. Nageak, Sr.First Vice-ChairAlaska Board of Education and

Early DevelopmentBarrow, Alaska

Debra PaulsonEighth-Grade Mathematics TeacherEl Paso, Texas

Honorable Jo Ann PottorffState LegislatorWichita, Kansas

Diane RavitchResearch ProfessorNew York UniversityNew York, New York

Sister Lourdes Sheehan, R.S.M.Secretary for EducationUnited States Catholic ConferenceWashington, DC

John H. StevensExecutive DirectorTexas Business and Education

CoalitionAustin, Texas

Migdania D. VegaPrincipalCoral Way Elementary Bilingual

SchoolMiami, Florida

Deborah VoltzAssistant ProfessorDepartment of Special EducationUniversity of LouisvilleLouisville, Kentucky

Honorable Michael E. WardState Superintendent of Public

InstructionNorth Carolina Public SchoolsRaleigh, North Carolina

Marilyn A. WhirryTwelfth-Grade English TeacherManhattan Beach, California

Dennie Palmer WolfDirector, Annenberg InstituteBrown UniversityProvidence, Rhode Island

Grover J. Whitehurst (Ex-Officio)Assistant Secretary of EducationOffice of Educational Research and

ImprovementU.S. Department of EducationWashington, DC

Roy TrubyExecutive Director, NAGBWashington, DC

Andrew R. Weiss

Anthony D. Lutkus

Barbara S. Hildebrant

Matthew S. Johnson

in collaboration with

Scott DavisWendy S. GriggMei-Jang LinFrank Jenkins

Yuxin Tang

June 2002

U.S. Department of EducationOffice of Educational Research and Improvement NCES 2002–484

Geography 2001

National Center fo r Educat ion Stat i s t ics

The Nation’s Report Card

U.S. Department of EducationRod PaigeSecretary

Office of Educational Research and ImprovementGrover J. WhitehurstAssistant Secretary

National Center for Education StatisticsGary W. PhillipsDeputy Commissioner

June 2002

SUGGESTED CITATIONU.S. Department of Education. Office of Educational Research and Improvement. National Center for Education Statistics.The Nation’s Report Card: Geography 2001, NCES 2002–484, by A. R.Weiss, A. D. Lutkus, B. S. Hildebrant, &M. S. Johnson. Washington, DC: 2002.

FOR MORE INFORMATIONContent contact:Arnold Goldstein202–502–7344

To obtain single copies of this report, limited number of copies available, or ordering information on other U.S.Department of Education products, call toll free 1–877–4ED-PUBS (877–433–7827), or write:

Education Publications Center (ED Pubs)U.S. Department of EducationP.O. Box 1398Jessup, MD 20794–1398

TTY/TDD 1–877–576–7734FAX 301–470–1244

Online ordering via the Internet: http://www.ed.gov/pubs/edpubs.htmlCopies also are available in alternate formats upon request.This report also is available on the World Wide Web: http://nces.ed.gov/nationsreportcard

The work upon which this publication is based was performed forthe National Center for Education Statistics by Educational Testing Service.

T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D iii

able of ContentsTThe

Nation’sReport

Card

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Chapter 1

NAEP 2001 Geography Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Overview of the 2001 National Assessment of Educational Progress . . . . . . . . . . . . . . . 2

The Geography Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Geography Assessment Instruments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Description of School and Student Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Reporting the Assessment Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

The Setting of Achievement Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Achievement-Level Descriptions for Each Grade . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

The Trial Status of Achievement Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Interpreting NAEP Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Overview of the Remaining Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Chapter 2

Average Scale Score and Achievement-Level Results for the Nation . . . . . . . . . . . . . . 17

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Average Scale Score Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Scale Scores by Percentile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Achievement-Level Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Chapter 3Average Scale Scores and Achievement-Level Results for Selected Subgroups . . . . . . 23

Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Region of the Country . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Parents’ Highest Level of Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Type of School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Type of Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Free/Reduced-Price School Lunch Program Eligibility . . . . . . . . . . . . . . . . . . . . . . . . 48

iv T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D

Chapter 4

Classroom Contexts for Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Teacher Background and Preparedness to Teach Geography . . . . . . . . . . . . . . . . . . . . 52

Geography Skills and Topics Taught in Grade 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Geography Skills Taught in Grades 8 and 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

The Extent of Students’ Social Studies and Geography Instruction . . . . . . . . . . . . . . . 61

The Use of Computers in the Social Studies Classroom in Grades 4 and 8 . . . . . . . . . 64

The Use of Computers in Grade 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Student Interest in Geography Grades 8 and 12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Chapter 5

Becoming a More Inclusive National Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Two Sets of 2001 NAEP Geography Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Results for the Nation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

National Results by Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

National Results by Race/Ethnicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Chapter 6

Sample Assessment Questions and Student Responses . . . . . . . . . . . . . . . . . . . . . . . 81

Grade 4 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . . . 82

Grade 8 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . . . 92

Grade 12 Sample Assessment Questions and Results . . . . . . . . . . . . . . . . . . . . . . . 100

Maps of Selected Item Descriptionson the NAEP Geography Scale—Grades 4, 8, and 12 . . . . . . . . . . . . . . . . . . . . . . . 108

Appendix AOverview of Procedures Used for the NAEP 2001 Geography Assessment . . . . . . . . . 113

Appendix B

Data Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

Appendix C

Members of the NAEP Geography Standing Committee . . . . . . . . . . . . . . . . . . . . . . 181

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D v

Chapter 1: Figures and Tables

Figure 1.1: Distribution of assessment time by geography content area,grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Figure 1.2: Descriptions of the three geography content areas . . . . . . . . . . . . . . . . . . 4

Table 1.1: Distribution of geography assessment time across cognitive areas,grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Figure 1.3: Policy definitions of the three NAEP achievement levels . . . . . . . . . . . . . . 9

Figure 1.4: Descriptions of NAEP geography achievement levels for grade 4 . . . . . . . 10

Figure 1.5: Descriptions of NAEP geography achievement levels for grade 8 . . . . . . . 11

Figure 1.6: Descriptions of NAEP geography achievement levels for grade 12 . . . . . . 12


Figure 2.1: Average geography scale scores,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Figure 2.2: Geography scale score percentiles,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Figure 2.3: Percentage of students within and at or abovegeography achievement levels, grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . 21


Figure 3.1: Average geography scale scores by gender,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Figure 3.2: Differences in average geography scale scores by gender,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 3.3: Percentage of students within and at or abovegeography achievement levels by gender, grades 4, 8, and 12: 1994 and 2001 . . . . 26

Figure 3.4: Average geography scale scores by race/ethnicity,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Figure 3.5: Differences in average geography scale scores by race/ethnicity,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Figure 3.6a: Percentage of students within and at or abovegeography achievement levels by race/ethnicity, grade 4: 1994 and 2001 . . . . . . . . 30

Figure 3.6b: Percentage of students within and at or abovegeography achievement levels by race/ethnicity, grade 8: 1994 and 2001 . . . . . . . . 31

Figure 3.6c: Percentage of students within and at or abovegeography achievement levels by race/ethnicity, grade 12: 1994 and 2001 . . . . . . . 32

Figure 3.7: Average geography scale scores by region of the country,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Figure 3.8a: Percentage of students within and at or abovegeography achievement levels by region of the country, grade 4: 1994 and 2001 . . . . . . 34

Figure 3.8b: Percentage of students within and at or abovegeography achievement levels by region of the country, grade 8: 1994 and 2001 . . . . . . 35

Figure 3.8c: Percentage of students within and at or abovegeography achievement levels by region of the country, grade 12: 1994 and 2001 . . . . . 36

Figure 3.9: Average geography scale scores byparents’ highest level of education, grades 8 and 12: 1994 and 2001 . . . . . . . . . . . 37

vi T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D

Figure 3.10a: Percentage of students within and at or above geographyachievement levels by parents’ highest level of education, grade 8: 1994 and 2001 . . . . 39

Figure 3.10b: Percentage of students within and at or above geographyachievement levels by parents’ highest level of education, grade 12: 1994 and 2001 . . . 40

Figure 3.11: Average geography scale scores by type of school,grades 4, 8, and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Figure 3.12a: Percentage of students within and at or abovegeography achievement levels by type of school, grade 4: 1994 and 2001 . . . . . . . . 43

Figure 3.12b: Percentage of students within and at or abovegeography achievement levels by type of school, grade 8: 1994 and 2001 . . . . . . . . 44

Figure 3.12c: Percentage of students within and at or abovegeography achievement levels by type of school, grade 12: 1994 and 2001 . . . . . . . 45

Table 3.1: Average geography scale scores by type of school location,grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Figure 3.13: Percentage of students within and at or above geographyachievement levels by type of school location, grades 4, 8, and 12: 2001 . . . . . . . . 47

Table 3.2: Average geography scale scores by student eligibility forFree/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001 . . . . . . . . . . 48

Figure 3.14: Percentage of students within and at or abovegeography achievement levels by Free/Reduced-Price School Lunchprogram eligibility, grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49


Table 4.1: Percentage of students and average geography scale scoresby teachers’ reported undergraduate/graduate major and minor/special emphasis,grades 4 and 8: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Table 4.2: Percentage of students and average geography scale scoresby teachers’ reports on how well prepared they felt they were to teach geography,grades 4 and 8: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Table 4.3: Percentage of students and average geography scale scoresby teachers’ reports on frequency of instruction of selected skills and topics,grade 4: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Table 4.4a: Percentage of students and average geography scale scoresby students’ reports on frequency of instruction of selected skills and topics,grade 8: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Table 4.4b: Percentage of students and average geography scale scoresby students’ reports on frequency of instruction of selected skills and topics,grade 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Table 4.5: Percentage of students and average geography scale scoresby students’ reports on grades in which geography was taken since the 6th grade,grade 8: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Table 4.6: Percentage of students and average geography scale scoresby students’ reports on grades in which geography was taken since 9th grade,grade 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Table 4.7a: Percentage of students and average geography scale scoresby teachers’ reports on computer use for social studies instruction,grade 4: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D vii

Table 4.7b: Percentage of students and average geography scale scoresby teachers’ reports on computer use for social studies instruction,grade 8: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Table 4.8: Percentage of students and average geography scale scoresby students’ reports on computer use for history and geography,grade 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Table 4.9: Percentage of students and average geography scale scoresby students’ reports on how much they like studying geography,grades 8 and 12: 1994 and 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69


Figure 5.1: The two sets of NAEP results based on a split-sample design . . . . . . . . . 74

Table 5.1: National average geography scale scores by type of results,grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Table 5.2: Percentage of students within and at or above geographyachievement levels by type of results, grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . 77

Table 5.3: National average geography scale scores by gender and type of results,grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Table 5.4: Percentage of students within and at or above geographyachievement levels by gender and type of results, grades 4, 8, and 12: 2001 . . . . . . 78

Table 5.5: National average geography scale scores by race/ethnicityand type of results, grades 4, 8, and 12: 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Table 5.6: Percentage of students within and at or above geography achievementlevels by race/ethnicity and type of results, grades 4, 8, and 12: 2001 . . . . . . . . . . . 80


Table 6.1: Grade 4 Sample Question 1 Results (Multiple-Choice) . . . . . . . . . . . . . . . 83


Table 6.3a: Grade 4 Sample Question 3 Results(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Table 6.3b: Grade 4 Sample Question 3 Results(“Partial” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Table 6.4a: Grade 4 Sample Question 4 Results(“Complete” Extended-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Table 6.4b: Grade 4 Sample Question 4 Results(“Essential” Extended-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Table 6.4c: Grade 4 Sample Question 4 Results(“Partial” Extended-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91





Table 6.9a: Grade 8 Sample Question 9 Results(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

viii T A B L E O F C O N T E N T S • G E O G R A P H Y R E P O R T C A R D


Table 6.10: Grade 12 Sample Question 10 Results (Multiple-Choice) . . . . . . . . . . . 100

Table 6.11: Grade 12 Sample Question 11 Results (Multiple-Choice) . . . . . . . . . . . 101

Table 6.12a: Grade 12 Sample Question 12 Results(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103


Table 6.13a: Grade 12 Sample Question 13 Results(“Complete” Short-Constructed-Response) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106


Figure 6.1: Grade 4 Item MapMap of selected item descriptions on theNational Assessment of Educational Progress geography scale for grade 4 . . . . . . . 109

Figure 6.2: Grade 8 Item MapMap of selected item descriptions on theNational Assessment of Educational Progress geography scale for grade 8 . . . . . . . 110

Figure 6.3: Grade 12 Item MapMap of selected item descriptions on theNational Assessment of Educational Progress geography scale for grade 12 . . . . . . 111

E X E C U T I V E S U M M A R Y • G E O G R A P H Y R E P O R T C A R D ix

xecutive SummaryEThe

Nation’sReport

Card

Major Findings atGrades 4, 8,

and 12

Results forStudent

Subgroups

ClassroomContexts for

Learning

Becoming aMore Inclusive

NAEP

The National Assessment of Educational Progress (NAEP) is

the nation’s only ongoing representative sample survey of

student achievement in core subject areas. In 2001, NAEP

conducted a geography assessment of the nation’s fourth-,

eighth-, and twelfth-grade students.

Authorized by Congress and administered by the National

Center for Education Statistics (NCES) in the U.S.

Department of Education, NAEP regularly reports to the

public on the educational progress of students in grades 4, 8,

and 12. This report presents the results of the NAEP 2001

geography assessment for the nation. Results in 2001 are

compared to results of the 1994 NAEP geography

assessment, which was the preceding NAEP geography

assessment and the only other geography assessment

conducted under the current framework. Students’

performance on the assessment is described in terms of

average scores on a 0–500 scale and in terms of the

percentage of students attaining three achievement levels:

Basic, Proficient, and Advanced. The achievement levels are

performance standards adopted by the National Assessment

Governing Board (NAGB) as part of its statutory

responsibilities. They represent collective judgments of what

students should know and be able to do.

x E X E C U T I V E S U M M A R Y • G E O G R A P H Y R E P O R T C A R D

As provided by law, the Deputy Com-missioner of Education Statistics, uponreview of a congressionally mandatedevaluation of NAEP, determined that theachievement levels are to be used on a trialbasis and should be interpreted withcaution. However, both the Deputy Com-missioner and the NAGB believe theseperformance standards are useful for under-standing trends in student achievement.They have been widely used by nationaland state officials as a common yardstick ofacademic performance.

In addition to providing average scoresand achievement-level performance ingeography for the nation’s fourth-, eighth-,and twelfth-graders, this report providesresults for subgroups of students at thosegrade levels defined by various backgroundcharacteristics (such as gender, race/ethnicity, region, parents’ education, etc.)and classroom contexts for learning. Asummary of major findings from the 2001NAEP geography assessment is presentedon the following pages. Differences be-tween results across years or betweengroups of students are discussed only if theyhave been determined to be statisticallysignificant. Readers are cautioned that therelationship between a contextual variableand student performance is not necessarilycausal.

Major Findings atGrades 4, 8, and 12� Average geography scores for fourth-

and eighth-graders were higher in 2001than in 1994, while the performance oftwelfth-graders was not significantlydifferent.

� At both grades 4 and 8, score increasesoccurred among the lower-performingstudents (at the 10th and 25th percentiles).

� The 2001 geography assessment showedthat 21 percent of fourth-graders, 30percent of eighth-graders, and 25 per-cent of twelfth-graders performed at orabove the Proficient level for theirrespective grades. These levels areidentified by NAGB as those at whichall students should perform.

� Both grades 4 and 8 showed an increasefrom 1994 to 2001 in the percentage ofstudents at or above Basic. There were nosignificant changes in the percentage ator above Proficient at any grade.

Results for Student SubgroupsIn addition to overall results, NAEP reportson the performance of various subgroupsof students. Observed differences betweenstudent subgroups in NAEP geographyperformance may reflect a range of socio-economic and educational factors notaddressed in this report or by NAEP.

Gender� There was no statistically significant

change at any grade in the average scoresof either male or female students be-tween 1994 and 2001.

� In 2001 as in 1994, male students atgrades 4, 8, and 12 had higher averagescores than female students.

Race/Ethnicity� At grade 4, Black students had higher

average scores in 2001 than in 1994.

� In 2001, White, Asian/Pacific Islander,and American Indian students hadhigher average scores than Black andHispanic students at all three grades.

� The 2001 results show a narrowing ofthe average score point difference be-tween White students and Black studentsat grade 4.

E X E C U T I V E S U M M A R Y • G E O G R A P H Y R E P O R T C A R D xi

Region of the Country� Between 1994 and 2001, the average

scores of fourth-graders increased in theNortheast, and the average scores ofeighth-graders increased in the Southeast.

� Fourth- and eighth-grade students in theNortheast and Central regions outper-formed students in the West in 2001, andstudents in the Central region alsooutperformed their counterparts in theSoutheast. Twelfth-graders in the Centralregion had higher average scores thantwelfth-graders in the Southeast.

Parents’ Highest Level of Education� Twelfth-graders whose parents had not

graduated from high school had higheraverage scores in 2001 than in 1994.

� The higher the parental education levelreported, the higher the average scoreattained by students at both grades 8 and12 in 2001.

Type of School� Eighth-grade public school students had

higher average scores in 2001 than in1994.

� In 2001, nonpublic school studentsoutperformed public school students atall three grades.

� In 2001, Catholic school students out-performed public school students atgrades 4, 8, and 12. Apparent differencesbetween public school and othernonpublic school students were notstatistically significant.

Type of Location� In 2001, students in rural and urban

fringe locations had higher averagescores than central city students at grades4, 8, and 12.

Eligibility for Free/Reduced-PriceLunch� At every grade in 2001, the average

score for students who were eligible forthe Free/Reduced-Price School Lunchprogram was lower than the average forstudents who were not eligible for theprogram (i.e., those above the povertyguidelines).

Classroom Contexts for LearningNAEP collects information about thecontexts for student learning by adminis-tering questionnaires to assessed students,their teachers, and their school administra-tors. Using the student as the unit ofanalysis, NAEP examines the relationshipbetween selected contextual variablesdrawn from these questionnaires andstudents’ average scores on the geographyassessment.

Teacher Preparation� Ninety-three percent of fourth-grade

students had teachers who indicatedtheir graduate/undergraduate major orminor was elementary education, andabout one-quarter (28 percent) ofeighth-grade students had teachers whoindicated they had a graduate/under-graduate major or minor in geographyor geography education.

� A higher percentage of fourth-gradestudents in 2001 had teachers whoreported they were very prepared toteach geography than did students in1994. Forty-four percent of eighth-gradestudents in 2001 had teachers whoreported they were very prepared toteach geography.

xii E X E C U T I V E S U M M A R Y • G E O G R A P H Y R E P O R T C A R D

Geography Skills Taught� The percentage of eighth-grade students

who studied maps and globes at leastonce or twice a week increased in 2001as compared to 1994.

� There was an increase in the percentageof eighth- and twelfth-grade studentswho studied natural resources once ortwice a week in 2001 as comparedwith 1994.

� The percentages of eighth-grade stu-dents who studied countries and culturesin their geography instruction at leastonce or twice a week were greater in2001 than in 1994.

Geography Course-Taking� A higher percentage of eighth-graders in

2001 reported taking geography in sixth,seventh, and eighth grades than did theircounterparts in 1994.

� The percentage of twelfth-gradestudents taking geography courses ateach grade level during their highschool years increased in 2001 fromthe percentage reported in 1994.

� In 2001 at grade 8, students who re-ported taking two or three years ofgeography had higher scores than thosewho took it for fewer years. Twelfth-graders who reported taking one year orless of geography had higher averagescores than those who took 3 or 4 yearsof geography.

Use of Computers� Students at grades 4, 8, and 12 who used

the Internet or CD-ROM materials to asmall or moderate extent had higherscores than students who did not usethese tools at all.

Becoming a More Inclusive NAEPIn the 2001 geography assessment, theNAEP program used a split-sample design,so that trends in students’ geographyachievement could be reported acrossassessment years and, at the same time, theprogram could continue to examine theeffects of including special-needs studentsassessed with accommodations. Included inthis report is an overview of the second setof results that include special-needs stu-dents who required and were providedaccommodations during the assessmentadministration.

� In the sample where accommodationswere not permitted, between 44 and 48percent of the special-needs students ateach of the three grade levels (between5 and 8 percent of all students) wereexcluded from NAEP testing by theirschools. In the sample where accommo-dations were offered, between 23 and24 percent of the special-needs studentswere excluded from the assessment(between 2 and 4 percent of the totalsample).

� At grade 8, the average score whenaccommodations were permitted waslower than the average score whenaccommodations were not permitted.At grades 4 and 12, there were nostatistically significant differences be-tween the average scores of studentswhen accommodations were permittedand when accommodations were notpermitted.

C H A P T E R 1 • G E O G R A P H Y R E P O R T C A R D 1

1Chapter

Contents

Overview

GeographyFramework

GeographyAssessment

School andStudent Samples

ReportingResults

NAEPAchievement

Levels

InterpretingNAEP Results

This Report

What is the NAEPgeographyassessment?

How does theNAEP geographyassessmentmeasure andreport studentprogress?

ChapterFocus

NAEP 2001 Geography AssessmentIntroductionAfter more than 50 years during which geography was

largely replaced by social studies in American public schools,

geography education began to experience a revival during

the 1980s and 1990s.1 Contributing to the change was a

growing belief in the relevance of geography to addressing

economic, political, and environmental issues at the

national and global level. Moreover, geography

education was increasingly seen as an essential tool in

the creation of effective citizens. This process gained

momentum through the work of various

organizations concerned with geography and

geography education. These groups encouraged a

more positive attitude toward geography and

provided important guidance for reestablishing

geography in the school curriculum.2 Two surveys of

geographic literacy, in 1988 and 1994, provided

statistical evidence that student knowledge and skills

fell far short of what was needed for responsible

citizenship.3 By the end of 1990, Congress had authorized

development of a broad-based National Assessment of

Educational Progress (NAEP) geography assessment at

1 Salter, C. L. (1990). Missing the magic carpet: The real significance of geographic ignorance.Princeton, NJ: Educational Testing Service.

2 Joint Committee on Geographic Education. (1984). Guidelines for geographic education:Elementary and secondary schools. Washington, DC: Association of American Geographersand the National Council for Geographic Education.

3 Allen, R., Bettis, N., Kurfman, D., MacDonald, W., Mullis, I. V. S., & Salter, C. (1990).The geography learning of high school seniors. Princeton, NJ: National Assessment ofEducational Progress, Educational Testing Service.

Persky, H. R., Reese, C. M., O’Sullivan, C. Y., Lazer, S., Moore, J. D., & Shakrani, S.(1996). NAEP 1994 geography report card. Washington, DC: National Center forEducation Statistics, Office of Educational Research and Improvement,U.S. Department of Education.

2 C H A P T E R 1 • G E O G R A P H Y R E P O R T C A R D

grades 4, 8, and 12, and the President andnation’s governors had declared geographyto be one of five core subjects in theirNational Education Goals.

Progress toward increasing the promi-nence of geography in the elementary andsecondary school curriculum has generallybeen good. The 1990s saw the publicationof the Geography Framework for the 1994National Assessment of Educational Progressand the NAEP geography assessment in1994, the introduction of the NationalGeography Standards, and the institution ofthe National Geographic Alliance Net-work.4 The alliance is a professional orga-nization encouraged and supported withgrants from the National GeographicSociety Education Foundation. Geo-graphic Alliances are present in all 50 states,and are comprised of primary, secondary,community college, and university geogra-phy educators interested in the enhance-ment of geography education. The numberof states with geography standards has beenincreasing steadily as well. According torecent data collected by the NationalGeographic Society, 48 states plus theDistrict of Columbia now have geographystandards in place, 37 of which are basedon the National Geography Standards.However, only 13 states require a geogra-phy course as a requirement for highschool graduation. Moreover, in 27 statesgeography is not tested in mandated stateexaminations, while in some other states

the portion of mandated tests devoted togeography is very small. As a result, therecould be little incentive for teachers toemphasize geography instruction whenhigher stakes are attached to other subjects.5

The results from the 2001 NAEP geographyassessment provide policymakers, educators,and the general public with a new, objectivetool with which to evaluate the country’sprogress toward geographic literacy.

Overview of the 2001National Assessment ofEducational ProgressFor over 30 years, the National Assessmentof Educational Progress (NAEP) has beenauthorized by Congress to collect, analyze,and report reliable and valid informationabout what American students know andcan do in core subject areas. NAEP assessesthe performance of public and nonpublicschool students in grades 4, 8, and 12. In2001, student performance in geographyand U.S. history was assessed at all threegrades. This report deals only with theresults of the geography assessment.

All NAEP assessments are based oncontent frameworks developed through anational consensus process. The NAEP2001 geography assessment was the secondadministration of an assessment based onthe Geography Framework for the 1994National Assessment of Educational Progress,which was originally developed for the1994 assessment.6 In both 1994 and 2001,

4 National Assessment Governing Board. (1994). Geography framework for the 1994 National Assessment of EducationalProgress. Washington, DC: Author.

Geography Education Standards Project. (1994). Geography for life: National geography standards. Washington, DC:National Geographic Research and Exploration.

5 Munroe, S. and Smith, T. (1998). State geography standards. Fordham Report, 2(2), http://www.edexcellence.net/standards/geography/geograph.htm.

Dean, A. (2002). Unpublished data. National Geographic Education Foundation.

Council of Chief State School Officers. (2000). Key state education policies on K-12 education: 2000. Washington,DC: Author.

6 National Assessment Governing Board. (1994). Geography framework for the 1994 National Assessment of EducationalProgress. Washington, DC: Author.


assessments based on the framework wereadministered to national samples of fourth-,eighth-, and twelfth-graders.

This report describes the results of the2001 geography assessment at grades 4, 8,and 12 and compares results in 2001 tothose in 1994. Comparisons across assess-ment years are possible because the assess-ments were developed under the samebasic framework and share a common setof geography questions. In addition, thepopulations of students were sampled andassessed using comparable procedures.

The Geography FrameworkAlthough NAEP had conducted a geogra-phy assessment at grade 12 in 1988, a morecomprehensive NAEP geography frame-work was developed for the 1994 assess-ment. The new framework provided theoperational specifications for both the 1994and 2001 assessments. The development ofthe framework was managed by the Coun-cil of Chief State School Officers(CCSSO) and adopted by the NationalAssessment Governing Board (NAGB).Approximately 50 professional geographers,educators, administrators, and other inter-ested individuals worked to achieve con-

sensus on the general goals as well as thespecific language of the framework. Inaddition, several hundred educationalexperts and interested members of thepublic contributed to the process, either byparticipating in public hearings or byreviewing drafts. The framework documentproduced by this consensus process calledfor the assessment of a broad range ofoutcomes. It represented an ambitiousvision both of what students should knowand be able to do in geography, and of theways in which those competencies shouldbe tested.

The geography framework is organizedalong two dimensions, a content dimensionand a cognitive dimension. The contentdimension forms the heart of the frame-work. It is divided into three main contentareas covering the breadth of geographylearning outcomes—knowledge andskills—that would flow from good geogra-phy instruction.

The geography framework specifies thepercentage of assessment time to bedevoted to each content area. Figure 1.1shows how the assessment time is distrib-uted for each of the three grades: 40

Distribution of assessment time by geography content area,grades 4, 8, and 12: 2001

Figure 1.1

Distribution ofAssessment Time

Space and PlaceEnvironment and Society

Spatial Dynamics and Connections

40% 30%

30%

SOURCE: National Assessment Governing Board, Geography Framework for the 2001 National Assessment of Educational Progress.


percent of assessment time goes to Spaceand Place, and 30 percent each to Environ-ment and Society and to Spatial Dynamicsand Connections. The percentages areimportant both because they guide the

development of test questions and becausethey determine how much weight eachcontent area receives in computing overalltest scores. Figure 1.2 provides descriptionsof each content area.

SOURCE: National Assessment Governing Board, Geography Framework for the 1994 and 2001 National Assessment of Educational Progress.

Space and Place:Knowledge of geography as it relates to particular places on Earth, to spatial patterns on Earth’s surface,and to physical and human processes that shape such spatial patterns.

Space is the basic resource and organizing element for geography. Patterns that are illustratedon maps reflect both natural features and human activities. This content area requires studentsto distinguish between and understand the spatial distribution of physical and human charac-teristics. Students must locate significant features and places on Earth, recognize existingpatterns in the distribution of features and places, and comprehend the reasons for thedevelopment and existence of these patterns.

Environment and Society:Knowledge of geography as it relates to the interactions between environment and society.

Geography is an integrative discipline that focuses on the interrelationships between the physicalenvironment and society. Human adaptation to and modification of the environment haveeconomic and political implications. Understanding the nature, scale, and ramifications of suchenvironmental transformations is fundamental in geography education, and is the core of thiscontent area. Students must be aware that every environmental issue lends itself to manyinterpretations, depending on the people’s perspectives. Students must consider such multipleperspectives as they evaluate decisions about issues, such as land use and resource develop-ment, because the results of such decisions often have complicated and unpredictable conse-quences. Learning to make wise decisions concerning the costs and benefits of environmentalmodification is an expressed goal of geography education.

Spatial Dynamics and Connections:Knowledge of geography as it relates to spatial connections among people, places, and regions.

This content area explores critical problems in human interaction. It requires students todemonstrate comprehension of cultural, economic, and political regions and the connectionsamong them. Students must understand how peoples and places are alike and how they differ.They should know that people of every country and every nation are increasingly connected to anddependent upon other peoples and places of the world for both human and natural resources. Inthis content area, students must demonstrate the knowledge that the world’s resources areunevenly distributed, and an understanding of how this contributes to the movements of people,patterns of trade, and conflict.

Descriptions of the three geography content areasFigure 1.2

Content AreaDescriptions


Three cognitive areas or levels comprisethe cognitive dimension of the geographyassessment. The framework labels them asKnowing, Understanding, and Applying,and defines them as follows.

Knowing—What is it? Where is it?

In this area, students are assessed on theirability to perform two related functionswith respect to information: a) an observa-tion function and b) a recall function.Students should be able to observe differ-ent elements of the landscape and answerquestions by recalling, for example, thename of a place or a resource indigenous toa particular country or by finding informa-tion about trading patterns among severalcountries.

Understanding—Why is it there? How didit get there? What is its significance?

In this area, students attribute meaningto what has been observed and explainevents. Putting events in context andexplaining them requires students to seeconnections among diverse pieces ofgeographic information and to use thatinformation to explain existing patternsand processes on Earth.

Applying—How can knowledge andunderstanding be used to solve geographicproblems?

Applying geography knowledge andunderstanding requires a range of higher-order thinking skills. Students classify,hypothesize, use inductive and deductivereasoning, and form problem-solvingmodels. They use many tools and skills ofgeography as they attempt to develop acomprehensive understanding en route toproposing viable solutions.

Student performance in the three cogni-tive areas was not reported on separatesubscales. Rather, the three areas were usedto help guide development of the assess-ment instrument. The percentages ofassessment time to be devoted to eachcognitive area, as specified in the frame-work, are displayed in table 1.1.

Together the content and cognitivedimensions of the assessment form a matrixin which each content area is measured ateach cognitive level.

Table 1.1 Geography Assessment Time Across Cognitive Areas

Distribution of geography assessment time across cognitive areas, grades 4, 8, and 12: 2001

Knowing Understanding Applying

Grade 4 45% 30% 25%

Grade 8 40% 30% 30%

Grade 12 30% 30% 40%



Geography AssessmentInstrumentsAs the only federally authorized ongoingassessment of geography achievement,NAEP must reflect the spirit of the frame-work as well as the specifications providedby it. In order to achieve those goals, theassessment development process involvedstages of review by measurement expertsand a committee of teachers, teachereducators, and curriculum specialists expertin geography. All components of theassessment were evaluated for curricularrelevance, developmental appropriateness,and fairness. The National AssessmentGoverning Board (NAGB) gave finalapproval for NAEP test questions. A list ofthe geography development committeemembers for the 2001 assessment is pro-vided in appendix C.

The 2001 geography assessment bookletsat grades 4, 8, and 12 contained either threeor four sections: a set of general backgroundquestions, a set of subject-related back-ground questions, and one or two sets, or“blocks,” of cognitive questions assessingknowledge and skills in geography. Thegeneral background questions are used tocollect some important basic informationabout students. These questions tend toremain fairly constant across differentNAEP assessments. The subject-relatedquestions are designed for specific assess-ments or for assessments given in an indi-vidual year. The questions in the geographyassessment asked students to give informa-tion about their school practices, such asthe frequency with which they used theInternet or a CD-ROM to study geogra-phy, how often they received instruction inusing maps and globes, and when they had

taken a geography course. All studentsparticipating in the geography assessment ata particular grade received the same back-ground questions.

The geography assessment as a wholecontained 91 questions at grade 4, 124questions at grade 8, and 123 questions atgrade 12. The grade 4 assessment wasdivided into six 25-minute blocks, whileboth the grade 8 and grade 12 assessmentscontained nine blocks, eight of which were25-minute blocks and one of which was a50-minute block. However, to reduce theburden on individuals, each student an-swered only a small portion of the totalnumber of questions—either two 25-minute blocks or one 50-minute block.The 50-minute blocks administered atgrades 8 and 12 focused on a particulargeographic topic. In addition, one block ateach grade was based entirely upon astudent atlas that was provided to students.The assessment time for each grade, there-fore, was 50 minutes plus the 10-15 min-utes needed to complete the backgroundquestions.

Each block of geography questionsconsisted of both multiple-choice and“constructed-response” questions. (“Con-structed response” is the term used todescribe test questions in which studentsproduce their own response, as distinctfrom multiple-choice questions, in whichstudents choose an answer from one ofseveral options.) Typically, a block willcontain about 16–18 questions, but there isconsiderable variation depending on thebalance between multiple-choice andconstructed-response questions. Overall,more than 50 percent of student assessmenttime was devoted to the latter question


type. In addition, of the time reserved forconstructed-response questions approxi-mately 20 percent was used for “produc-tion” questions in which students engagedin such tasks as indicating place locationson outline maps, drawing routes betweenpoints on a map, and drawing maps anddiagrams based upon written descriptions.Two types of constructed-response ques-tions were used:

■ short-constructed-response questionsthat required students to provide briefwritten answers of one or two sentencesor complete a limited production task;and

■ extended-constructed-response ques-tions that required students to provideanswers of a paragraph or more in lengthor engage in an extensive productiontask like producing a map.

Examples of multiple-choice, short- andextended-constructed-response and pro-duction questions are provided in chapter6. Additional information about the designof the 2001 geography assessment is pre-sented in appendix A.

Description of Schooland Student SamplesThe NAEP 2001 geography assessmentincluded representative samples of bothpublic and nonpublic schools. For thereporting sample, approximately 7,000fourth-graders, 9,000 eighth-graders, and9,000 twelfth-graders were assessed. Thenumber of schools in the reporting samplewere 365 at fourth grade, 369 at eighthgrade, and 374 at twelfth grade. Eachselected school that participated in theassessment and each student assessedrepresent a portion of the population ofinterest. For additional information on

sample sizes and participation rates, seeappendix A.

This report contains two different sets ofnational results based on two reportingsamples that differed in terms of whetheror not accommodations were made avail-able to special-needs students. The nationalresults presented in chapters 2, 3, 4, and 6of this report are based on a nationallyrepresentative sample that included special-needs students only if they could be as-sessed meaningfully without accommoda-tions. These results can be compared tothose from 1994 because accommodationswere also not made available in that assess-ment year. Chapter 5 presents a second setof national results from 2001 for a repre-sentative sample that includes the perfor-mance of students who required and wereprovided with accommodations (e.g.,bilingual dictionary, extended time, smallgroup testing). No comparison of theseresults to those from 1994 can be madebecause of the inclusion of these accom-modated special-needs students.

In the sample that did not permit ac-commodations, 8 percent of fourth-graders,8 percent of eighth-graders, and 5 percentof twelfth-graders were excluded from thegeography assessment in 2001. School stafffamiliar with these students made thedetermination, based upon NAEP’s inclu-sion criteria, that these students could notbe assessed meaningfully without accom-modations because of their disability and/or limited English proficiency. In 1994, 5percent at both the fourth- and eighth-grades, and 3 percent at the twelfth-gradewere excluded. Additional informationregarding exclusion rates is provided inappendix A.


Reporting theAssessment ResultsStudent performance on the NAEP geog-raphy assessment is presented in two ways:as average scores on the NAEP geographyscale, and in terms of the percentage ofstudents attaining NAEP geographyachievement levels. The average scalescores are a measure of what students knowand can do in geography. The achievement-level results indicate the degree to whichstudents’ performance meets expectations ofwhat they should know and be able to do.

Average scale score results are presentedon the NAEP geography composite scale,which ranges from 0–500. Students’ re-sponses on the NAEP 2001 geographyassessment were analyzed to determine thepercentages of students that respondedcorrectly to each multiple-choice questionand the percentages of students that re-sponded at each score level for the con-structed-response questions. Scales thatsummarize results for each of the threecontent areas described earlier were cre-ated. The composite scale is a weightedaverage of the separate subscales for thethree content areas. The weight for eachcontent area corresponds to its relativeimportance as prescribed in the NAEPgeography framework. A full description ofNAEP scale procedures can be found inthe forthcoming NAEP 2001 TechnicalReport.

Achievement-level results are presentedin terms of geography achievement levels

7 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.107-110 (H.R. 1).

National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297, 20, U.S.C. 1211.8 National Assessment of Educational Progress Improvement Act of 1988. Pub. L. No. 100-297, 20, U.S.C. 1211.9 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.

107-110 (H.R. 1).

as authorized by the NAEP legislation andadopted by the National AssessmentGoverning Board (NAGB).7 For eachgrade tested, NAGB has adopted threeachievement levels: Basic, Proficient, andAdvanced. For reporting purposes, theachievement-level cut scores are placed onthe geography scale, resulting in fourranges: below Basic, Basic, Proficient, andAdvanced.

The Setting ofAchievement LevelsThe 1988 NAEP legislation that createdthe National Assessment Governing Boarddirected the Board to identify “appropriateachievement goals…for each subject area”that NAEP measures.8 The 2001 NAEPreauthorization reaffirmed many of theBoard’s statutory responsibilities, includingdeveloping “appropriate student achieve-ment levels for each grade or age in eachsubject area to be tested . . . ”9 To followthis directive and achieve the mandate ofthe 1988 statute to “improve the form anduse of NAEP results,” NAGB undertookthe development of student performancestandards called “achievement levels.” Since1990 the Board has adopted achievementlevels in mathematics, reading, U.S. history,geography, science, writing, and civics.

The Board defined three levels for eachgrade: Basic, Proficient, and Advanced. TheBasic level denotes partial mastery of theknowledge and skills that are fundamentalfor proficient work at a given grade. TheProficient level represents solid academic


SOURCE: National Assessment Governing Board.

This level denotes partial mastery of prerequisite knowledge and skills that arefundamental for proficient work at each grade.

This level represents solid academic performance for each grade assessed. Studentsreaching this level have demonstrated competency over challenging subject matter,including subject-matter knowledge, application of such knowledge to real-worldsituations, and analytical skills appropriate to the subject matter.

This level signifies superior performance.

Figure 1.3

Achievement Levels

Basic

Proficient

Advanced

Policy definitions of the three NAEP achievement levels

performance. Students reaching this leveldemonstrate competency over challengingsubject matter. The Advanced level pre-sumes mastery of both the Basic and Profi-cient levels and superior performance.Figure 1.3 presents the policy definitions ofthe achievement levels that apply across allgrades and subject areas. The policy defini-tions guided the development of thegeography achievement levels, as well as

the achievement levels established in allother subject areas. Adopting three levelsof achievement for each grade signals theimportance of looking at more than onestandard of performance. The Boardbelieves, however, that all students shouldreach the Proficient level: the Basic level isnot the desired goal, but rather representspartial mastery that is a step toward Proficient.

The achievement levels in this reportwere adopted by the Board based on astandard-setting process designed andconducted under a contract with ACT, Inc.To develop these levels, ACT convened across section of educators and interestedcitizens from across the nation and askedthem to judge what students should knowand be able to do relative to a body ofcontent reflected in the NAEP frameworkfor geography. This achievement-level-setting process was reviewed by a variety ofindividuals including policymakers, repre-sentatives of professional organizations,teachers, parents, and other members of thegeneral public. Prior to adopting these

levels of student achievement, NAGBengaged a large number of persons tocomment on the recommended levels andto review the results.

The results of the achievement-level-setting process, after NAGB’s approval,became a set of achievement-level descrip-tions and a set of achievement-level cutpoints on the 0–500 NAEP geographyscale. The cut points are the scores thatdefine the boundaries between belowBasic, Basic, Proficient, and Advanced perfor-mance at grades 4, 8, and 12. The Boardestablished these geography achievementlevels based upon the geography contentframework.


Achievement-LevelDescriptions for Each GradeSpecific definitions of the Basic, Proficient,and Advanced geography achievement levelsfor grades 4, 8, and 12 are presented infigures 1.4 through 1.6. As noted previ-ously, the achievement levels are cumula-tive. Therefore, students performing at theProficient level also display the competenciesassociated with the Basic level, and students

at the Advanced level also demonstrate theskills and knowledge associated with boththe Basic and the Proficient levels. For eachachievement level listed in figures 1.4through 1.6, the scale score that corre-sponds to the beginning of that level isshown in parentheses. For example, infigure 1.4 the scale score of 240 corre-sponds to the beginning of the grade 4Proficient level of achievement.

Figure 1.4

AchievementLevels

Descriptions of NAEP geography achievement levels for grade 4


Basic Students should be able to use words or diagrams to define basic geography vocabulary;(187) identify personal behaviors and perspectives related to the environment, and describe some

environmental and cultural issues in their community; use visual and technology tools toaccess information; identify major geographic features on maps and globes; be able to readand draw simple maps, map keys, and legends; demonstrate how people depend upon, use,and adapt to the environment; and give examples of the movement of people, goods, services,and ideas from one place to another. In addition to demonstrating an understanding of howindividuals are alike and different, they should demonstrate a knowledge of the ways peopledepend on each other.

Proficient Students should be able to use fundamental geographic knowledge and vocabulary to identify(240) basic geographic patterns and processes; describe an environmental or cultural issue from

more than one perspective; and read and interpret information from visual and technologicaltools such as photograph maps and globes, aerial photography, and satellite images. Theyshould be able to use number and letter grids to plot specific locations; understand relativelocation terms; and sketch simple maps and describe and/or draw landscapes they haveobserved or studied. Proficient students should be able to illustrate how people depend upon,adapt to, and modify the environment, describe and/or illustrate geographic aspects of aregion using fundamental geographic vocabulary and give reasons for current humanmigration; discuss the impact a location has upon cultural similarities and differences; andbe able to demonstrate how an event in one location can have an impact upon anotherlocation.

Advanced Students should be able to use basic geographic knowledge and vocabulary to describe global(276) patterns and processes; describe ways individuals can protect and enhance environmental

quality; describe how modifications to the environment may have a variety of consequences;explain differing perspectives that apply to local environmental or cultural issues; anddemonstrate an understanding of forces that result in migration, changing demographics,and boundary changes. They should be able to solve simple problems by applying informationlearned through working with visual and technological tools such as aerial and otherphotographs, maps and globes, atlases, news media, and computers. They should be able toconstruct models and sketch and label maps of their own state, the United States, and theworld; use them to describe and compare differences, similarities, and patterns of change inlandscapes; and be able to predict the impact a change one location can have on another.They should be able to analyze the ways individuals and groups interact.


Figure 1.5

AchievementLevels



Basic Students should possess fundamental knowledge and vocabulary of concepts relating to(242) patterns, relationships, distance, directions, scale, boundary, site, and situation; solve

fundamental locational questions using latitude and longitude; interpret simple map scales;identify continents and their physical features, oceans, and various cities; respond accuratelyto descriptive questions using information obtained by use of visual and technological toolssuch as geographic models and/or translate that information into words; explain differencesbetween maps and globes; and find a wide range of information using an atlas or almanac.Students should be able to recognize and illustrate the relationships that exist betweenhumans and their environments, and provide evidence showing how physical habitat caninfluence human activity. They should be able to define a region and identify its distinguishingcharacteristics. Finally, they should be able to demonstrate how the interaction that takes placebetween and among regions is related to the movement of people, goods, services, and ideas.

Proficient Students should possess a fundamental geographic vocabulary; understand geography’s(282) analytical concepts; solve locational questions requiring integration of information from two

or more sources, such as atlases or globes; compare information presented at differentscales; and identify a wide variety of physical and cultural features and describe regionalpatterns. Students should be able to respond accurately to interpretive questions usinggeography’s visual and technological tools and translate that information into patterns;identify differences in map projections and select proper projections for various purposes; anddevelop a case study working with geography’s analytical concepts. In addition, studentsshould be able to describe the physical and cultural characteristics of places; explain howplaces change due to human activity; and explain and illustrate how the concept of regionscan be used as a strategy for organizing and understanding Earth’s surface. Students shouldbe able to analyze and interpret data bases and case studies, as well as use information frommaps to describe the role that regions play in influencing trade and migration patterns andcultural and political interaction.

Advanced Students should have a command of extensive geographic knowledge, analytical concepts,(315) and vocabulary; be able to analyze spatial phenomena using a variety of sources with

information presented at a variety of scales and show relationships between them; and usecase studies for special analysis and to develop maps and other graphics. Students should beable to identify patterns of climate, vegetation, and population across Earth’s surface andinterpret relationships between and among these patterns, and use one category of a map oraerial photograph to predict other features of a place such as vegetation based on climate orpopulation density based on topographic features. Students should also be able to relate theconcept of region to specific places and explain how regions change over time due to a varietyof factors. They should be able to profile a region of their own design using geographicconcepts, tools, and skills.


Figure 1.6

AchievementLevels



Basic Students should possess a knowledge of concepts and terms commonly used in physical and(270) human geography as well as skills enabling them to employ applicable units of measurement

and scale when solving simple locational problems using maps and globes. They should beable to read maps; provide examples of plains, plateaus, hills, and mountains; and locatecontinents, major bodies of water, and selected countries and cities. They should be able tointerpret geographic data and use visual and technological tools such as charts, tables,cartograms, and graphs; know the nature of and be able to identify several basic types ofmap projection; understand the basic physical structure of the planet; explain and applyconcepts such as continental drift and plate tectonics; and describe geography’s analyticalconcepts using case studies. Students should have a comprehensive understanding of spatialrelationships including the ability to recognize patterns that exist across Earth in terms ofphenomena, including climate regions, time zones, population distributions, availability ofresources, vegetation zones, and transportation and communication networks. They should beable to develop data bases about specific places and provide a simple analysis about theirimportance.

Proficient Students should have an extensive understanding and knowledge of the concepts and(305) terminology of physical and human geography. They should be able to use geographic

concepts to analyze spatial phenomena and to discuss economic, political, and social factorsthat define and interpret space. They should be able to do this through the interpretation ofmaps and other visual and technological tools, through the analysis of case studies, theutilization of data bases, and the selection of appropriate research materials. Studentsshould be able to design their own maps based on descriptive data; describe the physical andcultural attributes of major world regions; relate the spatial distribution of population toeconomic and environmental factors; and report both historical and contemporary eventswithin a geographic framework using tools such as special purpose maps, and primary andsecondary source materials.

Advanced Students should possess a comprehensive understanding of geographic knowledge and(339) concepts; apply this knowledge to case studies; formulate hypotheses and test geographic

models that demonstrate complex relationships between physical and human phenomena;apply a wide range of map skills; develop maps using fundamental cartographic principlesincluding translating narratives about places and events into graphic representations, anduse other visual and technological tools to perform locational analysis and interpret spatialrelationships. Students should also be able to undertake sophisticated analysis from aerialphotographs or satellite imagery and other visuals. Advanced students should be able todevelop criteria assessing issues relating to human spatial organization and environmentalstability and, through research skills and the application of critical thinking strategies,identify alternative solutions. They should be able to compile data bases from disparatepieces of information and from these data bases develop generalizations and speculationsabout outcomes when data change.


10 No Child Left Behind Act of 2001: Reauthorization of the Elementary and Secondary Education Act. Pub. L. No.107-110 (H.R. 1).

11 United States General Accounting Office. (1993). Education achievement standards: NAGB’s approach yields misleadinginterpretations. U.S. General Accounting Office Report to Congressional Requestors. Washington, DC: Author.

National Academy of Education. (1993). Setting performance standards for achievement: A report of the National Academyof Education Panel on the evaluations of the NAEP Trial State Assessment: An evaluation of the 1992 achievement levels.Stanford, CA: Author.

12 Cizek, G. (1993). Reactions to National Academy of Education report. Washington, DC: National Assessment GoverningBoard.

Kane, M. (1993). Comments on the NAE evaluation of the NAGB achievement levels. Washington, DC: NationalAssessment Governing Board.

13 American College Testing. (1995). NAEP reading revisited: An evaluation of the 1992 achievement level descriptions.Washington, DC: National Assessment Governing Board.

14 National Academy of Education. (1996). Reading achievement levels. In Quality and utility: The 1994 Trial StateAssessment in reading. The fourth report of the National Academy of Education Panel on the evaluation of the NAEP TrialState Assessment. Stanford, CA: Author.

15 National Academy of Education. (1997). Assessment in transition: Monitoring the nation’s educational progress (p. 99).Mountain View, CA: Author.

The Trial Status ofAchievement LevelsThe 2001 NAEP reauthorization lawrequires that the achievement levels beused on a trial basis until the Commis-sioner of Education Statistics determinesthat the achievement levels are “reasonable,valid, and informative to the public.”10

Until that determination is made, the lawrequires the Commissioner and the Boardto state clearly the trial status of theachievement levels in all NAEP reports.

In 1993, the first of several congression-ally mandated evaluations of the achieve-ment-level-setting process concluded thatthe procedures used to set the achievementlevels were flawed and that the percentageof students at or above any particularachievement-level cutpoint may be under-estimated.11 Others have critiqued theseevaluations, asserting that the weight of theempirical evidence does not support suchconclusions.12

In response to the evaluations andcritiques, NAGB conducted an additionalstudy of the 1992 reading achievementlevels before deciding to use those readingachievement levels for reporting 1994

NAEP results.13 When reviewing thefindings of this study, the National Acad-emy of Education (NAE) Panel expressedconcern about what it saw as a “confirma-tory bias” in the study and about theinability of this study to “address the panel’sperception that the levels had been set toohigh.”14 In 1997, the NAE Panel summa-rized its concerns with interpreting NAEPresults based on the achievement levels asfollows:

First, the potential instability of the levelsmay interfere with the accurate portrayal oftrends. Second, the perception that few Americanstudents are attaining the higher standards wehave set for them may deflect attention to thewrong aspects of education reform. The public hasindicated its interest in benchmarking againstinternational standards, yet it is noteworthy thatwhen American students performed very well ona 1991 international reading assessment, theseresults were discounted because they werecontradicted by poor performance against thepossibly flawed NAEP reading achievementlevels in the following year.15

The National Center for EducationStatistics and the National AssessmentGoverning Board have sought and con-


tinue to seek new and better ways to setperformance standards on NAEP.16 Forexample, NCES and NAGB jointly spon-sored a national conference on standardsetting in large-scale assessments, whichexplored many issues related to standardsetting.17 Although new directions werepresented and discussed, a proven alterna-tive to the current process has not yet beenidentified. The Deputy Commissioner ofEducation Statistics and the Board con-tinue to call on the research community toassist in finding ways to improve standardsetting for reporting NAEP results.

The most recent congressionally man-dated evaluation conducted by the Na-tional Academy of Sciences (NAS) reliedon prior studies of achievement levels,rather than carrying out new evaluations,on the grounds that the process has notchanged substantially since the initialproblems were identified. Instead, the NASPanel studied the development of the 1996science achievement levels. The NAS Panelbasically concurred with earlier congres-sionally mandated studies. The Panelconcluded that “NAEP’s current achieve-ment-level-setting procedures remainfundamentally flawed. The judgment tasksare difficult and confusing; raters’ judg-ments of different item types are internally

inconsistent; appropriate validity evidencefor the cut scores is lacking; and the processhas produced unreasonable results.”18

The NAS Panel accepted the continuinguse of achievement levels in reportingNAEP results on a developmental basis,until such time as better procedures can bedeveloped. Specifically, the NAS Panelconcluded that “....tracking changes in thepercentages of students performing at orabove those cut scores (or, in fact, anyselected cut scores) can be of use in de-scribing changes in student performanceover time.”19

The National Assessment GoverningBoard urges all who are concerned aboutstudent performance levels to recognizethat the use of these achievement levels is adeveloping process and is subject to variousinterpretations. The Board and the DeputyCommissioner believe that the achieve-ment levels are useful for reporting trendsin the educational achievement of studentsin the United States.20 In fact, achievement-level results have been used in reports bythe President of the United States, theSecretary of Education, state governors,legislators, and members of Congress.Government leaders in the nation and inmore than 40 states use these results intheir annual reports.

16 Reckase, Mark, D. (2000). The evolution of the NAEP achievement levels setting process: A summary of the research anddevelopment efforts conducted by ACT. Iowa City, IA: ACT, Inc.

17 National Assessment Governing Board and National Center for Education Statistics. (1995). Proceedings of the jointconference on standard setting for large-scale assessments of the National Assessment Governing Board (NAGB) and theNational Center for Education Statistics (NCES). Washington, DC: Government Printing Office.

18 Pellegrino, J.W., Jones, L.R., & Mitchell, K.J. (Eds.). (1998). Grading the nation’s report card: evaluating NAEP andtransforming the assessment of educational progress. Committee on the Evaluation of National Assessments of Educa-tional Progress, National Research Council. (p.182). Washington, DC: National Academy Press.

19 Ibid., page 176.20 Forsyth, Robert A. (2000). A description of the standard-setting procedures used by three standardized test

publishers. In Student performance standards on the National Assessment of Educational Progress: Affirmations andimprovements. Washington, DC: National Assessment Governing Board.

Nellhaus, Jeffrey M. (2000). States with NAEP-like performance standards. In Student performance standards on theNational Assessment of Educational Progress: Affirmations and improvements. Washington, DC: National AssessmentGoverning Board.


However, based on the congressionallymandated evaluations so far, the DeputyCommissioner agrees with the NationalAcademy’s recommendation that cautionneeds to be exercised in the use of thecurrent achievement levels. Therefore, theDeputy Commissioner concludes thatthese achievement levels should continueto be used on a trial basis and shouldcontinue to be interpreted with caution.

Interpreting NAEP ResultsThe average scores and percentages pre-sented in this report are estimates based onsamples of students rather than on entirepopulations. Moreover, the collection ofquestions used at each grade level is but asample of the many questions that couldhave been asked to assess student knowl-edge of the framework content. As such,the results are subject to a measure ofuncertainty, reflected in the standard errorof the estimates—a range of a few pointsplus or minus the score—which accountsfor potential score fluctuation due tosampling error and measurement error.The standard errors for the estimated scalescores and percentages in this report areprovided in appendix B.

The differences between scale scores andbetween percentages discussed in thefollowing chapters take into account thestandard errors associated with the esti-mates. Comparisons are based on statisticaltests that consider both the magnitude ofthe difference between the group averagescores or percentages and the standard

errors of those statistics. Estimates based onsmaller subgroups are likely to have rela-tively large standard errors. As a conse-quence, some seemingly large differencesmay not be statistically significant. Whenthis is the case, the term “apparent differ-ences” is used in this report. Throughoutthis report, differences between scores orbetween percentages are pointed out onlywhen they are significant from a statisticalperspective. All differences reported aresignificant at the 0.05 level with appropri-ate adjustments for multiple comparisons.The term “significant” identifies statisticallydependable population differences to helpinform dialogue among policymakers,educators, and the public.

Readers are cautioned against interpret-ing NAEP results in a causal sense. Infer-ences related to student subgroup perfor-mance or to the effectiveness of public andnonpublic schools, for example, should takeinto consideration the many socioeco-nomic and educational factors that mayalso affect performance in geography.

Overview of theRemaining ReportThe results in chapters 2, 4 and 6 of thisreport are based on the set of data with noaccommodations offered to students.Findings are presented for the nation andfor all the major reporting subgroupsincluded in all NAEP report cards. Com-parisons with results from the 1994 assess-ment are noted where the data permit.Chapter 4 examines contexts for learninggeography in terms of classroom practicesand student variables.


NAEP has sought to assess samples thatare as inclusive as possible. Nevertheless,there has always been some exclusion ofstudents with disabilities (SD) and limitedEnglish proficient (LEP) students whocould not be assessed meaningfully withoutaccommodations. Local school officialshave made decisions about exclusion inaccordance with explicit criteria providedby NAEP. In order to expand the propor-tion of students who can be assessedmeaningfully, NAEP began in recentassessments to explore the use of accom-modations with special-needs students.Chapter 5 presents an overview of a secondset of results—those that include studentswho were provided accommodationsduring the test administration. By includingthese results in the nation’s geographyreport card, NAEP continues a phasedtransition toward a more inclusive report-ing sample. Future assessment results willbe based solely on a student and schoolsample in which accommodations arepermitted.

Chapter 6 provides sample assessmentquestions and student responses from the2001 assessment. Also presented in chapter6 are item maps that position selectedquestion descriptions along the NAEPgeography scale where they are likely to beanswered successfully by students. Thedescriptions used on these item maps focuson the geography skills or knowledgeneeded to answer the question. The datapresented in both chapters 4 and 6 arebased on the set of results that did notinclude accommodated special-needsstudents.

This report also contains appendices thatsupport or augment the results presented.Appendix A contains an overview of theNAEP geography framework and specifica-tions, information on the national sample,and a more detailed description of themajor reporting subgroups featured inchapters 2 and 3. Appendix B contains thefull data with standard errors for all tablesand figures in this report. Appendix Ccontains a list of the NAEP geographycommittee members.


2 Average Scale Score andAchievement-Level Results for the NationOverviewThis chapter presents the NAEP 2001 geography assessment

results for the nation at grades 4, 8, and 12. Student

performance is described by average scale scores on the

NAEP geography composite scale, which ranges from 0 to

500, and in terms of percentages of students who attained

each of the three geography achievement levels: Basic,

Proficient, and Advanced. Results of the NAEP 2001

geography assessment are compared with results from

the NAEP geography assessment given in 1994. This

comparison is possible because the assessments share a

common set of geography exercises based on the

current geography framework and because the

populations of students were sampled and assessed

using comparable procedures. The results for this

chapter are based on testing conditions comparable to

those offered in 1994 when accommodations for

special-needs students were not offered. Special-needs

students who could participate without

accommodations were included. A second set of

results were obtained in 2001 that includes the

performance of students who required and were provided

accommodations. Results for the 2001 assessment that

include special-needs students tested with accommodations

are presented in chapter 5.

Are the nation’sfourth-, eighth-,and twelfth-graders makingprogress ingeography?

ChapterFocus

ChapterContents

Overview

Average ScaleScore Results

Scale Scores byPercentile

Achievement-Level Results


Average Scale Score ResultsThe results of the 2001 geography assess-ment show higher average scores than theresults in 1994 at grades 4 and 8, and nostatistically significant change at grade 12.

As seen in figure 2.1, the average score offourth-graders rose from 206 to 209, andthe average score of eighth-graders rosefrom 260 to 262.

Average geography scale scores, grades 4, 8, and 12: 1994 and 2001Figure 2.1

National Scale ScoreResults

Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

500

290

280

270

260

1994 2001

260

285

262

285

250

240

230

220

210

200

0

206 209

Grade 12

Grade 8

Grade 4


Scale Scores by PercentileAn examination of percentile scores pro-vides additional information about studentperformance across the score distribution.The percentile indicates the percentage ofstudents whose scores fell below a particu-lar point on the NAEP geography scale.The advantage of viewing percentile scoresis that they show how students with lower

or higher ability performed compared tothe national average. In addition, thepercentile data show whether trends in thenational average scores are reflected inscores at other levels of the performancedistribution. Figure 2.2 shows the geogra-phy scale scores for grades 4, 8, and 12 atthe 10th, 25th, 50th, 75th, and 90th per-centiles for the 1994 and 2001 assessments.


Figure 2.2

National PerformanceDistribution

Geography scale score percentiles, grades 4, 8, and 12: 1994 and 2001

270

260

250

240

230

220

210

0

244

265

247

267

500

330

320

310

300

290

280

1994 2001

287

306

321

287

305

319

250

240

230

220

210

200

190

0

213

237

217

241

500

310

300

290

280

270

260

1994 2001

263

285

302

265

286

303

200

190

180

170

160

150

140

0

146

179

158

185

500

260

250

240

230

220

210

1994 2001

211

237

257

212

236

254

Grade 4 Grade 8 Grade 12

Perc

entil

es

90th

75th

50th

25th

10th

Perc

entil

es

90th

75th

50th

25th

10th

Perc

entil

es

90th

75th

50th

25th

10th


At grades 4 and 8, scores at the twolowest percentiles (10th and 25th) werehigher in 2001 than in 1994, suggestingthat much of the improvement seen atgrades 4 and 8 was concentrated amongthe lower-performing students. Otherapparent changes at these two grades werenot statistically significant. At grade 12,consistent with national average scoreresults, none of the apparent differences inpercentile scores was statistically significant.

Achievement-Level ResultsThe results of student performance are notonly reported using scores on the NAEPgeography scale, but also using geographyachievement levels. The achievement levelsare performance standards adopted by theNational Assessment Governing Board(NAGB), based on the collective judgmentsof experts about what students should beexpected to know and be able to do.Viewing student performance from thisperspective provides some insight into theadequacy of students’ knowledge and skillsand the extent to which they achievedexpected levels of performance. A discus-

sion of the trial status of achievement levelsis in chapter 1.

Figure 2.3 presents achievement-levelresults for grades 4, 8, and 12. The resultsare shown in two ways: 1) the percentageof students within each achievement-levelinterval, and 2) the percentage of studentsat or above the Basic level and at or abovethe Proficient level. The text that followsdiscusses significant differences at or aboveBasic and Proficient, which are marked with* in the figures. Differences withinachievement levels are not discussed al-though they are shown in the figures. Inreading figure 2.3, it is necessary to keep inmind that the percentages at or abovespecific achievement levels are cumulative.Included among the percentage of studentsat or above the Basic level are also thosewho have achieved the Proficient andAdvanced levels of performance, and in-cluded among students at or above theProficient level are also those who haveattained the Advanced level of performance.


Figure 2.3

National Achievement-Level Results

Percentage of students within and at or above geography achievement levels,grades 4, 8, and 12: 1994 and 2001

Significantly different from 1994.NOTE: Percentages within each geography achievement level may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

3%

19% 22%

48%

70%

30%

1994

2%

19% 21%

53%

74%

26%

2001

Grade 4Advanced

At or aboveProficient

At or aboveBasic

Proficient

Basic

BelowBasic

4%

24%28%

43%

71%

29%

1994

4%

26%30%

44%

74%

26%

2001

Advanced

Proficient

Grade 8

BelowBasic

Basic


At or aboveBasic

2%

25%27%

43%

70%

30%

1994

1%

23%25%

47%

71%

29%

2001

Grade 12Advanced


At or aboveBasic

Proficient

Basic

BelowBasic


The NAGB has identified the Proficientachievement level as the minimum level atwhich all students should perform. In 2001,21 percent of fourth-graders, 30 percent ofeighth-graders, and 25 percent of twelfth-graders were at or above the Proficient level.Across years, the improvement in perfor-mance seen in the fourth- and eighth-grade average scale scores is reflected in

achievement-level performance. Bothgrades show an increase from 1994 to 2001in the percentage of students at or aboveBasic and a decrease in the percentage ofstudents below Basic. As in 1994, only asmall percentage of students at any gradereached the Advanced level in 2001: 2percent at fourth grade, 4 percent at eighthgrade, and 1 percent at twelfth grade.


3Chapter

Contents

Gender

Race/Ethnicity

Region of theCountry

Parents’Education

Type of School

Type of Location

Eligibility forFree/Reduced-

Price SchoolLunch Program

Are selectedsubgroups ofstudents makingprogress ingeography?

ChapterFocus

Average Scale Scores and Achievement-Level Results for Selected Subgroups

In addition to reporting on the performance of all students,

NAEP also provides results for various subgroups of students

at each grade. Examining subgroup results provides insight,

not only into how these groups of students performed in

comparison to one another, but also into how each group

has progressed over time. In light of recent educational

reform efforts that focus on improving the achievement of

all students, the information presented in this chapter

serves as a valuable indicator on the nation’s progress

in meeting its educational goals.

Results for the NAEP 2001 geography

assessment are presented by gender, race/ethnicity,

region of the country, parents’ highest level of

education, type of school, type of location, students’

eligibility for the Free/Reduced-Price School Lunch

program. For all subgroups except type of location

and free/reduced-price school lunch eligibility,

results of the 2001 assessment are compared with

those of the 1994 assessment.

Differences reported in this chapter between

demographic subgroups for the 2001 assessment and

between the 2001 and 1994 assessments are based on

statistical tests that consider both the magnitude of the

difference between the group average scores or percentages

and the standard errors of those statistics. Differences


Average geography scale scores by gender, grades 4, 8, and 12: 1994 and 2001Figure 3.1

National Scale ScoreResults by Gender

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

500

250

240

230

220

210

200

190

0 1994 2001

208203

212207


280

270

260

250

240

230

220

0 1994 2001

262258

264260

500

290

280

270

260

250

240

230

0 1994 2001

288281

287282

MaleFemale

between groups and between assessmentyears are discussed only if they have beendetermined to be statistically significant.Throughout this chapter, differencesbetween 1994 and 2001 are marked in thefigures. Differences within 2001 are notmarked, but where such differences arediscussed in the text, they are statisticallysignificant. Furthermore, the reader shouldbear in mind that differences in perfor-mance among subgroups of students mayreflect a range of socioeconomic andeducational factors not addressed in thisreport or by NAEP.

GenderFigure 3.1 presents average geographyscores for male and female students acrossassessment years. There was no statisticallysignificant change from 1994 to 2001 inthe average scores of either male or femalestudents at any of the three grades. Al-though the scale score differences acrossyears by gender were similar to the changeacross years for the population as a whole,the smaller sample sizes and the generallylarger standard errors in the two subgroupsprevented the results of the statistical testsfrom reaching the “significant” level in


NOTE: Score differences are calculated based on differences between unrounded average scale scores.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.2

National Scale ScoreDifferences by Gender

Differences in average geography scale scores by gender, grades 4, 8, and 12:1994 and 2001

100 20 30 40 50

2001

1994

Score Differences100 20 30 40 50

2001

1994

Score Differences

5 4 7

5 4 4

100 20 30 40 50

2001

1994

Score Differences

Male–FemaleGrade 4 Grade 8 Grade 12

these comparisons. In 2001, male studentsat all three grades had higher average scoresthan female students, just as they had in 1994.

The performance of subgroups on thegeography assessment can also be comparedby determining if a difference or “gap”exists between subgroups’ average scores

and, if it does, whether that gap increases ordecreases between assessment years. Figure3.2 shows that there was no statisticallysignificant change since 1994 in the differ-ences between the average scores of maleand female students at any of the threegrades.

The percentages of male and femalestudents at or above the geographyachievement levels and within eachachievement-level range are presented infigure 3.3. None of the apparent changesacross years in the percentages of male andfemale students at or above Basic and at orabove Proficient were statistically significant.

Looking at the differences in perfor-mance of male and female students in 2001shows higher percentages of fourth- andeighth-grade male students at or aboveProficient and at Advanced than their femalecounterparts. Among twelfth-graders, ahigher percentage of male students thanfemale students were at or above the Basicand Proficient levels.


Figure 3.3

National Achievement-Level Results by Gender

Percentage of students within and at or above geography achievement levels bygender, grades 4, 8, and 12: 1994 and 2001

Significantly different from 1994.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 4

Grade 12

Grade 8

4%

22%26%

46%

71%

29%

1994

3%

21% 24%

51%

75%

25%

2001

MaleAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic 25%

2%

17% 19%

49%

68%

32%

1994

1%17% 18%

54%

72%

28%

2001

FemaleAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

5%

25%30%

42%

72%

28%

1994

5%

29%33%

42%

75%

25%

2001

MaleAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

22%25%

44%

69%

31%

1994

3%

24%26%

47%

73%

27%

2001

FemaleAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

2%

29%32%

41%

73%

27%

1994

2%

26%28%

45%

73%

27%

2001

MaleAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

1%

21% 22%

45%

67%

33%

1994

1%

20% 21%

48%

70%

30%

2001

FemaleAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.4

National Scale ScoreResults by Race/Ethnicity

Average geography scale scores by race/ethnicity, grades 4, 8, and 12:1994 and 2001

Significantly different from 1994.NOTE: Sample size was insufficient to permit a reliable estimate for American Indian students at grade 12 in 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

500

220

210

200

190

180

170

160

0 1994 2001

218

183

214

168

193

184

212

181

199

222

500

280

270

260

250

240

230

220

0 1994 2001

270

239

264

229

248

273

240

266

234

261

500

300

290

280

270

260

250

240

0 1994 2001

291

268

285

258

291

270

286

260

288

White

Hispanic

Asian/PacificIslander

Black

American Indian


Race/EthnicityThe background questionnaire adminis-tered with the NAEP geography assess-ment asked students to indicate the racial/ethnic subgroup that best described them.The mutually exclusive subgroup catego-ries were White, Black, Hispanic, Asian/Pacific Islander, and American Indian(including Alaska Native). Figure 3.4 showsthe average scores for the five subgroups atgrades 4, 8, and 12. Only the results fromthe 2001 assessment are reported for

twelfth-grade American Indian studentsbecause the sample size in 1994 wasinsufficient to permit a reliable estimate.At grade 4, the average score of Blackstudents increased from 168 in 1994 to 181in 2001. There were no other statisticallysignificant changes in average scores amongthe five racial/ethnic groups. The signifi-cance of the apparent gains for AmericanIndian students at grades 4 and 8 could notbe determined because of insufficientsample sizes.

Scale score differences among subgroupswere evident at all three grades in 2001. Atfourth grade, White students had higherscores, on average, than students from theother four racial/ethnic groups. Asian/Pacific Islander students outperformedBlack, Hispanic, and American Indian

students, and American Indian studentsperformed better than Black and Hispanicstudents.

At grade 8, White students had higheraverage scores than Black, Hispanic, andAsian/Pacific Islander students. Asian/Pacific Islander and American Indian


Significantly different from 1994.NOTE: Score differences are calculated based on differences between unrounded average scale scores.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.5

National Scale ScoreDifferences by Race/Ethnicity

Differences in average geography scale scores by race/ethnicity, grades 4, 8,and 12: 1994 and 2001

100 20 30 40 50

2001

1994


2001

1994

Score Differences

50 40 33

100 20 30 40 50

2001

1994

Score Differences

White–BlackGrade 4 Grade 8 Grade 12

100 20 30 40 50

2001

1994


2001

1994

Score Differences

35 31 23

38 33 21

100 20 30 40 50

2001

1994

Score Differences

White–HispanicGrade 4 Grade 8 Grade 12

40 38 32

students outperformed Black and Hispanicstudents. Hispanic students had higheraverage scores than Black students.

Among twelfth-graders, White students,Asian/Pacific Islander students, andAmerican Indian students had higheraverage scores than Black students andHispanic students, and Hispanic studentsoutperformed Black students. Thesedifferences should, however, be interpretedwith caution. The average score of aselected subgroup does not represent theentire range of performance within thatgroup. Furthermore, differences between

groups of students cannot be attributedsolely to group identification. A complexarray of educational and social factorsinteracts to affect average studentperformance.

Score differences between Whitestudents and Black students and betweenWhite students and Hispanic students arepresented in figure 3.5. Results from the2001 geography assessment show anarrowing of the score difference be-tween White students and Black studentsat grade 4. Other apparent changes werenot statistically significant.


Achievement-level results for the racial/ethnic subgroups are presented in figures3.6a, b, and c. At grade 4, the percentagesof Black students and White studentsperforming at or above the Basic level werehigher in 2001 than in 1994. At grade 8,the percentage of White students at orabove Basic was higher in 2001. There wasno statistically significant change in the

percentage of twelfth-grade students at orabove the Basic and Proficient levels amongany of the racial/ethnic groups.

Comparing the performance of sub-groups in 2001 shows higher percentagesof White and Asian/Pacific Islanderstudents at or above the Basic and Proficientlevels than Black and Hispanic students atall three grades.


Figure 3.6a

National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levelsby race/ethnicity, grade 4: 1994 and 2001

Significantly different from 1994.# Percentage is between 0.0 and 0.5.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

4%

25%29%

53%

81%19%

1994

3%

26%29%

58%

87%13%

2001

WhiteAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

(2%) 3%

32%34%

66%

1994

(5%) 5%

39%

44%

56%

2001

BlackAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

##

1%(9%) 10%

39%

49%

51%

1994

(6%) 6%

43%

49%

51%

2001

HispanicAdvanced At or above

Proficient

At or aboveBasic

Proficient

Basic

BelowBasic

# 4%

23%27%

49%

76%

24%

1994

1%

23%25%

52%

77%

23%

2001

Asian/Pacific IslanderAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

(9%) 9%

53%

62%

38%

1994

13% 13%

53%

66%

34%

2001

American IndianAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

# #


Figure 3.6b




5%

30%36%

47%

82%18%

1994

5%

34%39%

48%

86%14%

2001

WhiteAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

(4%) 5%

30%34%

66%

1994

(6%) 6%

34%40%

60%

2001

BlackAdvanced At or above

Proficient

At or aboveBasic

Proficient

Basic

BelowBasic

# #

1%(10%) 10%

39%

50%

50%

1994

1%(9%) 10%

38%48%

52%

2001

HispanicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

6%

25%30%

43%

73%

27%

1994

4%

28%32%

47%

79%

21%

2001



At or aboveBasic

Proficient

Basic

BelowBasic

2%13% 15%

43%

59%

41%

1994

3%

29%31%

41%

72%

28%

2001



At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.6c



Significantly different from 1994.# Percentage is between 0.0 and 0.5.Sample size was insufficient to permit a reliable estimate for American Indian students at grade 12 in 1994.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

2%

31%33%

46%

78%

22%

1994

2%

29%31%

51%

81%19%

2001

WhiteAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

(5%) 5%

27%32%

68%

1994

(4%) 4%

31%35%

65%

2001

BlackAdvanced At or above

Proficient

At or aboveBasic

Proficient

Basic

BelowBasic

# #

(10%) 10%

38%48%

52%

1994

(10%) 10%

42%

52%

48%

2001

HispanicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

# # 3%

25%28%

41%

69%

31%

1994

1%

25%26%

45%

72%

28%

2001



At or aboveBasic

Proficient

Basic

BelowBasic

1%

31%32%

41%

74%

26%

2001



At or aboveBasic

Proficient

Basic

BelowBasic

1994

Nodata

reported



Figure 3.7

National Scale ScoreResults by Region ofthe Country

Average geography scale scores by region of the country, grades 4, 8, and 12:1994 and 2001

500

250

240

230

220

210

200

190

0 1994 2001

203200

215

205214207

219

200

Northeast

SoutheastCentralWest

500

280

270

260

250

240

230

220

0 1994 2001

266

252

268

255

266260

270

255

500

290

280

270

260

250

240

230

0 1994 2001

284278

289286 286

281

287

283


Region of the Country

NAEP assessments traditionally provideresults for four regions of the country:Northeast, Southeast, Central, and West.Appendix A (see page 137) contains adescription of the states and other jurisdic-tions that make up each region.

Figure 3.7 shows scale score results byregion of the country. Overall increases inaverage scores at grades 4 and 8 were notspread evenly across the four regions of thecountry. At grade 4, students in the North-east region showed a gain in the averagescore between 1994 and 2001. At grade 8,students in the Southeast region showed a

statistically significant increase. None of theother apparent changes between 1994 and2001 in regional average scores werestatistically significant.

In 2001, some differences in perfor-mance among regions are evident at allthree grades. At grades 4 and 8, students inthe Northeast and Central regions hadhigher average scores than their counter-parts in the West. Students in the Centralregion outperformed students in theSoutheast. Twelfth-graders in the Centralregion had higher average scores thantwelfth-graders in the Southeast region.


Achievement-level results for the fourregions appear in figures 3.8a, b, and c. Thepercentage of fourth-grade students fromthe West performing at or above the Profi-cient level decreased between 1994 and2001. Over the same period, the percent-age of both eighth- and twelfth-graders inthe Southeast performing at or above theBasic level increased.

Figures 3.8a, b, and c also show a num-ber of differences in achievement-levelperformance among the four regions forthe year 2001. At grades 4 and 8, therewere higher percentages of students in theNortheast region and the Central region ator above Basic and at or above Proficientthan in the West. At all three grades, thepercentage of students at or above Basic andat or above Proficient was higher in theCentral region than in the Southeast.


Figure 3.8a

National Achievement-Level Results byRegion of the Country

Percentage of students within and at or above geography achievement levelsby region of the country, grade 4: 1994 and 2001

3%

19% 22%

45%

67%

33%

1994

3%

22%24%

54%

78%

22%

2001

NortheastAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

2%14% 17%

48%

64%

36%

1994

1%17% 18%

54%

72%

28%

2001

SoutheastAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

19% 21%

48%

70%

30%

1994

1%13% 14%

52%

66%

34%

2001

WestAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

4%

25%28%

49%

78%

22%

1994

3%

27%30%

51%

82%18%

2001

CentralAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.8b




6%

28%33%

43%

76%

24%

1994

4%

29%34%

44%

78%

22%

2001

NortheastAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

19% 21%

40%

62%

38%

1994

3%

24%26%

46%

73%

27%

2001

SoutheastAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

6%

30%36%

44%

80%

20%

1994

6%

32%38%

43%

82%18%

2001

CentralAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

20% 23%

45%

67%

33%

1994

2%

21% 23%

44%

66%

34%

2001

WestAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.8c




2%

23%25%

44%

69%

31%

1994

2%

24%26%

46%

71%

29%

2001

NortheastAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

1%

19% 20%

41%

60%

40%

1994

1%

20% 21%

46%

67%

33%

2001

SoutheastAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

2%

30%32%

43%

75%

25%

1994

1%

27%28%

48%

76%

24%

2001

CentralAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

2%

27%29%

43%

72%

28%

1994

1%

22%23%

47%

70%

30%

2001

WestAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

Parents’ Highest Levelof EducationEighth- and twelfth-grade students whoparticipated in the NAEP geographyassessment were asked to indicate thehighest level of education completed byeach parent. Students chose from amongfour options: did not finish high school,graduated from high school, some educa-tion after high school, and graduated from

college. Students could also choose theresponse, “I don’t know.” The analysis hereuses the highest education level reportedfor either parent. Data were not collectedat grade 4 because in previous NAEPassessments fourth-graders’ responses abouttheir parents’ education were unreliable andcontained a large percentage of “I don’tknow” responses.


Figure 3.9

National Scale ScoreResults by Parents’Education

Average geography scale scores by parents’ highest level of education,grades 8 and 12: 1994 and 2001


500

290

280

270

260

250

240

230

0 1994 2001

238

250

265272

234241

253

266274

245

Less than high school

Graduated high school

Some education after high school

Graduated college

Unknown

Grade 8 Grade 12500

300

290

280

270

260

250

240

0 1994 2001

263

274

286294

257

269276284

293

257

The scale score results for student-reported parent education level appear infigure 3.9. In 2001, nearly one-half of allstudents at both grades reported that atleast one of their parents graduated fromcollege (48 percent at grade 8 and 46percent at grade 12). In the case of grade 8,that percentage is a statistically significantincrease over 1994. Only a small percent-age of students at either grade reported thattheir parents had not graduated from highschool (6 percent at grade 8 and 7 percentat grade 12). Additional informationon the percentage of students reporting

parents’ highest level of education is avail-able in appendix B.

Twelfth-graders who reported that theirparents had not graduated from highschool had higher average scores in 2001than in 1994. The results for both gradesin 2001 reveal a pattern similar to that fromthe 1994 geography assessment and fromother NAEP assessments. Overall there is apositive relationship between student-reported parental education and studentachievement: the higher the parentaleducation level reported, the higher theaverage score.


Achievement-level results by level ofparental education are presented in figures3.10a and b. As with the average scalescore results, the 2001 achievement-levelresults show a general pattern of higherpercentages of eighth- and twelfth-gradestudents at or above the Basic and Proficientlevels as the reported level of parental

education level increased. The one excep-tion to the pattern was at grade 12, wherethe percentage of students at or aboveProficient did not differ significantly be-tween those students who reported thattheir parents graduated high school andthose who reported their parents did not.


Figure 3.10a

National Achievement-Level Results byParents’ Education

Percentage of students within and at or above geography achievement levelsby parents’ highest level of education, grade 8: 1994 and 2001


1%(8%) 8%

39%

47%

53%

1994

(8%) 8%

41%

48%

52%

2001

Less than high schoolAdvanced At or above

Proficient

At or aboveBasic

Proficient

Basic

BelowBasic

# 1%14% 15%

47%

62%

38%

1994

1%16% 18%

48%

66%

34%

2001

Graduated high schoolAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

26%29%

50%

79%

21%

1994

2%

27%30%

51%

80%

20%

2001

Some education after high schoolAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

7%

34%41%

41%

82%18%

1994

6%

37%43%

42%

86%14%

2001

Graduated collegeAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

1%(8%) 8%

35%44%

56%

1994

1%(11%) 12%

44%

56%

44%

2001

UnknownAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.10b

National Achievement-Level Results byParents’ Education

Percentage of students within and at or above geography achievement levelsby parents’ highest level of education, grade 12: 1994 and 2001


0%(7%) 7%

34%41%

59%

1994

(10%) 10%

38%48%

52%

2001

Less than high schoolAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

#13% 14%

42%

56%

44%

1994

12% 12%

50%

62%

38%

2001

Graduated high schoolAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

##

1%

23%24%

51%

75%

25%

1994

1%

20% 21%

52%

73%

27%

2001

Some education after high schoolAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

37%40%

41%

81%19%

1994

3%

34%36%

46%

82%18%

2001

Graduated collegeAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

0%(7%) 7%

29%36%

64%

1994

0%(5%) 5%

28%33%

67%

2001

UnknownAdvanced At or above

Proficient

At or aboveBasic

Proficient

Basic

BelowBasic


Type of SchoolThe schools that participate in the NAEPassessment are classified as either public ornonpublic. A further distinction is thenmade within the nonpublic classificationbetween nonpublic Catholic schools andother nonpublic schools. In 2001, as inprevious NAEP assessments, fourth-,eighth-, and twelfth-grade students attend-ing nonpublic schools had higher averagescores than did their public school peers.However, readers are cautioned againstmaking assumptions about the relativequality of public- and nonpublic-schoolinstruction based on these findings. Socio-

economic and sociological factors that mayaffect student performance should also beconsidered when interpreting the results.

Figure 3.11 shows average geographyscores by type of school. At grade 8, publicschool students had higher average scoresin 2001 than in 1994. All other apparentdifferences across years were not statisticallysignificant. The data for performance in2001 reveal that, at all three grades, studentsfrom Catholic schools had higher averagescores than students from public schools.The apparent differences between studentsin public schools and other nonpublicschools were not statistically different.


500

250

240

230

220

210

200

190

0 1994 2001

204

221

207

226

500

250

240

230

220

210

200

190

0 1994 2001

222220

230

221Grade 4

Public

Nonpublic Nonpublic: OtherNonpublic: Catholic

500

290

280

270

260

250

240

230

0 1994 2001

258

276

261

274Grade 8

Grade 12

500

310

300

290

280

270

260

250

0 1994 2001

283

294

284291

500

290

280

270

260

250

240

230

0 1994 2001

277271

500

310

300

290

280

270

260

250

0 1994 2001

291298

294287

Significantly different from 1994.NOTE: Italicized scale score values indicate that two or more groups had the same rounded average score. The average scores, when rounded, were the samefor Nonpublic: Catholic and Nonpublic: Other at grade 8 in 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Figure 3.11

National Scale ScoreResults by Type ofSchool

Average geography scale scores by type of school, grades 4, 8, and 12:1994 and 2001


Achievement-level results by type ofschool are presented in figures 3.12a, b, andc. At grade 4, the percentage of Catholicschool students performing at or above theBasic level increased between 1994 and2001. At grade 8, the percentage of publicschool students at or above the Basic levelincreased across years.

In 2001 there were a number of differ-ences in achievement-level performancebetween subgroups. At all three grades,there were higher percentages of nonpublicschool students and, more specifically,Catholic school students at or above theBasic and Proficient levels than public schoolstudents.

Figure 3.12a

National Achievement-Level Results by Typeof School

Percentage of students within and at or above geography achievement levelsby type of school, grade 4: 1994 and 2001


3%

19% 21%

47%

68%

32%

1994

2%

18% 20%

52%

72%

28%

2001

PublicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

5%

26%30%

53%

84%16%

1994

3%

28%31%

59%

91%

2001(9%)

NonpublicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

5%

(7% )

25%30%

54%

85%15%

1994

4%

32%36%

56%

93%

2001

Nonpublic: CatholicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

4%

26%30%

52%

82%18%

1994

2%

23%25%

63%

88%12%

2001

Nonpublic: OtherAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.12b




4%

22%26%

43%

69%

31%

1994

3%

25%28%

44%

72%

28%

2001

PublicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

8%

36%44%

43%

87%13%

1994

5%

36%41%

46%

87%13%

2001

NonpublicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

8%

35%44%

45%

89%11%

1994

6%

40%

46%

44%

89%11%

2001



At or aboveBasic

Proficient

Basic

BelowBasic

7%

38%45%

41%

86%14%

1994

4%

33%37%

48%

85%15%

2001



At or aboveBasic

Proficient

Basic

BelowBasic


Figure 3.12c




1%

24%26%

42%

68%

32%

1994

1%

23%24%

47%

71%

29%

2001

PublicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

3%

33%36%

47%

83%17%

1994

2%

30%32%

48%

80%

20%

2001

NonpublicAdvanced


At or aboveBasic

Proficient

Basic

BelowBasic

1%

32%33%

47%

80%

20%

1994

2%

32%34%

51%

85%15%

2001



At or aboveBasic

Proficient

Basic

BelowBasic

5%

35%40%

47%

87%13%

1994

2%

26%29%

44%

73%

27%

2001



At or aboveBasic

Proficient

Basic

BelowBasic


Average geography scale scores by type of school location, grades 4, 8, and 12: 2001

Central city Urban fringe/large town Rural/small town

Table 3.1 National Scale Score Results by Type of Location

Grade 4199 212 215

Grade 8 255 265 265

Grade 12 279 288 284

SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Type of LocationThe schools from which NAEP draws itssamples of students are classified accordingto their type of location. Based on CensusBureau definitions of metropolitan statisti-cal areas, including population size anddensity, the three mutually exclusive cat-egories are central city, rural/small town,and urban fringe/large town. Because ofnew methods used by NCES to identifythe type of location assigned to each schoolin the Common Core of Data, schoolswere not classified in exactly the same way

in 2001 as in 1994. Therefore, comparisonsbetween the two assessment years are notpossible, and only the data from the 2001assessment are reported. More informationon the definitions of the 2001 assessmentclassifications for location type appears inappendix A.

The performance of students by type ofschool location is shown in table 3.1. At allthree grades, students attending schools inrural and urban fringe locations had higheraverage scale scores than students in centralcity schools.

Achievement-level results by type ofschool location appear in figure 3.13. Atgrade 4, higher percentages of urban fringeand rural students performed at or aboveBasic and at or above Proficient than didtheir central city counterparts. There werehigher percentages of eighth-graders fromurban fringe and rural locations at or aboveBasic than those from central city locations.There were also higher percentages of

students at or above Proficient in urbanfringe locations than central city locations.At grade 12, higher percentages of urbanfringe and rural students than central citystudents performed at or above the Basiclevel, and there were higher percentages ofstudents at or above Proficient in urbanfringe locations than in central city or rurallocations.


Figure 3.13

National Achievement-Level Results by Typeof Location

Percentage of students within and at or above geography achievement levelsby type of school location, grades 4, 8, and 12: 2001

2%14% 16%

46%

62%

38%

2%

21% 23%

53%

76%

24%

Central city Urban fringe/large town

Grade 4

Rural/small town


At or aboveBasic

Advanced

Basic

Proficient

BelowBasic

2%

21% 23%

58%

81%19%

3%

22%25%

39%

64%

36%

4%

29%32%

45%

78%

22%


Grade 8

Rural/small town


At or aboveBasic

Advanced

Basic

Proficient

BelowBasic

3%

27%30%

48%

78%

22%

1%

19% 20%

42%

63%

37%

2%

28%30%

45%

75%

25%


Grade 12

Rural/small town


At or aboveBasic

Advanced

Basic

Proficient

BelowBasic

1%

21% 22%

52%

74%

26%

NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.


Table 3.2 National Scale Score Results by Free/Reduced-Price School Lunch Eligibility

Average geography scale scores by student eligibility for Free/Reduced-Price School Lunchprogram, grades 4, 8, and 12: 2001

Eligible Not eligible Info not available

Grade 4186 221 218

Grade 8 242 270 266

Grade 12 269 287 289


Free/Reduced-Price School LunchProgram EligibilityFunded by the U.S. Department of Agri-culture (USDA) as part of the NationalSchool Lunch Program, the Free/Re-duced-Price School Lunch program isdesigned to assure that children at or nearthe poverty line receive nourishing meals.Eligibility guidelines for the lunch programare based on the federal income povertyguidelines and are stated by householdsize.1 NAEP first began collecting data onstudent eligibility for this program in 1996;therefore cross-year comparisons back to1994 are not possible.

Table 3.2 presents the average scale scoreresults for grades 4, 8, and 12. The scoresfor the substantial number of students forwhom eligibility information is not avail-able appear in the “Info not available”column (see the percentages for eachcategory in the table B.18). Students whoseschools do not participate in the Free/Reduced-Price School Lunch program areincluded in this category. At each grade,students eligible for the Free/Reduced-Price School Lunch program (i.e., thosemeeting the poverty guidelines) had loweraverage scores than did ineligible studentsand students for whom information wasnot available.

Achievement-level results reflected thescale score results, as seen in figure 3.14. Atall three grades, higher percentages ofstudents who were ineligible for the Free/Reduced-Price School Lunch program orfor whom information was not available

were at or above Basic and at or aboveProficient than were students who wereeligible for the lunch program. At grade 8,this pattern extended to the Advancedachievement level.

1 U.S. General Services Administration. (2001). Catalog of federal domestic assistance. Washington, DC: Executive Officeof the President, Office of Management and Budget. http://www.cfda.gov/default.htm.


# Percentage is between 0.0 and 0.5.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Figure 3.14National Achievement-Level Results by Free/Reduced-Price SchoolLunch Program Eligibility

Percentage of students within and at or above geography achievement levels byFree/Reduced-Price School Lunch program eligibility, grades 4, 8, and 12: 2001

(6%) 6%

45%

51%

49%

3%

27%29%

56%

86%14%

Eligible Not eligible

Advanced


At or aboveBasic

Proficient

Basic

BelowBasic

Grade 4

Info notavailable

3%

24%27%

57%

84%16%

#

1%(10%) 11%

39%

50%

50%

5%

32%37%

46%

83%17%


Advanced


At or aboveBasic

Proficient

Basic

BelowBasic

Grade 8

Info notavailable

4%

29%33%

46%

79%

21%

(10%) 11%

40%

51%

49%

1%

25%26%

49%

75%

25%


Advanced

Proficient

Basic

BelowBasic

Grade 12

Info notavailable

# 2%

29%31%

45%

76%

24%


At or aboveBasic


4Chapter

Contents

TeacherBackground and

Preparedness

Geography SkillsTaught

Extent ofGeographyInstruction

Computer Use

Student Interestin Geography

How does teacherpreparednessrelate to students’geographyperformance?

How do classroomactivities andcomputer userelate to studentachievement?

ChapterFocus

Classroom Contexts for Learning

This chapter examines teacher and student variables related

to the teaching and learning of geography, such as teacher

preparedness and classroom practices, student interest in

geography, and the context in which students learn the

concepts of geography. The results presented in this chapter

are based on teachers’ and students’ responses to

questionnaires administered as part of the NAEP 2001

geography assessment. NAEP administers

background and instructional questionnaires to

students at grades 4, 8, and 12 and to the teachers of

participating fourth- and eighth-grade students.

Teachers at grade 12 were not administered a

questionnaire because of the difficulty of linking

students to teachers across the diversity of courses at

this grade level. In this examination of contexts for

learning, students are the unit of analysis. Thus, for

questions answered by students, the percentage of

students choosing each response option is presented.

For questions answered by teachers, the percentage of

students whose teachers chose each option is

presented. Students’ average NAEP geography scores

for each response are also presented in order to examine the

relationship between each variable and students’ geography

performance. Readers are reminded that the relationship

between a contextual variable and geography performance is

not necessarily causal. Many factors contribute to student

performance. NAEP data can identify relationships between

contextual variables and student performance, but cannot

explain why the relationships exist.


1 Geography Education Standards Project. (1994). Geography for life: national geography standards. Washington, DC:National Geographic Research and Exploration.

2 Gregg, M. (2001). River views of beginning pre-service teachers: content knowledge use. Journal of Geography100, 61–68.

Brophy, J. (1991). Advances in research on teaching. (Vol. 2) Teacher’s knowledge of subject matter as it relates to theirteaching practice. Greenwich, CT: JAI Press.

Reynolds, M. C., Ed. (1989). Knowledge base for the beginning teacher. Oxford: Pergamon Press.3 Rynne, E. (1997) The continuing mismatch between student’s undergraduate experience and the teaching

demands of the geography classroom: experience of pre-service secondary geography teachers. Journal ofGeography in Higher Education 21, 65–77.

Teacher Background andPreparedness to Teach GeographyCompetency in geography is positivelyassociated with extent of geography educa-tion and begins with well-prepared teach-ers.1 Teachers of geography in grades 4, 8,and 12 approach instruction with variedbackgrounds in geography or social studies,which necessarily results in diverse learningexperiences for their students. Research onteaching and learning shows that the extentof a teacher’s knowledge of a subject canhave a marked effect on the quality ofteaching or even the willingness to teachthat subject.2 A number of educators areconcerned that poorly trained geographyteachers—those teaching “out of field”—may lack the knowledge or skills to providestudents with the knowledge necessary tomeet geography standards.3 Thus, it isinstructive to explore the educationalbackgrounds of the teachers who arecurrently teaching geography to thenation’s fourth- and eighth-grade students.

As part of the NAEP 2001 assessment,teachers of participating students in grades4 and 8 were asked about their under-graduate and graduate majors and minors.The NAEP teacher questionnaire gavefourth- and eighth-grade teachers a num-ber of different majors/minors from which

to choose. Table 4.1 shows results for fivemajors/minors asked of elementary teach-ers and four majors/minors asked ofeighth-grade teachers. Both the fourth-and eighth-grade teachers were asked ifthey had a major or minor in geography orgeography education, history or historyeducation, general social science or socialstudies education, or other social science(for example, political science, economics,sociology, psychology, anthropology).Fourth-grade teachers were also askedwhether they had a major or minor inelementary education. Although teacherswere asked separately about their under-graduate and graduate education, andabout whether they had majored or mi-nored in each subject, the data are pre-sented here in a simplified form. The firstcolumn in table 4.1 shows the percentagesand average scores of students whosegeography teachers either majored orminored or had a special emphasis in asubject at either the undergraduate orgraduate level. The second column showsthe corresponding data for students whoseteachers did not indicate that major orminor. Note that the columns can sum tomore than 100 percent because it is pos-sible for college students to complete morethan one major or minor.


At grade 4, nearly all students (93 percent)had teachers who majored or minored inelementary education in undergraduate orgraduate school. While 15 to 20 percent ofstudents were taught by teachers who had asocial studies or history major or minor,only 7 percent of fourth-graders weretaught by teachers who had majored orminored in geography. Average geographyscores for fourth-grade students taught by

teachers with an elementary educationmajor or minor were higher than thosetaught by teachers who did not.

At grade 8, just over one-quarter (28percent) of students were taught by teacherswith a graduate or undergraduate major orminor in geography. Within this grade,there was no statistically significant relation-ship between teachers’ major/minor andtheir students’ NAEP geography scores.

Table 4.1Percentage of students and averagegeography scale scores by teachers’reported undergraduate/graduatemajor and minor/special emphasis,grades 4 and 8: 2001

G r a d e s

4&8Grade 4

Geography or geography education 7 93204 210

History or history education 15 85206 211

Social science or social studies education 20 80208 210

Other social science 57 43210 209

Elementary education 93 7211 197

Grade 8Geography or geography education 28 72

263 263History or history education 71 29

263 261Social science or social studies education 55 45

263 262Other social science 51 49

261 264

The percentage of students is listed first with the corresponding average scale score presented below.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Yes NoDid you have a major, minor, or special emphasis in anyof the following subjects as part of your undergraduateor graduate course work?

Teachers’Majors/Minors

At grade 4, studentswhose teachersmajored in elemen-tary education hadhigher averagescores than thosewhose teachers didnot.


Teachers of fourth- and eighth-gradestudents participating in the geographyassessment were asked how prepared theyfelt they were to teach geography. Thequestion was asked in both 1994 and 2001,so cross-year comparisons are possible.Table 4.2 shows that a higher percentage ofthe fourth-grade students in 2001 hadteachers who reported they were veryprepared to teach geography (31 percent)than did students in 1994 (23 percent).These results may be associated with theincrease in workshops in geography offeredto teachers through various geographicorganizations, and an increase in the litera-ture available to teachers related to teach-ing geographic concepts.4 In addition,there has been a call for geographers as agroup, specifically college and universityfaculty, to develop an open dialogue with

K-12 teachers that would aid in providingelementary and high school teachers withthe teaching tools necessary for them toenter the classroom with some degree ofconfidence.5

In 2001, approximately 84 percent offourth-grade students were taught byteachers who reported that they were veryprepared or adequately prepared to teachgeography. Only 1 percent of fourth-gradestudents had teachers who reported theywere unprepared to teach geography.Approximately 87 percent of eighth-gradestudents had teachers who reported theywere at least adequately prepared to teachgeography, and only 2 percent had teacherswho felt unprepared. The level of teachers’self-reported preparedness had no statisti-cally significant relationship to students’average geography scores.

4 Gibbs, G. (1999). Improving teaching, learning, and assessment. Journal of Geography in Higher Education 23, 147–155.

McDougall, W. A. (2001). Why geography matters. American Educator 25,10–15.

McAlonan, S. Hotchkiss, H., Roark, K., Kenney, M., & Jackson, J. (2001) Making standards work! Geography. Ateacher’s guide to contextual learning: integrating academic content standards with career development and workplace competen-cies. Denver: Colorado State Department of Education.

5 Bettis, N. C. (2001). Assessment issues in geographic education for the twenty-first century. Journal of Geography100, 172–174.

Welford, M. & Fouberg, E.H. (2000). Theory and research in geography education. Journal of Geography 99,183–184.


G r a d e s

4&8Table 4.2Percentage of students and averagegeography scale scores by teachers’reports on how well prepared theyfelt they were to teach geography,grades 4 and 8: 1994 and 2001

Teachers’Preparedness toTeach Geography

The percentage offourth-graderswhose teachersreported they werevery prepared toteach geographyincreased between1994 and 2001.

Grade 4Very prepared 23 31 *

209 211Adequately prepared 57 53

206 210Somewhat prepared 18 15

207 206Unprepared 2 1

200 209

Grade 8Very prepared 36 44

260 263Adequately prepared 48 43

262 262Somewhat prepared 13 11

265 261Unprepared 2 2

260 264

The percentage of students is listed first with the corresponding average scale score presented below.* Significantly different from 1994.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001Regardless or whether you are currently teaching thetopic, how well prepared do you feel you are to teachgeography at the elementary /middle school level?


Geography Skills andTopics Taught in Grade 4Many of the instructional guides andstandards for geography educationemphasize the importance of using mapsand globes as tools to visualize space andgain a spatial perspective—necessary skillsfor an understanding of and competency ingeography.6 A number of articles alsodiscuss the importance of exposingchildren to cultural geography, andenvironmental geography.7, 8 As part of the1994 and 2001 NAEP geographyassessments, fourth-grade teachers wereasked about the frequency with which theytaught about maps and globes, naturalresources, foreign countries and cultures,and environmental issues as a part ofgeography instruction. The results aredisplayed in table 4.3.

There were a few changes between 1994and 2001 in the frequency of instructionreported by teachers of fourth-gradestudents in these four skill and topic areas.The percentage of fourth-grade studentswhose teachers reported they never taughtabout natural resources increased from 9percent in 1994 to 14 percent in 2001,while the percentage of students whoseteachers reported teaching about naturalresources as frequently as once or twice

weekly decreased from 38 percent to 31percent. These data may suggest that thetopic of natural resources has lost favorduring the past decade in teaching geogra-phy to fourth-graders.

Of the four skills and topics, maps andglobes were most frequently used duringgeography instruction during 2001. Ap-proximately three-quarters of fourth-gradestudents received instruction about mapsand globes on at least a weekly basis,according to their teachers. However, lessthan one-half of the students had teacherswho included natural resources as part ofinstruction once a week or more, and onlyabout one-quarter received instruction inforeign countries and cultures and environ-mental issues at least once a week. Twenty-nine percent of students received noinstruction in foreign countries and cul-tures. Readers should be aware that teach-ers were asked only to indicate the fre-quency and not the total amount of timethey devoted to the skills and topics dis-cussed here “as part of geography instruc-tion.” Therefore, students may have receivedmore instruction in these four areas than isreadily apparent from the percentagesshown, though not necessarily from ageographic perspective.

6 Oldakowski, R. K. (2001). Activities to develop a spatial perspective among students in introductory geographycourses. Journal of Geography 100, 243–250.

Thompson, G. (1999). I thought the world was flat, like the maps showed it! Social Education 63, 269–271.

Trifonoff, K. M. (1998). Introducing thematic maps in the primary grades. Social Studies and the Young Learner 11,17–22.

7 Crampton, J. (1998). A regional geography class in a distributed learning environment. Journal of Higher Education22, 417–423.

8 Graf, M. (2000). The world’s best places: classroom explorations in geography & environmental science. Portsmouth, NH:Heinemann.

Steinberg, P. E. (1997). Political geography and the environment. Journal of Geography 96, 113–118.


G r a d e

4Table 4.3Percentage of students and averagegeography scale scores by teachers’reports on frequency of instruction ofselected skills and topics, grade 4:1994 and 2001

Using maps and globesAlmost every day 29 28

210 213Once or twice a week 54 47

208 209Once or twice a month 17 22

199 206Never or hardly ever 1 3

*** 209Natural resources

Almost every day 9 9201 217

Once or twice a week 38 31 *209 208

Once or twice a month 44 46208 210

Never or hardly ever 9 14 *198 208

Foreign countries and culturesAlmost every day 6 3




209 212Environmental issues

Almost every day 4 7201 212

Once or twice a week 27 21206 205

Once or twice a month 56 56208 211

Never or hardly ever 13 16208 211

The percentage of students is listed first with the corresponding average scale score presented below.* Significantly different from 1994.*** Sample size is insufficient to permit a reliable estimate.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001How often do you teach the following skills and topics asa part of geography instruction with this class?

Frequency ofInstruction inFourth-GradeGeography Skillsand Topics

Natural resourcestopics were taughtless frequently in2001.


Geography Skills Taught inGrades 8 and 12Eighth-and twelfth-grade students partici-pating in the 1994 and 2001 geographyassessments were asked about the frequencywith which they studied the followingskills and topics in school: using maps andglobes, natural resources (for example, oil,forests, and water), countries and cultures,and environmental issues (for example,pollution and recycling). The results ofthese questions are shown intables 4.4a and 4.4b.

At grade 8, students in 2001 reportedhaving devoted more time to two of thefour areas than did their counterparts in1994. Approximately 46 percent of stu-dents reported using maps and globes atleast once or twice a week compared to 39percent in 1994. At the same time, thepercentage that reported never using mapsand globes dropped from 28 percent to 21percent. For the topic of countries andcultures, approximately 63 percent ofstudents in 2001 said they studied it one ortwo times per week or more compared to52 percent in 1994, and the percentage thatreported never studying it fell from 20 to 13percent.

The geography performance of eighth-graders varied somewhat depending on theamount of time they spent studying eachtopic, but in general, higher frequency ofstudy did not translate into significantlyhigher performance. In the case of mapsand globes and natural resources, studentsperformed best when they studied thetopics once or twice a month. Studentswho studied natural resources and environ-mental issues almost every day had lower

average scores than students who studiedthese topics less frequently or never. Stu-dents who never studied countries andcultures had lower average scores thanstudents who did study that topic.

Twelfth-graders’ reports indicated ageneral increase between 1994 and 2001 inthe amount of time they spent studying thefour geography skills and topics listed inthe beginning of this section. The easiestway to see this change among the data intable 4.4b is to note that the percentage ofstudents responding “never” declined foreach of the four topics (from 40 percent to36 percent for maps and globes, from 45percent to 39 percent for natural resources,from 28 percent to 19 percent for coun-tries and cultures, and from 37 percent to30 percent for environmental issues, respec-tively). But as these results also show, asubstantial percentage of twelfth-gradersstill did not study these topics in 2001.

Again, as was seen at the eighth-gradelevel, more instruction was not necessarilylinked to better student performance.Students who reported studying maps andglobes and natural resources almost everyday had lower average scores than studentswho reported less frequent study. The samewas true with the study of environmentalissues except that the performance ofstudents in the “almost every day” and“never” categories did not differ signifi-cantly. The one instructional topic that didhave a somewhat positive relationship totwelfth-graders’ geography scores wascountries and cultures. Students studyingcountries and cultures at least once amonth outperformed those who neverstudied countries and cultures.


G r a d e

8Table 4.4aPercentage of students and averagegeography scale scores by students’ reportson frequency of instruction of selected skillsand topics, grade 8: 1994 and 2001

Using maps and globesAlmost every day 9 12 *

261 259Once or twice a week 30 34 *


263 268Never or hardly ever 28 21 *

253 258

Natural resourcesAlmost every day 9 9




260 263

Countries and culturesAlmost every day 23 31 *


261 266Once or twice a month 28 24 *


256 254

Environmental issuesAlmost every day 12 11




260 262

1994 2001How often have you studied the following geography skillsand topics in school?

The percentage of students is listed first with the corresponding average scale score presented below.* Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Frequency ofInstruction inEighth-GradeGeography Skillsand Topics

Students performedbest when instruc-tion on these topicswas once or twice amonth.

Eighth-graders whonever studiedcountries andcultures had lowerscores than thosewho did.


Grade

12


Using maps and globesAlmost every day 7 6




283 284

Natural resourcesAlmost every day 7 7




284 285

Countries and culturesAlmost every day 16 20 *




280 277

Environmental issuesAlmost every day 11 11




282 282

Table 4.4bPercentage of students and averagegeography scale scores by students’ reportson frequency of instruction of selected skillsand topics, grade 12: 1994 and 2001

Frequency ofInstruction inTwelfth-GradeGeography Skillsand Topics

All topics receivedmore frequentinstruction in 2001.

Students who neverstudied countriesand cultures scoredlower than thosewho did.



The Extent of Students’Social Studies andGeography InstructionAs part of the NAEP 2001 geographyassessment, eighth- and twelfth-gradestudents were asked a series of questionsconcerning the amount of geographyinstruction they had received. The resultsfor the student responses are shown intables 4.5 and 4.6 and are summarizedbelow.

Geography Course Takingat Grades 8 and 12Eighth-grade students were asked whetherthey had previously taken a geographycourse in grades 6 and 7, and whether theywere currently enrolled in a geographycourse in eighth grade. The results of theiranswers to these questions are presented incumulative form in table 4.5. This table

shows the percentages and average scores ofeighth-grade students who reported takingzero, one, two, and three years of geographyfrom the sixth through eighth grades. Intotal, about 59 percent of eighth-gradestudents in 2001 took two or three years ofgeography. In 2001, 63 percent of eighth-graders reported taking a geography coursein eighth grade (data not shown). A higherpercentage of students in 2001 than in1994 reported taking three years of geogra-phy and, conversely, a lower percentage in2001 than in 1994 reported taking nogeography courses. In addition, there was apositive association in 2001 between morecourse-taking and higher geography scores.Students who took three years of coursework had higher scores, on average, thanthose who took two years. Those studentswho took two years had higher scores thanthose who took one year.


Twelfth-grade students were askedwhether they had previously taken ageography course in grades 9, 10, 11, orwere currently enrolled in a geographycourse. Table 4.6 displays percentages oftwelfth-grade students and their averagegeography scores by their cumulativeamount of high school geography course-taking. Twelfth-graders in 2001 were morelikely to report having taken three and fourcourses than their counterparts in 1994.The majority of twelfth-grade students (73percent) indicated they were not currentlytaking geography in twelfth grade (data notshown). While taking more years ofgeography was related to higher geography

scores for the eighth-graders in table 4.5,this pattern did not hold for the twelfth-grade data in table 4.6. In fact, those whoreported taking no geography courses inhigh school had higher average scores thanthose who had taken 2, 3, or 4 years, andwere not significantly different from thosewho took one year of geography. Aboutone-half of the twelfth-grade students (53percent) took one year or less of geographyin high school. This group may representstudents who are following a differentacademic curriculum than the studentswho reported taking geography in multipleyears.

G r a d e

8Table 4.5Percentage of students and averagegeography scale scores by students’reports on grades in which geographywas taken since the 6th grade, grade 8:1994 and 2001


Number of grades selectedNone 18 12 *

250 255

One 30 20 *257 256

Two 14 16269 263

Three 26 43 *274 272

Don’t know 13 9 *243 246

1994 2001Did you take or do you expect to take a geography coursein 6th, 7th, or 8th grade?

Eighth-GradeFrequency ofGeography CourseTaking

More frequentcourse taking wasassociated withhigher scores foreighth-graders.


Number of grades selectedNone 31 21 *

286 289

One 35 32288 288

Two 16 15286 285

Three 10 18 *281 280

Four 5 10 *277 281

Don’t know 3 3268 265

G r a d e

12Table 4.6Percentage of students and averagegeography scale scores by students’reports on grades in which geographywas taken since 9th grade, grade 12:1994 and 2001

Twelfth-GradeFrequency ofGeography CourseTaking

Students taking oneyear or less ofgeography hadhigher scores, onaverage, thanstudents taking 3 or4 years.


1994 2001Did you take or do you expect to take a geography coursein 9th, 10th, 11th, or 12th grade?


9 Shepard, I. (1998). Teaching and learning geography with information and communication technologies. Cheltenham, UK:Cheltenham and Gloucester College of Higher Education

Crampton, J. W. (1999). Integrating the web and the geography curriculum: the Bosnian virtual fieldtrip. Journal ofGeography 98, 155–168.

10 Mosely, W. G. (2001). Computer assisted comprehension of distant worlds: understanding hunger dynamics inAfrica. Journal of Geography 100, 32–45.

11 Sharma, M. B. & Elbow, G. S. (2000). Using internet primary sources to teach critical thinking skills in geography.Westport, CT: Greenwood Professional Guides in School Librarianship.

Cross, J.A. (1997). Natural hazards and disaster information on the internet. Journal of Geography 96, 307–314.Barta-Smith, N.A. & Hathaway, J.T. (2000). Making cyberspaces into cyberplaces. Journey of Geography 99, 253–265.

The Use of Computers inthe Social Studies Classroomin Grades 4 And 8Computers can be used to enhance geog-raphy instruction. The role of informationand communication technologies in theclassroom is to effectively supplement, notreplace, human contact in the teaching andlearning process.9 Some research hasshown that there is a quantifiable improve-ment in student understanding of geo-graphic concepts when computers are usedto enhance the learning experience.10 Theuse of computers in geography instructioncan also be employed to increase criticalthinking skills for oral and written presen-tations, as well as for specific topics ingeography.11 While most geography in-struction takes place under the curriculumcategory of “social studies,” such instruc-tion also takes place in classes with geogra-phy titles. The 2001 NAEP geographyassessment asked teachers of fourth- andeighth-grade students how frequentlycomputers were used when working onsocial studies to locate and retrieve socialstudies information through the Internet,look up social studies information in CD-ROM reference works, use exploration or

simulation software, and organize socialstudies information using spreadsheets ordatabases. Table 4.7 presents results corre-sponding with teachers’ reports on thefrequency of these activities.

According to their teachers, the majorityof fourth-grade students used computersfor the four tasks either to a small extent ornot at all. At fourth grade, students occa-sionally retrieved information through theInternet and used CD-ROMs for refer-ence, but only rarely used exploration/simulation software or spreadsheets anddatabases. Nearly two-thirds of fourth-graders used the Internet or CD-ROMs toat least a small extent while less than halfused simulation software or spreadsheets/databases at least to a small extent. How-ever, 34 percent and 37 percent of students,respectively, did not engage in these com-puter activities at all. Students who usedthe Internet and CD-ROMs to either asmall or a moderate extent had higheraverage scores than students who did notuse them at all, and students who usedsimulation software to a small extentoutperformed students who did not usesuch software.


G r a d e

4Table 4.7aPercentage of students and averagegeography scale scores by teachers’reports on computer use for socialstudies instruction, grade 4: 2001

Grade 4

Use CD-ROM to look up reference worksNot at all 37

205Small extent 47

211Moderate extent 14

216Large extent 2

214

Retrieve information through the InternetNot at all 34

203Small extent 45


216Large extent 4

211

Use exploration/simulation softwareNot at all 54

207Small extent 37


211Large extent 1

***

Organize information using spreadsheets/databasesNot at all 89

209Small extent 9


213Large extent #

***

The percentage of students is listed first with the corresponding average scale score presented below.*** Sample size is insufficient to permit a reliable estimate.# Percentage is between 0.0 and 0.5.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

2001When students in this class work on social studies, to whatextent do they use computers to do each of the following?

Fourth-GradeComputer Use

Fourth-graders whodid not use thesetechnologies hadlower scores thanthose who usedthem to a small ormoderate extent.


The percentage of students is listed first with the corresponding average scale score presented below.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Grade 8

Use CD-ROM to look up reference worksNot at all 31

258Small extent 48


266Large extent 4

268

Retrieve information through the InternetNot at all 20

255Small extent 47


266Large extent 4

273


261Small extent 32


259Large extent 1

257

Organize information using spreadsheets/databasesNot at all 74

261Small extent 22


262Large extent 1

249

G r a d e

8Table 4.7bPercentage of students and averagegeography scale scores by teachers’reports on computer use for socialstudies instruction, grade 8: 2001


Eighth-GradeComputer Use

Eighth-graders whodid not use thesetechnologies hadlower scores thanthose who usedthem to a small ormoderate extent.


At the eighth-grade level, the pattern offrequency of use of the information tech-nology tools was similar to that in grade 4:information retrieval through the Internetand use of CD-ROMs for reference oc-curred to a small or moderate extent fortwo-thirds to three-quarters of the students,while exploration/simulation software andspreadsheets/databases were rarely used.Twenty-nine percent of eighth-gradestudents used the Internet to a moderateextent for social studies and 47 percentused it to a small extent, according to theirteachers. Forty-eight percent used CD-ROMs to a small extent, nearly one-third(32 percent) used simulation software to asmall extent, and 22 percent used spread-sheets or databases to a small extent. Aboutthree-quarters of students in eighth grade(74 percent) did not use spreadsheets ordatabases at all.

Greater usage of the Internet and CD-ROMs was generally associated withhigher performance among eighth-graders.Students whose teachers reported a largeextent of Internet usage had higher averagescores than students who used the Internetto a small extent or not at all. Students whoused the Internet or CD-ROMs to amoderate or small extent had higher scoresthan students who did not use these toolsat all.

The Use of Computersin Grade 12Twelfth-grade students participating in the2001 NAEP geography assessment alsoanswered questions on the extent of use ofseveral types of computer technology. Inanswering the questions, students were toconsider both work in class and homeworkassignments. The results are shown intable 4.8.

Forty-two percent of students used aCD or the Internet for research to at least amoderate extent. About one-third ofstudents (32 percent) used these tools to asmall extent, and 26 percent did not usethem at all. Students who reported using aCD or the Internet for research to a smallor moderate extent had higher averagescores than those who never used them,and moderate use was associated withhigher scores than a small amount of use.There was no statistically significant differ-ence between the average scores of studentsusing CDs and the Internet to a moderateextent and those of students using them toa large extent.

Thirty-four percent of twelfth-gradestudents reported some use of simulationsoftware. Twenty-three percent used it to asmall extent, 9 percent to a moderateextent, and 2 percent to a large extent. The66 percent of the students who reportednot using simulation software at all hadhigher average scores than students whoreported using it to any extent.

Thirty percent of the students reportedusing a computer to a small extent to puthistory or geography information intotables, charts, or graphs. Sixteen percent ofstudents used a computer to a moderate orlarge extent for this purpose and over one-half (55 percent) of students never used acomputer for this purpose. Students whoreported carrying on these activities to asmall extent had higher average scores thanthose who engaged in them either more ornot at all. Students who said they did notdo these activities at all outperformedstudents who did them to a large extent.


Research projects using a CD or the InternetNot at all 26

274Small extent 32


290Large extent 13

292


287Small extent 23


276Large extent 2

278

Tables, charts, or graphs on the computerNot at all 55

284Small extent 30


281Large extent 4

277

The percentage of students is listed first with the corresponding average scale score presented below.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

G r a d e

12Table 4.8Percentage of students and averagegeography scale scores by students’reports on computer use for historyand geography, grade 12: 2001

Twelfth-GradeComputer Use

Students whoused electronicinformation retrievalhad higher scoresthan those whodid not.

2001Think about all the courses since the 9th grade in which youhave studied history or geography. To what extent have youused computers to do the following? For this question includeboth work in class and homework assignments.


Student Interest in GeographyGrades 8 and 12Interest in geography may increase astudent’s efforts to learn the subject, eitherin school or independently. The NAEP2001 assessment asked students in grades 8and 12 to indicate whether geography wasone of their favorite subjects, whether theylike most other subjects better thangeography, or whether they never studiedgeography. As shown in table 4.9, mosteighth-grade students in 2001 preferredsubjects other than geography. Only 20percent indicated that geography was oneof their favorites. However, these students

outperformed students who liked othersubjects better.

At grade 12, the percentage of studentswho preferred subjects other than geogra-phy increased from 63 percent to 72percent between 1994 and 2001 even asthe percentage of students who reportednever taking a geography class declined(from 23 percent to 13 percent). Only 15percent of students in 2001 chose geogra-phy as one of their favorite subjects. How-ever, those students had higher averagescores than the students who did not favorgeography.

The percentage of students is listed first with the corresponding average scale score presented below.* Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

G r a d e s

8&12Table 4.9Percentage of students andaverage geography scale scoresby students’ reports on howmuch they like studying geography,grades 8 and 12: 1994 and 2001

Grade 8One of my favorite subjects 19 20

274 275Like other subjects better 67 69

260 263Never studied geography 14 11 *

241 247

Grade 12One of my favorite subjects 14 15

297 293Like other subjects better 63 72 *

285 285Never studied geography 23 13 *

277 278

1994 2001How much do you like studying geography?

How Much Eighth-and Twelfth-GradeStudents LikeGeography

Students whofavored geographyhad the highestscores.


5Chapter

Contents

Two Sets of 2001Geography

Results

Results for theNation

National Resultsby Gender

National Resultsby Race/Ethnicity

1 See appendix A for a description of specific criteria provided to assist them in makingexclusion decisions.

How wouldthe NAEP resultsdiffer ifaccommodationswere permittedfor special-needsstudents?

ChapterFocus

Becoming a More InclusiveNational Assessment

In its efforts to assess a representative sample of all students

in the nation, NAEP has consistently striven to include

special-needs students—those with disabilities (SD) and/or

limited English proficiency (LEP). A certain percentage of

such students, however, has always been excluded because

they could not be assessed meaningfully without

accommodations. Schools that participate in NAEP have

been permitted to exclude certain students who have

been classified as having a disability under the

Individuals with Disabilities Education Act (IDEA),

based upon their Individualized Education Programs

(IEP) and Section 504 of the Rehabilitation Act of

1973. Similarly, schools have been permitted to

exclude some students they identify as being limited

English proficient.1

In order to assess a more inclusive sample, and

in an attempt to remain consistent with state- and

district-level testing policies that increasingly offer

accommodations to special-needs students, NAEP

began to explore the use of accommodations in the

1996 and 1998 assessments. A split-sample design

was used to identify a portion of schools that could provide

accommodations to their special-needs students who

required them, and a portion of schools in which

accommodations would not be offered (the standard

administration procedure prior to 1996). The split-sample


2 Olson, J. F. & Goldstein, A. A. (1997). The inclusion of students with disabilities and limited-English-proficient students inlarge-scale assessments: A summary of recent progress. (NCES Publication No. 97-482). Washington, DC: NationalCenter for Education Statistics.

Mazzeo, J., Carlson, J. E., Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of special needs students inNAEP: A report on 1996 research activities. (NCES Publication No. 2000-473). Washington, DC: National Centerfor Education Statistics.

design made it possible to study the effectson NAEP results of including special-needsstudents who required and were providedaccommodations, while at the same time,obtaining results that were comparable tothose from previous assessments. Based onresearch conducted and published sincethat time, it was determined that NAEPcould begin a transition to reporting resultsthat included the performance of accom-modated special-needs students.2 It isanticipated that in the near future, NAEPwill only report results that include accom-modated special-needs students.

Two Sets of 2001 NAEPGeography ResultsThis report is the first to display twodifferent sets of NAEP geography resultsbased on the split-sample design: 1) thosethat reflect the performance of regular andspecial-needs students when accommoda-tions were not permitted; and 2) those thatreflect the performance of regular andspecial-needs students—both those whowere accommodated and those who couldbe tested without accommodations—whenaccommodations were permitted. It shouldbe noted that accommodated studentsmake up a small proportion of the totalweighted number of students assessed (seetable A.6, page 124 in appendix A fordetails). Making accommodations availablemay change the overall assessment results in

subtle and different ways. For example,when accommodations are permitted, theremay be some occurrences of students beingaccommodated who might have taken thetest under standard conditions if accommo-dations were not permitted. This couldlead to an overall increase in the averageassessment results if accommodations wereto increase special-needs students’ perfor-mance. Conversely, when accommodationsare permitted, special-needs students whocould not have been tested without ac-commodations could be included in thesample. Assuming that these are generallylower-performing students, their inclusionin the sample—even with accommoda-tions—could result in an overall loweraverage score.

The two sets of results presented in thischapter were obtained by administering theassessment to a nationally representativesample of students and schools. In onesample, no accommodations were permit-ted; all students were assessed under thesame conditions that were the basis forreporting results from the 1994 NAEPgeography assessment. In another sample,accommodations were permitted for SDand/or LEP students who normally receiveaccommodations in their district or stateassessment programs. Most accommoda-tions that schools routinely provide fortheir own testing programs were permitted.


The permitted accommodations included,but were not limited to, the following:

■ one-on-one testing;

■ bilingual dictionary;

■ small-group testing;

■ extended time;

■ oral reading of questions/directions; and

■ use of an aide for transcribing responses.

(See appendix A, table A.7, page 126, forgreater detail on the numbers and percent-ages of students accommodated by accom-modation type in the 2001 assessment.)

Figure 5.1 provides a visual representa-tion of how the two sets of results werebased on the two samples in 2001. In-cluded in both sets of results (accommoda-tions not permitted and accommodationspermitted) are those students from both

samples of schools who were not identifiedas either SD and/or LEP. In addition, thefirst set of results (accommodations notpermitted) includes SD and/or LEP stu-dents from the sample of schools whereaccommodations were not permitted (seemiddle portion of figure 5.1). This is theset of results that allows for trend compari-sons back to 1994 and are presented in theother chapters of this report.

The second set of results, accommoda-tions permitted (see bottom portion offigure 5.1), includes SD and/or LEPstudents from the sample of schools whereaccommodations were permitted. This isthe set of results that form the new, moreinclusive baseline for future reporting oftrend comparisons for the NAEP geogra-phy assessment.


The two sets of NAEP results based on a split-sample design

Figure 5.1 Split-Sample Design

Split-sample designThe national sample was split. In part of theschools, accommodations were not permittedfor students with disabilities (SD) and limitedEnglish proficient (LEP) students. In the otherschools, accommodations were permitted forSD and LEP students who routinely receivedthem in their school assessments.

Accommodations-not-permitted resultsThe accommodations-not-permitted resultsinclude the performance of students from bothsamples who were not classified as SD or LEPand the performance of SD and LEP studentsfrom the sample in which no accommodationswere permitted.

Accommodations-permitted resultsThe accommodations-permitted results alsoinclude the performance of students from bothsamples who were not classified as SD or LEP;however, the SD and LEP students whoseperformance is included in this set ofresults were from the sample in whichaccommodations were permitted. Sincestudents who required testing accommodationscould be assessed and represented in theoverall results, it was anticipated that theseresults would include more special-needsstudents and reflect a more inclusive sample.

Sample with no Sample withaccommodations permitted accommodations permitted

Non-SD/LEP Non-SD/LEPstudents students

SD/LEP SD/LEPstudents students








In the NAEP 2001 sample where ac-commodations were not permitted, 16percent of the students in fourth grade, 16percent in eighth grade, and 11 percent intwelfth grade were identified by theirschools as having special needs (i.e., eitheras SD or LEP students). In the other samplewhere accommodations were offered, 17percent of the students in the fourth grade,16 percent of students in the eighth grade,and 10 percent in the twelfth grade wereidentified as having special needs. In thesample where accommodations were notpermitted, between 44 and 48 percent ofthe special-needs students at each of thethree grade levels (between 5 and 8 percentof all students—see appendix A, table A.5,page 123) were excluded from NAEPtesting by their schools. In the samplewhere accommodations were offered,between 23 and 24 percent of the special-needs students were excluded from theassessment (between 2 and 4 percent of thetotal sample).

Because the split-sample design was notused in 1994, trend data for accommodatedstudents are not available. Therefore, thischapter compares only the two sets ofresults from the 2001 geography assess-

ment. Overall results are provided for thenation and for student subgroups bygender and by race/ethnicity. These resultsare discussed in terms of statistically signifi-cant differences between the two sets ofresults and differences between subgroupsof students within each set of results.Throughout this chapter, the assessmentresults that include SD and/or LEP stu-dents for whom accommodations were notpermitted will be referred to as the “ac-commodations-not-permitted” results.The set of results that includes SD and/orLEP students for whom accommodationswere permitted will be referred to as the“accommodations-permitted” results.

Results for the NationAccommodations Not Permitted andAccommodations Permitted

Table 5.1 displays the average geographyscale scores for the nation in 2001 for twosets of results: 1) accommodations notpermitted, and 2) accommodations permit-ted. There were no statistically significantdifferences in the average scores betweenthe two sets of results at grades 4 and 12.At grade 8, however, the average scorewhen accommodations were permittedwas lower than the average score whenaccommodations were not permitted.


Table 5.1 Comparison of Two Sets of National Scale Score Results

National average geography scale scores by type of results, grades 4, 8, and 12: 2001

Accommodations not permitted Accommodations permitted

Grade 4 209 208

Grade 8 262 260 �

Grade 12 285 284

� Significantly different from the sample where accommodations were not permitted.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

As noted in the introduction to thischapter, NAEP has always sought to in-clude special-needs students proportionalto their representation in the U.S. popula-tion. Offering accommodations tends toreduce exclusion rates for special-needsstudents, and therefore allows NAEP tooffer a fairer and more accurate picture ofthe status of American education. Becausespecial-needs students are typically classi-fied as eligible for special educationalservices after having shown some difficultyin the regular learning environment, somemay assume that including the perfor-mance of these students would tend tolower the overall results. This assumptionappears to have been justified only in theobserved difference between the two setsof geography results in 2001 in grade 8,where the accommodations-permittedresults, which included slightly morespecial-needs students because of the

availability of accommodations, were lowerthan the accommodations-not-permittedresults. It is important to examine thepercentages of students attaining the NAEPachievement levels, however, to see if therewere higher percentages at the lowerperformance ranges (i.e., below Basic andBasic), when students were assessed withaccommodations.

Table 5.2 shows the percentages ofstudents attaining each of the achievementlevels. The percentages are similar acrossthe two sets of results for grades 4 and 12;apparent differences between the accom-modations-not-permitted and the accom-modations-permitted results were notsignificantly different. At grade 8, however,the percentage of students at or aboveBasic was higher when accommodationswere not permitted than when they werepermitted.


Percentage of students within and at or above geography achievement levels by type of results,grades 4, 8, and 12: 2001

Table 5.2 Comparison of Two Sets of National Achievement-Level Results

At or above At or above

Below Basic At Basic At Proficient At Advanced Basic Proficient

Grade 4Accommodations were not permitted 26 53 19 2 74 21

Accommodations were permitted 27 52 19 2 73 20


Accommodations were permitted 28 � 43 25 4 72 � 29



� Significantly different from the sample where accommodations were not permitted.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

National average geography scale scores by gender and type of results, grades 4, 8, and 12: 2001

Table 5.3 Comparison of Two Sets of National Scale Score Results by Gender

Male Female

Grade 4Accommodations were not permitted 212 207

Accommodations were permitted 210 206


Accommodations were permitted 262 � 258


Accommodations were permitted 287 281

�� Significantly different from the sample where accommodations were not permitted.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

National Results by GenderAccommodations Not Permitted andAccommodations Permitted

The average geography scale scores bygender for both sets of results in 2001 are

provided in table 5.3. Male students atgrade 8 had higher geography scores whenaccommodations were not permitted thanwhen accommodations were permitted.


As noted in chapter 3, in 2001, malestudents at all three grades where accom-modations were not offered had higherscale scores than female students. Thesame pattern continued where accommo-dations were offered—male students hadhigher average scale scores than femalestudents in all three grades.

The percentages of male and femalestudents attaining the Basic, Proficient, andAdvanced levels are provided in table 5.4.Comparing the two sets of results in 2001,there were no statistically significant differ-ences by accommodation status in thepercentages of male or female studentsattaining each of the achievement levels atgrades 4, 8, or 12.

National Results byRace/EthnicityAccommodations Not Permitted andAccommodations Permitted

NAEP assessments across academic subjectshave typically reported large score differ-

Table 5.4 Comparison of Two Sets of National Achievement-Level Results by Gender



Percentage of students within and at or above geography achievement levels by gender andtype of results, grades 4, 8, and 12: 2001

Grade 4MaleAccommodations were not permitted 25 51 21 3 75 24


FemaleAccommodations were not permitted 28 54 17 1 72 18










NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

ences according to race and ethnic groupmembership. If SD and/or LEP studentsare over-represented in a particular racial orethnic group, that group’s assessment scoresmay decrease. Table 5.5 provides the


National average geography scale scores by race/ethnicity and type of results, grades 4, 8, and 12:2001

Table 5.5 Comparison of Two Sets of National Scale Score Results by Race/Ethnicity

Asian/Pacific AmericanWhite Black Hispanic Islander Indian

Grade 4Accommodations were not permitted 222 181 184 212 199

Accommodations were permitted 220 181 185 216 199






average geography scale scores for each ofthe race/ethnicity categories for the twosets of results in 2001. There were nostatistically significant differences in average

As noted in chapter 3, a pattern ofperformance differences by race/ethnicitycan be seen in the accommodations-not-permitted results in 2001. Both White andAsian/Pacific Islander students at all threegrades scored higher, on average, thanBlack and Hispanic students. The samepattern can be observed at all three gradesin the accommodations-permitted results.In addition, Hispanic students had higherscores than Black students regardless ofaccommodations condition at grades 8 and12. At grade 4, however, regardless ofaccommodation conditions, there was nosignificant difference between the scores ofBlack and Hispanic students.

The relative standing of the performanceof American Indian students changes bygrade level. At grade 4, American Indianstudents had lower average scores thanWhite students regardless of accommoda-tion condition. At grades 8 and 12, how-ever, the scores of American Indian stu-dents were not significantly different thanWhite students within either accommoda-tion condition.

The percentages of students in eachrace/ethnicity category who attained theBasic, Proficient, and Advanced levels areprovided in table 5.6. No statisticallysignificant differences were found at any ofthe three grades between the accommoda-tions-not-permitted results and the accom-modations-permitted results for the per-centages of students attaining each of theachievement levels in 2001.

scores for any racial/ethnic group at anygrade, between the samples where accom-modations were not permitted and whereaccommodations were permitted.


Table 5.6 Comparison of Two Sets of National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levels by race/ethnicity andtype of results, grades 4, 8, and 12: 2001



# Percentage is between 0.0 and 0.5.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Grade 4WhiteAccommodations were not permitted 13 58 26 3 87 29

Accommodations were permitted 15 57 25 3 85 28BlackAccommodations were not permitted 56 39 5 # 44 5

Accommodations were permitted 56 40 4 # 44 4HispanicAccommodations were not permitted 51 43 6 # 49 6

Accommodations were permitted 49 45 5 # 51 6Asian/Pacific IslanderAccommodations were not permitted 23 52 23 1 77 25

Accommodations were permitted 18 57 24 2 82 25American IndianAccommodations were not permitted 34 53 13 # 66 13

Accommodations were permitted 37 51 12 # 63 12



Accommodations were permitted 62 32 6 # 38 6HispanicAccommodations were not permitted 52 38 9 1 48 10

Accommodations were permitted 54 37 9 1 46 9Asian/Pacific IslanderAccommodations were not permitted 21 47 28 4 79 32

Accommodations were permitted 20 49 28 4 80 32American IndianAccommodations were not permitted 28 41 29 3 72 31




Accommodations were permitted 67 30 3 # 33 3HispanicAccommodations were not permitted 48 42 10 # 52 10

Accommodations were permitted 50 42 9 # 50 9Asian/Pacific IslanderAccommodations were not permitted 28 45 25 1 72 26

Accommodations were permitted 29 46 23 1 71 25American IndianAccommodations were not permitted 26 41 31 1 74 32



6Chapter

Contents

SampleQuestions

StudentResponses

Item Maps

Sample materialsfrom the 2001geographyassessment

ChapterFocus

Sample Assessment QuestionsAnd Student Responses

This chapter presents sample questions from the 2001

NAEP geography assessment. Four sample questions at each

grade are provided, including multiple-choice and

constructed-response questions. Each sample is classified

according to its geography content area, as described in the

geography framework. The constructed-response questions

are accompanied by actual student responses, reproduced

from test booklets, that illustrate work at different

rating levels. The constructed-response samples were

rated using either a three-point or four-point scoring

rubric. Three-point questions were rated as

“Complete,” “Partial,” or “Inappropriate.” Four-point

questions were rated as “Complete,” “Essential,”

“Partial,” or “Inappropriate. ” Sample responses are

included for each level except “Inappropriate.”

The table accompanying each sample question

presents two types of performance data: the overall

percentage of students who answered successfully,

and the percentage of students who answered

successfully within a specific score range on the

NAEP geography scale. The score ranges correspond

to the three achievement-level intervals—Basic, Proficient, and

Advanced—as well as the range below Basic. These

percentages give some indication of how difficult the

question was for students who performed within each of the

achievement-level ranges.


Many additional sample questionsreleased from the 1994 and 2001 NAEPgeography assessments are available forviewing on the NAEP Web Site at http://www.nces.ed.gov/nationsreportcard/itmrls/. The item-viewing feature of theWeb Site includes student performancedata for all questions, detailed scoringguides (rubrics), and sample student re-sponses for the constructed-responsequestions.

Grade 4 Sample AssessmentQuestions and ResultsQuestions in the grade 4 assessment cover awide variety of geographic concepts andskills across the three geography contentareas. A somewhat higher percentage ofquestions is devoted to United Statesgeography than at the two higher gradeswhere increasing emphasis is placed onworld geography.

Many of the questions at all three gradesare based upon visual or textual stimulidesigned to make the assessment moreinteresting and more authentic. Visualstimuli include maps, charts, graphs,diagrams, cartoons, and, as in samplequestion 1, photographs.

The sample questions are also markedon the item maps on pages 110-112. Theitem map location of each question identi-fies the scale score at which that questionwas answered successfully by at least 65percent of the students for constructed-response questions and 74 percent of thestudents for four-option multiple-choicequestions.


Look at the photograph above. What would help farmers in this area growmore food?

A Cutting down forests

B Making terraces

C Building houses

Irrigating the land

In sample question 1, students are assessed on whether they can recognize a photographicrepresentation of a landscape and associate irrigation with the landscape depicted. Thisquestion is mapped at scale score 216.

Table 6.1 Sample Question 1 Results (Multiple-Choice)

Overall percentage correct and percentages correct within each achievement-level range: 2001

*NAEP geography composite scale range.***Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Percentage correct withinachievement-level intervals

Overall percentage Below Basic Basic Proficient Advancedcorrect 186 and below* 187–239* 240–275* 276 and above*

70 50 74 84 ***

Grade 4

© Fritz Henle/Photo Researchers, Inc.

Geography Content Area: Environment and Society

Grade 4 Sample Question 1:


Which two nations are most likely to have a conflict over mineral resources?

A Nation A and Nation B

B Nation A and Nation C

Nation A and Nation D

D Nation C and Nation D

Sample question 2 measures students’ understanding of how geography plays a role inconflict among countries. Students found this question to be quite difficult, with onlyone-third answering correctly. This question appears on the item map at scale score 271.






33 22 28 56 ***

Grade 4

Geography Content Area: Spatial Dynamics and Connections



Sample question 3 is one of a number of production tasks included in the NAEP geographyassessment in which students are asked to locate a place on a map or draw a map in their testbooklet. Responses to this question were scored with a three-level rubric as “Complete,” “Par-tial,” or “Inappropriate.” Two-thirds of students could correctly identify where they lived. Thisquestion appears on the item map at scale score 192. (Note that the circled numbers on themap were used in a different question that was also based on this map.)

Write down the name of the state or district where you live.

I live in______________________________________________ .

Directly on the map, draw an “X” on the state or district where you live.

Geography Content Area: Space and Place



To earn a score of “Complete” on this question, students had to write the name of the stateor district where they live and correctly mark the location on the map.

Sample “Complete” Response:


I live in______________________________________________ .


Table 6.3a Sample Question 3 Results (“Complete” Short-Constructed-Response)

Overall percentage “Complete” and percentages “Complete” within each achievement-level range:2001


Percentage “Complete” withinachievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced“Complete” 186 and below* 187–239* 240–275* 276 and above*

66 38 71 88 ***

Grade 4



I live in______________________________________________ .


To earn a score of “Partial,” students could indicate their state or district and mark a border-ing state, or they could indicate the city or town in which they live and mark the correct statein which that city lies. In the sample below, the student lives in North Carolina but markedVirginia on the map.

Sample “Partial” Response:

Table 6.3b Sample Question 3 Results (“Partial” Short-Constructed-Response)

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-level range: 2001


Percentage “Partial” or better withinachievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced“Partial” or better 186 and below* 187–239* 240–275* 276 and above*

72 43 78 93 ***

Grade 4


LITTLE TOWN

– Width: 4.0 miles east to west– Length: 3.0 miles north to south– Main Street runs east to west through the town.– The school is on the northeast side of town.– Phelps Park is on the southwest side of town.– Runt River runs north to south through the town.

On the grid below, each square is one mile wide and one mile long. Draw amap of Little Town on the grid. Draw the town’s borders. Then, use thesymbols in the key below to draw the features listed above.

A more complex production task is seen in sample question 4. Here, students must usewritten descriptions of features of a town to sketch a map on a grid. They must understandscale, distance, and direction, and be able to read and use a map key. Responses werescored with a four-level rubric as “Complete,” “Essential,” “Partial,” or “Inappropriate.” Thequestion was difficult for fourth-graders, with only 28 percent providing an “Essential” orbetter response. The item map scale score point for this question is 295.

N

S

EW

Scale

= 1 mile

KeyS

P

School

Street

Park

River




Responses scored “Complete” correctly located all four features and drew the length andwidth to scale in the correct directions.


N

S

EW

Scale

= 1 mile

KeyS

P

School

Street

Park

River


Table 6.4a Sample Question 4 Results (“Complete” Extended-Constructed-Response)



11 0 6 32 ***


Grade 4


Responses scored “Essential” correctly located four features but not to scale, or correctlylocated three features and had the scale correct.

Sample “Essential” Response:

N

S

EW

Scale

= 1 mile

Key

S

P

School

Street

Park

River

Overall percentage “Essential” or better and percentages “Essential” or better within each achievement-level range: 2001

Table 6.4b Sample Question 4 Results (“Essential” Extended-Constructed-Response)

Percentage “Essential” or better withinachievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced“Essential” or better 186 and below* 187–239* 240–275* 276 and above*

28 1 25 65 ***


Grade 4


N

S

EW

Scale

= 1 mile

KeyS

P

School

Street

Park

River

Overall percentage “Partial” or better and percentages “Partial” or better within each achievement-levelrange: 2001

Table 6.4c Sample Question 4 Results (“Partial” Extended-Constructed-Response)



38 4 36 78 ***


Grade 4

Responses scored “Partial” located only one or two features and had the scale correct, orlocated three features with an incorrect scale.



Grade 8 Sample AssessmentQuestions and ResultsThe assessment at grade 8, like that atgrade 4, covers a wide range of geographyskills and concepts. The questions, on

average, look for a deeper understanding ofthe material and require students to grapplewith more sophisticated stimuli, comparemultiple maps, and apply geographicunderstanding to solving problems.




*NAEP geography composite scale range.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



70 37 74 91 97

Grade 8

Which question could you answer based only on the information in the map?

A At what times do the public trains arrive?

B How much time does it take to go from Forest Hills to Oak Grove?

C How many miles is it from one station to another?

How can one travel from Alewife to the Aquarium by public train?



In this multiple-choice question students are asked to interpret a kind of map they may neverhave seen to determine exactly what kind of information it provides and doesn’t provide. Itwas a fairly easy task for students. The scale score point for this question on the eighth-gradeitem map is 257.







50 36 47 64 ***

Grade 8

How do the forces listed below affect the natural environment?

GravityIce

WaterWind

They are major causes of erosion.

B They are important influences on human settlement.

C They are responsible for seismic activity.

D They cause continental drift.



Sample question 6 asks about an important aspect of physical geography. One-half of eighth-graders knew that the four forces contribute to erosion. This question maps at scale scorepoint 316.







74 40 80 93 100

Grade 8

Florida is an example of

A an isthmus

B an island

a peninsula

D a plateau



Sample question 7 tests students’ knowledge of landforms as well as their skill with whatgeographers call “mental mapping”—the ability to visualize spatial patterns in one’s mind.Students had to create an image of Florida in their minds before they could identify it as apeninsula. Nearly three-quarters of the students answered correctly. The question maps at ascale score of 256.







60 40 57 79 96

Grade 8



Sample question 8 measures students’ understanding of why countries join trans-regionalorganizations, a topic related to the larger theme of how people from different places worktogether across space to address common issues. Sixty percent of students answered thismoderately difficult question correctly. The item map scale score for this question is 285.

What is an important reason that countries join internationalorganizations like the United Nations?

A Countries who do not join usually lose their independence.

Many of the world’s problems involve more than one country.

C Most citizens want their countries to join as many internationalorganizations as possible.

D Such organizations force countries to join.


Tropical forests are being destroyed at the rate of at least eleven millionhectares each year, an area the size of Pennsylvania. About half of alltropical forests are already gone.

Discuss two major reasons for this high rate of tropical deforestation.



Sample question 9 is a short-constructed-response question designed to measure students’understanding of the interaction between human beings and the environment. Responseswere scored on a three-level rubric as “Complete,” “Partial,” or “Inappropriate.” The ques-tion was quite difficult for students, with only 22 percent giving a “Complete” response. Onthe item map for eighth grade this question appears as scale score 328.


Responses scored “Complete” provided two reasons for the high rate of tropical deforesta-tion. Reasons could relate to demand for land and resources or to the lack of regulation thatallows deforestation to occur.





22 6 18 38 ***




Grade 8


Responses scored “Partial” provided only one reason for the high rate of tropical deforesta-tion, thereby revealing a more limited knowledge of the subject.





60 26 62 84 ***




Grade 8







78 46 86 99 ***

Grade 12

The varying widths of the lines on the map most probably indicate the

A strength of ocean currents

B type of trade

volume of trade

D type of transportation used



Grade 12 Sample AssessmentQuestions and ResultsThe grade 12 assessment included higherpercentages of extended-constructed-

response questions and questions devotedto non-U.S. geography than the assessmentsat grades 4 and 8. It also contained the mostcomplex stimuli and challenging concepts.

Sample question 10 is a skills question designed to measure whether students understandthe conventions used in what is known as a flow map. A majority of students (78 percent)successfully answered the question. This question appears on the twelfth-grade item map atscale score 272.


What religion is practiced by most people who live in India?

A Confucianism

B Buddhism

C Christianity

Hinduism






61 46 62 76 ***

Grade 12



This straightforward multiple-choice question helps measure students’ knowledge of thedistribution of world religions. Six out of ten students answered correctly. The item map scalescore point for this question is 318.


Sample question 12 is a map-based, short-constructed-response question dealing with theinteraction between humans and the natural environment. Although some students may havebeen able to answer without referring to the map, others could use it to gain valuable infor-mation about the region. Responses were scored on a three-level rubric as “Complete,”“Partial,” or “Inappropriate.” The question was moderately difficult, with 47 percent ofstudents providing a “Complete” response. This question “maps” at scale score 300 for“Complete.”

Give two reasons why early civilizations flourished in the valley ofthe Tigris and Euphrates rivers.



Eu phra tes

0 600 miN

S

EW

SOUTHWEST ASIA

Egypt

Turkey

SyriaLebanon

JordanIsrael Iraq

SaudiArabia

Iran

PersianGulf

Tigris

Qatar

United Arab

Emirates

Oman

Afg

hani

stan

600 km0

Key

= National Boundaries = Rivers

Kuwait


Responses scored “Complete” gave two valid reasons why river valleys were important to theearly civilization of Iraq.



Percentage “Complete”withinachievement-level intervals


47 17 52 70 ***



*NAEP Geography composite scale range.***Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12


Responses scored “Partial” gave only one valid reason for the importance of the river valleyto the early civilization of Iraq.





76 42 85 96 ***



*NAEP Geography composite scale range.***Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12


Describe the difference in population patterns for people age 60and over in countries 1 and 2. Give one possible explanation for thedifference you have identified.



Sample question 13 is a short-constructed-response that measures students’ ability to readand understand population pyramids. Responses were scored on a three-point rubric as“Complete,” “Partial,” or “Inappropriate.” Students found this question to be very difficult.Sixteen percent received a score of “Complete.” This question maps at scale score 347 for“Complete.”

Age% ofTotalPop’n

70 + 1.0%60-69 1.6%50-59 2.6%40-49 3.9%30-39 5.6%20-29 7.7%10-19 10.4%0-9 17.0%

Male

COUNTRY 1 Age Distribution

Female

Age% ofTotalPop’n

70 + 1.2%60-691.8%50-592.7%40-494.0%30-395.5%20-297.7%10-1910.4%0-916.9%

Age% ofTotalPop’n

70 + 2.9%60-69 3.7%50-59 4.7%40-49 5.8%30-39 8.2%20-29 9.3%10-19 7.5%0-9 7.3%

Male

COUNTRY 2 Age Distribution

Female

Age% ofTotalPop’n

70 + 4.2%60-694.3%50-594.8%40-495.7%30-398.3%20-299.2%10-197.1%0-97.0%




16 2 15 33 ***




Grade 12

Responses scored “Complete” had to accurately describe the difference between the popula-tion patterns for people age 60 and over in the two countries and give a plausible explanationfor the difference.


Describe the difference in population patterns for people age 60and over in countries 1 and 2. Give one possible explanation forthe difference you have identified.




51 18 57 79 ***




Grade 12

Responses scored “Partial” either described the difference between the two populationpyramids but did not explain the difference or, as in the following example, incorrectlydescribed the difference as one of absolute numbers rather than percentages of the popula-tion and gave a plausible explanation.


Describe the difference in population patterns for people age 60and over in countries 1 and 2. Give one possible explanation forthe difference you have identified.


Maps of Selected ItemDescriptions on theNAEP Geography Scale –Grades 4, 8, and 12The geography performance of fourth-,eighth-, and twelfth-graders can be illus-trated by maps that position item descrip-tions along the NAEP geography scalewhere they are likely to be answeredsuccessfully by students.1 The descriptionsused on the item maps focus on the geog-raphy knowledge or skill needed to answerthe question. For multiple-choice ques-tions, the description indicates the knowl-edge or skill demonstrated by selection ofthe correct option; for constructed-re-sponse questions, the description takes intoaccount the knowledge or skill specified bythe different levels of scoring criteria forthat question. The questions described onthe item maps include the 12 samplequestions in the preceding section.

Figures 6.1 through 6.3 are item mapsfor grades 4, 8, and 12, respectively. Theitem map location of each questionidentifies the scale score at which thatquestion was answered successfully by atleast 65 percent of the students forconstructed-response questions and 74percent of the students for four-option,multiple-choice questions. For eachquestion indicated on the item map,students whose average score fell at orabove the scale point had a higherprobability of successfully answering thequestion. Students whose average scorefell below that scale point had a lowerprobability of successfully answering thequestion.

As an example of how to interpret theitem maps, consider the multiple-choicequestion in figure 6.1 that maps at scorepoint 271. This question appeared assample question 2 earlier in the chapter,and was shown to have been a difficultquestion answered correctly by 33 percentof students. Students whose geographyability corresponds to a score of 271 orabove on the scale had at least a 74 percentprobability of answering this questioncorrectly. Students whose ability is repre-sented by a score below 271 had less than a74 percent probability of answering cor-rectly. This does not mean that all of theformer students answered the questioncorrectly or that all of the latter studentsanswered it incorrectly. Rather, the itemmap indicates higher or lower probabilityof answering correctly depending onstudents’ overall geography ability as mea-sured on the NAEP scale.

The three geography achievement levelsare indicated on the item map for eachgrade. It is important to note that, althoughthe same 0-500 geography scale is used ateach grade, the achievement levels aregrade-specific, and each achievement levelbegins at a different score point at eachgrade. Returning to the example of thequestion mapping at score point 271, theitem map is useful in showing how thisdifficult question maps relatively high upon the scale. In terms of achievement levels,one sees that students with a 74 percentprobability of answering the questioncorrectly performed near the upper end ofthe Proficient achievement-level range.

1 Details on the procedures used to develop item maps are provided in appendix A.


NOTE: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question.* Each grade 4 geography question in the 2001 assessment was mapped onto the NAEP 0–500 geography scale. The position of the question on the scale represents the scale scoreattained by students who had a 65 percent probability of successfully answering a constructed-response question, or a 74 percent probability of correctly answering a four-optionmultiple-choice question. Only selected questions are presented. Scale score ranges for geography achievement levels are referenced on the map. For constructed-response questions, thequestion description represents students’ performance at the scoring criteria level being mapped.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of Educational Progress (NAEP),2001 Geography Assessment.

NAEP Geography Scale

320310300290280270260250240230220210200190180170160

Figure 6.1

Grade 4Item Map

Map of selected itemdescriptions on theNational Assessmentof EducationalProgress (NAEP)geography scale forgrade 4

This map describesthe knowledge or skillassociated withanswering individualgeography questions.The map identifiesthe score point atwhich students had ahigh probability ofsuccessfullyanswering thequestion.*

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Proficient240

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Advanced276

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Basic187

320 Interpret resource map to determine likely location for large city to develop

295 Draw map based on written description of its features—Sample Question 4

288 Identify mountain range in which Switzerland is located

276 Use multiple maps to locate states where crops grow year round

271 Use map to determine which countries might have a conflict over resources—Sample Question 2269 Use multiple maps to compare conditions for farming in two countries

259 Interpret information given in a transit map

255 Find and draw specified route on a transit system map251 Identify a megalopolis on a population map249 Determine elevation of a region on a physical map

244 Identify world’s largest ocean242 Interpret a specialized map of irrigation

230 Compare climate and land use of two countries based on three maps

225 Locate bordering countries on a political map

221 Identify Mississippi River on map of North America218 Locate place with specified features on physical map218 Use resource map to explain where steel industry would develop216 Recognize desert landscape in a photograph and need for irrigation to grow food

—Sample Question 1207 Identify capital city on a political map

202 Distinguish activities associated with large cities and small towns

195 Recognize features of dry climate in a photograph

192 Identify exact or approximate location of home state on map of United States—Sample Question 3

187 Use map to determine products traded between two countries

182 Identify some land forms on map

175 Recognize type of land use shown in photograph

161 Interpret a simple population pie chart




360350340330320310300290280270260250240230220210200

Figure 6.2

Grade 8Item Map


This map describesthe knowledge or skillassociated withanswering individualgeography questions.The map identifiesthe score point atwhich students had ahigh probability ofsuccessfullyanswering thequestion.*

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Basic242

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Proficient282

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Advanced315

359 Describe and explain differences in population pyramids

348 Use a time zone map

338 Use map to explain international trade in oil

328 Explain two reasons for high rate of tropical deforestation—Sample Question 9

319 Use multiple maps to explain land use in Canada region316 Recognize the natural forces that cause erosion—Sample Question 6314 Use map to explain historical shift in center of U.S. population309 Identify purpose of OPEC309 Understand and compare different views on land ownership303 Interpret resource map to determine likely location for large city to develop301 Use map to help explain two reasons why early civilizations developed in Fertile Crescent

297 Use atlas to find some information about urbanization295 Identify an economic impact of EL Niño on Peru291 Apply concept of interior to locate capital city on political map288 Use map to determine which countries might have a conflict over resources285 Recognize why countries join international organizations—Sample Question 8283 Use a political map and a land use map to locate an African city

278 Compare states using rainfall and growing season maps

271 Explain one reason for high rate of tropical deforestation270 Understand how to read a population pyramid

267 Use map to explain one reason why early civilizations developed in Fertile Crescent

262 Recognize fault line on a map

257 Determine direct or nearly direct route between two points on a transit map—Sample Question 5256 Identify Florida as a peninsula—Sample Question 7

250 Locate Lake Superior on map of North America

240 Identify city closest to earthquake epicenter on map

227 Use map to identify large U.S. trading partner

217 Use resource map to explain where steel industry would develop

207 Locate home state on map




380

370

360

350

340

330

320

310

300

290

280

270

260

250

240

Figure 6.3

Grade 12Item Map


This map describesthe knowledge orskill associated withanswering individualgeography questions.The map identifiesthe score point atwhich students had ahigh probability ofsuccessfullyanswering thequestion.*

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Advanced339

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Proficient305

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Basic270

377 Use data and maps to explain Mongolia’s economic development

350 Explain reasons for international trade in oil347 Explain differences between two countries using population pyramids—Sample Question 13345 Use atlas to explain regional variations in land use

337 Use map to explain historical shift in center of U.S. population333 Use map to explain economic impact of Mid-East War331 Use multiple maps to describe regions where most Australians live

325 Explain high rate of tropical deforestation

321 Use map and charts to compare urbanization in two European countries319 Use a time zone map318 Identify most widely-practiced religion in India—Sample Question 11314 Identify oil as product depicted on map of international trade311 Use climate map to locate countries in tropical zone

305 Use map and charts to identify source of forest products for Japan301 Identify purpose of OPEC300 Explain reasons why Fertile Crescent was home to early civilizations—Sample Question 12299 Use multiple maps to determine U.S. region with highest population density

295 Use map to explain geographic distribution of languages

287 Identify an economic impact of El Niño on Peru284 Define the characteristic of a region282 Locate natural hazards on map and explain their impact280 Use map and data to evaluate an environmental threat

276 Explain siting of cities272 Recognize how to read a flow map—Sample Question 10271 Read a population pyramid

267 Draw partially accurate map based on written description

258 Identify an area of flat land on contour map

253 Recognize that Richter scale is used for measuring earthquake intensity

242 Use map to locate area likely to suffer earthquake damage

A P P E N D I X A • G E O G R A P H Y R E P O R T C A R D 113

AAppendixContents

The Assessment

The Samples

Students withDisabilities (SD)

andLimited EnglishProficient (LEP)

Students

Data Collection

Data Analysis

NAEP ReportingGroups

Cautions inInterpretations

1 National Assessment Governing Board (1994). Geography framework for the 1994 and 2001National Assessment of Educational Progress. Washington, DC: Author.

Technical aspects ofthe NAEP 2001geographyassessment

AppendixFocus

Appendix AOverview of Procedures Used for theNAEP 2001 Geography Assessment

This appendix provides an overview of the NAEP 2001

geography assessment’s primary components—framework,

development, administration, scoring, and analysis. A more

extensive review of the procedures and methods used in the

geography assessment will be included in the forthcoming

NAEP 2001 Technical Report.

The NAEP 2001 Geography AssessmentThe National Assessment Governing Board (NAGB),

created by Congress in 1988, is responsible for

formulating policy for NAEP. The NAGB is

specifically charged with developing assessment

objectives and test specifications through a national

consensus approach. That consensus approach results

in the development of an assessment framework. The

design of the NAEP 2001 geography assessment

followed the guidelines provided in the framework

developed for the 1994 assessment.1

The framework underlying both the NAEP

1994 and 2001 assessments reflects consensus among

educators and researchers about the study of geography. Its

purpose is to present a comprehensive overview of the most

essential outcomes of students’ geography education.

Developing this framework and the specifications that

guided development of the assessment involved the critical

114 A P P E N D I X A • G E O G R A P H Y R E P O R T C A R D

Target and actual percentage distribution of questions by grade and geography content area, grades4, 8, and 12: 1994 and 2001

Table A.1 Distribution of Questions

Space and Place 40 42 48 40 39 40 40 42 38

Environment andSociety 30 28 24 30 30 32 30 30 35

Spatial Dynamicsand Connections 30 31 28 30 32 28 30 29 27



Actual Actual Actual Actual Actual ActualContent Areas Target 1994 2001 Target 1994 2001 Target 1994 2001

input of hundreds of individuals across thecountry, including representatives ofnational education organizations, teachers,parents, policymakers, business leaders, andthe interested general public. This consen-sus process was managed by the Council ofChief State School Officers for NAGB.

The assessment framework specified notonly the particular content areas of geogra-phy to be measured (see chapter 1 for adescription of these dimensions), but alsothe percentage of assessment questions thatshould be devoted to each. The targetpercentage distribution of content areas, asspecified in the framework, along with theactual percentage distributions in the 1994and 2001 assessments, are presented in tableA.1. The targeted content mix of 40percent Space and Place, 30 percent Envi-ronment and Society, and 30 percentSpatial Dynamics and Connections washeld constant across all three grades. Theactual content of the assessment in terms ofpercentage of time spent by students wasgenerally within a few percentage points ofthe targeted distribution in both assessmentyears. Such variation across years in item

classification distribution does not affectthe reporting of trends in student perfor-mance. Trend reporting is based upon theunderlying scale, which uses the commonitems (i.e., those used in both assessmentyears), but maintains its stability even ifsome items are dropped or replaced. More-over, the weighting of subscales in derivingthe composite scale is based on the targetitem classification distribution.

The Assessment DesignEach student who participated in thegeography assessment received a bookletcontaining three or four sections: a set ofgeneral background questions, a set ofsubject-specific background questionsdealing largely with the student’s use oftechnology, and one or two sets, or“blocks,” of cognitive questions assessingknowledge and skills in geography asoutlined in the framework. At grades 8and 12, students were given either two 25-minute blocks or one 50-minute block. Atgrade 4, however, only 25-minute blockswere used. At each grade, one of the 25-minute blocks of questions required the useof an atlas, which was provided.


2 These blocks were distributed across the student booklets in a Balanced Incomplete Block (BIB) design that isdescribed later in this section.

At grade 4, a total of six 25-minuteblocks of cognitive questions were given,while at grades 8 and 12, seven blocks (six25-minute blocks and one 50-minuteblock) were administered.2 Some of theblocks at each grade (three at grade 4, andfour at grades 8 and 12) were carriedforward from the 1994 assessment to the2001 assessment to allow for the measure-ment of changes across time. Each blockconsisted of both multiple-choice andconstructed-response questions. Short-constructed-response questions required afew sentences for an answer, whileextended-constructed-response questionsgenerally required a paragraph or more.Some of the constructed-responsequestions required students to createmaps or graphics. It was expected thatstudents could adequately answer theshort-constructed-response questions inabout two to three minutes and theextended-constructed-response questionsin about five minutes. The 50-minute

Distribution of questions administered by question type, grades 4, 8, and 12: 1994 and 2001

Table A.2 Distribution of Questions by Question Type

Multiple-choice 59 63 84 85 85 86

Short constructed-response 23 21 32 30 25 24

Extended constructed-response 8 7 9 9 13 13

Total 90 91 125 124 123 123



1994 2001 1994 2001 1994 2001

blocks contained questions focusing ona particular theme, and included twoextended-constructed-response questions.Only one 50-minute block was adminis-tered at each of grades 8 and 12.

Table A.2 displays the number of ques-tions by type and by grade level for the1994 and 2001 assessments. Some of thequestions were used at more than onegrade level; thus, the sum of the questionsthat appears at each grade level is greaterthan the total number of unique questions.The total number of questions at eachgrade level varied little from 1994 to 2001,despite the release to the public of severalblocks at each grade level and attendantreplacement with new blocks of questions.It should be noted that any such variationacross years does not affect NAEP’s abilityto report on changes in students’ perfor-mance across years because this reporting isbased on the presence of blocks that werecommon to the assessment in two years.


3 For further details on the booklet design, see the forthcoming NAEP 2001 Technical Report.

The assessment design allowed for maxi-mum coverage of geography content atgrades 4, 8, and 12, while minimizing thetime burden for any one student. This wasaccomplished through the use of matrixsampling of cognitive questions, in whichrepresentative samples of students tookdifferent portions of the entire pool ofassessment questions. The aggregate resultsacross the entire assessment allowed forbroad reporting of the geography perfor-mance of the targeted population. Matrixsampling did not apply to backgroundquestions; each student received all thebackground questions appropriate for hisor her grade.

In addition to matrix sampling, theassessment design utilized a procedure fordistributing test booklets that controlled forposition and context effects. Studentsreceived different blocks of questions intheir booklets according to a procedurecalled “Balanced Incomplete Block (BIB)spiraling.” This procedure assigns blocks ofquestions so that every block appears in thefirst or second position within a booklet anequal number of times. Every block ofquestions is paired with every other block,with the exception of the 50-minutetheme block, which appears on its ownwithout another block of cognitive ques-tions. The spiraling aspect of this proce-dure cycles the booklets for administration,so that typically only a few students in anyassessment session receive the same booklet.

This design allows for some balancing ofthe impact of context and fatigue effects tobe measured and reported, but makesallowance for the difficulties of administer-ing the 50-minute blocks.3

In addition to the student assessmentbooklets, three other instruments provideddata relating to the assessment: a teacherquestionnaire, a school questionnaire, and aStudents with Disabilities and/or LimitedEnglish Proficiency (SD and/or LEP)questionnaire. The teacher questionnairewas administered to the geography or socialstudies teachers of fourth- and eighth-grade students participating in the assess-ment. The questionnaire consisted of threesections and took approximately 20 min-utes to complete. The first section focusedon the teacher’s general background andexperience; the second section on com-puter resources available in the school; andthe third section on classroom informationabout geography/social studies instruction.

The school characteristics and policyquestionnaire was given to the principal orother administrator in each participatingschool and took about 20 minutes tocomplete. The questions asked aboutschool policies, programs, facilities, anddemographic composition and backgroundof the student body.

The SD and/or LEP student question-naire was completed by a school staffmember knowledgeable about those


students who were selected to participatein the assessment and who were identifiedas: 1) having an Individualized EducationProgram (IEP) or equivalent program (forreasons other than being gifted and tal-ented) or 2) being limited English profi-cient (LEP). A questionnaire was com-pleted for each SD and/or LEP studentsampled regardless of whether the studentparticipated in the assessment. Each ques-tionnaire took approximately 3 minutes tocomplete and asked about the student andthe special programs in which he or sheparticipated.

National SampleThe national results presented in this reportare based on nationally representativeprobability samples of fourth-, eighth-, andtwelfth-grade students. The sample waschosen using a multistage design thatinvolved sampling students from selectedschools within selected geographic areasacross the country. The sample design hadthe following stages:

1) selection of geographic areas (a county,group of counties, or metropolitanstatistical area);

2) selection of schools (public andnonpublic) within the selected areas; and

3) selection of students within selectedschools.

4 Additional details regarding the design and structure of the national and state samples will be included in theforthcoming NAEP 2001 Technical Report. In addition, the reader may consult the NAEP 2000 Technical Report for adiscussion of sampling procedures that are mostly common to all NAEP assessments.

Each selected school that participated inthe assessment and each student assessedrepresents a portion of the population ofinterest. Sampling weights are needed tomake valid inferences between the studentsamples and the respective populationsfrom which they were drawn. Samplingweights account for disproportionaterepresentation due to the oversampling ofstudents who attend schools with highconcentrations of Black and/or Hispanicstudents and students who attendnonpublic schools. Among other uses,sampling weights also account for lowersampling rates for very small schools andare used to adjust for school and studentnonresponse.4

Unlike the 1994 national assessment, aspecial feature of the 2001 national assess-ment was the collection of data fromsamples of students where assessmentaccommodations for special-needs studentswere not permitted and from samples ofstudents where accommodations forspecial-needs students were permitted.NAEP inclusion rules were applied, andaccommodations were offered only when astudent had an Individualized EducationProgram (IEP) because of a disability and/or was identified as being a limited Englishproficient student (LEP); all other studentswere asked to participate in the assessmentunder standard conditions.


Table A.3 shows the number of studentsincluded in the national samples for theNAEP 1994 and 2001 geography assess-ments at each grade level. For the 2001assessment, the table includes the numberof students in the sample where accommo-dations were not permitted and the num-ber of students in the sample whereaccommodations were permitted. Thetable shows that the same non-SD and/orLEP students were included in both

National student sample size by type of results, grades 4, 8, and 12: 1994 and 2001

Table A.3 National Student Sample Size

1994 2001Accommodations Accommodations Accommodations

not permitted not permitted permittedsample sample sample

Grade 4Non SD/LEP students assessed 5,045 6,375

SD/LEP students assessedwithout accommodations 462 551 476

SD/LEP students assessedwith accommodations NA NA 368

Total students assessed 5,507 6,926 7,219



SD/LEP students assessed with accommodations NA NA 397




SD/LEP students assessedwith accommodations NA NA 188


SD = Students with Disabilities.LEP = Limited English Proficient students.NA = Not applicable. No accommodations were permitted in this sample.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

samples in 2001; only the SD and/or LEPstudents differed between the two samples.The 1994 design differed somewhat in thatthe SD and/or LEP students were assessedin standard conditions and accommoda-tions were not permitted.

Table A.4 provides a summary of thenational school and student participationrates for the geography assessment sampleswhere accommodations were not per-mitted and where accommodations were


5 The initial base sampling weights were used in weighting the percentages of participating schools and students. Anattempt was made to preselect (before field processes began) a maximum of two substitute schools for each sampledpublic school (one in-district and one out-of-district) and each sampled Catholic school, and one for each samplednonpublic school (other than Catholic). To minimize bias, a substitute school resembled the original selection asmuch as possible on affiliation, estimated number of grade-eligible students, and minority composition.

National school and student participation rates for public schools, nonpublic schools, and publicand nonpublic schools combined, grades 4, 8, and 12: 2001

Table A.4 Participation Rates

Samples where accommodations Samples where accommodationsWeighted school participation were not permitted were permitted

Student participation Overall participation rate Student participation Overall participation rate

Weighted Total Weighted TotalPercentage Percentage Total percentage number of percentage number of

before after number student students Before After student students Before Aftersubstitution substitution of schools participation assessed substitution substitution participation assessed substitution substitution

Grade 4Public 83 88 276 95 5,895 79 84 95 6,181 79 84

Nonpublic 83 91 89 96 1,031 80 87 96 1,038 80 88Combined 83 88 365 95 6,926 79 84 95 7,219 79 84

Grade 8Public 79 87 259 92 7,728 73 80 92 8,063 72 80


Grade 12Public 73 80 311 76 7,977 55 61 76 8,112 55 61



permitted. Participation rates are presentedfor public and nonpublic schools, individu-ally and combined. The first rate is theweighted percentage of schools participat-ing in the assessment before substitution ofdemographically similar schools.5 This rateis based only on the number of schools thatwere initially selected for the assessment.The numerator of this rate is the sum ofthe number of students represented by eachinitially selected school that participated inthe assessment. The denominator is thesum of the number of students representedby each of the initially selected schools thathad eligible students enrolled.

The second school participation rate isthe weighted participation rate after substi-tution. The numerator of this rate is thesum of the number of students representedby each of the participating schools,whether originally selected or selected as asubstitute for a school that chose not toparticipate. The denominator is the sameas that for the weighted participation ratefor the initial sample. Because of thecommon denominators, the weightedparticipation rate after substitution is atleast as great as the weighted participationrate before substitution.


6 As described in the following section, a second sample in the 2001 national assessments was assessed that includedstudents who required and were provided with accommodations.

Also presented in table A.4 are weightedstudent participation rates. The numeratorof this rate is the sum across all studentsassessed (in either an initial session or amakeup session) of the number of studentsthat each represents. The denominator ofthis rate is the sum across all eligiblesampled students in participating schools ofthe number of students that each repre-sents. The overall participation rates takeinto account the weighted percentage ofschool participation before or after substi-tution and the weighted percentage ofstudent participation after makeup sessions.

For the grade 12 national sample, whereschool and student response rates did notmeet NCES standards, an extensive analysiswas conducted that examined, amongother factors, the potential for nonresponsebias at both the school and student level.No evidence of any significant potential foreither school or student nonresponse biaswas found. Results of these analyses, as wellas nonresponse bias analyses for the grade 4and grade 8 national samples, will beincluded in the forthcoming NAEP 2001Technical Report.

Students with Disabilities (SD)and/or Limited English Proficient(LEP) StudentsIt is NAEP’s intent to assess all selectedstudents from the target population.Therefore, every effort is made to ensurethat all selected students who are capable ofparticipating in the assessment are assessed.Some students sampled for participation inNAEP can be excluded from the sampleaccording to carefully defined criteria.

These criteria were revised in 1996 tocommunicate more clearly a presumptionof inclusion except under special circum-stances. According to these criteria, stu-dents with Individualized EducationPrograms (IEPs) were to be included in theNAEP assessment except in the followingcases:

1) The school’s IEP team determined thatthe student could not participate, OR,

2) The student’s cognitive functioning wasso severely impaired that she or he couldnot participate, OR,

3) The student’s IEP required that thestudent had to be tested with an accom-modation or adaptation and that thestudent could not demonstrate his or herknowledge without that accommoda-tion.6

All LEP students receiving academicinstruction in English for three years ormore were to be included in the assess-ment. Those LEP students receivinginstruction in English for fewer than threeyears were to be included unless schoolstaff judged them to be incapable of par-ticipating in the assessment in English.

Participation of SD and/or LEPStudents in the NAEP Samples

Testing all sampled students is the best wayfor NAEP to ensure that the statisticsgenerated by the assessment are as repre-sentative as possible of the performance ofthe entire national population and thepopulations of participating jurisdictions.However, all groups of students includecertain proportions that cannot be tested in


7 Office of Special Education Programs (1997). Nineteenth annual report to Congress on the implementation of theindividuals with disabilities education act. Washington, DC: U. S. Department of Education.

8 Section 504 of the Rehabilitation Act of 1973 is a civil rights law designed to prohibit discrimination on the basis ofdisability in programs and activities, including education, that receive federal financial assistance.

large-scale assessments (such as studentswho have profound mental disabilities), orwho can only be tested through the use of“accommodations” such as extra time, one-on-one administration, or use of magnify-ing equipment.

Some students with disabilities and someLEP students cannot show on a test whatthey know and can do unless they areprovided accommodations. When suchaccommodations are not allowed, studentsrequiring such adjustments are oftenexcluded from large-scale assessments suchas NAEP. This phenomenon has becomemore common in the last decade andgained momentum with the passage of theIndividuals with Disabilities Education Act(IDEA), which led schools and states toidentify increasing proportions of studentsas needing accommodations on assessmentsto best show what they know and can do.7

Furthermore, Section 504 of the Rehabili-tation Act of 1973 requires that, whenstudents with disabilities are tested, schoolsmust provide them with appropriateaccommodations so that the test resultsaccurately reflect students’ achievement.8 Inaddition, as the proportion of limitedEnglish proficient students in the popula-tion has increased, some states have startedoffering accommodations, such as trans-lated versions of assessments or the use ofbilingual dictionaries as part of assessments.

Before 1996, NAEP did not allow anytesting under nonstandard conditions(i.e., accommodations were not permitted).At that time, NAEP samples were able toinclude almost all sampled students in“standard” assessment sessions. However, asthe influence of IDEA grew more wide-spread, the failure to provide accommoda-tions led to increasing levels of exclusion inthe assessment. Such increases posed twothreats to the program: 1) they threatenedthe stability of trend lines (because exclud-ing more students in one year than thenext might lead to apparent rather than realgains), and 2) they made NAEP samplesless than optimally representative of targetpopulations.

NAEP reacted to this challenge byadopting a multipart strategy. It becameclear that, to ensure that NAEP sampleswere as inclusive as possible, the programhad to move toward allowing the sameassessment accommodations that wereafforded students in state and districttesting programs. However, allowingaccommodations represents a change intesting conditions that may affect measure-ment of changes over time. Therefore,beginning with the 1996 national assess-ments and the 1998 state assessments,NAEP has assessed a series of parallelsamples of students. In one set of samples,testing accommodations were not permit-ted; this has allowed NAEP to maintain the


9 The two samples are described as “overlapping” because in 2001 the same group of non-SD and/or LEP studentswere included in both samples.

measurement of achievement trends. Inaddition to the samples where accommo-dations were not permitted, parallel samplesin which accommodations were permittedwere also assessed. By having two overlap-ping samples and two sets of related datapoints, NAEP could meet two core pro-gram goals.9 First, data trends could bemaintained. Second, parallel trend linescould be set in ways that ensure that infuture years the program will be able to usethe most inclusive practices possible andmirror the procedures used by most stateand district assessments. Beginning in 2002,NAEP will use only the more inclusivesamples in which assessment accommoda-tions are permitted.

In geography, national data from 1994and 2001 are reported for the sample inwhich accommodations were not per-mitted. National data for the second sample,in which accommodations were permitted,are reported at all grades for 2001 only.

In order to make it possible to evaluatethe impact of increasing exclusion rates,data on exclusion in both assessment yearsare included in this appendix. Since theexclusion rates may affect average scalescores, readers should consider the magni-tude of exclusion rate changes wheninterpreting score changes.

Percentages of SD and/or LEP studentsfor the national sample where accommo-dations were not permitted are presentedin table A.5. The data in this table includethe percentages of students identified as SDand/or LEP, the percentage of studentsexcluded, and the percentage of assessed SDand/or LEP students. Percentages of thesestudents in the national sample whereaccommodations were permitted arepresented in table A.6. The data in thistable include the percentages of studentsidentified as SD and/or LEP, the percentageof students excluded, the percentage ofassessed SD and/or LEP students, thepercentage assessed without accommodations,and the percentage assessed with accommo-dations.

In the 2001 accommodations-not-permitted national sample, 8 percent ofstudents at grades 4 and 8, and 5 percent ofstudents at grade 12 were excluded fromthe assessment. The comparable percent-ages in the 2001 accommodations-permit-ted national sample were 4 percent atgrades 4 and 8, and 2 percent at grade 12.


Percentage of students identified as SD and/or LEP where accommodations were not permitted,grades 4, 8, and 12: 1994 and 2001

Table A.5 Students Identified as SD and/or LEP Where Accommodations Were Not Permitted

Grade 4SD and/or LEP students

Identified 1,487 14 100 1,051 16 100Excluded 1,025 5 41 500 8 48Assessed 462 8 59 551 8 52

SD students onlyIdentified 974 10 100 611 11 100Excluded 685 4 43 378 6 58Assessed 289 6 57 233 4 42

LEP students onlyIdentified 546 4 100 489 6 100Excluded 368 1 35 157 2 32Assessed 178 3 65 332 4 68


Identified 1,674 10 100 1,379 16 100Excluded 1,278 5 46 658 8 48Assessed 396 5 54 721 8 52

SD students onlyIdentified 1,254 8 100 947 12 100Excluded 979 4 49 546 7 54Assessed 275 4 51 401 6 46

LEP students onlyIdentified 450 2 100 489 4 100Excluded 323 1 38 153 1 31Assessed 127 1 62 336 3 69


Identified 1,238 8 100 1,096 11 100Excluded 948 3 43 574 5 44Assessed 290 4 57 522 6 56

SD students onlyIdentified 967 6 100 772 8 100Excluded 776 3 47 483 4 49Assessed 191 3 53 289 4 51

LEP students onlyIdentified 285 2 100 373 3 100Excluded 184 # 29 121 1 31Assessed 101 1 71 252 2 69

# Percentage is between 0.0 and 0.5.SD = Students with Disabilities. LEP = Limited English Proficient students.NOTE: Within each grade level, the combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students wereidentified as both SD and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.Within each portion of the table, percentages may not sum properly due to rounding. In 1994, the geography assessment was conducted at the same time asthe 1994 U.S. history assessment. The identification and exclusion of special-needs students occurred after they were sampled, but before they could beassigned either a history or geography session. As a consequence, the 1994 sample sizes for identified and excluded students appear larger than would beexpected given the weighted percentages that were calculations based on the geography sample only.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

1994 2001Weighted Weighted

Number of Weighted percentage of Number of Weighted percentage ofstudents percentage students students percentage studentssampled of all students identified sampled of all students identified


Grade 4SD and/or LEP students Identified 1,137 17 100

Excluded 293 4 24Assessed 844 13 76

Assessed without accommodations 476 6 36Assessed with accommodations 368 7 41

SD students only Identified 641 13 100Excluded 138 3 21Assessed 503 10 79


LEP students only Identified 576 5 100Excluded 175 2 31Assessed 401 4 69


Grade 8SD and/or LEP students Identified 1,453 16 100

Excluded 381 4 23Assessed 1,072 12 77




LEP students only Identified 545 4 100Excluded 140 1 27Assessed 405 3 73

Assessed without accommodations 348 3 63Assessed with accommodations 57 # 10

Grade 12SD and/or LEP students Identified 956 10 100

Excluded 301 2 23Assessed 655 8 77




LEP students only Identified 334 2 100Excluded 63 # 17Assessed 271 2 83

Assessed without accommodations 242 2 75Assessed with accommodations 29 # 8

# Percentage is between 0.0 and 0.5.SD = Students with Disabilities. LEP = Limited English Proficient students.NOTE: Within each grade level, the combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students wereidentified as both SD and LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.Within each portion of the table, percentages may not sum properly due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Number of Weighted percentage Weighted percentagestudents sampled of all students of students identified

Percentage of students identified as SD and/or LEP where accommodations were permitted,grades 4, 8, and 12: 2001

Table A.6 Students Identified as SD and/or LEP Where Accommodations Were Permitted


10 For information on DIF studies of items assessed with accommodations in the 1996 mathematics and scienceassessments, see Mazzeo, J. M., Carlson, J. E., Voelkl, K. E., & Lutkus, A. D. (1999). Increasing the participation of specialneeds students in NAEP; A report on 1996 NAEP research activities. Washington, DC: National Center for EducationStatistics.

Investigating the Effects of ExclusionRates on Assessment Results

As indicated by the data in the previoussection, exclusion rates have tended toincrease across assessment years in thesamples that did not permit accommoda-tions. In considering the effects of exclu-sion rates on assessment results, at least onemajor issue becomes evident. If exclusionrates vary substantially across assessmentyears, then the ability to report trends (i.e.,compare results between years) may beaffected by the fact that the results fromdifferent years are based on differentproportions of the population.

NCES has funded research into ways inwhich excluded students might be in-cluded in the estimation of scores for totalpopulations and has also commissionedstudies of the impact of assessment accom-modations on overall scores. Severalstatistical adjustment approaches for esti-mating full populations (including estimatesfor excluded students) have been proposed,but none has yet been judged ready foroperational use. Regarding the impact ofassessment accommodations on overall

scores, ETS has conducted differential itemfunctioning (DIF) studies of items assessedwith accommodations in the 1996 assess-ment.10 In these studies, ETS researchersfound little evidence that accommodationschanged the functioning of test questions.

Types of Accommodations Permitted

Table A.7 displays the number and thepercentages of SD and/or LEP studentsassessed with the variety of availableaccommodations. It should be noted thatstudents assessed with accommodationstypically received some combination ofaccommodations. The numbers and per-centages presented in the table reflect onlythe primary accommodation provided. Forexample, students assessed in small groups(as compared to standard NAEP sessions ofabout 30 students) usually receivedextended time. In one-on-one administra-tions, students often received assistance inrecording answers and were afforded extratime. Extended time was considered theprimary accommodation only when it wasthe sole accommodation provided.


Percentage of students identified as SD and/or LEP by type of accommodation where accommodationswere permitted, grades 4, 8, and 12: 2001

Table A.7 Students Identified as SD and/or LEP by Type of Accommodation


Weighted Weighted WeightedNumber Weighted percentage Number Weighted percentage Number Weighted percentage

of students percentage of students of students percentage of students of students percentage of studentssampled of all students identified sampled of all students identified sampled of all students identified

SD and/or LEP studentsBilingual dictionary 41 0.38 2.2 6 0.04 0.3 21 0.14 1.4

Large-print book 3 0.02 0.1 3 0.02 0.2 3 0.05 0.5Extended time 40 0.59 3.4 70 0.82 5.1 68 0.86 8.6

Read aloud 15 0.27 1.5 14 0.15 0.9 10 0.13 1.3Small group 230 4.97 28.6 286 4.02 25.1 83 1.43 14.4One-on-one 27 0.50 2.9 6 0.09 0.6 3 0.03 0.4

Scribe/computer 10 0.31 1.8 3 0.08 0.5 0 0.00 0.0Other 2 0.03 0.2 9 0.16 1.0 0 0.00 0.0

SD students onlyBilingual dictionary 5 0.05 0.4 0 0.00 0.0 0 0.00 0.0




LEP students onlyBilingual dictionary 41 0.38 6.9 6 0.04 1.0 21 0.14 5.7




SD = Students with Disabilities. LEP = Limited English Proficient students.NOTE: The combined SD/LEP portion of the table is not a sum of the separate SD and LEP portions because some students were identified as both SDand LEP. Such students would be counted separately in the bottom portions, but counted only once in the top portion.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment of EducationalProgress (NAEP), 2001 Geography Assessment.


11 These procedures are described more fully in the “Weighting and Variance Estimation” section later in this docu-ment. For additional information about the use of weighting procedures, see the forthcoming NAEP 2001 TechnicalReport. In addition, the reader may consult the NAEP 2000 Technical Report for a discussion of weighting proceduresthat are common to all NAEP assessments.

12 Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence ErlbaumAssociates.

Data Collection and ScoringThe 2001 geography assessment wasconducted from January through March2001, with some makeup sessions in earlyApril. As with all NAEP assessments, datacollection for the 2001 assessment wasconducted by a trained field staff. This wasaccomplished by staff from Westat, Inc.

Materials from the 2001 assessment wereshipped to NCS Pearson, where trainedstaff evaluated the responses to the con-structed-response questions using scoringrubrics or guides prepared by ETS. Eachconstructed-response question had a uniquescoring rubric that defined the criteriaused to evaluate students’ responses. Theextended-constructed-response questionswere evaluated with four-level rubrics,and almost all of the short-constructed-response questions were rated according tothree-level rubrics that permitted partialcredit. Other short-constructed-responsequestions were scored as either acceptableor unacceptable.

For the 2001 geography assessment,approximately 303,000 constructedresponses were scored. This numberincludes rescoring to monitor inter-raterreliability. The within-year average per-centage of agreement for the 2001 nationalreliability sample was 95 percent at grade 4,94 percent at grade 8, and 93 percent atgrade 12.

Data Analysis and IRT ScalingSubsequent to the professional scoring, allinformation was transcribed to the NAEPdatabase at ETS. Each processing activitywas conducted with rigorous qualitycontrol. After the assessment informationwas compiled in the database, the data wereweighted according to the populationstructure. The weighting for the nationalsample reflected the probability of selectionfor each student as a result of the samplingdesign, adjusted for nonresponse. Throughpost-stratification, the weighting assuredthat the representation of certain sub-populations corresponded to figures fromthe U.S. Census and the CurrentPopulation Survey.11

Analyses were then conducted to deter-mine the percentages of students who gavevarious responses to each cognitive andbackground question. In determining thesepercentages for the cognitive questions, adistinction was made between missingresponses at the end of a block (i.e., missingresponses subsequent to the last questionthe student answered) and missing responsesprior to the last observed response. Missingresponses before the last observed responsewere considered intentional omissions. Inanalysis, omitted responses to multiple-choice items were scored as fractionallycorrect.12 For constructed-response items,


omitted responses were placed into thelowest score category. Missing responses atthe end of the block were considered “notreached” and treated as if the questions hadnot been presented to the student. Incalculating response percentages for eachquestion, only students classified as havingbeen presented the question were includedin the denominator of the statistic.

It is standard NAEP practice to treat allnonrespondents to the last question in ablock as if they had not reached the ques-tion. For multiple-choice and short-constructed-response questions, this prac-tice produces a reasonable pattern of resultsin that the proportion reaching the lastquestion is not dramatically smaller thanthe proportion reaching the next-to-lastquestion. However, for geography blocksthat ended with extended-constructed-response questions, the standard practicecould result in extremely large drops in theproportion of students attempting some ofthe final questions. Therefore, for blocksending with an extended-constructed-response question, students who answeredthe next-to-last question but did notrespond to the extended-constructed-response question were classified as havingintentionally omitted the last question.

Item Response Theory (IRT) was usedto estimate average geography scale scoresfor the nation and for various subgroups ofinterest within the nation. IRT models theprobability of answering a question in acertain way as a mathematical function ofproficiency or skill. The main purpose of

IRT analysis is to provide a common scaleon which performance can be comparedacross groups such as those defined bycharacteristics, including gender and race/ethnicity.

In producing the geography scales, threedistinct IRT models were used. Multiple-choice questions were scaled using thethree-parameter logistic (3PL) model;short-constructed-response questions ratedas acceptable or unacceptable were scaledusing the two-parameter logistic (2PL)model; and short-constructed-responsequestions rated according to a three-levelrubric, as well as extended-constructed-response questions rated on a four-levelrubric, were scaled using a GeneralizedPartial-Credit (GPC) model.13 Developedby ETS and first used in 1992, the GPCmodel permits the scaling of questionsscored according to multipoint ratingschemes. The model takes full advantage ofthe information available from each of thestudent response categories used for thesemore complex constructed-responsequestions.14

The geography scale is composed ofthree types of questions: multiple-choice,short-constructed-response (scored eitherdichotomously or allowing for partialcredit), and extended-constructed-response(scored according to a partial-credit model).Unfortunately, the question of how muchinformation different question-typescontribute to the geography scale has nosimple answer. The information providedby a given question is determined by the

13 Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied PsychologicalMeasurement, (16)2, 159–176.

14 More detailed information regarding the IRT analyses used in NAEP assessments will be provided in the forth-coming NAEP 2001 Technical Report. In addition, the reader may consult the NAEP 2000 Technical Report for adiscussion of analysis procedures that are common to all NAEP assessments.


15 Donoghue, J. R. (1994). An empirical examination of the IRT information of polytomously scored reading itemsunder the generalized partial credit model. Journal of Educational Measurement, (31)4, 295–311.

16 For theoretical and empirical justification of the procedures employed, see Mislevy, R. J. (1988). Randomization-based inferences about latent variables from complex samples. Psychometrika, (56)2, 177–196.

For computational details, see the forthcoming NAEP 2001 Technical Report.

IRT model used to scale the question. It isa function of the item parameters andvaries by level of geography proficiency.15

Thus, the answer to the query “How muchinformation do the different types ofquestions provide?” will differ for eachlevel of geography performance. Whenconsidering the composite geography scale,the answer is even more complicated. Thegeography data are scaled separately by thethree themes (space and place; environmentand society; and, spatial dynamics andconnections), resulting in three separatesubscales at each grade. The compositescale is a weighted combination of thesesubscales. IRT information functions areonly strictly comparable when the itemparameters are estimated together. Becausethe composite scale is based on threeseparate estimation runs, there is no directway to compare the information providedby the questions on the composite scale.

Because of the BIB-spiraling design usedby NAEP, students do not receive enoughquestions about a specific topic to providereliable information about individualperformance. (For more information onBIB-spiraling, see “The Assessment Design”section presented earlier in this appendix.)Traditional test scores for individual stu-dents, even those based on IRT, would leadto misleading estimates of populationcharacteristics, such as subgroup means andpercentages of students at or above a

certain scale-score level. Consequently,NAEP constructs sets of plausible valuesdesigned to represent the distribution ofperformance in the population. A plausiblevalue for an individual is not a scale scorefor that individual, but may be regarded asa representative value from the distributionof potential scale scores for all students inthe population with similar characteristicsand identical patterns of item response.Statistics describing performance on theNAEP geography scale are based on theplausible values. Under the assumptions ofthe scaling models, these population esti-mates will be consistent, in the sense thatthe estimates approach the model-basedpopulation values as the sample sizeincreases, which would not be the case forpopulation estimates obtained by aggre-gating optimal estimates of individualperformance.16

Item Mapping ProceduresThe geography performance of fourth-,eighth-, and twelfth-graders can be illus-trated by “item maps,” which positionquestion or “item” descriptions along theNAEP geography scale at each grade. Eachquestion shown is placed at the point onthe scale where questions are likely to beanswered successfully by students. Thedescriptions used on these item maps focuson the geography knowledge or skillneeded to answer the question. For multiple-choice questions, the description indicates


17 Huynh, H. (1994, October). Some technical aspects of standard setting. Paper presented at the Joint Conference on StandardSetting for Large-Scale Assessment, Washington, DC.

the knowledge or skill demonstrated byselection of the correct option; for con-structed-response questions, the descriptiontakes into account the knowledge or skillspecified by the different levels of scoringcriteria for that question.

To map questions to particular points onthe NAEP geography scale, a responseprobability convention was adopted thatwould divide those who had a higherprobability of success from those who hada lower probability. Establishing a responseprobability convention has an impact onthe mapping of the test questions onto thegeography scale. A lower boundary con-vention maps the geography questions atlower points along the scale, and a higherboundary convention maps the samequestions at higher points on the scale.The underlying distribution of geographyskills in the population does not change,but the choice of a response probabilityconvention does have an impact on theproportion of the student population that isreported as “able to do” the questions onthe geography scales.

There is no obvious choice of a pointalong the probability scale that is clearlysuperior to any other point. If the conven-tion were set with a boundary at 50 per-cent, those above the boundary would bemore likely to get a question right than getit wrong, while those below the boundarywould be more likely to get the questionwrong than right. Although this conven-tion has some intuitive appeal, it wasrejected on the grounds that having a

50/50 chance of getting the question rightshows an insufficient degree of mastery. Ifthe convention were set with a boundary at80 percent, students above the criterionwould have a high probability of successwith a question. However, many studentsbelow this criterion show some level ofgeography ability that would be ignored bysuch a stringent criterion. In particular,those in the range between 50 and 80percent correct would be more likely toget the question right than wrong, yetwould not be in the group described as“able to do” the question.

In a compromise between the 50 per-cent and the 80 percent conventions,NAEP has adopted two related responseprobability conventions for all its subjects:65 percent for constructed-response ques-tions (where guessing is not a factor) and74 percent for multiple-choice questions(to correct for the possibility of answeringcorrectly by guessing). These probabilityconventions were established, in part, basedon an intuitive judgment that they wouldprovide the best picture of students’geography skills.

Some additional support for the dualconventions adopted by NAEP was pro-vided by Huynh.17 He examined the IRTinformation provided by items, accordingto the IRT model used in scaling NAEPquestions. (“Information” is used here in atechnical sense. See the forthcomingNAEP 2001 Technical Report for details.)Following Bock, Huynh decomposed theitem information into that provided by a


18 Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more latentcategories. Psychometrika, 37, 29–51.

19 Donoghue, J. R. (1997, March). Item mapping to a weighted composite scale. Paper presented at the annual meeting ofthe American Educational Research Association, Chicago, IL.

correct response [P(q) I(q)] and that pro-vided by an incorrect response [(1– P(q))I(q)].18 Huynh showed that the iteminformation provided by a correct responseto a constructed-response item is maxi-mized at the point along the geographyscale at which the probability of a correctresponse is 0.65 (for multiple-choice items,the information provided by a correctresponse is maximized at the point atwhich the probability of getting the itemcorrect is 0.74). It should be noted, how-ever, that maximizing the item informationI(q), rather than the information providedby a correct response [P(q) I(q)], wouldimply an item mapping criterion closer to50 percent.

Results are presented in terms of thecomposite geography scale. However, thegeography assessment was scaled separatelyfor the three themes in geography at grades4, 8, and 12. The composite scale is aweighted combination of the threesubscales for the three themes in geography.To obtain item map information, a proce-dure developed by Donoghue was used.19

This method models the relationshipbetween the item response function for thesubscale and the subscale structure toderive the relationship between the itemscore and the composite scale (i.e., an itemresponse function for the composite scale).This item response function is then used toderive the probability used in the mapping.

Weighting andVariance EstimationA multistage sampling design was used toselect the students who were assessed.The properties of a sample selectedthrough such a design could be verydifferent from those of a simple randomsample, in which every student in thetarget population has an equal chance ofselection and in which the observationsfrom different sampled students can beconsidered to be statistically independentof one another. Therefore, the propertiesof the sample for the data collection designwere taken into account during the analysisof the assessment data.

One way that the properties of thesample design were addressed was by usingsampling weights to account for the factthat the probabilities of selection were notidentical for all students. All populationand subpopulation characteristics based onthe assessment data were estimated usingsampling weights. These weights includedadjustments for school and studentnonresponse.

Not only must appropriate estimates ofpopulation characteristics be derived, butappropriate measures of the degree ofuncertainty must be obtained for thosestatistics. Two components of uncertaintyare accounted for in the variability ofstatistics based on student ability: 1) theuncertainty due to sampling only a rela-


20 For further details, see Johnson, E. G. & Rust, K. F. (1992). Population inferences and variance estimation for NAEPdata. Journal of Educational Statistics, (17)2, 175–190.

tively small number of students, and 2) theuncertainty due to sampling only a portionof the cognitive domain of interest. Thefirst component accounts for the variabilityassociated with the estimated percentagesof students who had certain backgroundcharacteristics or who answered a certaincognitive question correctly.

Because NAEP uses multistage samplingprocedures, conventional formulas forestimating sampling variability that assumesimple random sampling are inappropriate.NAEP uses a jackknife replication proce-dure to estimate standard errors. Thejackknife standard error provides a reason-able measure of uncertainty for any studentinformation that can be observed withouterror. However, because each studenttypically responds to only a few questionswithin any theme of geography, the scalescore for any single student would beimprecise. In this case, plausible valuesmethodology can be used to describe theperformance of groups and subgroups ofstudents. Multiple plausible values (5) aredrawn for each student in order to estimatethe variance of the posterior scale scoredistribution. This component of variabilityis included in the standard errors of NAEPscale scores.20

Typically, when the standard error isbased on a small number of students orwhen the group of students is enrolled in asmall number of schools, the amount ofuncertainty associated with the estimationof standard errors may be quite large.

Estimates of standard errors subject to alarge degree of uncertainty are followed bythe “!” symbol to indicate that the natureof the sample does not allow accuratedetermination of the variability of thestatistic. In such cases, the standard errors—and any confidence intervals or significancetests involving these standard errors—should be interpreted cautiously. Addi-tional details concerning procedures foridentifying such standard errors are dis-cussed in the forthcoming NAEP 2001Technical Report.

Drawing Inferencesfrom the ResultsThe reported statistics are estimates and aretherefore subject to a measure of uncer-tainty. There are two sources of suchuncertainty. First, NAEP uses a sample ofstudents rather than testing all students.Second, all assessments have some amountof uncertainty related to the fact that theycannot ask all questions that might beasked in a content area. The magnitude ofthis uncertainty is reflected in the standarderror of each of the estimates. When thepercentages or average scale scores ofcertain groups are compared, the standarderror should be taken into account, andobserved similarities or differences shouldnot be relied on solely. Therefore, thecomparisons are based on statistical teststhat consider the standard errors of thosestatistics and the magnitude of the differ-ence among the averages or percentages.


Using confidence intervals based on thestandard errors provides a way to take intoaccount the uncertainty associated withsample estimates and to make inferencesabout the population averages and percent-ages in a manner that reflects that uncer-tainty. An estimated sample average scalescore plus or minus 1.96 standard errorsapproximates a 95 percent confidenceinterval for the corresponding populationquantity. This statement means that onecan conclude with approximately a 95percent level of confidence that the averageperformance of the entire population ofinterest (e.g., all fourth-grade students inpublic and nonpublic schools) is withinplus or minus 1.96 standard errors of thesample average.

As an example, suppose that the averagegeography scale score of the students in aparticular group was 256 with a standarderror of 1.2. An approximate 95 percentconfidence interval for the populationquantity would be as follows:

Average � 1.96 standard errors256 � 1.96 � 1.2

256 � 2.35(253.65, 258.35)

Thus, one can conclude with a 95percent level of confidence that the averagescale score for the entire population ofstudents in that group is between 253.65and 258.35. It should be noted that thisexample, and the examples in the followingsections are illustrative. More preciseestimates carried out to one or moredecimal places are used in the actualanalyses.

Similar confidence intervals can beconstructed for percentages, if the percent-ages are not extremely large or extremelysmall. Extreme percentages should beinterpreted with caution. Adding orsubtracting the standard errors associatedwith extreme percentages could cause theconfidence interval to exceed 100 percentor go below 0 percent, resulting in num-bers that are not meaningful. The forth-coming NAEP 2001 Technical Report willcontain a more complete discussion ofextreme percentages.

Analyzing Group Differences inAverages and PercentagesStatistical tests determine whether theevidence, based on the data from thegroups in the sample, is strong enough toconclude that the averages or percentagesare actually different for those groups inthe population. If the evidence is strong(i.e., the difference is statistically signifi-cant), the report describes the groupaverages or percentages as being different(e.g., one group performed higher than orlower than another group), regardless ofwhether the sample averages or percentagesappear to be approximately the same.

The reader is cautioned to rely on theresults of the statistical tests rather than onthe apparent magnitude of the differencebetween sample averages or percentageswhen determining whether the sampledifferences are likely to represent actualdifferences among the groups in thepopulation.


To determine whether a real differenceexists between the average scale scores (orpercentages of a certain attribute) for twogroups in the population, one needs toobtain an estimate of the degree of uncer-tainty associated with the differencebetween the averages (or percentages) ofthese groups for the sample. This estimateof the degree of uncertainty, called the“standard error of the difference” betweenthe groups, is obtained by taking the squareof each group’s standard error, summingthe squared standard errors, and taking thesquare root of that sum.

Standard Error of the Difference �

SEA-B

� √(SEA

2 � SEB

2)

Similar to how the standard error for anindividual group average or percentage isused, the standard error of the differencecan be used to help determine whetherdifferences among groups in the populationare real. The difference between theaverages or percentages of the two groupsplus or minus two standard errors of thedifference represents an approximate 95percent confidence interval. If the resultinginterval includes zero, there is insufficientevidence to claim a real difference betweenthe groups in the population. If the intervaldoes not contain zero, the differencebetween the groups is statistically signifi-cant (different) at the 0.05 level.

As an example of comparing groups,consider the problem of determiningwhether the average geography scale scoreof group A is higher than that of group B.Suppose that the sample estimates of the

average scale scores and standard errorswere as follows:

The difference between the estimates ofthe average scale scores of groups A and Bis two points (218 � 216). The standarderror of this difference is

√(0.92 � 1.12) � 1.4

Thus, an approximate 95 percent confi-dence interval for this difference is plus orminus two standard errors of the difference

2 � 1.96 � 1.42 � 2.74

(�0.74, 4.74)

The value zero is within the confidenceinterval; therefore, there is insufficientevidence to claim that group A outper-formed group B.

Conducting Multiple TestsThe procedures in the previous section andthe certainty ascribed to intervals (e.g., a 95percent confidence interval) are based onstatistical theory that assumes that only oneconfidence interval or test of statisticalsignificance is being performed. However,many different groups are being compared(i.e., multiple sets of confidence intervalsare being analyzed). In sets of confidenceintervals, statistical theory indicates that the

AverageGroup Scale Score Standard Error

A 218 0.9

B 216 1.1


21 Miller, R. G. (1966). Simultaneous statistical inference. New York: Wiley.22 Benjamini, Y. & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to

multiple testing. Journal of the Royal Statistical Society, Series B, No. 1., pp 289–300.23 Williams, V. S. L., Jones, L. V., & Tukey, J. W. (1999). Controlling error in multiple comparisons with examples from state-to-

state differences in educational achievement. Journal of Educational and Behavioral Statistics, 24(1), 42–69.

certainty associated with the entire set ofintervals is less than that attributable toeach individual comparison from the set.To hold the significance level for the set ofcomparisons at a particular level (e.g., 0.05),adjustments (called “multiple comparisonprocedures”21) must be made to the meth-ods described in the previous section. Onesuch procedure, the False Discovery Rate(FDR) procedure22 was used to control thecertainty level.

Unlike the other multiple comparisonprocedures (e.g., the Bonferroni procedure)that control the familywise error rate(i.e., the probability of making even onefalse rejection in the set of comparisons),the FDR procedure controls the expectedproportion of falsely rejected hypotheses.Furthermore, familywise procedures are

considered conservative for large families ofcomparisons.23 Therefore, the FDR proce-dure is more suitable for multiple compari-sons in NAEP than other procedures. Adetailed description of the FDR procedureappears in the forthcoming NAEP 2001Technical Report.

To illustrate how the FDR procedure isused, consider the comparisons of currentand previous years’ average geography scalescores for the five groups presented in tableA.8. Note that the difference in averagescale scores and the standard error of thedifference are calculated in a way compa-rable with that of the example in theprevious section. The test statistic shown isthe difference in average scale scoresdivided by the standard error of thedifference.

Example of FDR comparisons of average scale scores for different groups of students

Table A.8 FDR Comparisons of Average Scale Scores

Previous year Current year Previous year and current year

StandardAverage Standard Average Standard Difference error of Test Percent

scale score error scale score error in averages difference statistic confidence*

Group 1 224 1.3 226 1.0 2.08 1.62 1.29 20

Group 2 187 1.7 193 1.7 6.31 2.36 2.68 1

Group 3 191 2.6 197 1.7 6.63 3.08 2.15 4

Group 4 229 4.4 232 4.6 3.24 6.35 .51 62

Group 5 201 3.4 196 4.7 –5.51 5.81 –.95 35

* The percent confidence is 2(1–F(x)) where F(x) is the cumulative distribution of the t-distribution with the degrees of freedom adjusted to reflect thecomplexities of the sample design.


24 The level of confidence times the number of comparisons minus one divided by the number of comparisons is0.05�(5�1)/5 � 0.04�100 � 4 percent.

The difference in average scale scoresand its standard error can be used to findan approximate 95 percent confidenceinterval as in the example in the previoussection or they can be used to identify aconfidence percentage. In the example inthe previous section, because an approxi-mate 95 percent confidence interval wasdesired, the number 1.96 was used tomultiply the standard error of the differ-ence to create the approximate confidenceinterval. In the current example, the confi-dence interval for the test statistics isidentified from statistical tables. Instead ofchecking to see if zero is within the 95percent confidence interval about themean, the significance level from thestatistical tables can be directly comparedto 100�95 � 5 percent.

If the comparison of average scale scoresacross two years were made for only one ofthe five groups, there would be a significantdifference between the average scale scoresfor the two years if the significance levelwere less than 5 percent. However, becausewe are interested in the difference inaverage scale scores across the two years forall five of the groups, comparing each ofthe significance levels to 5 percent is notadequate. Groups of students defined byshared characteristics, such as race/ethnicitygroups, are treated as sets or families whenmaking comparisons. However, compari-sons of average scale scores for each pair ofyears were treated separately. So the stepsdescribed in this example would be repli-cated for the comparison of other currentand previous year average scale scores.

To use the FDR procedure to take intoaccount that all comparisons are of interestto us, the percents of confidence in theexample are ordered from largest to small-est: 62, 35, 20, 4, and 1. In the FDR proce-dure, 62 percent confidence for the Group4 comparison would be compared to 5percent, 35 percent for the Group 5comparison would be compared to0.05�(5�1)/5 � 0.04�100 � 4 percent,24

20 percent for the Group 1 comparisonwould be compared to 0.05�(5�2)/5 �0.03�100 � 3 percent, 4 percent for theGroup 3 comparison would be comparedto 0.05�(5�3)/5 � 0.02�100 � 2 percent,and 1 percent for the Group 2 comparison(actually slightly smaller than 1 prior torounding) would be compared to0.05�(5�4)/5 � 0.01�100 � 1 percent.The last of these comparisons is the onlyone for which the percent confidence issmaller than the FDR procedure value.The difference in the current year andprevious years’ average scale scores for theGroup 2 students is significant; for all of theother groups, average scale scores forcurrent and previous year are not signifi-cantly different from one another. Inpractice, a very small number of counter-intuitive results occur when using the FDRprocedures to examine between-yeardifferences in subgroup results by jurisdic-tion. In those cases, results were not in-cluded in this report. NCES is continuingto evaluate the use of FDR and multiple-comparison procedures for future reporting.


25 For the national assessment, a PSU is a selected geographic region (a county, group of counties, or metropolitanstatistical area). Further details about the procedure for determining minimum sample size appear in the NAEP 2000Technical Report and the forthcoming NAEP 2001 Technical Report.

NAEP Reporting GroupsResults are provided for groups of studentsdefined by shared characteristics—regionof the country, gender, race or ethnicity,school’s type of location, eligibility for thefree/reduced-price school lunch program,and type of school. Based on participationrate criteria, results are reported for sub-populations only when sufficient numbersof students and adequate school representa-tion are present. The minimum require-ment is at least 62 students in a particularsubgroup from at least five primary sam-pling units (PSUs).25 However, the data forall students, regardless of whether theirsubgroup was reported separately, wereincluded in computing overall results.

Definitions of the subpopulations arepresented below.

Region

Results in NAEP are reported for fourregions of the nation: Northeast, Southeast,Central, and West. Figure A.1 shows howstates are subdivided into these NAEPregions. All 50 states and the District ofColumbia are listed. Other jurisdictions,including territories and the two Depart-ment of Defense Educational Activitiesjurisdictions are not assigned to any region.

Gender

Results are reported separately for malesand females.

ConnecticutDelawareDistrict of ColumbiaMaineMarylandMassachusettsNew HampshireNew JerseyNew YorkPennsylvaniaRhode IslandVermont

* Virginia

AlabamaArkansasFloridaGeorgiaKentuckyLouisianaMississippiNorth CarolinaSouth CarolinaTennessee

*VirginiaWest Virginia

IllinoisIndianaIowaKansasMichiganMinnesotaMissouriNebraskaNorth DakotaOhioSouth DakotaWisconsin

AlaskaArizonaCaliforniaColoradoHawaiiIdahoMontanaNevadaNew MexicoOklahomaOregonTexasUtahWashingtonWyoming

* The part of Virginia that is included in the Northeast region is the Washington, DC metropolitan area; the remainder of the state is included in the Southeastregion.

Northeast Southeast Central West

Figure A.1

States byRegion

States included in the four NAEP regions: 2001


Race/Ethnicity

The race/ethnicity variable is derived fromtwo questions asked of students and fromschool records, and it is used for race/ethnicity subgroup comparisons. Twoquestions from the set of general studentbackground questions were used to deter-mine race/ethnicity:

If you are Hispanic, what is your Hispanicbackground?❏ I am not Hispanic❏ Mexican, Mexican American, or Chicano❏ Puerto Rican❏ Cuban❏ Other Spanish or Hispanic background

Students who responded to this questionby filling in the second, third, fourth, orfifth oval were considered Hispanic. Forstudents who filled in the first oval, did notrespond to the question, or providedinformation that was illegible or could notbe classified, responses to the followingquestion were examined to determine theirrace/ethnicity.

Which best describes you?❏ White (not Hispanic)❏ Black (not Hispanic)❏ Hispanic (“Hispanic” means someone

who is Mexican, Mexican American,Chicano, Puerto Rican, Cuban, or otherSpanish or Hispanic background)

❏ Asian or Pacific Islander (“Asian orPacific Islander” means someone who isfrom a Chinese, Japanese, Korean,Filipino, Vietnamese, Asian American orsome other Asian or Pacific Islanderbackground.)

❏ American Indian or Alaskan Native(“American Indian or Alaskan Native”means someone who is from one of theAmerican Indian tribes or one of theoriginal people of Alaska.)

❏ Other (specify)

Students’ race/ethnicity was then assignedon the basis of their responses. For studentswho filled in the sixth oval (“Other”),provided illegible information or informa-tion that could not be classified, or did notrespond at all, race/ethnicity was assignedas determined by school records.

Race/ethnicity could not be determinedfor students who did not respond to eitherof the demographic questions and whoseschools did not provide information aboutrace/ethnicity.

Also, some students indicated that theywere from a Hispanic background(e.g., Puerto Rican or Cuban) and that aracial/ethnic category other than Hispanicbest described them. These students wereclassified as Hispanic based on the rulesdescribed above.

Type of Location

Results from the 2001 assessment arereported for students attending schools inthree mutually exclusive location types:central city, urban fringe/large town, andrural/small town:

Central City: This category includes centralcities of all Standard Metropolitan Statisti-cal Areas (SMSA) as defined by the Officeof Management and Budget. Central Cityis a geographical term and is not synony-mous with “inner city.”

Urban Fringe/Large Town: The urban fringecategory includes all densely settled placesand areas within SMSA’s that are classifiedas urban by the Bureau of the Census, butwhich do not qualify as Central City. ALarge Town is defined as a place outside aSMSA with a population greater than orequal to 25,000.


Rural/Small Town: Rural includes all placesand areas with populations of less than2,500 that are classified as rural by theBureau of the Census. A Small Town isdefined as a place outside a SMSA with apopulation of less than 25,000, but greaterthan or equal to 2,500.

Results for each type of location are notcompared across years. This was due tonew methods used by NCES to identifythe type of location assigned to each schoolin the Common Core of Data (CCD).The new methods were put into place byNCES in order to improve the quality ofthe assignments and they take into accountmore information about the exact physicallocation of the school. The variable wasrevised in NAEP beginning with the 2000assessments.

Eligibility for the Free/Reduced-PriceSchool Lunch Program

Based on available school records, studentswere classified as either currently eligiblefor the free/reduced-price school lunchcomponent of the Department of Agri-culture’s National School Lunch Programor not eligible. Eligibility for the programis determined by students’ family incomein relation to the federally establishedpoverty level. Free lunch qualification is setat 130 percent of the poverty level, andreduced-price lunch qualification is set at170 percent of the poverty level. Theclassification applies only to the school yearwhen the assessment was administered(i.e., the 2000–2001 school year) and is not

based on eligibility in previous years. Ifschool records were not available, thestudent was classified as “Information notavailable.” If the school did not participatein the program, all students in that schoolwere classified as “Information notavailable.”

Type of School

Results are reported by the type of schoolthat the student attends—public or non-public. Nonpublic schools include Catholicand other private schools.26 Because theyare funded by federal authorities, not state/local governments, Bureau of Indian Affairs(BIA) schools and Department of DefenseDomestic Dependent Elementary andSecondary Schools (DDESS) are notincluded in either the public or nonpubliccategories; they are included in the overallnational results.

Grade 12 Participation RatesNAEP has been described as a “low-stakes”assessment. That is, students receive noindividual scores, and their NAEP perfor-mance has no effect on their grades, pro-motions, or graduation. There has beencontinued concern that this lack of conse-quences affects participation rates of stu-dents and schools, as well as the motivationof students to perform well on NAEP. Ofparticular concern has been the perfor-mance of twelfth-graders, who typicallyhave lower student participation rates thanfourth- and eighth-graders, and who aremore likely to omit responses compared tothe younger cohorts.

26 Through a pilot study, more detailed breakdowns of nonpublic school results are available on the NAEP Web Site(http://nces.ed.gov/nationsreportcard/geography/results/index.asp).


In NAEP, there has been a consistentpattern of lower participation rates forolder students. In the 2001 NAEP assess-ments, for example, the student partici-pation rates were 95 percent and 93 per-cent at grades 4 and 8, respectively. Atgrade 12, however, the participation ratewas 77 percent. School participation rates(the percentage of sampled schools thatparticipated in the assessment) have alsotypically decreased with grade level. Againciting the 2001 assessments, the schoolparticipation rate was 88 percent for thefourth grade, 87 percent for the eighthgrade, and 80 percent for the twelfth grade.

The effect of participation rates onstudent performance, however, is unclear.Students may choose not to participate inNAEP for many reasons, such as desire toattend regular classes so as not to missimportant instruction or conflict withother school-based activities. Similarly,there are a variety of reasons for whichvarious schools do not participate. Thesampling weights and nonresponse adjust-ments, described earlier in this document,provide an approximate statistical adjust-ment for nonparticipation. However, theeffect of some school and student non-participation may have some undeterminedeffect on results.

More research is needed to delineate thefactors that contribute to nonparticipationand lack of motivation. To that end, NCESis currently investigating how varioustypes of incentives can be effectively usedto increase participation in NAEP. Onereport that examines the impact of mon-etary incentives on student effort andperformance is available on the NCES WebSite at http://nces.ed.gov/pubsearch/.Enter NCES#: 2001024.

Cautions in InterpretationsAs described earlier, the NAEP geographyscale makes it possible to examine relation-ships between students’ performance andvarious background factors measured byNAEP. However, a relationship that existsbetween achievement and another variabledoes not reveal its underlying cause, whichmay be influenced by a number of othervariables. Similarly, the assessments do notcapture the influence of unmeasuredvariables. The results are most useful whenthey are considered in combination withother knowledge about the student popu-lation and the educational system, such astrends in instruction, changes in theschool-age population, and societaldemands and expectations.

A P P E N D I X B • G E O G R A P H Y R E P O R T C A R D 141

Appendix BData Appendix

This appendix contains complete data for all the tables and

figures presented in this report, including average scores,

achievement-level results, and percentages of students. In

addition, standard errors appear in parentheses next to each

scale score and percentage. The comparisons presented in

this report are based on statistical tests that consider the

magnitude of the difference between group averages

or percentages and the standard errors of those

statistics. Because NAEP scores and percentages are

based on samples rather than the entire population(s),

the results are subject to a measure of uncertainty

reflected in the standard errors of the estimates. It can

be said with 95 percent certainty that for each

population of interest, the value for the whole

population is within plus or minus two standard

errors of the estimate for the sample. As with the

figures and tables in the chapters, significant

differences between results of previous assessments

and the 2001 assessment are highlighted.

BAppendixContents

Average Scores

Achievement-Level Results

Percentages ofStudents

Standard Errors

Complete datafor all tablesand figures.

AppendixFocus

142 A P P E N D I X B • G E O G R A P H Y R E P O R T C A R D

Average geography scale scores, grades 4, 8, and 12: 1994 and 2001


1994 206 (1.2) 260 (0.7) 285 (0.7)

2001 209 (1.0) * 262 (0.9) * 285 (0.8)

Standard errors of the estimated scale scores appear in parentheses.* Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Table B.1: Data for Figure 2.1 National Scale Score Results

National geography scale score percentiles, grades 4, 8, and 12: 1994 and 2001

Table B.2: Data for Figure 2.2: National Performance Distribution

10th 25th 50th 75th 90th

Grade 4 1994 146 (1.9) 179 (1.5) 211 (1.1) 237 (1.3) 257 (2.0)

2001 158 (1.7) * 185 (1.9) * 212 (1.1) 236 (1.0) 254 (0.9)

Grade 8 1994 213 (1.3) 237 (1.0) 263 (1.1) 285 (0.9) 302 (1.9)

2001 217 (1.0) * 241 (0.9) * 265 (1.1) 286 (0.9) 303 (1.2)

Grade 12 1994 244 (0.9) 265 (1.1) 287 (0.9) 306 (1.0) 321 (1.0)

2001 247 (1.7) 267 (1.2) 287 (0.9) 305 (0.9) 319 (1.0)

Standard errors of the estimated scale scores appear in parentheses.* Significantly different from 1994.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.


Table B.3: Data for Figure 2.3: National Achievement-Level Results

Percentage of students within and at or above geography achievement levels, grades 4, 8, and 12:1994 and 2001



Grade 4 1994 30 (1.1) 48 (1.0) 19 (1.1) 3 (0.4) 70 (1.1) 22 (1.2)

2001 26 (1.2) * 53 (1.4) * 19 (1.1) 2 (0.3) 74 (1.2) * 21 (1.0)

Grade 8 1994 29 (1.0) 43 (1.1) 24 (0.9) 4 (0.4) 71 (1.0) 28 (1.0)

2001 26 (0.9) * 44 (0.9) 26 (1.1) 4 (0.6) 74 (0.9) * 30 (1.2)

Grade 12 1994 30 (0.9) 43 (1.0) 25 (1.0) 2 (0.5) 70 (0.9) 27 (1.2)

2001 29 (0.9) 47 (0.9) * 23 (1.0) 1 (0.3) 71 (0.9) 25 (1.1)

Standard errors of the estimated percentages appear in parentheses.* Significantly different from 1994.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.


Table B.5: Data for Figure 3.2 National Scale Score Differences by Gender

Differences in average geography scale scores by gender, grades 4, 8, and 12: 1994 and 2001

Standard errors of the estimated difference in scale scores appear in parentheses.Score differences are calculated based on differences between unrounded average scale scores.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Male-Female

Grade 4 1994 5 (2.0)

2001 5 (1.7)

Grade 8 1994 4 (1.2)

2001 4 (1.4)

Grade 12 1994 7 (1.2)

2001 4 (1.2)

Table B.4: Data for Figure 3.1 National Scale Score Results by Gender

Percentage of students and average geography scale scores by gender, grades 4, 8, and 12:1994 and 2001

Male Female

Grade 4 1994 51 (1.0) 49 (1.0)208 (1.4) 203 (1.4)

2001 51 (0.8) 49 (0.8)212 (1.1) 207 (1.2)

Grade 8 1994 51 (0.7) 49 (0.7)262 (0.9) 258 (0.8)

2001 51 (0.6) 49 (0.6)264 (1.0) 260 (1.1)

Grade 12 1994 50 (1.0) 50 (1.0)288 (0.8) 281 (0.9)

2001 48 (0.8) 52 (0.8)287 (0.9) 282 (0.8)

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.


Table B.6: Data for Figure 3.3 National Achievement-Level Results by Gender

Percentage of students within and at or above geography achievement levels by gender,grades 4, 8, and 12: 1994 and 2001



Grade 4 Male 1994 29 (1.3) 46 (1.4) 22 (1.6) 4 (0.7) 71 (1.3) 26 (1.7)2001 25 (1.3) 51 (1.6) * 21 (1.4) 3 (0.5) 75 (1.3) 24 (1.4)

Female 1994 32 (1.4) 49 (1.3) 17 (1.2) 2 (0.5) 68 (1.4) 19 (1.3)2001 28 (1.6) 54 (1.7) * 17 (1.2) 1 (0.4) 72 (1.6) 18 (1.1)

Grade 8 Male 1994 28 (1.3) 42 (1.4) 25 (1.2) 5 (0.6) 72 (1.3) 30 (1.2)2001 25 (1.0) 42 (1.3) 29 (1.7) 5 (0.7) 75 (1.0) 33 (1.5)

Female 1994 31 (1.1) 44 (1.2) 22 (1.2) 3 (0.4) 69 (1.1) 25 (1.1)2001 27 (1.2) 47 (1.1) 24 (1.3) 3 (0.6) 73 (1.2) 26 (1.4)

Grade 12 Male 1994 27 (1.1) 41 (1.1) 29 (1.1) 2 (0.7) 73 (1.1) 32 (1.4)2001 27 (1.1) 45 (1.3) * 26 (1.4) 2 (0.4) 73 (1.1) 28 (1.5)

Female 1994 33 (1.2) 45 (1.5) 21 (1.2) 1 (0.4) 67 (1.2) 22 (1.4)2001 30 (1.0) 48 (1.0) * 20 (0.9) 1 (0.3) 70 (1.0) 21 (1.0)



Table B.8: Data for Figure 3.5 National Scale Score Differences by Race/Ethnicity

Differences in average geography scale scores by race/ethnicity, grades 4, 8, and 12: 1994 and 2001

White-Black White-Hispanic

Grade 4 1994 50 (2.9) 35 (2.9)

2001 40 (2.0) * 38 (3.0)

Grade 8 1994 40 (1.9) 31 (2.0)

2001 38 (2.0) 33 (2.0)

Grade 12 1994 33 (1.6) 23 (1.7)

2001 32 (1.7) 21 (1.8)

Standard errors of the estimated difference in scale scores appear in parentheses.*Significantly different from 1994.Score differences are calculated based on differences between unrounded average scale scores.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Table B.7: Data for Figure 3.4 National Scale Score Results by Race/Ethnicity

Percentage of students and average geography scale scores by race/ethnicity, grades 4, 8, and 12:1994 and 2001

Asian/ AmericanWhite Black Hispanic Pacific Islander Indian

Grade 4 1994 69 (0.2) 15 (0.1) 12 (0.2) 3 (0.2) 1 (0.2)218 (1.5) 168 (2.5) 183 (2.5) 214 (3.8) 193 (3.6)

2001 64 (0.3) 14 (0.2) 16 (0.3) 3 (0.2) 2 (0.2)222 (1.0) 181 (1.8) * 184 (2.8) 212 (2.7) 199 (3.6)

Grade 8 1994 69 (0.2) 15 (0.1) 11 (0.1) 3 (0.4) 2 (0.4)270 (0.8) 229 (1.7) 239 (1.9) 264 (5.2) 248 (3.4) !

2001 66 (0.3) 14 (0.2) 14 (0.2) 4 (0.2) 1 (0.2)273 (1.0) 234 (1.7) 240 (1.7) 266 (2.5) 261 (5.8)

Grade 12 1994 74 (0.3) 12 (0.4) 8 (0.2) 4 (0.2) 1 (0.2)291 (0.8) 258 (1.4) 268 (1.5) 285 (2.7) *** (***)

2001 70 (0.3) 13 (0.3) 12 (0.2) 5 (0.2) 1 (0.2)291 (0.9) 260 (1.4) 270 (1.5) 286 (2.9) 288 (3.6) !

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.* Significantly different from 1994.! The nature of the sample does not allow accurate determination of the variability of the statistic.***(***) Sample size is insufficient to permit a reliable estimate.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.




Table B.9: Data for Figure 3.6a, b, c National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levels by race/ethnicity,grades 4, 8, and 12: 1994 and 2001

Grade 4 White 1994 19 (1.3) 53 (1.3) 25 (1.5) 4 (0.6) 81 (1.3) 29 (1.6)2001 13 (1.3) * 58 (1.8) 26 (1.6) 3 (0.5) 87 (1.3) * 29 (1.5)

Black 1994 66 (2.4) 32 (2.4) 2 (0.6) # (***) 34 (2.4) 3 (0.6)2001 56 (2.1) * 39 (2.1) 5 (0.8) # (***) 44 (2.1) * 5 (0.9)

Hispanic 1994 51 (2.7) 39 (2.0) 9 (1.7) 1 (0.4) 49 (2.7) 10 (1.7)2001 51 (3.0) 43 (2.5) 6 (1.0) # (***) 49 (3.0) 6 (1.0)

Asian/Pacific Islander 1994 24 (4.0) 49 (4.3) 23 (3.9) 4 (2.2) 76 (4.0) 27 (4.4)2001 23 (3.4) 52 (4.4) 23 (3.1) 1 (0.9) 77 (3.4) 25 (3.0)

American Indian 1994 38 (5.7) 53 (5.8) 9 (3.6) # (***) 62 (5.7) 9 (3.9)2001 34 (4.9) 53 (6.3) 13 (4.2) # (***) 66 (4.9) 13 (4.1)

Grade 8 White 1994 18 (0.9) 47 (1.2) 30 (1.2) 5 (0.5) 82 (0.9) 36 (1.3)2001 14 (0.9) * 48 (1.2) 34 (1.5) 5 (0.8) 86 (0.9) * 39 (1.7)

Black 1994 66 (2.9) 30 (2.8) 4 (0.7) # (0.3) 34 (2.9) 5 (0.7)2001 60 (2.3) 34 (1.9) 6 (0.8) # (***) 40 (2.3) 6 (0.8)

Hispanic 1994 50 (3.6) 39 (3.1) 10 (1.2) 1 (0.5) 50 (3.6) 10 (1.2)2001 52 (1.9) 38 (1.6) 9 (1.1) 1 (0.2) 48 (1.9) 10 (1.0)


American Indian 1994 41 (5.1) ! 43 (4.9) ! 13 (3.5) ! 2 (1.3) ! 59 (5.1) ! 15 (3.6) !2001 28 (6.8) 41 (11.1) 29 (8.9) 3 (***) 72 (6.8) 31 (11.2)

Grade 12 White 1994 22 (0.9) 46 (1.3) 31 (1.2) 2 (0.6) 78 (0.9) 33 (1.5)2001 19 (0.9) 51 (1.1) * 29 (1.2) 2 (0.4) 81 (0.9) 31 (1.4)

Black 1994 68 (2.3) 27 (2.1) 5 (1.0) # (***) 32 (2.3) 5 (1.0)2001 65 (2.3) 31 (2.1) 4 (0.7) # (***) 35 (2.3) 4 (0.7)

Hispanic 1994 52 (2.8) 38 (2.4) 10 (1.7) # (***) 48 (2.8) 10 (1.8)2001 48 (2.6) 42 (2.5) 10 (1.4) # (0.1) 52 (2.6) 10 (1.4)


American Indian 1994 *** (***) *** (***) *** (***) *** (***) *** (***) *** (***)2001 26 (6.0) ! 41 (7.0) ! 31 (5.3) ! 1 (***) ! 74 (6.0) ! 32 (4.9) !

Standard errors of the estimated percentages appear in parentheses.* Significantly different from 1994.# Percentage is between 0.0 and 0.5.! The nature of the sample does not allow accurate determination of the variability of the statistic.(***) Standard error estimates cannot be accurately determined.***(***) Sample size is insufficient to permit a reliable estimate.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.


Table B.10: Data for Figure 3.7 National Scale Score Results by Region of the Country

Percentage of students and average geography scale scores by region of the country,grades 4, 8, and 12: 1994 and 2001

Northeast Southeast Central West

Grade 4 1994 22 (0.8) 23 (1.0) 25 (0.8) 30 (0.7)203 (2.7) 200 (2.5) 215 (3.2) 205 (1.7)

2001 21 (0.8) 24 (1.3) 24 (0.4) 31 (1.3)214 (2.8) * 207 (2.1) 219 (1.8) 200 (2.5)

Grade 8 1994 20 (0.8) 25 (1.0) 24 (0.6) 31 (0.7)266 (1.9) 252 (1.6) 268 (1.6) 255 (1.8)

2001 21 (0.8) 22 (1.0) 25 (0.6) 32 (1.2)266 (2.4) 260 (2.0) * 270 (2.5) 255 (1.5)

Grade 12 1994 21 (0.5) 23 (0.8) 28 (0.7) 29 (0.7)284 (1.6) 278 (1.1) 289 (1.8) 286 (1.9)

2001 20 (0.9) 21 (1.2) 27 (0.6) 31 (1.4)286 (2.8) 281 (1.0) 287 (1.3) 283 (1.3)

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.* Significantly different from 1994.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.




Table B.11: Data for Figure 3.8a, b, c National Achievement-Level Results by Region of the Country

Percentage of students within and at or above geography achievement levels by region of thecountry, grades 4, 8, and 12: 1994 and 2001

Grade 4 Northeast 1994 33 (2.7) 45 (2.5) 19 (2.1) 3 (0.8) 67 (2.7) 22 (2.5)2001 22 (3.7) 54 (3.7) 22 (2.1) 3 (0.9) 78 (3.7) 24 (2.2)

Southeast 1994 36 (2.6) 48 (2.2) 14 (1.9) 2 (0.5) 64 (2.6) 17 (2.0)2001 28 (2.5) 54 (2.7) 17 (1.8) 1 (0.6) 72 (2.5) 18 (1.9)

Central 1994 22 (2.6) 49 (2.3) 25 (3.2) 4 (1.3) 78 (2.6) 28 (3.3)2001 18 (1.7) 51 (1.8) 27 (2.3) 3 (0.7) 82 (1.7) 30 (2.5)

West 1994 30 (1.7) 48 (1.9) 19 (2.0) 3 (0.6) 70 (1.7) 21 (1.7)2001 34 (2.7) 52 (2.4) 13 (1.5) 1 (0.3) 66 (2.7) 14 (1.7) *

Grade 8 Northeast 1994 24 (2.2) 43 (1.6) 28 (1.8) 6 (1.0) 76 (2.2) 33 (2.0)2001 22 (2.5) 44 (2.1) 29 (3.2) 4 (1.3) 78 (2.5) 34 (3.3)

Southeast 1994 38 (2.1) 40 (1.8) 19 (1.4) 3 (0.5) 62 (2.1) 21 (1.6)2001 27 (2.4) * 46 (1.7) 24 (1.5) 3 (0.6) 73 (2.4) * 26 (1.6)

Central 1994 20 (1.7) 44 (1.9) 30 (1.9) 6 (0.9) 80 (1.7) 36 (2.1)2001 18 (2.3) 43 (2.4) 32 (3.1) 6 (1.3) 82 (2.3) 38 (3.7)

West 1994 33 (2.4) 45 (1.8) 20 (1.8) 3 (0.7) 67 (2.4) 23 (2.0)2001 34 (1.7) 44 (1.7) 21 (1.6) 2 (0.6) 66 (1.7) 23 (1.7)

Grade 12 Northeast 1994 31 (2.3) 44 (2.1) 23 (1.8) 2 (0.6) 69 (2.3) 25 (2.1)2001 29 (2.3) 46 (2.4) 24 (3.2) 2 (1.1) 71 (2.3) 26 (4.1)

Southeast 1994 40 (1.5) 41 (1.5) 19 (1.4) 1 (0.6) 60 (1.5) 20 (1.3)2001 33 (1.6) * 46 (1.4) 20 (1.2) 1 (0.3) 67 (1.6) * 21 (1.3)

Central 1994 25 (2.1) 43 (2.3) 30 (2.2) 2 (1.0) 75 (2.1) 32 (2.9)2001 24 (1.8) 48 (1.8) 27 (1.8) 1 (0.5) 76 (1.8) 28 (1.9)

West 1994 28 (2.1) 43 (1.6) 27 (2.3) 2 (0.7) 72 (2.1) 29 (2.6)2001 30 (1.9) 47 (1.7) 22 (1.7) 1 (0.4) 70 (1.9) 23 (1.8)



Table B.12: Data for Figure 3.9 National Scale Score Results by Parents’ Education

Percentage of students and average geography scale scores by parents’ highest level of education,grades 8 and 12: 1994 and 2001

Some educationLess than Graduated after Graduated

high school high school high school college Unknown

Grade 8 1994 7 (0.5) 22 (0.9) 19 (0.7) 42 (1.2) 10 (0.5)238 (1.7) 250 (1.2) 265 (1.0) 272 (1.0) 234 (1.5)

2001 6 (0.4) 18 (0.5) 19 (0.6) 48 (1.2) 9 (0.6)241 (1.7) 253 (1.2) 266 (1.0) 274 (0.9) 245 (1.5) *

Grade 12 1994 7 (0.4) 22 (0.8) 25 (0.7) 44 (1.2) 3 (0.2)263 (1.2) 274 (1.1) 286 (1.0) 294 (0.9) 257 (2.8)

2001 7 (0.4) 19 (0.7) 25 (0.7) 46 (1.1) 3 (0.3)269 (1.7) * 276 (0.9) 284 (0.9) 293 (1.1) 257 (2.9)



Table B.13: Data for Figure 3.10a, b National Achievement-Level Results by Parents’ Education

Percentage of students within and at or above geography achievement levels by parents’ highestlevel of education, grades 8 and 12: 1994 and 2001



Standard errors of the estimated percentages appear in parentheses.* Significantly different from 1994.# Percentage is between 0.0 and 0.5.(***) Standard error estimates cannot be accurately determined.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Grade 8Less than high school 1994 53 (3.4) 39 (3.4) 8 (1.4) 1 (***) 47 (3.4) 8 (1.6)

2001 52 (2.6) 41 (3.6) 8 (2.6) # (***) 48 (2.6) 8 (2.6)

Graduated high school 1994 38 (2.0) 47 (2.1) 14 (1.6) 1 (0.6) 62 (2.0) 15 (1.5)2001 34 (2.1) 48 (1.5) 16 (2.1) 1 (0.6) 66 (2.1) 18 (1.9)

Some education after high school 1994 21 (1.3) 50 (2.7) 26 (2.5) 3 (0.8) 79 (1.3) 29 (2.3)2001 20 (1.6) 51 (1.8) 27 (1.7) 2 (0.8) 80 (1.6) 30 (1.8)

Graduated College 1994 18 (1.2) 41 (1.4) 34 (1.3) 7 (0.7) 82 (1.2) 41 (1.4)2001 14 (0.9) 42 (1.5) 37 (1.3) 6 (0.9) 86 (0.9) 43 (1.5)

Unknown 1994 56 (2.9) 35 (3.2) 8 (1.6) 1 (***) 44 (2.9) 8 (1.5)2001 44 (2.6) * 44 (3.1) 11 (1.9) 1 (0.5) 56 (2.6) * 12 (1.9)

Grade 12Less than high school 1994 59 (2.9) 34 (3.5) 7 (2.0) 0 (***) 41 (2.9) 7 (2.0)

2001 52 (3.5) 38 (3.2) 10 (1.5) # (***) 48 (3.5) 10 (1.5)

Graduated high school 1994 44 (2.0) 42 (2.5) 13 (1.6) # (0.3) 56 (2.0) 14 (1.6)2001 38 (2.0) 50 (1.8) * 12 (1.6) # (***) 62 (2.0) 12 (1.6)

Some education after high school 1994 25 (1.5) 51 (1.7) 23 (1.4) 1 (***) 75 (1.5) 24 (1.8)2001 27 (1.2) 52 (1.7) 20 (1.6) 1 (0.3) 73 (1.2) 21 (1.7)

Graduated College 1994 19 (1.1) 41 (1.3) 37 (1.5) 3 (0.8) 81 (1.1) 40 (1.6)2001 18 (1.1) 46 (1.4) * 34 (1.5) 3 (0.6) 82 (1.1) 36 (1.8)

Unknown 1994 64 (4.4) 29 (4.1) 7 (1.6) 0 (***) 36 (4.4) 7 (1.6)2001 67 (4.2) 28 (4.3) 5 (2.0) 0 (***) 33 (4.2) 5 (2.0)


Table B.14: Data for Figure 3.11 National Scale Score Results by Type of School

Percentage of students and average geography scale scores by type of school, grades 4, 8, and 12:1994 and 2001

Public Nonpublic Nonpublic: Catholic Nonpublic: Other

Grade 4 1994 90 (0.8) 10 (0.8) 6 (0.7) 4 (0.5)204 (1.4) 221 (2.2) 222 (2.6) 220 (3.8)

2001 89 (1.2) 11 (1.2) 6 (0.8) 5 (0.9)207 (1.1) 226 (2.2) 230 (1.7) 221 (4.4)

Grade 8 1994 90 (0.8) 10 (0.8) 6 (0.6) 4 (0.6)258 (0.8) 276 (1.3) 276 (1.6) 276 (2.6)

2001 90 (0.9) 10 (0.9) 5 (0.6) 5 (0.7)261 (1.0) * 274 (2.5) 277 (2.0) 271 (4.7)

Grade 12 1994 89 (1.0) 11 (1.0) 6 (0.9) 4 (0.6)283 (0.8) 294 (1.6) 291 (3.0) 298 (2.0)

2001 92 (0.8) 8 (0.8) 4 (0.6) 3 (0.6)284 (0.8) 291 (2.3) 294 (2.0) 287 (4.3)



Table B.15: Data for Figure 3.12a, b, c National Achievement-Level Results by Type of School

Percentage of students within and at or above geography achievement levels by type of school,grades 4, 8, and 12: 1994 and 2001




Grade 4 Public 1994 32 (1.2) 47 (1.1) 19 (1.2) 3 (0.5) 68 (1.2) 21 (1.3)2001 28 (1.3) 52 (1.6) * 18 (1.1) 2 (0.3) 72 (1.3) 20 (1.1)

Nonpublic 1994 16 (2.2) 53 (2.3) 26 (2.3) 5 (1.2) 84 (2.2) 30 (2.5)2001 9 (2.2) 59 (2.1) 28 (2.6) 3 (0.9) 91 (2.2) 31 (2.8)

Nonpublic: Catholic 1994 15 (2.8) 54 (3.2) 25 (2.4) 5 (1.9) 85 (2.8) 30 (3.0)2001 7 (1.2) * 56 (2.3) 32 (2.0) 4 (1.6) 93 (1.2) * 36 (2.5)

Nonpublic: other 1994 18 (3.5) 52 (3.6) 26 (3.7) 4 (1.2) 82 (3.5) 30 (4.1)2001 12 (5.1) 63 (4.1) 23 (4.9) 2 (1.1) 88 (5.1) 25 (5.5)

Grade 8 Public 1994 31 (1.0) 43 (1.1) 22 (1.0) 4 (0.4) 69 (1.0) 26 (1.0)2001 28 (0.9) * 44 (1.0) 25 (1.2) 3 (0.6) 72 (0.9) * 28 (1.2)

Nonpublic 1994 13 (1.7) 43 (2.3) 36 (2.3) 8 (1.4) 87 (1.7) 44 (2.2)2001 13 (2.6) 46 (3.6) 36 (3.5) 5 (1.3) 87 (2.6) 41 (4.2)

Nonpublic: Catholic 1994 11 (1.8) 45 (2.2) 35 (2.8) 8 (1.6) 89 (1.8) 44 (2.6)2001 11 (2.5) 44 (3.4) 40 (2.9) 6 (1.3) 89 (2.5) 46 (3.4)

Nonpublic: other 1994 14 (3.2) 41 (4.7) 38 (4.6) 7 (2.0) 86 (3.2) 45 (4.8)2001 15 (4.7) 48 (5.1) 33 (6.2) 4 (1.8) 85 (4.7) 37 (7.2)

Grade 12 Public 1994 32 (1.0) 42 (1.1) 24 (1.0) 1 (0.5) 68 (1.0) 26 (1.3)2001 29 (1.0) 47 (1.0) * 23 (1.1) 1 (0.3) 71 (1.0) 24 (1.2)

Nonpublic 1994 17 (2.2) 47 (1.7) 33 (2.3) 3 (0.7) 83 (2.2) 36 (2.3)2001 20 (3.0) 48 (2.0) 30 (2.6) 2 (0.7) 80 (3.0) 32 (3.0)

Nonpublic: Catholic 1994 20 (3.9) 47 (2.0) 32 (3.6) 1 (0.5) 80 (3.9) 33 (3.8)2001 15 (2.5) 51 (2.5) 32 (3.4) 2 (0.6) 85 (2.5) 34 (3.4)

Nonpublic: other 1994 13 (2.4) 47 (2.9) 35 (2.9) 5 (1.5) 87 (2.4) 40 (3.1)2001 27 (5.5) 44 (3.1) 26 (4.7) 2 (1.3) 73 (5.5) 29 (5.6)


Table B.17: Data for Figure 3.13 National Achievement-Level Results by Type of Location

Percentage of students within and at or above geography achievement levelsby type of school location, grades 4, 8, and 12: 2001



Percentage of students and average geography scale scores by type of school location, grades 4, 8,and 12: 2001

Central city Urban fringe/large town Rural/small town

Table B.16: Data for Table 3.1 National Scale Score Results by Type of Location

Grade 4 27 (1.6) 44 (2.9) 29 (2.8)199 (2.3) 212 (2.1) 215 (2.0)

Grade 8 27 (2.0) 45 (2.8) 28 (2.5)255 (2.0) 265 (1.6) 265 (2.0)

Grade 12 26 (2.0) 40 (2.8) 34 (2.5)279 (1.6) 288 (1.6) 284 (1.0)

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Standard errors of the estimated percentages appear in parentheses.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Grade 4 Central city 38 (2.4) 46 (1.8) 14 (1.6) 2 (0.4) 62 (2.4) 16 (1.8)

Urban fringe/large town 24 (2.1) 53 (1.7) 21 (1.5) 2 (0.6) 76 (2.1) 23 (1.7)

Rural/small town 19 (2.1) 58 (3.0) 21 (2.4) 2 (0.5) 81 (2.1) 23 (2.4)



Rural/small town 22 (2.2) 48 (2.0) 27 (2.4) 3 (0.7) 78 (2.2) 30 (2.7)



Rural/small town 26 (1.5) 52 (1.5) 21 (1.4) 1 (0.3) 74 (1.5) 22 (1.4)


Table B.19: Data for Figure 3.14 National Achievement-Level Results by Free/Reduced-Price SchoolLunch Program Eligibility

Percentage of students within and at or above geography achievement levels by student eligibilityfor the Free/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001



Standard errors of the estimated percentages appear in parentheses.# Percentage is between 0.0 and 0.5.(***) Standard error estimates cannot be accurately determined.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Grade 4 Eligible 49 (2.2) 45 (2.1) 6 (0.9) # (***) 51 (2.2) 6 (0.9)

Not eligible 14 (1.1) 56 (1.6) 27 (1.4) 3 (0.6) 86 (1.1) 29 (1.5)

Info not available 16 (2.5) 57 (2.9) 24 (3.1) 3 (0.8) 84 (2.5) 27 (3.2)

Grade 8 Eligible 50 (1.8) 39 (1.6) 10 (1.1) 1 (0.3) 50 (1.8) 11 (1.2)

Not eligible 17 (0.9) 46 (1.3) 32 (1.5) 5 (0.8) 83 (0.9) 37 (1.7)


Grade 12 Eligible 49 (2.3) 40 (1.7) 10 (1.5) # (***) 51 (2.3) 11 (1.6)

Not eligible 25 (1.2) 49 (1.3) 25 (1.4) 1 (0.4) 75 (1.2) 26 (1.6)


Table B.18: Data for Table 3.2 National Scale Score Results by Free/Reduced-Price School LunchProgram Eligibility

Percentage of students and average geography scale scores by student eligibility forFree/Reduced-Price School Lunch program, grades 4, 8, and 12: 2001

Eligible Not eligible Info not available

Grade 4 33 (1.4) 48 (2.3) 18 (2.4)186 (1.7) 221 (1.2) 218 (2.5)

Grade 8 25 (1.1) 53 (2.1) 22 (2.3)242 (1.4) 270 (1.1) 266 (1.8)

Grade 12 16 (1.0) 64 (2.2) 21 (2.4)269 (1.6) 287 (1.0) 289 (1.5)



Percentage of students and average geography scale scores by teachers’ reported undergraduate/graduate major and minor/special emphasis, grades 4 and 8: 2001

Table B.20: Data for Table 4.1 Teachers’ Majors/Minors

Grade 4Geography or Geography Education 7 (1.0) 93 (1.0)

204 (5.2) 210 (1.1)History or History Education 15 (1.5) 85 (1.5)

206 (3.6) 211 (1.1)Social Science or Social Studies Education 20 (1.7) 80 (1.7)

208 (2.6) 210 (1.3)Other Social Science 57 (2.5) 43 (2.5)

210 (1.6) 209 (1.4)Elementary Education 93 (1.0) 7 (1.0)

211 (1.1) 197 (4.3)

Grade 8Geography or Geography Education 28 (2.5) 72 (2.5)

263 (2.1) 263 (1.2)History or History Education 71 (2.7) 29 (2.7)

263 (1.3) 261 (1.9)Social Science or Social Studies Education 55 (3.0) 45 (3.0)

263 (1.3) 262 (1.5)Other Social Science 51 (2.1) 49 (2.1)

261 (1.5) 264 (1.4)The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Yes NoDid you have a major, minor, or special emphasis in anyof the following subjects as part of your undergraduateor graduate course work?


Percentage of students and average geography scale scores by teachers’ reports on how well pre-pared they felt they were to teach geography, grades 4 and 8: 1994 and 2001

Table B.21: Data for Table 4.2 Teachers’ Preparedness to Teach Geography

Grade 4Very prepared 23 (2.0) 31 (2.1) *

209 (2.1) 211 (2.1)Adequately prepared 57 (2.0) 53 (2.3)

206 (1.8) 210 (1.3)Somewhat prepared 18 (1.8) 15 (1.5)

207 (2.8) 206 (2.4)Unprepared 2 (0.5) 1 (0.3)

200 (8.9) ! 209 (8.6) !

Grade 8Very prepared 36 (2.9) 44 (2.5)

260 (2.2) 263 (1.4)Adequately prepared 48 (3.6) 43 (2.4)

262 (1.4) 262 (1.3)Somewhat prepared 13 (2.3) 11 (1.7)

265 (2.9) 261 (2.5)Unprepared 2 (***) 2 (0.6)

260 (3.7) ! 264 (8.9) !The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.! The nature of the sample does not allow accurate determination of the variability of the statistic.* Significantly different from 1994.(***) Standard error estimates cannot be accurately determined.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Regardless or whether you are currently teaching thetopic, how well prepared do you feel you are to teachgeography at the elementary/middle school level?

1994 2001


Using maps and globesAlmost every day 29 (2.3) 28 (2.3)

210 (2.1) 213 (1.9)Once or twice a week 54 (2.5) 47 (2.2)

208 (1.7) 209 (1.5)Once or twice a month 17 (2.1) 22 (1.7)

199 (3.2) 206 (2.4)Never or hardly ever 1 (0.3) 3 (0.7)

*** (***) 209 (8.9) !

Natural resourcesAlmost every day 9 (1.8) 9 (1.4)

201 (4.1) 217 (4.6)Once or twice a week 38 (2.5) 31 (1.6) *


208 (2.2) 210 (1.7)Never or hardly ever 9 (1.4) 14 (1.4) *

198 (4.8) 208 (3.1)

Foreign countries and culturesAlmost every day 6 (1.2) 3 (0.8)

206 (5.7) 206 (6.3) !Once or twice a week 19 (1.9) 23 (1.7)



209 (1.9) 212 (2.2)

Environmental issuesAlmost every day 4 (0.9) 7 (1.5)

201 (5.6) ! 212 (3.6) !Once or twice a week 27 (2.1) 21 (1.7)



208 (3.9) 211 (3.0)

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.! The nature of the sample does not allow accurate determination of the variability of the statistic.* Significantly different from 1994.*** (***) Sample size is insufficient to permit a reliable estimate.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 1994 and 2001 Geography Assessments.

Percentage of students and average geography scale scores by teachers’ reports on frequency ofinstruction of selected skills and topics, grade 4:1994 and 2001

Table B.22: Data for Table 4.3 Frequency of Instruction in Fourth-Grade Geography Skills and Topics

1994 2001How often do you teach the following skills and topics as apart of geography instruction with this class?


Using maps and globesAlmost every day 9 (0.6) 12 (0.7) *




253 (1.0) 258 (1.2)





260 (0.9) 263 (1.2)

Countries and culturesAlmost every day 23 (0.8) 31 (1.0) *


261 (1.1) 266 (1.2)Once or twice a month 28 (0.9) 24 (0.6) *


256 (1.3) 254 (1.6)





260 (0.9) 262 (1.3)

Percentage of students and average geography scale scores by students’ reports on frequency ofinstruction of selected skills and topics, grade 8 : 1994 and 2001

Table B.23a: Data for Table 4.4a Frequency of Instruction in Eighth-Grade Geography Skills and Topics





Percentage of students and average geography scale scores by students’ reports on frequency ofinstruction of selected skills and topics, grade 12: 1994 and 2001

Table B.23b: Data for Table 4.4b Frequency of Instruction in Twelfth-Grade Geography Skills and Topics

Using maps and globesAlmost every day 7 (0.4) 6 (0.4)

284 (2.0) 277 (1.5)Once or twice a week 22 (0.7) 24 (0.5)



283 (1.1) 284 (1.0)





284 (0.9) 285 (1.0)

Countries and culturesAlmost every day 16 (0.6) 20 (0.5) *




280 (1.0) 277 (1.0)





282 (0.9) 282 (0.9)



Percentage of students and average geography scale scores by students’ reports on grades in whichgeography was taken since the 6th grade, grade 8: 1994 and 2001

Table B.24: Data for Table 4.5 Eighth-Grade Frequency of Geography Course Taking


Number of grades selectedNone 18 (1.0) 12 (0.7)*

250 (1.6) 255 (1.7)

One 30 (0.7) 20 (0.6)*257 (1.1) 256 (1.5)

Two 14 (0.9) 16 (0.6)269 (1.4) 263 (1.3)

Three 26 (0.9) 43 (1.1)*274 (0.9) 272 (1.1)

Don’t know 13 (0.6) 9 (0.5)243 (1.5) 246 (1.5)

1994 2001Did you take or do you expect to take a geography coursein 6th, 7th, or 8th grade?


Percentage of students and average geography scale scores by students’ reports on grades in whichgeography was taken since 9th grade, grade 12: 1994 and 2001

Table B.25: Data for Table 4.6 Twelfth-Grade Frequency of Geography Course Taking


Number of grades selectedNone 31 (1.6) 21 (1.2) *

286 (1.4) 289 (1.3)

One 35 (1.4) 32 (1.4)288 (0.9) 288 (1.2)

Two 16 (0.8) 15 (0.7)286 (1.5) 285 (1.3)

Three 10 (0.6) 18 (0.9)*281 (1.7) 280 (1.1)

Four 5 (0.5) 10 (0.6) *277 (2.5) 281 (1.3)

Don’t know 3 (0.4) 3 (0.3)268 (2.1) 265 (2.6)

Did you take or do you expect to take a geography coursein 9th, 10th, 11th, or 12th grade?

1994 2001


Percentage of students and average geography scale scores by teachers’ reports on computer use forsocial studies instruction, grade 4: 2001

Table B.26a: Data for Table 4.7a Fourth-Grade Computer Use

Grade 4

Use CD-ROM to look up reference worksNot at all 37 (2.5)

205 (2.0)Small extent 47 (2.4)

211 (1.6)Moderate extent 14 (1.8)

216 (2.4)Large extent 2 (0.9)

214 (7.9) !

Retrieve information through the InternetNot at all 34 (2.3)

203 (2.3)Small extent 45 (2.3)


216 (3.0)Large extent 4 (1.1)

211 (6.3) !

Use exploration/simulation softwareNot at all 54 (2.4)

207 (1.4)Small extent 37 (2.4)


211 (3.7)Large extent 1 (0.2)

*** (***)

Organize information using spreadsheets/databasesNot at all 89 (1.4)

209 (1.2)Small extent 9 (1.3)


213 (8.5) !Large extent # (0.2)

*** (***)


The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.# Percentage is between 0.0 and 0.5.! The nature of the sample does not allow accurate determination of the variability of the statistic.*** (***) Sample size is insufficient to permit a reliable estimate.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.


Percentage of students and average geography scale scores by teachers’ reports on computer use forsocial studies instruction, grade 8: 2001

Table B.26b: Data for Table 4.7b Eighth-Grade Computer Use

2001When students in this class work on social studies, to whatextent do they use computers to do each of the following?Grade 8

Use CD-ROM to look up reference worksNot at all 31 (2.0)

258 (1.3)Small extent 48 (2.3)


266 (2.4)Large extent 4 (0.9)

268 (4.7) !

Retrieve information through the InternetNot at all 20 (1.9)

255 (2.0)Small extent 47 (2.4)


266 (1.9)Large extent 4 (0.8)

273 (3.8)


261 (1.2)Small extent 32 (2.3)


259 (3.4) !Large extent 1 (0.4)

257 (11.0) !

Organize information using spreadsheets/databasesNot at all 74 (2.7)

261 (1.2)Small extent 22 (2.7)


262 (6.2) !Large extent 1 (0.5)

249 (5.5) !

The percentage of students is listed first with the corresponding average scale score presented below.Standard errors of the estimated percentages and scale scores appear in parentheses.! The nature of the sample does not allow accurate determination of the variability of the statistic.NOTE: Percentages may not add to 100 due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.


Percentage of students and average geography scale scores by students’ reports on computer use forhistory and geography, grade 12: 2001

Table B.27: Data for Table 4.8 Twelfth-Grade Computer Use

Research projects using a CD or the InternetNot at all 26 (0.8)

274 (1.0)Small extent 32 (0.5)


290 (1.1)Large extent 13 (0.6)

292 (1.3)


287 (0.8)Small extent 23 (0.5)


276 (1.4)Large extent 2 (0.2)

278 (3.3)

Tables, charts or graphs on the computerNot at all 55 (0.9)

284 (0.7)Small extent 30 (0.7)


281 (1.6)Large extent 4 (0.3)

277 (2.7)


2001Think about all the courses since the 9th grade in which youhave studied history or geography. To what extent haveyou used computers to do the following? For this questioninclude both work in class and homework assignments.


Percentage of students and average geography scale scores by students’ reports on how much theylike studying geography, grades 8 and 12: 1994 and 2001

Table B.28: Data for Table 4.9 How Much Eighth- and Twelfth-Grade Students Like Geography

Grade 8One of my favorite subjects 19 (0.8) 20 (0.6)

274 (1.2) 275 (1.3)Like other subjects better 67 (0.9) 69 (0.6)

260 (0.7) 263 (1.0)Never studied geography 14 (0.6) 11 (0.5) *

241 (1.9) 247 (1.7)

Grade 12One of my favorite subjects 14 (0.6) 15 (0.6)

297 (1.3) 293 (1.2)Like other subjects better 63 (1.1) 72 (0.8) *

285 (0.8) 285 (0.8)Never studied geography 23 (1.2) 13 (0.8) *

277 (1.3) 278 (1.8)


1994 2001How much do you like studying geography?


Table B.29: Data for Table 5.1 Comparison of Two Sets of National Scale Score Results

National average geography scale scores by type of results, grades 4, 8, and 12: 2001

Accommodations not permitted Accommodations permitted

Grade 4 209 (1.0) 208 (0.9)

Grade 8 262 (0.9) 260 (1.0) �

Grade 12 285 (0.8) 284 (0.8)

Standard errors of the estimated scale scores appear in parentheses.� Significantly different from the sample where accommodations were not permitted.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Percentage of students within and at or above geography achievement levels by type of results,grades 4, 8, and 12: 2001

Table B.30: Data for Table 5.2 Comparison of Two Sets of National Achievement-Level Results



Grade 4Accommodations were not permitted 26 (1.2) 53 (1.4) 19 (1.1) 2 (0.3) 74 (1.2) 21 (1.0)

Accommodations were permitted 27 (1.0) 52 (1.1) 19 (0.9) 2 (0.3) 73 (1.0) 20 (0.9)


Accommodations were permitted 28 (1.2) � 43 (0.9) 25 (1.0) 4 (0.5) 72 (1.2) � 29 (1.3)



Standard errors of the estimated percentages appear in parentheses.� Significantly different from the sample where accommodations were not permitted.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.


National average geography scale scores by gender and type of results, grades 4, 8, and 12: 2001

Table B.31: Data for Table 5.3 Comparison of Two Sets of National Scale Score Results by Gender

Male Female

Grade 4Accommodations were not permitted 212 (1.1) 207 (1.2)

Accommodations were permitted 210 (1.0) 206 (1.3)


Accommodations were permitted 262 (1.2) � 258 (1.0)


Accommodations were permitted 287 (1.0) 281 (0.8)

Standard errors of the estimated scale scores appear in parentheses.� Significantly different from the sample where accommodations were not permitted.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Table B.32: Table 5.4 Comparison of Two Sets of National Achievement-Level Results by Gender



Percentage of students within and at or above geography achievement levels by gender andtype of results, grades 4, 8, and 12: 2001

Grade 4MaleAccommodations were not permitted 25 (1.3) 51 (1.6) 21 (1.4) 3 (0.5) 75 (1.3) 24 (1.4)


FemaleAccommodations were not permitted 28 (1.6) 54 (1.7) 17 (1.2) 1 (0.4) 72 (1.6) 18 (1.1)










Standard errors of the estimated percentages appear in parentheses.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.


National average geography scale scores by race/ethnicity and type of results, grades 4, 8, and 12:2001

Table B.33: Data for Table 5.5 Comparison of Two Sets of National Scale Score Results by Race/Ethnicity

Asian/Pacific AmericanWhite Black Hispanic Islander Indian

Grade 4Accommodations were not permitted 222 (1.0) 181 (1.8) 184 (2.8) 212 (2.7) 199 (3.6)

Accommodations were permitted 220 (1.0) 181 (1.9) 185 (2.1) 216 (2.6) 199 (3.4)

Grade 8Accommodations were not permitted 273 (1.0) 234 (1.7) 240 (1.7) 266 (2.5) 261 (5.8)

Accommodations were permitted 271 (1.4) 232 (1.6) 238 (1.8) 267 (2.2) 259 (4.9)

Grade 12Accommodations were not permitted 291 (0.9) 260 (1.4) 270 (1.5) 286 (2.9) 288 (3.6) !

Accommodations were permitted 292 (0.8) 258 (1.5) 269 (1.4) 285 (5.0) 286 (3.5) !

Standard errors of the estimated scale scores appear in parentheses.! The nature of the sample does not allow accurate determination of the variability of the statistic.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.


Table B.34: Data for Table 5.6 Comparison of Two Sets of National Achievement-Level Results by Race/Ethnicity

Percentage of students within and at or above geography achievement levels by race/ethnicity and typeof results, grades 4, 8, and 12: 2001



Standard errors of the estimated percentages appear in parentheses.# Percentage is between 0.0 and 0.5.(***) Standard error estimates cannot be accurately determined.! The nature of the sample does not allow accurate determination of the variability of the statistic.NOTE: Percentages within each geography achievement-level range may not add to 100, or to the exact percentages at or above achievement levels, due to rounding.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics, National Assessment ofEducational Progress (NAEP), 2001 Geography Assessment.

Grade 4White

Accommodations were not permitted 13 (1.3) 58 (1.8) 26 (1.6) 3 (0.5) 87 (1.3) 29 (1.5)Accommodations were permitted 15 (1.0) 57 (1.4) 25 (1.3) 3 (0.5) 85 (1.0) 28 (1.3)

BlackAccommodations were not permitted 56 (2.1) 39 (2.1) 5 (0.8) # (***) 44 (2.1) 5 (0.9)

Accommodations were permitted 56 (2.7) 40 (2.6) 4 (0.5) # (***) 44 (2.7) 4 (0.6)Hispanic

Accommodations were not permitted 51 (3.0) 43 (2.5) 6 (1.0) # (***) 49 (3.0) 6 (1.0)Accommodations were permitted 49 (2.5) 45 (2.1) 5 (0.9) # (***) 51 (2.5) 6 (0.9)

Asian/Pacific IslanderAccommodations were not permitted 23 (3.4) 52 (4.4) 23 (3.1) 1 (0.9) 77 (3.4) 25 (3.0)

Accommodations were permitted 18 (3.4) 57 (4.0) 24 (3.9) 2 (0.8) 82 (3.4) 25 (3.7)American Indian

Accommodations were not permitted 34 (4.9) 53 (6.3) 13 (4.2) # (***) 66 (4.9) 13 (4.1)Accommodations were permitted 37 (5.7) 51 (5.7) 12 (3.1) # (***) 63 (5.7) 12 (3.3)

Grade 8White







Accommodations were not permitted 28 (6.8) 41 (11.1) 29 (8.9) 3 (***) 72 (6.8) 31 (11.2)Accommodations were permitted 30 (5.2) 46 (5.9) 21 (6.0) 3 (***) 70 (5.2) 24 (7.2)

Grade 12White




Accommodations were not permitted 48 (2.6) 42 (2.5) 10 (1.4) # (0.1) 52 (2.6) 10 (1.4)Accommodations were permitted 50 (2.4) 42 (2.2) 9 (1.1) # (***) 50 (2.4) 9 (1.1)



Accommodations were not permitted 26 (6.0) ! 41 (7.0) ! 31 (5.3) ! 1 (***) 74 (6.0) ! 32 (4.9) !Accommodations were permitted 29 (7.6) ! 41 (7.6) ! 29 (6.9) ! 1 (***) 71 (7.6) ! 30 (6.9) !


Table B.36: Data for Table 6.2 Grade 4 Sample Question 2 Results (Multiple-Choice)



Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.***(***)Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



70 (1.4) 50 (2.8) 74 (1.7) 84 (2.5) *** (***)

Grade 4


Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.***(***)Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



33 (1.1) 22 (1.8) 28 (1.8) 56 (3.2) *** (***)

Grade 4


Table B.37: Data for Table 6.3a Grade 4 Sample Question 3 Results (“Complete” Short Constructed-Response)


Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



66 (1.4) 38 (2.3) 71 (2.0) 88 (2.3) *** (***)

Grade 4


Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.*** (***) Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



72 (1.4) 43 (2.5) 78 (1.8) 93 (2.3) *** (***)

Grade 4

Table B.38: Data for Table 6.3b Grade 4 Sample Question 3 Results (“Partial” Short Constructed-Response)


Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 2001


Overall percentage Below Basic Basic Proficient Advanced“Complete” or better 186 and below* 187–239* 240–275* 276 and above*

11 (0.8) 0 (***) 6 (1.2) 32 (3.4) *** (***)


Grade 4

Table B.39: Data for Table 6.4a Grade 4 Sample Question 4 Results (“Complete” Extended Constructed-Response)

Table B.40: Data for Table 6.4b Grade 4 Sample Question 4 Results (“Essential” Extended Constructed-Response)

Overall percentage “Essential” or better and percentages “Essential” or better within each achievement-level range: 2001

Percentage “Essential” or better withinachievement-level intervals

Overall percentage Below Basic Basic Proficient Advanced“Essential” or better 186 and below* 187–239* 240–275* 276 and above*

28 (1.3) 1 (0.6) 25 (2.1) 65 (3.8) *** (***)


Grade 4

Table B.41: Data for Table 6.4c Grade 4 Sample Question 4 Results (“Partial” Extended Constructed-Response)




38 (1.3) 4 (1.5) 36 (2.2) 78 (2.5) *** (***)


Grade 4




Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.(***) Standard error estimates cannot be accurately determined.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



70 (1.2) 37 (2.3) 74 (1.7) 91 (1.5) 97 (***)

Grade 8






50 (1.3) 36 (2.2) 47 (2.3) 64 (3.1) *** (***)

Grade 8




Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



60 (1.4) 40 (2.6) 57 (2.0) 79 (2.7) 96 (1.7)

Grade 8



Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.(***) Standard error estimates cannot be accurately determined.SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



74 (1.4) 40 (2.3) 80 (2.0) 93 (1.9) 100 (***)

Grade 8





22 (1.4) 6 (2.1) 18 (1.9) 38 (2.7) *** (***)



Grade 8




60 (1.3) 26 (2.5) 62 (2.1) 84 (2.3) *** (***)



Grade 8







61 (1.4) 46 (2.3) 62 (2.2) 76 (3.3) *** (***)

Grade 12



Standard errors of the estimated percentages appear in parentheses.*NAEP geography composite scale range.(***) Standard error estimates cannot be accurately determined.***(***)Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.



78 (1.2) 46 (2.3) 86 (1.6) 99 (***) *** (***)

Grade 12



Overall percentage “Complete” and percentages “Complete” within each achievement-level range: 2001



47 (1.3) 17 (1.9) 52 (2.1) 70 (3.2) *** (***)

Standard errors of the estimated percentages appear in parentheses.*NAEP Geography composite scale range.***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12



76 (1.2) 42 (2.8) 85 (1.4) 96 (1.4) *** (***)


Standard errors of the estimated percentages appear in parentheses.*NAEP Geography composite scale range.***(***) Sample size is insufficient to permit a reliable estimate (see appendix A).SOURCE: U.S. Department of Education, Office of Educational Research and Improvement, National Center for EducationStatistics, National Assessment of Educational Progress (NAEP), 2001 Geography Assessment.

Grade 12

Table B.51 Data for Table 6.12b Grade 12 Sample Question 12 Results (“Partial” Short Constructed-Response)




16 (0.9) 2 (0.9) 15 (1.3) 33 (3.0) *** (***)



Grade 12




51 (1.7) 18 (2.1) 57 (2.0) 79 (2.7) *** (***)



Grade 12


A P P E N D I X C • G E O G R A P H Y R E P O R T C A R D 181

Appendix CMembers of theNAEP Geography Standing CommitteeC

Sarah BednarzDepartment of GeographyTexas A&M UniversityCollege Station, TX

Kristi DesaulniersTea Primary SchoolTea, SD

Norman BettisDept. of Curriculum & InstructionIllinois State UniversityNormal, IL

Richard BoehmDepartment of Geography and PlanningSouthwest Texas State UniversitySan Marcos, TX

Leah Bug-TownsendClair E. Gale Jr. High SchoolIdaho Falls, IDPresent affiliation:Idaho/Oregon NASA RepresentativeNASA Ames Research CenterIdaho Falls, ID

Karen ConeyStone Scholastic AcademyChicago, IL

Roger DownsChairman, Department of GeographyThe Pennsylvania State UniversityUniversity Park, PA

Briavel HolcombDepartment of Urban Studies and Community HealthE. J. Bloustein School of PlanningRutgers UniversityNew Brunswick, NJ

Marianne KenneyColorado Department of EducationDenver, COPresent affiliation:McRELAurora, CO

R. Keith LuceroEast High SchoolDenver, CO

James Marran(Retired) New Trier Twp. High SchoolWinnetka, IL

Ines MiyaresDepartment of GeographyHunter CollegeNew York, NY

Barbara PattyArkansas Department of EducationLittle Rock, AR

Tom WissingerHarrington Middle SchoolMt. Laurel, NJ

182 A C K N O W L E D G M E N T S • G E O G R A P H Y R E P O R T C A R D

AcknowledgmentsThis report is the culmination of the effort of many individuals who contributed their considerableknowledge, experience, and creativity to the NAEP 2001 geography assessment. The assessmentwas a collaborative effort among staff from the National Center for Education Statistics (NCES),the National Assessment Governing Board (NAGB), Educational Testing Service (ETS), Westat, andNational Computer Systems (NCS). Most importantly, NAEP is grateful to the students andschool staff who made the assessment possible.

The 2001 NAEP geography assessment was funded through NCES, in the Office of EducationalResearch and Improvement of the U.S. Department of Education. The Deputy Commissioner ofEducation Statistics, Gary W. Phillips, and the NCES staff—Peggy Carr, Arnold Goldstein, CarolJohnson, and Andrew Kolstad—worked closely and collegially with the authors to produce thisreport.

The NAEP project at ETS is directed by Stephen Lazer and John Mazzeo, with assistance fromJohn Barone. Sampling and data collection activities were conducted by Westat under the directionof Rene Slobasky, Nancy Caldwell, Keith Rust, and Dianne Walsh. Printing, distribution, scoring,and processing activities were conducted by NCS under the direction of Brad Thayer, ConnieSmith, and William Buckles.

Test development activities took place at ETS under the direction of Andy Weiss with assistancefrom Barbara Hildebrant and John Kasbarian.

The complex statistical and psychometric activities necessary to report results for the NAEP2001 geography assessment were directed by Catherine Hombo and Matthew Johnson, withassistance from Hui Deng. Frank Jenkins, advised by Matthew Johnson, supervised the analyses thatproduced results for the data for which student accommodations were permitted.

The extensive data processing and computer programming activities underlying the statisticaland psychometric analyses conducted at ETS are under the direction of David Freund, EdwardKulick, Bruce Kaplan, and Steven Isham. Data Analyses presented in this report were managed byScott Davis with assistance from Mei-jang Lin, Christina Tang, Norma Norris, and Alfred Rogers.The complex database work for this assessment was managed by Katherine Pashley with assistancefrom Gerry Kokolis.

The design and production of this report was overseen by Loretta Casalaina. Joseph Kolodey andRick Hasney contributed invaluable design and production expertise to the effort. Wendy Griggcoordinated the documentation and data checking procedures with assistance from Janice Goodis,Andrea Bergen, and Alice Kass. Shari Santapau coordinated the editorial and proofreading proce-dures with assistance from Valerie Mukuna and Julia McGuire. The Web version of this report wascoordinated by Pat O’Reilly with assistance from Rick Hasney.

Many thanks are due to the numerous reviewers, both internal and external to NCES and ETS.The comments and critical feedback of the following reviewers are reflected in the final version ofthis report: Ghedam Bairu, Teresa Calahan, Young Chun, Douglas Cohrane, Mary Crovo, RogerDowns, Steven Gorman, Carol Johnson, Dana Kelly, Andrew Kolstad, Marilyn McMillen, BarbaraPatty, Taslima Rahman, Patrick Rooney and Sharif Shakrani.

United StatesDepartment of EducationED Pubs8242-B Sandy CourtJessup, MD 20794-1398

Official Business OnlyPenalty for Private Use, $300

Postage and Fees PaidU.S. Department of Education

Permit No. G-17

Parcel

The Nation's Report Card: Geography 2001

Documents

Transcript of The Nation's Report Card: Geography 2001