STATISTICAL RESEARCH DIVISION FY 2008 FIRST QUARTER REPORT Q1.pdf · FY 2008 FIRST QUARTER REPORT...

STATISTICAL RESEARCH DIVISIONFY 2008 FIRST QUARTER REPORT

–October, November, December 2007–

COLLABORATION

DECENNIAL DIRECTORATE

Decennial Management Division/Decennial Statistical Studies Division/American Community Survey Office(Sponsors)

Project ProjectNumber Title FTEs5210801 Forms Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.965210802 Content Planning and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

A. Census Questionnaire Design Features B. Short Form Questionnaire Content Other Than Race and Ethnicity C. Development of Race and Ethnicity Questions

5210803 Language Planning and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.235310801 Data Collection Planning and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.43

A. Making Large Databases AccessibleB. Usability Input to Coverage Follow-up (CFU) User-Interface RequirementsC. Usability Input to the Field Data Collection Automation (FDCA) Program

5310808 Special Place/Group Quarters (GQ) Planning and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Group Quarters Operational Integration Team (OIT) for 2008 Dress Rehearsal

5610802 Statistical Design and Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.25A. Decennial Editing and ImputationB. Decennial Record LinkageC. Research on Item and Count Imputation for Implementation in Census 2010D. Decennial Disclosure AvoidanceE. Census Unduplication Research

5610803 Coverage Measurement Planning and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.35A. Coverage Measurement ResearchB. Accuracy of Coverage MeasurementC. Questionnaire Wording and Automation Team

5610805 Coverage Improvement Planning and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.085610806 Evaluations/Planning Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.06

A. Decennial Privacy ResearchB. Development of Questionnaires for Decennial Coverage ImprovementC. Inter-Divisional Decennial 2010 Working Groups on Residence Rules and Coverage ImprovementD. Evaluations, Experiments, and Assessments Operational Integration Team (EEA OIT)

5385860 American Community Survey (ACS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.29A. ACS Questionnaire Design MeasurementB. ACS Missing Data and ImputationC. ACS Group Quarters Item Imputation and Micro Data Disclosure Avoidance ResearchD. ACS Language ResearchE. ACS Applications for Time Series MethodsF. ACS VariancesG. ACS Data Products – Display of Variability MeasuresH. ACS Additional Mail TestI. ACS Website: Card-sorting StudyJ. AFF/ACS Low Fidelity Usability StudyK. ACS Multiyear Estimates: User Guidelines for Use of ThemL. ACS 3-year Estimates: Methods for Review of ThemM. ACS: 2005 and 2006 Item Nonresponse Rates

DEMOGRAPHIC DIRECTORATE

Population Division (Sponsor)

Project ProjectNumber Title FTEs1443000 Race and Ethnic Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.00TBA Use of the Empirical Bayes Approach in the Housing Unit Method for Population Estimates . . . . . TBA

Demographic Surveys Division (Sponsor)

Project ProjectNumber Title FTEs0906/7374 Data Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66TBA Quick Turnaround Pretesting of Household Surveys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TBATBA Migration Supplement to the Current Population Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TBA

Housing and Household Economic Statistics Division (Sponsor)

Project ProjectNumber Title FTEs1465001 Re-Engineered Survey of Income and Program Participation (RE-SIPP) Research . . . . . . . . . . . . . . 1.227558111 SIPP Measurement of Wealth: Assets/Liabilities Imputation Research/Software Design . . . . . . . . . .33TBA Civic Engagement Supplement to the Current Population Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . TBATBA Cognitive Pretesting of American Housing Survey (AHS) Spanish Language Questionnaire . . . . . TBA

Demographic Statistical Methods Division (Sponsor)

Project ProjectNumber Title FTEsTBA 2010 NSCG Research to Model Field of Degree Information for College Graduates in the ACS . . . TBA4000801 Sample Redesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .74

Data Integration Division (Sponsor)

Project ProjectNumber Title FTEs7165 Research for Small Area Income and Poverty Estimates (SAIPE) . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.39TBA Small Area Health Insurance Estimates (SAHIE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TBA

ECONOMIC DIRECTORATE

Project ProjectNumber Title FTEs2370854 Editing Methods Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21

Investigation of Selective Editing Procedures for Foreign Trade Programs 2470851 Disclosure Avoidance Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.142370852 Time Series Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.95

A. Seasonal Adjustment SupportB. Seasonal Adjustment Software Development and EvaluationC. Research on Seasonal Time Series - Modeling and Adjustment Issues

D. Supporting Documentation and Software for X-12-ARIMA and X-13A-STBA Survey of Research and Development in Industry, Imputation and Sampling Research and Software

Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TBA

STRATEGIC PLANNING AND INNOVATION

Project ProjectNumber Title FTEs0359999 Remote Access Synectics - Microdata Analysis System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80

STATISTICAL RESEARCH DIVISION

Project ProjectNumber Title FTEs8150000 Postal Regulatory Commission/Statistical Consulting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .09

CENSUS BUREAU

Project ProjectNumber Title FTEs 0251000 Program Division Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.00

A. Division Leadership and SupportB. Research Computing

GENERAL RESEARCH AND SUPPORT

Project ProjectNumber Title FTEs0351000 General Research and Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26.351871000 General Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.69STATISTICAL METHODOLOGY

A. Disclosure Avoidance B. Disclosure Avoidance for MicrodataC. Seasonal AdjustmentD. Small Area Estimation - Decennial/Demographic ApplicationsE. Nonresponse in Longitudinal SurveysF. Household Survey Design and EstimationG. Sampling and Estimation Methodology: Economic SurveysH. Research and Development Contracts

STATISTICAL COMPUTING METHODOLOGYA. Record Linkage and Analytic Uses of Administrative ListsB.1 EditingB.2 Editing and ImputationC. Developed Software Support – General Variance Estimation Development and SupportD. Statistical Computation for Longitudinal Employer-Household Dynamics (LEHD)E. Missing Data and Imputation: Multiple Imputation Feasibility StudyF. Optimizing Field OperationsG. Modeling, Analysis and Quality of Data

SURVEY METHODOLOGYA. Usability Research and TestingB. Questionnaire PretestingC. Questionnaire Design Experimental Research Survey (QDERS)D. Language: Interdisciplinary Research on Language and Sociolinguistic Issues Relevant to Survey

Methodology

E. Training for Cognitive InterviewingF. Research on Cognitive Testing of Non-English Language Survey InstrumentsG. Interviewer-Respondent InteractionsH. Research on Cognitive Testing of Housing Questions from the American Community Survey (ACS)

and the American Housing Survey (AHS)I. Q-Bank: A Database of Pretested QuestionsJ. Health Insurance Measurement

RESEARCH ASSISTANCE

PUBLICATIONS- Journal Articles, Publications- Books/Book Chapters- Proceedings Papers- Statistical Research Division Research Reports- Statistical Research Division Studies - Other Reports

TALKS AND PRESENTATIONS

STATISTICAL RESEARCH DIVISION SEMINAR SERIES

PERSONNEL ITEMS- Honors/Awards/Special Recognition- Significant Service to Profession- Personnel Notes

1

1. COLLABORATION

1.1 – 1.2 DECENNIAL TOPICS(Decennial Projects 5210801 and 5210802)

A. Census Questionnaire Design Features

Description: The project represents ongoing researchusing the behavior coding method to analyzeinterviewer/respondent interactions to evaluate thedecennial Nonresponse Follow-up (NRFU) questions inthe 2008 Census Dress Rehearsal (the 2004 and 2006NRFU instruments were also evaluated using thismethod). The redesign of the 2008 instrument wasinformed by the 2004 and 2006 behavior coding,cognitive testing, usability testing, and observationalstudy results. These changes include a revised approachto gather a roster incorporating the residence rules intoa question/answer sequence, rather than using aflashcard. The behavior coding of the 2008 instrumentwill evaluate the effectiveness of the revised approach,as well as continue to evaluate the questions being usedto collect data from mail nonrespondents. The 2008NRFU behavior coding data will be collected from theFayetteville and Stockton area test sites and both theEnglish and Spanish language instruments will beevaluated.

Highlights: During this quarter, plans were made forthis upcoming project, including coordination with FieldDivision and the Tucson Telephone Center. Staff alsobegan to prepare materials for the project. Staff: Jennifer Hunter Childs (x34927), Patti Goerman,Dawn Norris

B. Short Form Questionnaire Content Other ThanRace and Ethnicity

Description: This project involves participation indecennial content team meetings, including Content andForms Design IPT, Housing Unit OIT, NRFUInstrument Subteam, Mode Consistency Subteam, andCPEX Methods Team. It also includes cognitivepretesting of census questionnaires.

Highlights: Staff participated in the activities of theContent and Forms Design IPT, Housing Unit OIT,NRFU Instrument Subteam, Mode ConsistencySubteam, and CPEX Methods Teams.

Staff made preparations for cognitive testing of the BeCounted form and census mailing package.

Future Plans: Staff will complete testing of the BeCounted form and begin conduct pretesting of the censusmailing package in March.

Staff: Terry DeMaio (x34894), Jenny Hunter Childs,Eleanor Gerber, Patti Goerman, Dawn Norris, LorraineRandall

C. Development of Race and Ethnicity Questions

Description: Staff will participate in planning andpretesting alternative versions of the race and ethnicityquestions used in the Decennial Census. We willdevelop proposals for cognitive testing of new questionformats in conjunction with decennial staff, and lead orengage in cognitive research as needed.

Highlights: No significant progress this quarter.

Staff: Eleanor Gerber (x34890), Jenny Hunter Childs

1.3 LANGUAGE PLANNING AND DEVELOPMENT

(Decennial Project 5210803)

Description: Staff members participate in the inter-divisional Decennial Task Force, or language team,which focuses on developing and planning the LanguageProgram for the 2010 Census, pre-census tests, and theDress Rehearsal. In addition, staff members in ourdivision provide consultation and technical support in thedesign, development and conduct of research forDecennial language-related projects.

Highlights: Staff members have attended inter-divisional, bi-weekly Decennial Language IntegratedProduct Team meetings to discuss and plan language-related research for the Dress Rehearsal. We have alsoworked on a number of language related researchprojects.

During this quarter, we have worked as technicalmanagers and collaborators on the cognitive testing ofthe 2010 census form in five languages (English,Chinese, Korean, Russian, and Vietnamese). Staff work

2

as the Census Bureau research analyst for this project,providing technical guidance and management for thecontractor (RTI). Staff co-authored an interviewprotocol for cognitive testing, reviewed and providedconsultation for the creation of the staffing andrecruitment plan, the cognitive interviewer trainingagenda and other materials. Staff also served as theteam lead for the Chinese team for this project.

In addition, we have worked on the cognitive testing ofthe Spanish Coverage Measurement Personal Interviewinstrument. During this quarter, staff presented finalrecommendations to the sponsor and wrote up a reporton problems with the Spanish translation of groupquarters (GQ) terms included in the Census CoverageMeasurement (CCM) instrument and recommendationson how to reword these terms. Staff presented thedocument to people from the CCM area, GQ and thePopulation Division. Based on feedback and commentsfrom these areas, staff created a consolidatedrecommendations document to be presented to thesponsor and the translation office. Staff also worked onthe write up of the final project report.

During this quarter, staff served as reviewers for theSpanish and Chinese terms contained in the newDecennial Language Reference Dictionary. We providedcomments on two different drafts of the dictionary andattended meetings to discuss the project.

Future Plans: Work will continue on the cognitivetesting of the Decennial questionnaire in five languages.Staff will complete the final project report on the CCMproject. We will provide one more round of commentson the Language Reference Dictionary project.

Staff: Patricia Goerman (x31819), Yuling Pan, Manuelde la Puente, Diana Simmons

1.4 DATA COLLECTION PLANNING AND DEVELOPMENT


A. Making Large Databases Accessible

Description: This project will investigate theaccessibility of tables with varying levels of columnheaders and row stubs which are generated fromquerying large databases such as the LongitudinalEmployer-Household Dynamics Quarterly WorkforceIndicators and the American FactFinder. This project isa collaborative effort with Westat.

Highlights: This project is complete.

Staff: Larry Malakhoff (x33688), Sid Schneider(Westat)

B. Usability Input to Coverage Follow-up (CFU)User-Interface Requirements

Description: Our division’s role was to provide usabilityreview of user-interface requirements for an Internet-based CFU instrument to be administered online bytelephone interviewers. The CFU user-interface teamdeveloped the requirements in cooperation with thecontractors, Z-Tech Corporation and GunnisonConsulting Group. When these requirements weredelivered to the Decennial Response Integration System(DRIS) contractor, our division’s role was to respond toany usability questions raised by the contractor and toconduct usability testing on interim developmentproducts.

Highlights: Staff completed the final report on thisproject: “Usability Evaluation of the 2008 CoverageFollow-up (CFU) Online User Interface,” HCIMemorandum Series #116. This project is complete. Staff: Betty Murphy (x34858), Jenna Beck

C. Usability Input to the Field Data CollectionAutomation (FDCA) Program

Description: Our division’s role was to provideusability review of FDCA contractor documents andother products, such as screen flows and drafts of theuser-interface design style guide.

Highlights: Staff attended reviews of testing conductedby the contractor to evaluate the usability of screendesigns for the handheld computer (HHC) in the contextof simulated Non-Response Follow-up (NRFU)operations; consulted with Field staff concerned aboutthe Person Interview (PI) Reinterview (RI) portion of theinstrument design; and provided comments on screendesigns for use by office-based supervisors of the NRFUfield operation. We attended demonstrations of softwareand screen designs developed for the NRFU HHC andfor the PI/PIRI operations.

Staff: Betty Murphy (x34858), Jennifer Hunter Childs,Erica Olmsted-Hawala

1.5 SPECIAL PLACE/GROUP QUARTERS (GQ)PLANNING AND DEVELOPMENT

3


Group Quarters Operational Integration Team(OIT) for 2008 Dress Rehearsal

Description: The Group Quarters EnumerationOperational Integration Team (GQE OIT) developsdetailed plans, procedures, schedules, and operationalassessments for the Group Quarters Enumeration (GQE)in the 2008 Dress Rehearsal.

Highlights: Staff attended GQE OIT meetings. Weprovided comments on methods for processing usualhome elsewhere (UHE) cases and provided commentson methods for dealing with GQE adds. Due to fundingcuts, GQE and Service Based Enumeration assessmentswere cut from the 2008 Dress Rehearsal. We providedcomments on formatting the ICR and on operationaldocuments, and we also read CPEX proposedexperiments for testing group quarters in the 2010Census.

Staff: George Carter, III (x31774)

1.6 STATISTICAL DESIGN AND ESTIMATION(Decennial Project 5610802)

A. Decennial Editing and Imputation [See Projects0351000 and 1871000 (B), General Research -Statistical Methodology]

B. Decennial Record Linkage [See Projects 0351000and 1871000 (A), General Research - StatisticalComputing Methodology]

C. Research on Item and Count Imputation for Implementation in Census 2010

Description: Research and studies will be undertakenon item and count imputation for implementation inCensus 2010.


Staff: Bor-Chung Chen (x34857), Yves Thibaudeau

D. Decennial Disclosure Avoidance

Description: The purpose of this research is to developdisclosure avoidance methods to be used for CensusBureau publicly available decennial census andAmerican Community Survey (ACS) data products.Emphasis will be placed on techniques to implement

disclosure avoidance at the stage of processing.Disclosure research will be conducted on alternativemethods to protect both tabular data and microdata fromthe decennial census and the ACS. Methods will bedeveloped, tested, evaluated, and documented. We willalso aid in the implementation of the methods.

Highlights: Staff met with Decennial Statistical StudiesDivision and Housing and Household EconomicStatistics Division staff to discuss research into whetherthe synthetic data procedure could be used for imputingdata that are currently “allocated” (imputed) via avariety of methods. The issue is how to streamline thecombined editing/imputation and disclosure avoidanceprocesses to be more efficient and more accurate.

Staff conducted computational analyses for the purposeof increasing the proportion of the data and the numberof variables to be synthesized for the AmericanCommunity Survey Group Quarters data.

Staff worked on disclosure avoidance procedures for the2008 Dress Rehearsal.

Staff: Laura Zayatz (x34955), Paul Massell, Phil Steel,Sam Hawala, Jeremy Funk, Rolando Rodríguez

E. Census Unduplication Research

Description: The goal of this project is to conductresearch to guide the development and assessment ofmethods for conducting nationwide matching andunduplication in the 2010 Decennial Census. One of themajor problems is how to incorporate the effects of namefrequency into the unduplication procedures. Our staffalso provides assistance in specifying and reviewingoutput from the matching and unduplication proceduresfor test Censuses and eventually for Census 2010. Webegan this project in May of 2004.

Highlights: Our staff continued testing a nationalmatching using the 2000 Census data and continuedexamining results. The document “Initial Results froma Nationwide BigMatch Matching of 2000 Census Data,”which looks at the results of Across Response Matching[BigMatch matching of individuals across Censushousing units (HUs)] is currently in the review processto become an SRD Research Report. The documentsuggests that the most serious problems withcoincidental matches are concentrated in the mostcommon last names and the most common Hispanic lastnames.

4

The report “Additional Results from a NationwideMatching of 2000 Census Data” is also in the reviewprocess to become an SRD Research Report. This reportlooks at two other stages of results: within-responsemodeling (which evaluates housing unit pairs withmultiple links), and the remaining residual person links.The results confirm the finding that problems withapparent false matches are concentrated in the mostcommon surnames and the most common Hispanicsurnames. However, name frequency does not appear tohave much effect when there are multiple links ofreasonable quality between housing units or when thephone number matches.

In addition, staff looked into the effect on the matchingresults of the potential duplicates identified in the 2000Housing Unit Duplicate Operation (HUDO). Removingthe potential duplicates from the results reduces thenumber of links, especially at lower geographic levels,but the apparent problems with coincidental matchesremain. Note that if a matching and modeling operationhad been done in 2000, it would have been beforeHUDO.

Staff also began work on a new national matching on the2000 Census data. The new matching incorporates thespecifications for the 2008 Census Dress Rehearsal.

Staff: Michael Ikeda (x31756), Ned Porter

1.7 COVERAGE MEASUREMENT PLANNINGAND DEVELOPMENT

(Decennial Projects 5610803)

A. Coverage Measurement Research

Description: Conduct research on model-based smallarea estimation of census coverage. Consult andcollaborate on modeling coverage measurement.

Highlights: The paper titled “Small area random effectmodels for capture/recapture methods with applicationsto estimating coverage error in the U.S. DecennialCensus,” which documents the staff results on usingrandom effect models for Census Local Officeestimation was accepted for publication in the peer-reviewed journal, Statistics in Medicine. Thepublication date is to be determined. Staff continueworking on marginal small area models (Maples) andconditional models (Malec) for improved methods ofsmall area estimation. Our staff continues attending theweekly meetings of the Coverage Estimation ResearchGroup and working and meeting weekly with staff from

the Decennial Statistical Studies Division on issues offixed-effect logistic models of census coverage.

Staff: Don Malec (x31718), Jerry Maples

B. Accuracy of Coverage Measurement

Description: 2010 Census Coverage MeasurementResearch conducts the research necessary to developmethodology for evaluating the coverage of the 2010Census. This includes planning, designing, andconducting the research, as well as analyzing andsynthesizing the results to evaluate their accuracy andquality. The focus is on the design of the censuscoverage measurement survey and estimation ofcomponents of coverage error with secondary emphasison the estimation of net coverage error. The estimationof overcount and undercount separately has not beendone for previous censuses because of the difficulty ofobtaining adequate data for unbiased estimates. The firstattempt to implement the new methodology is in the2006 Census Test.

Highlights: Our staff provided technical expertise andexperience in the planning and implementation ofcoverage measurement research for the 2010 Census.We served on three teams formed to plan and implementcensus coverage measurement research for the 2010Census in the 2006 Census Test, the 2008 DressRehearsal, and with data from Accuracy and CoverageEvaluation Revision II and Census 2000. We alsoserved on an Executive Steering Committee Subgroupformed to identify high-level research topics andquestions for the 2010 Census Program for Evaluationsand Experiments. We reviewed the Interim Report bythe Panel that contained recommendations forexperiments to conduct during the 2010 Census andprepared recommendations for the Executive SteeringCommittee to consider. Also, our staff served on theIntegrated Communications Program Research Groupformed to provide research products and oversight toinform the 2010 Integrated Communication Program.

Staff: Mary Mulry (x31759)

C. Questionnaire Wording and Automation Team

Description: The purpose of this project is to design thecoverage measurement survey instruments for the 2010Census. These instruments will gather enough data tomeasure both person and household coverage of the2010 Census. In preparation for 2010, there will be a2006 Test of the coverage measurement operation in

5

specific sites in conjunction with the 2006 Census Test.For 2006, there will be an automated person interview(PI) collecting an independent roster of people living atpre-selected sample addresses in the sites and theirresidency. There will also be a paper-based personfollow-up questionnaire which collects additionalresidency information about some people collected inthe census or the independent roster, but for whom wedid not collect enough residency information todetermine where they should have been counted for thecensus. Both these instruments will be used to measureperson coverage. Our immediate goals are to create andtest these two instruments given requirements from otherteams working on coverage measurement planning. This team is further tasked with developing theindependent housing unit listing booklet, and housingunit follow-up forms in order to measure housing unitcoverage in 2008/2010.

Highlights: With regard to the 2006 Census Test, ourstaff documented the Person Followup (PFU)observation and respondent debriefings findings in aformal report; “#2006-D7-13, Subject: 2006 CensusCoverage Measurement Person Followup Interview:Trip Report: January 2007, Prepared by: Beth Nichols,Jennifer Hunter Childs, Joanne Pascale, Laurie Schwede,Julie Bibb, Vicki Smith, Sandy Norton, Jamie Burnham& Patricia Sanchez .” The audiotaped 2006 PFU caseswere analyzed and a draft report was written. Cognitivetesting of the 2008 Spanish Person Interview wasconducted and recommendations were accepted. Staffreviewed and commented on the 2008 PI specification.

Staff: Beth Nichols (x31724), Joanne Pascale, JennyHunter Childs, Laurie Schwede, Dawn Norris, PattiGoerman, Diana Simmons, Manuel de la Puente

1.8 COVERAGE IMPROVEMENT PLANNINGAND DEVELOPMENT

(Decennial Projects 5610805 and 5610806)

A. Decennial Privacy Research

Description: The purpose of this project is to serve onand assist the work of the Privacy Policy and ResearchCommittee (PPRC), and to conduct research to assesspublic opinion on privacy-related issues, including theincreased use of administrative records to assistDecennial Census enumeration.

Highlights: In addition to serving on and assisting thework of the Privacy Policy and Research Committee,staff also participated in the meetings of the 2010

Integrated Communication Plan (ICP) Research Group,chaired by Nancy Bates (DIR). We reviewed andprovided comments on several internal research effortswhich have focused on trying to anticipate 2010 self-enumeration issues through an examination of Census2000 and ACS results. We also reviewed andcommented on materials prepared by DraftFCB, theprime contractor for the 2010 ICP effort, including suchthings as focus group plans, draft outreach “platforms,”and a draft “Census Participation Survey” questionnaire.Staff also began preparations for testing privacylanguage in decennial letters and other communicationsin the context of 2010 messaging; we anticipate thatinterviewing for this project will start in March.

Staff: Jeff Moore (x34975), Anna Chan, Terry DeMaio,Eleanor Gerber

B. Development of Questionnaires for DecennialCoverage Improvement

Description: We will consult on the development ofquestions and questionnaires designed to improve withinhousehold coverage in the Decennial Census. We willparticipate in the development and pretesting ofhousehold and individual-level coverage questions in thedecennial short form and the Coverage Follow-up (CFU)reinterview instrument.

Highlights: Cognitive interviews were carried out to testthe Be Counted form, and recommendations for changeswere proposed. Staff continued to work on new layoutsfor the Update Enumerate form.

Staff: Eleanor Gerber (x34890), Jenny Hunter Childs,Elizabeth Ann Dimler, George R. Carter, III

C. Inter-divisional Decennial 2010 Working Groupson Residence Rules and Coverage Improvement

Description: These overall inter-divisional workinggroups provide input to the Decennial ManagementDivision (DMD) for planning successive operations, andtest broadly related coverage research during the decadeleading up to the 2010 Census. These groups receiveproposals from various subgroups on: within-householdcoverage, residence rules, imputation, and unduplication.

Highlights: No significant progress this quarter. Staff: Laurie Schwede (x32611), Eleanor Gerber

D. Evaluations, Experiments, and Assessments

6

Operational Integration Team (EEA OIT)

Description: The purpose of the EEA OIT is tofacilitate planning and timely implementation of 2008Census Dress Rehearsal and 2010 Census evaluations,experiments, and assessments. The group specifies thegeneral scope of the 2008 Census Dress RehearsalAssessment Program; questions to be answered; the datewhen the final results are needed to inform 2010; andpresents recommendations to the Census IntegrationGroup. The group ensures that program integration andimplementation of the 2010 Census Program ofEvaluations and Experiments (CPEX) meet the guidanceprovided by the Executive Steering Committee andprepares the 2010 Census Program for Evaluations andExperiments Master Plan.

Highlights: Our division’s representative is serving asthe evaluation consultant for her proposed 2010 CPEXethnographic evaluation of enumeration methods andcoverage in hard-to-enumerate race/ethnic groups. Shedrafted a preliminary proposal for this study andsolicited comments. She gave an informal presentationon this evaluation to the CPEX Executive SteeringCommittee at the January, 2008 offsite meeting. Inresponse to the executives’ request that she expand thescope of this study to include more race/ethnic sites, sherevised and resubmitted the proposal to DecennialManagement Division. She participated in EEAmeetings and responded to requests by team members. Staff: Laurie Schwede (x32611), Manuel de la Puente

1.9 AMERICAN COMMUNITY SURVEY (ACS)(Decennial Project 5385860)

A. ACS Questionnaire Design Measurement

Description: This project provides technical andresearch support for the development and improvementof ACS data collection instruments used in all modes ofdata collection available in the ACS. Staff membersserve on inter-divisional working groups, and providetechnical support in the design and conduct ofquestionnaire design research for the ACS.

Highlights: Staff continued to attend inter-divisionalworking groups and continued to provide technicalsupport.

Staff: Eleanor Gerber (x34890), Yuling Pan, PattiGoerman, Laurie Schwede

B. ACS Missing Data and Imputation

Description: This project undertakes research andstudies on missing data and imputation for the AmericanCommunity Survey.

Highlights: Staff implemented statistical matchingalgorithms to match records from the NationalAssessment of Educational Progress (NAEP) to recordsof the American Community Survey. These algorithmsimpute NAEP information to add to the ACSinformation. Staff translated STATA code to SAS toprocess the matched records.

Staff: María García (x31703), Yves Thibaudeau

C. ACS Group Quarters (GQ) Item Imputation andMicro Data Disclosure Avoidance Research

See project “D. Decennial Disclosure Avoidance” (1.6Statistical Design and Estimation).

D. ACS Language Research

Description: This project provides technical andresearch support for addressing language issues in ACSdata collection instruments and supporting documents.Staff members serve on inter-divisional working groupsand provide consultation and technical support in thedesign and development of language research for theACS.

Highlights: Our staff is working closely with the ACSLanguage Team to develop and pretest multilingualdocuments for the ACS, and to pretest the ACS Spanishinstrument. We started three projects in this quarter: a)cognitive testing of the ACS multilingual brochure infour languages (Spanish, Chinese, Korean, and Russian),b) cognitive testing of the ACS CAPI materials inmultiple languages (Arabic, French, Polish, Portuguese,Haitian Creole, and Vietnamese), and c) pretesting of theACS Spanish CAPI/CATI instrument.

For Project a and Project b, staff work as the CensusBureau research analyst for the project, providingtechnical guidance, management, and consultation forthe contractor (RTI). In collaboration with RTI, staff co-authored interview protocols for cognitive testing, provided consultation on the staffing and recruitmentplan, on the cognitive interviewer training agenda andother materials. Staff also provided training forlanguage experts for Project a. In addition, staff serve asthe team lead for Chinese team, and conducted English

7

and Chinese interviews for the first round of interviewsfor Project a. In collaboration with RTI, we completedthe interim report for Project a and held an interimmeeting with the sponsor.

During this quarter, we have worked as technicalmanagers and collaborators on the cognitive testing ofthe Spanish CAPI/CATI version of the AmericanCommunity Survey (Project c above). SRD staffworked with ACS staff on planning the project. Weconducted an expert review of the Spanish surveyinstrument (about 300 questions) to identify potentialproblems in need of testing and presented results to theACS language team. We looked at feedback on thesurvey instrument from the Field and Regional Officesand Advisory Committees and created a spread sheetshowing which questions were identified as problematicby different sources. SRD staff participated in meetingsto choose the questions to be tested. SRD staffcoordinate the creation of 3 lists of questions to be testedin the 3 phases of the project and added issues to beprobed to the final question lists before providing themto the contractor. SRD staff created a draft projectschedule, co-authored the interview protocol for Phase1, Round 1of the project, and reviewed the respondentrecruitment plan and other interviewing documentationand materials.

Future Plans: We will work with the ACS LanguageTeam to continue the three projects to pretest the ACSsupporting materials in multiple languages and to pretestthe ACS Spanish instrument. We will work closely withthe contractor to ensure successful completion of theseprojects.

In addition, staff will draft a checklist to provideguidance for the use of interpreters in field interviews.Staff will also issue an SRD report to documenttranslation review process.

Staff: Yuling Pan (x34950), Patti Goerman, Manuel dela Puente, Diana Simmons

E. ACS Applications for Time Series Methods

Description: This project undertakes research andstudies on applying time series methodology in supportof the American Community Survey.

Highlights: Staff continued a consultation with the ACSOffice, whereby various multi-year estimates are madecompatible. Staff investigated cases in the ACS

database where the methods are plausible andimplausible.

Future Plans: Staff will contribute to a paper giving anoverview of the American Community Survey for TheAmerican Statistician.

Staff: Tucker McElroy (x33227)

F. ACS Variances

Description: The purpose of this short-term project is tocompare variances for survey totals based on severalalternative methods of controlling to population totals inthe ACS.

Highlights: Staff continued work on planning dataanalyses for a Decennial Statistical Studies Divisionprojection evaluating variances of ACS estimates at thelevel of weighting-area by demographic class, based onalternative methods of applying population controls.This work included research on the validity of BRRvariance estimation methodology when nonresponse isestimated using slightly misspecified adjustment classes.

Staff: Eric Slud (x34991), Yves Thibaudeau

G. ACS Data Products – Display of VariabilityMeasures

Description: This project has two parts: (1) determinewhich measure of variability should be displayed foreach ACS data product and how it should be displayed;and (2) for the web, develop a simpler and clearerdescription of variability measures and how they can beused with ACS data products.

Highlights: The survey about use of sampling error waschanged from a web user’s survey to a survey ofrecipients of the ACS Alert, and a sample of e-mailaddresses was selected. The survey instrument wasrevised and an introduction drafted. Materials are beingdeveloped to send to OMB for clearance to conduct thesurvey.

Staff: Lynn Weidman (x34902), Kathy Ashenfelter,Betty Murphy, Jenna Beck

H. ACS Additional Mail Test

Description: This new inter-divisional ACS team isplanning a split-panel test to determine if response rates

8

of mail nonrespondents without known phone numberscan be improved by means of an additional mailing. Ifsuccessful, this could cut the workload during the finalCAPI phase, saving time and money and improving theaccuracy of the data. The three proposed test treatmentsinclude: sending an additional postcard with amotivational message, mailing a third questionnaire witha revised cover letter, or using the current method.

Highlights: We finalized all of the cognitiveinterviewing materials and had completed eightinterviews when we realized that slight differences in thebrightness of the two sets of green reminder cardsappeared to be influencing respondents’ evaluations oftheir effectiveness. We stopped testing, reprinted allcards on identical green cardstock and completed the lastseven interviews. We did a preliminary analysis of theresults and gave an oral presentation of results andrecommendations to the sponsor and working group. Thefinal results will be presented in a sponsor report and inan AAPOR paper in 2008.

Staff: Laurie Schwede (x32611)

I. ACS Website: Card-sorting Study

Description: The purpose of this study is to identify auser-centered information architecture of the AmericanCommunity Survey (ACS) domain of the Census.govWeb site. The goal for the project is to come up with abasis for a re-design of the ACS portion of the Web site.

Highlights: Four external novice participants werebrought in for the closed sort, Round 2 of the ACS card-sorting study. Participants were encouraged to place 16predetermined subheading-level terms under 6 high-level terms, and then the 95 terms into the subheading-level categories. The subheading-level and high-levelcategories were derived from the results of the open sort(Round 1). At the conclusion of each session, the testadministrator listed which cards had been placed undereach subheading-level and high-level category. Theseresults were analyzed and interpreted. Staff found thatterms were consistently placed in many of the samehigh-level categories as in Round 1. These results helpvalidate SRD’s analysis from Round 1; and we concludethat the 16 subheading categories are usable linkcategories for the main page of the ACS web site. The16 subheading categories are as follows: use the ACS,information on the population/housing, specificinformation on the population/housing, communityprofile, FAQs, questions about the ACS, what areas get

surveyed, handy informational tools, things of interestrelated to the data, get to the data, putting data intotables, make year-to-year comparisons, definitions ofterms and list of topics, specialized datastatistical,considerations, and timeline. A final report was writtenand sent to the client.

Staff: Erica Olmsted-Hawala (x34893), AlexTrofimovsky, Betty Murphy

J. AFF/ACS Low Fidelity Usability Study

Description: The purpose of this study is to get userfeedback on the prototype pages of the AmericanFactFinder (AFF) release of the 2008 August andNovember American Community Survey (ACS) data onthe AFF prototype Web site pages. The site is testing theusability of the content and user comprehension of datathat is released in stages with Poverty and Income dataavailable in August and all data content available byNovember. The development team is interested ingetting user feedback on the help content of the 1-yearand 3-year estimates which are new product releasesfrom the ACS. The DADSO staff and ACS staff wouldalso like to have usability feedback on the new pages.

Highlights: Staff worked with DADSO staff to createtasks. Staff recruited six novice users, and six expertusers. Staff ran the participants through the usabilitystudy with at least half of the participants doing eye-tracking. Staff wrote a quick report and met with theclients to discuss findings and recommendations. Twoexamples of our usability findings withrecommendations follow:• In the August data sets pages, users did not

understand that all the data was not available. Werecommend that the page directly states whatcontent is available, e.g., “Get the latest Poverty andIncome data.”

• In the 2005-2007 Fact Sheet (November), usersexperienced some confusion about what the 3-yearestimate was as well as confusion about the tablabel. We recommend that the page give feedbackon what and why we are putting out the 3-yearestimates. Some possible text to use include: “Weaccumulate data over 3-years so we could estimateit more accurately.” “We combined our data overthree years” to indicate that the 3-year data was asingle number for each data point. Or that we have“accumulated data for data all 3-years.”

9

Staff: Erica Olmsted-Hawala (x34893)

L. ACS Multiyear Estimates: User Guidelines forUse of Them

Description: Working with Decennial Statistical StudiesDivision staff, we are to develop website documentationthat describes and compares 1-, 3-, and 5-year ACSestimates and their standard errors, discusses their usage,and presents corresponding examples.

Highlights: A version of a paper was sent to severaldivisions for review. Staff are analyzing standard errorsof 1- and 3-year estimates to develop recommendationsas to when it is appropriate to use 1-year rather than 3-year and 3-year rather than 5-year estimates based on theprecision desired, an area’s population, and whatproportion the estimate is of the size of the area.

Staff: Lynn Weidman (x34902), Michael Ikeka, JulieTsay

M. ACS 3-year Estimates: Methods for Review ofThem

Description: An interdivisional team is developingrules, combinations of rules, and systems forimplementing a tool to allow efficient analyst review in2008 of the first ACS 3-year estimates for geographiesof less than 65,000. The purpose of the review is toclear the estimates for public release but at the same timeidentify ‘unusual’ estimates and understand why theyshould or should not be released. One SRD staffmember is on this team.

Highlights: A staff member proposed two rules to beused in the analysis – comparing estimates toCensus 2000 and to expected values from linearregressions on other estimates. Staff developed andimplemented software to perform stepwise regressionson a subset of geographies for all analysis estimates,calculate their expected values for all geographies, andflag potential outliers.

Staff: Lynn Weidman (x34902), Michael Ikeda, JulieTsay

N. ACS: 2005 and 2006 Item Nonresponse Rates

Description: The Census Bureau calculates andpublishes item allocation rates for all data items

collected in the American Community Survey as part ofits data quality measures. However, these rates areactually composed of two separate components of itemnonresponse—items that can be assigned a value throughthe use of related items on the survey questionnaire anditems that require a statistical procedure to allocate thevalue of the missing item. The published allocation ratescombine the two measures into one rate. It is desirableto look at both components separately and to see if theserates vary across mode of data collection, individualitems, geography, certain population groups, location ofcall centers (for data collected by CATI), and otheritems. In addition to the item nonresponse rates, acompleteness index can be computed for the entirequestionnaire and examined by the same levels as theitem nonresponse rates.

Highlights: Programs to calculate the item nonresponserates for both assigned and allocated items have beenwritten for housing and person questions for the varioussubgroups for the 2005 and 2006 ACS data files.Results are being documented, compiled, andsummarized. Programming has begun to calculatecompleteness indices and to use logistic regressionmethods to examine the correlation of certain items tothe item nonresponse.

Staff: Pam Ferrari (x34993)

1.10 RACE AND ETHNIC GROUPS(Demographic Project 1443000)

Description: Staff will provide technical consultationservices and programming support addressing theredesign and content of SAS programs that produce thetable packages for the 2007 Current Population Survey(CPS) Annual Social and Economic Supplement (ASEC)that will feature information at the national and regionallevels for special population/topics.

Highlights: Work will begin in February 2008.

Staff: Aref Dajani (x31797)

1.11 USE OF THE EMPIRICAL BAYESAPPROACH IN THE HOUSING UNIT METHOD

FOR POPULATION ESTIMATES (Demographic Project TBA)

Description: Investigate the use of Empirical Bayes(EB) methods for estimating the change across years in

10

county level vacancy rate (%vac) and persons perhousehold (pph). The product of these two is then usedas the estimate of change in housing unit populationacross years.

Highlights: The EB models were fit via an R routine tocompare to SAS results. When both converged theyagreed, but a large number of regressions still did notconverge due to variance component estimates of 0. Atwo level model will be fit in order to allow us to obtainEB estimates for the states with small numbers ofcounties.

Staff: Lynn Weidman (x34902), Don Malec, Julie Tsay,Michael Ikeda, Rob Creecy

1.12 DATA INTEGRATION(Demographic Project 0906/7374)

Description: The purpose of this research is to identifymicrodata records at risk of disclosure due to publiclyavailable databases. Microdata from all Census Bureausample surveys and censuses will be examined.Potentially linkable data files will be identified.Disclosure avoidance procedures will be developed andapplied to protect any records at risk of disclosure.

Highlights: Research on datasets available on datamarts extended to occupations, prescription drugs, andhobbies. Software documentation for an SRD ResearchReport was submitted titled “Software for Web-Miningavailable Databases”. The software creates a web pagefor easy access to the data products available onlistfinder.directmag.com. A result was that single wordrequests to the website produced the most efficientresults.

Staff: Ned Porter (x31798), Phil Steel

1.13 QUICK TURNAROUND PRETESTING OFHOUSEHOLD SURVEYS

(Demographic Project TBA)

Description: This project involves pretesting new orrevised series of questions for insertion into householdsurveys. The projects are of the short-term, quickturnaround variety rather than long-term research effortsto redesign a survey. Methods used include cognitivetesting and other techniques as appropriate.

Highlights: Staff completed and distributed the finalreport of cognitive pretesting on the NCVS IdentityTheft Supplement (DeMaio, Beck, and Norris, 2007).The results showed that respondents think about theiridentity theft incidents as a single unit, and when theyare asked separately about actual identity thefts and thenattempted identity thefts, they tend to overreportincidents of actual identity thefts. Respondents had avery strict definition of what it means to know somethingabout the person who misused their information andwere reluctant to incriminate people, which resulted inunderreports of knowledge of information about theidentity thief. Also, respondents consistentlymisinterpreted the sponsor’s definition of an informationbreach and were unable to accurately report whether theyhad been the victim of an information breach.

Staff also began preparations for the third round ofcognitive testing of proposed Internet predationquestions for the National Crime Victimization Survey.

Future Plans: Staff will complete research on theInternet predation questions in April.

Staff: Terry DeMaio (x34894), Jen Beck

1.14 MIGRATION SUPPLEMENT TO THECURRENT POPULATION SURVEY


Description: This project conducts cognitive pretestingfor a new supplement for the August Rotation of the2008 Current Population Survey (CPS) interviewproposed by the Population Division’s ImmigrationStatistic Staff at the end of FY 2007. The supplementcontains five sections and its primary objective is togather data that are currently unavailable to analysts. Ifsuccessful, these new data will fill a critical gap,permitting a more complete understanding of themigration and emigration patterns of CPS samplehouseholds.

Highlights: During the first quarter of FY2008, staffexecuted the project plan as scheduled, led andcoordinated the cognitive pretesting efforts of thisproject. Staff from our division, Population Division, andDemographic Surveys Division conducted two rounds ofcognitive interviews to (1) pretest the survey anddocument respondents’ ability to comprehend and torespond to the survey questions as designed; and (2) toprovide recommendations for question wording revision.

11

Results from the first testing served as recommendationsfor question wording revisions for Round 2 of cognitivetesting. A total of 25 cognitive interviews was completedand summarized; and a draft report titled “Report on theCognitive Pretest Study for the Current PopulationSurvey’s New Migration Supplement Questions” wasdistributed to the sponsor at the end of the first quarter.

Staff: Anna Chan (x38462)

1.15 RE-ENGINEERED SURVEY OF INCOMEAND PROGRAM PARTICIPATION (RE-SIPP)

RESEARCH(Demographic Project 1465001)

Re-Engineered SIPP Methodological Research

Description: This project conducts long-termmethodological research to evaluate the Survey ofIncome and Program Participation (SIPP), and to informthe design of the re-engineered SIPP, which willeventually replace the current SIPP program. The twomajor components of this project are (1) the evaluationand documentation of the impacts of the many andsubstantial revisions to the 2004 panel SIPPquestionnaire made as a result of the multi-year SIPP“Methods Panel” research and development effort; and(2) the development of instruments and procedures forthe new RE-SIPP program, which will replace SIPPstarting in 2011.

Highlights: Staff continued to serve on several re-engineered SIPP planning groups – the Survey Group,the Research Group (chaired by SRD staff), and thegeneral planning group known as the Integration Group.The Research Group continued to focus on planning andimplementing a field test of event history calendar(EHC) interviewing methods, with a planned start ofinterviewing in April. The test will include a directcomparison of information gathered using an EHC, witha 12-month reference period, and data gathered from thesame set of respondents in three successive SIPPinterviews, covering the same period, but using in eachcase a 4-month reference period. A major effort in thefirst quarter of FY08 involved working with acontractor, RTI International, to develop and test aprototype paper-and-pencil EHC instrument and anaccompanying interviewer training package. That workwas completed, generally successfully, in lateNovember. We devoted substantial time late in thequarter to improving the RTI-produced forms, modifying

training materials to reflect those changes to the forms,implementing other improvements, and to workingthrough the myriad of details requiring attention in orderto implement the field test.

Staff presented two papers (and co-authored a third) at aspecial Census Bureau/University of Michigan-sponsored conference on EHC methods, as follows:Fields, J. and Moore, J. (2007); Moore, J. (2007); andPascale, J. and McGee, A. (2007).

Staff: Jeff Moore (x34975), Anna Chan, Joanne Pascale

1.16 SIPP MEASUREMENT OF WEALTH: ASSETS/LIABILITIES IMPUTATION

RESEARCH/SOFTWARE DESIGN(Demographic Project 7558111)

Description: Staff initiated new research on couplingwhen imputing assets and liabilities for households inSIPP panels.

Highlights: Staff identified structural flaws in the SIPPPUFF files for the 2004 panel. Staff found that theimputation of business value was decoupled from theimputation of business debt. That led to severelydistorted estimates for these two items. Staff helped theHousing and Household Economic Statistics Divisionidentify repairs for the inconsistencies. TheDemographic Surveys Division reprocessed the PUFFfiles using the new specifications. Staff programed newapplications for the imputation of propertyvalue/mortgage, mobile home value/debt, individualchecking account and joint checking account. Theapplications use SAS Proc IML to ensure each missingvalue case had a hot-deck donor as opposed to getting a“cold deck” donor, which is usually not representative ofthe universe in the scope of the survey.

Staff: Yves Thibaudeau (x31706), Julie Tsay

1.17 CIVIC ENGAGEMENT SUPPLEMENTTO THE CURRENT POPULATION SURVEY


Description: A new supplement on civic engagementhas been proposed for the Current Population Survey(CPS) by the Corporation for National and CommunityService (CNCS.) The primary objective of thesupplement is to gather information on the level of socialcapital and the extent to which American communities

12

are places where individuals are active citizens. Staffwill conduct two rounds of cognitive interviews to testproposed questions for the supplement.

Highlights: Project is finished. The following draftreport was shared with the Demographic SurveysDivision: Rothgeb, J., Okon, A., and Dusch, G.,(November 6, 2007), “Voting and Civic Engagement(VCE) Supplement to CPS: Cognitive Testing (Round 1)Summary Report.”

Staff: Jennifer Rothgeb (x34968), Gianna Dusch (DSD),and Aniekan Okon (DSD)

1.18 COGNITIVE PRETESTING OFAMERICAN HOUSING SURVEY (AHS)

SPANISH LANGUAGE QUESTIONNAIRE (Demographic Project TBA)

Description: The 2009 AHS will mark the first time thesurvey will be conducted with both an English andSpanish Questionnaire. Staff plans to conduct pretestingresearch with a contractor on the Spanish AHSQuestionnaire to evaluate the quality of the translationand ensure consistency between English and Spanishversions of the 2009 instruments.

Highlights: Staff met about AHS translation project inearly October 2007. Staff discussed experiences onACS translation project with ACS staff. Lead staffwrote statement of work (SOW) and schedule forAmerican Housing Survey translation project incollaboration with Manuel de la Puente, PatriciaGoerman, Ann Dimler, and Eleanor Gerber.

Staff: George Carter (x31774), Manuel de la Puente,Patricia Goerman

1.19 2010 NSCG RESEARCH TO MODELFIELD OF DEGREE INFORMATION FOR

COLLEGE GRADUATES IN THE ACS (Demographic Project TBA)

Description: Predict Field of Degree (FOD) for insertionon the 2005-2006 ACS files. FOD will be used by thestaff of the Demographic Statistical Methods Division(DSMD) as an aid to formulating a design for the 2010National Survey of College Graduates using the ACSsample as a sampling frame for the National Survey ofCollege Graduates (NSCG).

Highlights: Staff continues to develop strategies forpredicting a 12 part category FOD variable using theclassification tree methodology available in the software“rpart” In particular, we are evaluating predicting instages by broad FOD categories, first. We have alsodeveloped methods to predict and evaluate models (oncefitted) using SAS. DSMD staff is currently trying to addone more ACS design variables into the modelling dataset. After the inclusion of this variable is resolved, wewill begin making final predications and evaluations.

Staff: Don Malec (x31718), Elizabeth Huang, LynnWeidman

1.20 SAMPLE REDESIGN(Demographic Project 4000801)

Description: Following each decennial census a newsample design is put in place for each of the CensusBureau’s demographic surveys. Our division assists theDemographic Statistical Methods Division in performingresearch to determine how to improve the efficiency ofthese sample surveys through a coordinated jointredesign.

Highlights: One staff member is on the 2010 SampleRedesign Optimal Sample Design Strategy Team led bythe Demographic Statistical Methods Division. Theteam looked at overall approaches using or not usingprimary sampling unit (PSU)-based designs and finalizeda report with recommendations for the next phase ofstudy.

Staff: Lynn Weidman (x34902), Michael Ikeda

13

1.21 RESEARCH FOR SMALL AREA INCOMEAND POVERTY ESTIMATES (SAIPE)

(Demographic Project 7165000)

Description: The purpose of this research is to develop,in collaboration with the Data Integration Division(DID) (The Small Area and Poverty Estimates branch isnow in DID, but was previously in the Housing andHousehold Economic Statistics Division), methods toproduce “reliable” income and poverty estimates forsmall geographic areas and/or small demographicdomains (e.g., poor children age 5-17 for counties). Themethods should also produce realistic measures of theaccuracy of the estimates (standard errors). Theinvestigation will include assessment of the value ofvarious auxiliary data (from administrative records orsurveys) in producing the desired estimates. Alsoincluded would be an evaluation of the techniquesdeveloped, along with documentation of themethodology.

Highlights: A report on “ An Empirical Study on UsingCPS and ACS Survey Data in Bivariate State PovertyModels” by Huang and Bell was released in theResearch Report Series (Statistics #2007-18) of theStatistical Research Division. The paper examines thepotential benefits of jointly modeling state poverty ratioestimates from the CPS Annual Social and EconomicSupplement (ASEC) and ACS (demonstration surveysor full production) to improve model-based estimates ofpoverty ratios. We examined alternative bivariatemodels for doing this using CPS estimates for incomeyears (IYs) 2000-2002 with ACS data from the ACSdemonstration surveys of these years, and also usingCPS estimates for IY 2004 with full production ACS2005 data. The models included regression variablesconstructed from administrative records data and Census2000 poverty ratios (or Census 2000 residuals for IY2004), along with state random effects and samplingerror components. The results showed: 1) In comparingthe general bivariate model with the CPS univariatemodel, there is, at best, a small average improvement instate posterior variances of estimates from the CPSequation for most age groups and income years. Resultsare variable for individual states, with some statesshowing larger improvements, but a few showingsubstantial variance increases. Overall, these results donot suggest worthwhile benefits to the CPS equationestimates from using a bivariate model to bring in theACS data. 2) For many of the combinations of age-groupand IY, chi-squared tests rejected the assumption that theCPS and ACS estimate the same state poverty ratio, but

failed to reject a model that assumed the regressioncoefficients (except intercept) are the same in the twoequations. In fact, this model had substantially lowerposterior variances (for the CPS equation), on average,than did the general bivariate model, but it did noteliminate the occasional large posterior varianceincreases. 3) The occasional moderate to large posteriorvariance increases in the CPS equation from use of thebivariate models corresponded to large regressionresiduals from the ACS equation. 4) Use of either of thebivariate models makes no improvements of substance inestimates for the ACS equation. There is too muchsampling error in the CPS direct estimates for them toconvey much useful information for estimation in theACS equation.

We used 2005 ACS data instead of 2006 CPS data in theSAIPE state models for 2007 production. With its muchlarger sample size, the direct ACS estimates were clearlyadequate for the larger states (and were changed verylittle by the models), but we found that the models gavesubstantially lower variances of the estimates for thesmaller states. For example, the model based statepoverty estimates of poverty ratios for age 5-17 relatedchildren substantially improved standard errors for about10-20 states (depending on how one interprets“substantially”). We also examined regressiondiagnostics for the models and quality checks for theproduction of state estimates.

We used IRS tax data for the first time in the 2007SAIPE production to improve the school district povertyestimation procedure. We provided support to DID forthe production run and assisted with the data qualitychecks on the production of the school district estimatesto understand reasons why certain districts had a largeincrease or decrease in their estimate of poor school agedchildren from the previous year.

We developed an experimental method to constructcoefficient of variation (CV) estimates for the schooldistrict poverty estimates. We did this by comparing theestimates using IRS tax data for income years 1989 and1999 to the direct long form survey estimates fromCensus 1990 and 2000 respectively, computing meansquares of these differences over groups of schooldistricts, taking out the effects on these means squares ofsampling variances of the long form estimates, andconverting the resulting estimates of mean squared errorsto CVs. The CVs seemed to vary by population sizeand the degree to which exemptions would be geocodedto a census block. To account for this we used

14

alternative parametric forms for the CVs, though wecannot assess which assumed parametric form is better.Comparing these CV estimates with similarlyconstructed CV estimates for the previous estimationmethod showed improvements from the new method (inagreement with results of a previous evaluation).

Staff: Elizabeth Huang (x34923), Jerry Maples, WilliamBell (DIR)

1.22 SMALL AREA HEALTH INSURANCEESTIMATES (SAHIE)


Description: At the request of staff from the DataIntegration Division, our staff will review currentmethodology for making small area estimates for healthinsurance coverage by states and poverty level.


Staff: Don Malec (x31718), Elizabeth Huang

1.23 EDITING METHODS DEVELOPMENT(Economic Project 2370854)

Investigation of Selective Editing Procedures forForeign Trade Programs

Description: The purpose of this project is to developselective editing strategies for the U. S. Census Bureauforeign trade statistics program. The Foreign TradeDivision (FTD) processes more than 5 milliontransaction records every month using a parameter filecalled the Edit Master. In this project, we investigate thefeasibility of using selective editing for identifying themost erroneous records without the use of parameters.This process would allow a more efficient target ofrecords for review and lead to an expected reduction inrejects.

Highlights: We researched the feasibility of using scorefunctions for ranking failing records earlier in the editingprocess. The domain of records for the current researchis the full data set rather than the set of edit failingrecords. The objective is to allow a more efficient targetof records for review and reduce the number of rejectsby using selective editing techniques for identifying themost erroneous records without the use of the EditMaster parameter file. We tested different ways ofpartitioning the full data and selected a test data setwhich takes advantage of the availability of more recentdata and more historical data for calculation of mediansand quartiles needed for calculating scores. We re-wroteand adjusted all routines within the selective editing

legacy software to calculate scores using detail data. Wecompleted feasability tests for one section (Sundries).

Staff: María García (x31703), Yves Thibaudeau, AlisonGajcowski (FTD), Andrew Jennings (FTD)

1.24 DISCLOSURE AVOIDANCE METHODS(Economic Project 2470851)

Description: The purpose of this research is to developdisclosure avoidance methods to be used for CensusBureau publicly available economic data products.Emphasis will be placed on techniques to implementdisclosure avoidance at the stage of data processing. Disclosure avoidance research will be conducted onalternative methods to cell suppression for selectedeconomic surveys. We will also aid in theimplementation of the methods.

Highlights: The noise project has grown significantlyand is now under consideration for many economicprograms, so there is need to plan ahead for futuremanagement of noise related software. Staff has beenworking to organize the various SAS programs currentlybeing used for both research and production applications,and to determine what types of procedures are beingconsidered for each program. Because there aredifferences in certain aspects of how noise is going to beapplied to different programs, as well as the fact thatmuch of the research has been done independently, thereare many different versions of SAS programs currentlybeing used. Staff has also began working to developstandardized SAS code so that different economicprograms are able to use similar software to performsimilar tasks. This will allow us to ensure that noise isbeing used correctly and to maintain staff that is familiarwith these programs.

The results of the initial testing of noise for use withCounty Business Pattern (CBP) data has been deliveredand is currently being evaluated. Standardized noisesoftware was used to apply noise to CBP data. Theresults were as expected based upon earlier investigationand applications to other data products.

Staff is working on a year to year noise evaluation inorder to determine what types of long term issues mayarise and what effects nosie may have on time seriesdata. A systematic way in which to assign noise forsubsequent years has been developed and tested. Theseprocedures have been incorporated into the standardizedSAS programs. Two years of collected CBP data wereused in a test that applied noise to the first year, and thenapplied noise to the second year according to the factorretention rules that were developed. Initial results werepresented to the noise research group, and suggestions

15

for further research were gathered and are currentlybeing investigated.

Staff has worked to develop a procedure for applyingnoise to Non Employer Statistics data for the secondyear in a row. Standardized SAS programs have beendeveloped and used in this work as well. Because newtypes of programs will be used, a new system of testingand production application is under development and iscurrently being tested with the help of NE staff.

Staff is working to determine what types of informationwe should publish about noise along with publishedCensus data products. Outside users have beencontacted in order to get an idea about what types ofinformation would be useful for them. Staff is workingto develop methods of describing the noise and its effecton published estimates, and staff is working to determinehow much and what types of information may bereleased without compromising the protection affordedby the use of noise.

Staff worked with two economic area groups to developmethods for generating synthetic test data for use withStEPS and for use at WESTAT. The Disclosure ReviewBoard approved the procedures and the resulting data.

During this quarter, the main goals of the noise researchgroup were to (1) specify the workload structure for themajor components of noise work for all economicprograms that are likely to use noise within the next 3years. (Lavelle Lee of EPCD heads this effort; staffparticipated in a major meeting that he called) and to (2)focus on the specific goals for the CBP (CountyBusiness Patterns) program. Specifically, work wasdone on the task to determine exactly what informationcould be revealed to data users about the way noise wasadded to protect the data. This was motivated by arecommendation of the Census Advisory Committee(specifically the American Economic Associationrepresentatives). Opinions of economists who aresophisticated users of CBP and other Census economicdata were solicited; e.g. Tom Holmes, Lars Vilhuber,and Javier Miranda. Of course, the needs of data usersfor noise level information (that could be viewed as “anoise variance”) are limited by the need to protect theconfidentiality of the data.

Staff: Laura Zayatz (x34955), Paul Massell, Phil Steel,Sam Hawala, Jeremy Funk

1.25 TIME SERIES RESEARCH(Economic Project 2370852)

A. Seasonal Adjustment Support

Description: This is an amalgamation of projects whosecomposition varies from year to year, but alwaysincludes maintenance of the seasonal adjustment andbenchmarking software used by the EconomicDirectorate.

Highlights: Seasonal adjustment and X-12-ARIMAsupport was provided to: European Central Bank, Bankof Canada, Bank of England, Hendyplan, Business CycleResearch Institute, SAS, Beacon Economics, NORC,Bureau of Labor Statistics, U.S. Department of Energy,Office of National Statistics (UK), Australian Bureau ofStatistics, OECD, Statistics Sweden.

Staff met with staff from the Office of StatisticalMethods and Research for Economic Programs(OSMREP) to discuss methodology developed at theBureau of Economic Analysis concerning the source ofrevisions due to seasonal adjustment and how itcompared to what is done at the Census Bureau.

Staff taught two time series classes, which werewell-attended by seasonal adjusters at the CensusBureau.

Staff: Brian Monsell (x31721), Tucker McElroy, DavidFindley (DIR)

B. Seasonal Adjustment Software Development andEvaluation

Description: The goal of this project is a multi-platformcomputer program for seasonal adjustment, trendestimation, and calendar effect estimation that goesbeyond the adjustment capabilities of the Census X-11and Statistics Canada X-11-ARIMA programs, andprovides more effective diagnostics. This fiscal year’sgoals include: (1) developing a Windows programminginterface for the X-12/X-13 seasonal adjustmentsoftware in collaboration with analysts from the Bank ofBelgium; (2) finishing a version of the X-13ARIMA-SEATS program with accessible output and improvedperformance so that, when appropriate, SEATSadjustments can be produced by the EconomicDirectorate; and (3) incorporating further improvementsto the X-12-ARIMA/X-13A-S user interface, output anddocumentation. In coordination and collaboration withthe Time Series Methods Staff of the Office of StatisticalMethods and Research for Economic Programs(OSMREP), the staff will provide internal and/orexternal training in the use of X-12-ARIMA and the

16

associated programs, such as X-12-Graph, whenappropriate.

Highlights: Staff repaired minor defects found in the X-12-ARIMA source code affecting the processing of fileswith blanks in their names and saving outlier iterationfiles. Staff also added new information to the diagnosticoutput of the program denoting the number of outliersadded and deleted during an outlier identification, andadded an option to set a differential for the critical valueof “almost” outliers listed by the program.

Staff generated a 64-bit version of X-12-ARIMA forEuropean Central Bank for their testing purposes,updated X-12-ARIMA source code used in theDEMETRA package distributed by Eurostat, and madechanges in the source code of X-13A-S to allow it to becompiled by the Gnu Fortran 77 compiler used by theBureau of Labor Statistics.

Staff continued to develop the X-13A-S seasonaladjustment software, beginning to incorporate changesmade in the latest versions of SEATS.

Staff developed Linux versions of the seasonaladjustment software, and made available to staff in theEconomic Statistical Methods and ProgrammingDivision, for testing purposes, Linux source code thatcould be compiled on their Linux machines.

Future Plans: Prepare an update of Version 0.3 of X-12-ARIMA for distribution and release it to the public.The source code of X-13A-S will be updated with themost recent version of the SEATS code and thisimplementation will be compiled and tested before abeta release to interested parties. An option will beadded to X-13A-S that allows the program to use OLSregression to generate the AR coefficients used in theAR-spectrum. Restructuring of the X-12-ARIMAsoftware to create an X-12-ARIMA programminginterface will begin.

Staff: Brian Monsell (x31721)

C. Research on Seasonal Time Series - Modelingand Adjustment Issues

Description: The main goal of this research is todiscover new ways in which time series models can beused to improve seasonal and calendar effectadjustments. An important secondary goal is thedevelopment or improvement of modeling andadjustment diagnostics. This fiscal year’s projectsinclude: (1) continuing research on seasonal adjustmentdiagnostics; (2) studying further the effects of modelbased seasonal adjustment filters; (3) in collaborationwith the Time Series Methods Staff, making further

improvements to the automatic modeling procedure ofX-12-ARIMA Version 0.3; (4) determining ifinformation from the direct seasonally adjusted series ofa composite seasonal adjustment can be used to modifythe components of an indirect seasonal adjustment; (5)studying the modeling of seasonality using Bayesianmethods, and determine if using such a method isfeasible for short time series; (6) evaluating and refininga nonlinear approach to seasonal adjustment usingempirical mode decomposition; and (7) examining anapproach for performing signal extraction for correlatedcomponents.

Highlights: During the quarter, the staff conductedresearch on several research projects:

(a) Staff investigated further the frequency domaineffects of linear filters on nonstationary data. This isimportant for the understanding of the gain function ofseasonal adjustment filters.

(b) Staff continued work on empirical revisionvariances, which can be used to assess modelmisspecification pertinent to model-based seasonaladjustment procedures, forming a model-based analog ofthe sliding-span procedures. Staff conducted extensiveempirical testing of these diagnostics, and modified thediagnostics to improve power.

(c) Staff completed research on methods to recast theX-11 seasonal adjustment filters into a finite-samplemodel-based signal extraction context, wherebytime-varying filters could be obtained - allowing for fluidand coherent handling of edge effects - as well as signalextraction mean squared errors. Determination of decentestimates of the signal extraction mean squared errors forX-11 seasonal adjustment is a long-standing problem ofexceptional importance to the U.S. Census Bureau. Staffdeveloped two paradigms for recasting the X-11 filtersand has begun implementation of the methods.

(d) Staff examined the performance of a one-coefficienttrading day model for flow series where daily weightsfor Monday through Saturday are considered the sameand Sunday is different. A brief analysis of this modelfit to 17 Services division series showed that this modelfit better than the default trading day model inX-12-ARIMA for 3 of the 17 series, according to AICCstatistics.

(e) Staff completed research on a nonlinear seasonaladjustment algorithm that generates seasonal adjustmentestimates with less bias, thus counter-acting the typicaldownward bias in time series requiring Box-Coxtransforms.

17

(f) Staff completed investigation of a newgoodness-of-fit diagnostic based on the log spectraldensity.

(g) Staff continued investigating variance estimation forprocesses with a unit root, which is pertinent to themodeling of economic data.

(h) Staff conducted empirical investigations of signalextraction goodness-of-fit diagnostics andfrequency-specific seasonal time series models.

(i) Staff continued developing algorithms for extractingempirical modes as a non-model based seasonaladjustment method.

(j) Staff initiated joint research with colleagues at theUniversity of Missouri to examine the effect of usingcountry specific calendar regressors as used by Eurostaton U. S. Economic series.

Future Plans: Continue seasonal adjustment research,begin examining Bayesian methodology for modelingtime series, begin examining an automatic ARIMAmodel selection procedure that uses an empiricalinformation criteria. Present results on modelingnonlinear time series and using seasonal adjustmentdiagnostics on structural models.

Staff: Tucker McElroy (x33227), Brian Monsell,Christopher Blakely, Irma Hindrayanto, William Bell(DIR), David Findley (DIR)

D. Supporting Documentation and Software for X-12-ARIMA and X-13A-S

Description: The purpose of this project is to developsupplementary documentation and supplementaryprograms for X-12-ARIMA and X-13A-S that enableboth inexperienced seasonal adjustors and experts to usethe program as effectively as their backgrounds permit.This fiscal year’s goals include improving thedocumentation of X-12-ARIMA, improving thedocumentation of X-12-ARIMA, rendering the outputfrom X-13A-S accessible, and exploring the use ofcomponent and Java software developed at the NationalBank of Belgium.

Highlights: Our staff continued to work with acontractor to develop an accessible version of the X-12-ARIMA Reference Manual that will satisfy Section 508requirements. Staff members have finished evaluatingthe accessible versions of the documents, and developeda report on what work is needed to make them fullyaccessible.

Staff revised a utility that converts X-12-ARIMA outputinto accessible HTML. Another utility was developed toconvert output files to XHTML.

Staff updated the Seasonal Adjustment Papers website,allowing access to several additional papers.

Staff developed a Census Bureau contribution to a reportdealing with modernizing seasonal adjustment software,specifically creating software modules for the X-12-ARIMA seasonal adjustment program.

Future Plans: The effort to create an accessible versionof the Reference Manual for Version 0.3 will becontinue. Development of the documentation for X-13A-S will continue.

Staff: Brian Monsell (x31721), Larry Malakhoff, SaraWade

1.26 SURVEY OF RESEARCH ANDDEVELOPMENT IN INDUSTRY IMPUTATIONAND SAMPLING RESEARCH AND SOFTWARE

DESIGN(Economic Project TBA)

Description: This project undertakes research on theimputation of unreported mandatory items in the Surveyof Research and Development in Industry. It alsoexamines what estimators are more appropriate underalternative sampling plans, in particular it evaluatesusing balanced sampling.

Highlights: Staff wrote software for imputing R&Dinvestment for companies which do not report R&D inthe current year of the survey, but reported R&D in aprevious year of the survey. Staff implementedregression estimation technique to perform theimputation. Staff is currently implementing the methodof Xu, Shao, Palta, and Wang (2008) as an alternativemethod. This last method may be advantageous becauseit uses nonparametric regression to impute missing data,which may be more robust to extreme values.

Staff: Yves Thibaudeau (x31706), Jun Shao

1.27 REMOTE ACCESS - MICRODATAANALYSIS SYSTEM

(Strategic Planning and Innovation Project0359999)

Description: Researchers and sophisticated data users’demand for Census Bureau microdata, both for generalresearch and programmatic needs, continues to grow.Microdata allows virtually any type of analysis, and it isthe desired form of data that allows modeling. InternalCensus Bureau microdata files contain levels of detail,

18

and variables, which are not available in public use files.Methods are applied to reduce detail, both bysuppressing and coarsening variables in public use files,in order to protect the identity of respondents and toensure confidentiality of responses under Title 13 of theU.S. Code. As data on individuals accumulate, andidentifiable public and commercial data becomes moreand more accessible, the ability to publish qualitymicrodata while maintaining a sufficient level ofambiguity is becoming an issue.

One solution is to allow researchers and public users torun models against internal microdata. The result of themodel is the object of interest, not the underlying data.Over the past ten years, the Census Bureau’s DisclosureReview Board (DRB) has examined model outputs fromthe Census Bureau’s Research Data Centers (RDC) andthey have been virtually without disclosure problems.This triggered the development of the MicrodataAnalysis System (MAS). MAS allows researchers torun analysis against internal microdata and view modelresults. At the same time, it protects the underlyingsensitive data by applying a number of confidentialityrules.

Highlights: Staff worked on designing and developinga XML based communication protocol in order to linkthe new Microdata Analysis System into the existingTheDataWeb system. The new XML schema providesincreased security and a standard interface for futureenhancements.We also continued working on the front-end Java Servletand its integration with the back-end R engine usingRserve and Zelig.

Staff: Mohammed Chaudhry (x34989)

1.28 POSTAL REGULATORY COMMISSION/STATISTICAL CONSULTING

(Statistical Research Division Project 8150000)

Description: The work associated with this project willentail the review of testimony, interrogatories, decisions,and other documentation relating to proceedings of theCommission in order to identify major statistical issuesand provide relevant consultation. The consultation willinclude: 1) the briefing of the commissioners and othercommission officials on the ramifications and desirableapproaches to the identified statistical questions; and 2)the presentation of written summaries of the majorfindings from all assigned reviews.

Highlights: Staff conducted technical reviews ofproposals and related documentation and participated inmethodology conferences associated with the PostalService’s recommendation for a comprehensive revisionof the mail delivery performance measurement system.

Staff: Leroy Bailey (x34917)

1.29 PROGRAM DIVISION OVERHEAD(Census Bureau Project 0251000)

A. Division Leadership and Support

This staff provides leadership and support for the overallcollaborative consulting, research, and operation of thedivision.

Staff: Tommy Wright (x31702), Tina Arbogast, AliceBell, Pat Cantwell, Robert Creecy, Manuel de la Puente,Michael Hawkins, Judi Norvell, Barbara Palumbo,Gloria Prout, Diana Simmons, Kelly Taylor

B. Research Computing

Description: This ongoing project is devoted to ensuringthat Census Bureau researchers have the computers andsoftware tools they need to develop new statisticalmethods and analyze Census Bureau data.

Highlights: The SGI Altix (research1.srd.census.gov) isundergoing certification and accreditation (C&A) as partof CEN16 (Network Services). All federal informationsystems are required by law to undergo C&A, whichinvolves a careful analysis of the risks to the system andthe selection and implementation of security controls tomitigate those risks. During this first quarter, we havebeen meeting with the CEN 16 certification team weeklyand providing them with inventory and other supportinginformation. We have been scanning our systems withscanning tools from the Center for Information Security,and working with ISSRO to determine which itemsfound by the scans are actual concerns, and which arefalse positives.

Staff: Chad Russell (x33215), Mohammed Chaudhry

19

2. RESEARCH

2.1 – 2.2 GENERAL RESEARCH ANDSUPPORT TOPICS

(Census Bureau Projects 0351000, 1871000)

Statistical Methodology

A. Disclosure AvoidanceDescription: The purpose of this research is to developdisclosure avoidance methods to be used for all CensusBureau publicly available data products. Emphasis willbe placed on techniques to implement disclosureavoidance at the stage of processing. Methods will bedeveloped, tested, evaluated, and documented. We willalso aid in the implementation of the methods.

Highlights: Staff used Monte Carlo simulations todetermine the sampling distributions of various teststatistics for analytic validity of synthetic data. Staffworked with Synectics to develop an action plan for itsremaining work under contract on the MicrodataAnalysis System (MAS).

Staff have worked with the contractor for the finalprototype of the microdata analysis system (MAS).Two new rules have been implemented, “drop two” and“no marginal 1's or 2's”. “drop two” is a subsamplingrule designed to prevent differencing. For any givenpopulation, the system selects 2 observations to bedeleted in random and consistent manner. Random inthe usual sense; consistent in that the same 2observations are always dropped for a particular subset,regardless of how (or when) it is defined. The other rulelooks at the context of the population defined by theuser and determines if any nearby populations areproblematic (1s or 2s). Implementation of both ruleshas been difficult. The random number scheme ran intoa SAS limitation on the range of the seed. We had tokeep in range but carry enough digits to get a trulyrandom number. The margin rule required thegeneration of a new set of formats for all numericvariables. The new format led to the discovery of anendpoint problem in the cutpoint generation. The delaysare being coordinated with Census Bureau ITrequirements and the rules testing contract. The paperdescribing the system and its confidentiality provisionshas been revised and should appear in the 06 FCSMpolicy conference proceedings.

A staff member represents the disclosure review boardon the Privacy Policy Research Committee. This groupcomes under the writ of the Chief Privacy Officer,Nancy Gordon acting. This committee is hammering

out the details for our compliance to various (and oftenoverlapping) governmental initiatives on the handling ofPublicly Identifiable Information.

Staff: Laura Zayatz (x34955), Phil Steel, Paul Massell,Sam Hawala, Jeremy Funk

B. Disclosure Avoidance for Microdata

Description: Our staff investigates methods ofmicrodata masking that preserves analytic properties ofpublic-use microdata and avoid disclosure.

Highlights: Staff completed the report “Examples ofEasy-to-implement, Widely Used Methods of Maskingfor which Analytic Properties are not Justified” that willappear in the Research Report Series. Staff studied anextensive cryptographic literature on differentialprivacy. Although the methods can be alternatively beapplied to the methods given in “Analytically ValidDiscrete Microdata Files and Re-identification”(Research Report Series: Statistics #2007-19) to re-identify in a number of situations, the methods do notpresently appear to assure analytic properties of thereleased data. Further, the extensions to epsilon-differential privacy may only allow analytic propertiesin a very narrow range of situations. Staff: William Winkler (x34729), Yves Thibaudeau,William Yancey

C. Seasonal Adjustment (See Economic Project2370852)

D. Small Area Estimation-Decennial/ DemographicApplications

Description: A meeting designed as an open forum forsmall area research and topics on small area censuscoverage evaluation was established and meets twiceeach month. These meetings are designed as a way todisseminate ideas on small area estimation at the earlystages of development. The group includes researchersin the Decennial Statistical Studies Division, theStatistical Research Division, and the Director’s Officewho are working on small area coverage projects.


Staff: Jerry Maples (x32873), Don Malec

20

E. Nonresponse in Longitudinal Surveys

Description: The purpose of this continuing project isto develop methodology to evaluate alternative (cell-based and logistic regression) models for nonresponseadjustment in longitudinal surveys, especially in the re-engineered SIPP.

Highlights: Staff continued work on an FCSM talk(delivered at FCSM07) and paper (posted to theFCSM07 website) on the computation and interpretationof metrics for the effectiveness of weight adjustmentsfor attrition within longitudinal studies like SIPP, withthe objective of judging between alternative modelsused for adjustment. Additional research this quarterexplored how the metric results on SIPP 96 data wereaffected by raking of the adjusted weights, as in done inSIPP production. Staff worked to incorporate these newresults along with the FCSM paper to prepare a paperfor journal submission.

Staff: Leroy Bailey (x34917), Eric Slud, Julie Tsay

F. Household Survey Design and Estimation

Description: The household surveys of the CensusBureau cover a wide range of topics but use similarstatistical methods to calculate estimation weights. It isdesirable to carry out a continuing program of researchto improve the accuracy and efficiency of the estimatesof characteristics of persons and households. Amongthe methods of interest are sample designs, adjustmentsfor nonresponse, proper use of population estimates asweighting controls, and the effects of imputation onvariances.


Staff: Lynn Weidman (x34902)

G. Sampling and Estimation Methodology:Economic Surveys

Description: The Economic Directorate of the CensusBureau encounters a number of issues in sampling andestimation in which changes might increase theaccuracy or efficiency of the survey estimates. Theseinclude estimates of low-valued exports not currentlyreported, alternative estimation for the QuarterlyFinancial Report, and procedures to addressnonresponse and reduce respondent burden in thesurveys. Further, general simulation software might becreated and structured to eliminate various individualresearch efforts.

Highlights: The investigation methodology for treatingan influential observation in the estimation of total

revenue from the Monthly Survey of Retail Trade didnot progress because of the departure of a team member,but the search for a replacement is underway. Anobservation is considered influential if the estimate oftotal monthly revenue is dominated by its weightedcontribution. The goal is to find methodology that usesthe observation but in a manner that assures itscontribution does not dominate the total. Staff: Pat Cantwell (x34982), Mary Mulry, RoxanneFeldpausch (ADEP), Lynn Weidman, Don Malec

H. Research and Development Contracts

Description: The Research and Development Contractsare indefinite delivery, indefinite quantity task ordercontracts for the purpose of obtaining contractorservices in highly technical areas to support researchand development activities across all Census Bureauprograms. The contracts provide a pool of contractorsto assist the Census Bureau in conducting research onall survey and census methods and processes to improveour products and services through FY 2007. The primecontractors include educational institutions, university-supported firms and privately owned firms thatconcentrate in sample survey research, methodology,and applications to create a pool of specialists/experts totackle some of the Census Bureau’s most difficultproblems through research. Many of the primecontractors are teamed with one or more organizationsand/or have ar rangement with outs ideexperts/consultants to broaden their ability to meet all ofthe potential needs of the Census Bureau. These five-year contracts allow Census Bureau divisions andoffices to obtain outside advisory and assistanceservices to support their research and developmentefforts quickly and easily. The multiple contracts wereawarded during FY2002 in six technical areas: 1)assessment, planning, and analysis; 2) data analysis anddissemination; 3) statistical analysis, 4) methodologicalresearch, 5) sub-population research, and 6) surveyengineering.

Highlights: During the first quarter of FY2008, 5 taskorders were modified, and 2 were completed. To date,there have been 67 task orders awarded under theR&D2007 contracts, with a monetary value over $68million (over $60.5 million obligated). Forty-three taskorders have been completed and 1 task orders had a stopwork order, leaving 23 active tasks.

Work continues on the preparation and planning for thesolicitation and award of the next series of 5-year R&Dcontracts. A Pre-Solicitation conference was held onDecember 17, 2007 where interested vendors learnedabout the plans for the new contract and was given theopportunity to interact with program managers across

21

the Census Bureau. Also, meetings were held with theDemographic, Economic, and Decennial Directorates todiscuss the upcoming solicitation, request their help inpreparing the draft request for proposal and requestparticipation in the evaluation of the proposals for thenew contracts. Because of the past success of the R&D2007 services, the Census Bureau has a continuing needfor the research and development services.

Staff: Ann Dimler (x34996)

Statistical Computing Methodology

A. Record Linkage and Analytic Uses ofAdministrative Lists

Description: Under this project, our staff will provideadvice, develop computer matching systems, anddevelop and perform analytic methods for adjustingstatistical analyses for computer matching error.

Highlights: We completed a second versiondocumentation for the variant of BigMatch that is beingused during the 2008 Dress Rehearsal. Staff did a fullproduction match of 300 million against 300 million.The software is approximately 80 times as fast ascommercial software from IBM and 50 times as fast asa parallel version of matching software underdevelopment at Stanford University. We still maintainthe highest accuracy of record linkage software with thetypes of lists in the Decennial Census or certainadministrative lists.

Staff debugged a very subtle error in BigMatch that wascausing errors as much as 0.1% of the match projects(but only with an exceptionally small subset of pairs ofrecords). The error was due to an exceptionally minordifference in the inline string comparison function usedin the quick sort algorithm of Bentley and Sedgewick(ACM-SIAM 1993 Conference on Discrete Algorithms)and the standard C string comparison function strcmp().

Future Plans: Work on BigMatch software and testing.Work on record linkage theory and computationalmethods.

Staff: William Winkler (x34729), William Yancey, NedPorter

B.1 Editing

Description: This project covers development ofmethods for statistical data editing. Good methodsallow us to produce efficient and accurate estimates andhigher quality microdata for analyses.

Highlights: No significant progress this quarter. Staff: María García (x31703)

B.2 Editing and Imputation

Description: Under this project, our staff will provideadvice, develop computer edit/imputation systems insupport of demographic and economic projects,implement prototype production systems, andinvestigate edit/imputation methods.

Highlights: With the help of the Manufacturing andConstruction Division, we set up a database includingframe information along with survey records. Thedatabase contains the information needed for imputingand editing missing or inconsistent items for the Surveyof Research and Development (R&D), a semilongitudinal survey.

Staff developed experimental software to performlongitudinal imputation of R&D. The imputationmethod was based on calibration. A new method (Xu,Shao, Palta, and Wang, 2008) is currently beingimplemented. Staff presented the research at FCSM. Apaper was published in the proceedings.

Staff: Yves Thibaudeau (x31706), Jun Shao

C. Developed Software Support – GeneralVariance Estimation Development and Support

Description: This project will develop new methodsand interfaces for general variance estimation softwareincluding VPLX, WesVar, and SUDAAN. Our staffwill provide support for complex applications such asthe Survey of Income and Program Participation and theSurvey of Construction, create training materials, andprovide training for variance estimation softwareapplications.

Highlights: We continued to offer ongoing Hotlinesupport for variance estimation software to the fourprogram directorates at the Census Bureau. Staffcontinued to provide specific, long-term support to theManufacturing and Construction Division. In addition,staff began undertaking discussions on how best tosupport VPLX Bureau-wide beyond Bob Fay'supcoming retirement.

Staff: Aref Dajani (x31797), Ned Porter, Bob Fay(DIR)

22

D. Statistical Computation for LongitudinalEmployer-Household Dynamics (LEHD)

Description: The Longitudinal Employer-HouseholdDynamics project is a cooperative effort among severalareas of the Census Bureau to combine federal and stateeconomic data with demographic data. Sources of datainclude the American Community Survey, IRS, andSocial Security data. Using this data, researchers willnow be able to perform analyses that help disentanglethe effects of choices that firms make from the choicesworkers make.


Staff: Rob Creecy (x33207)

E. Missing Data and Imputation: MultipleImputation Feasibility Study

Description: Methods for imputing missing data areclosely related to methods used for synthesizing missingitems for disclosure limitation. One method currentlyapplied to both issues is multiple imputation. Althoughthe two issues may be addressed separately, techniqueshave been developed that allow data users to analyzedata in which both missing data imputation anddisclosure limitation synthesis have been accomplishedvia multiple-imputation techniques. This projectascertains the effectiveness of applying multipleimputation to both missing data and disclosurelimitation in American Community Survey (ACS) groupquarters data. Statistical models are used to generateseveral synthetic data sets for use in the multiple-imputation framework.

Highlights: Work continues on improving the synergyof group quarters disclosure methods and ACS editrequirements. We are actively working with sponsorsto establish a coherent framework for disclosuresoftware in the statistical language R that will ensureconformity with edit requirements and seemlessintegration with the ACS production workflow.

Staff: Rolando Rodríguez (x31816)

F. Optimizing Field Operations

Description: This project is intended to provide theField Division with a resource for new research in areasthat will improve its processes. Over time, researchtopics may include modeling or forecasting. The firstproject will study the use of operations researchtechniques to improve the ability to predict survey costsand response rates in the field.

Highlights: Staff worked to finalize a report onStochastic Simulation of Field Operations in Surveys.A report will be released next quarter. Work on thisproject is suspended for now.

Staff: Bor-Chung Chen (x34857)

G. Modeling, Analysis and Quality of Data

Description: Our staff investigates methods of thequality of microdata primarily via modeling methodsand new software techniques that accurately describeone or two of the analytic properties of the microdata.

Highlights: As a member of the National Academies ofScience committee that is studying voter registrationdatabases, one staff member helped create a surveyasking detailed background information related to Statevoter registration databases. The staff member alsowrote background documents on elementary recordlinkage methods that could be quickly implemented andon list maintenance procedures with examples.

One staff member co-presented the special session“Data Quality and Record Linkage” at the Society ofActuaries annual meeting.

Future Plans: We hope to extend a general dataanalysis framework so that it includes applications forgeneral modeling, edit/imputation and non-trivial recordlinkage.

Staff: William Winkler (x34729), Rob Creecy

Survey Methodology A. Usability Research and Testing

Census Usability DayOn December 5, 2007, staff sponsored Census UsabilityDay with the following posters on display: KathleenAshenfelter, “Eye Tracking Research-An Overview”;Larry Malakhoff, “Accessible Software at the CensusBuerau”; Erica Olmsted-Hawala and CarollynnHammersmith, “Incorporating Information ArchitectureActivities into the Evolution of the U.S. CensusBureau’s Web Site”; Michelle Rusch, “The UsabilityStudy of the Census in Schools Web Applications forGrades K-6"; and Beth Nichols, “Usability ofInterviewer Materials for the 2006 Census CoverageMeasurement Person Followup.”

A.1. Web Applications Accessibility

Description: This project focuses on the accessibility ofInternet and Intranet applications by blind and low

23

vision users in accordance with the Section 508regulations.

Highlights:

AESDirect (Foreign Trade Division): AESDirectpermits exporters to declare the value of goods they aresending to foreign countries. This quarter staff followedup on recommendations made about the AESDirect website in the accessibility report submitted to the ForeignTrade Division in 2007. Staff provided an alternative tothe color coding scheme for mandatory, conditional, andoptional data-entry fields with geometric symbols (reddiamond, blue square, and black circle). The AESDirectweb site conforms with the Section 508 regulations.

Staff: Larry Malakhoff (x33688)

Support for X-12 ARIMA Documentation & Software(Statistical Research Division):This quarter staff continued to review work done byNetCentric Logo to make X-12 ARIMA PDFdocumentation accessible. Staff found tables and plaintext accessible, but equations and equation termsembedded within the text were not always accessible.Staff submitted the final corrected X-12 ARIMA PDFdocumentation specification to NetCentric Logo fortheir action. A review will be conducted next quarterwhen the corrected documentation becomes available.

Staff: Larry Malakhoff (x33688), Brian Monsell, SaraWade

Classification Analytical Processing System (CAPS)(Economic Statistical Methods & ProgrammingDivision):This quarter staff evaluated the CAPS portal web site.This analysis revealed the CAPS header logo was nottagged with ALT text and the displayed label text andtext vocalized by the screen-reader software did notagree. This project is complete.

Staff: Larry Malakhoff (x33688), Danny Lee (ESMPD)

Secure Message Center (SMC) (Systems SupportDivision):The SMC web site is to be used by survey respondentswho need to communicate with the Census Bureau in asecure manner. The survey respondent could then sendand receive messages in the SMC as needed. Theevaluation revealed links are read from right to left onthe bottom of the screen and two sort functions areinaccessible on the mailbox screen. This project iscomplete and the report was submitted to the SRDResearch Report Series.


Title 13 Awareness E-Learning Application (PolicyOffice):This application permits Census Bureau staff to refreshtheir knowledge about working with Title 13 data. Theevaluation revealed visual focus is not shown whentabbing and using the arrow keys and users are directedto a “check answer” button which did not exist. Thisproject is complete.


A.2. Desktop Applications Accessibility

Description: This project focuses on accessibility ofdesktop applications by blind and low vision users inaccordance with the Section 508 regulations. Desktopapplications are either downloaded or sent to therespondent on disk.

A.2.a X-12 ARIMA Graphing Application: Thisapplication generates graphs from X-12 ARIMA data.The application can be downloaded by the X-12 usercommunity.

Highlights: Corrections were made to the report basedon comments received from the reviewer. The reportwas submitted to the SRD report series.

Staff: Larry Malakhoff (x33688), Demetra Lytras(ADEP)

A.3. Census.gov Template Development

Description: The purpose of this study is to develop aset of templates with a consistent and usable look andfeel for the Census.gov website. The template isintended to be used by both the demographic andeconomic domains of Census.gov. Some of thetechniques to develop the template include card sorting,low-fidelity prototype testing, and usability testing.Currently, the focus of the study is card sorting.

Highlights: We took the results from the open sort andcreated 16 top level categories for the closed sort. The16 top level categories used in the closed sort follow:Business, Census and Surveys, Economy/EconomicIndicators, Education, Geographic Areas (regions,states, cities), Health, More about your Questionnaire,Neighborhoods and Communities, Occupations andEmployment, Population Numbers, Poverty andWelfare, Real Estate, Retail Sales, Society and Culture,Voting and Politics, and Your Money.

We ran 17 users through the closed sort. The analysisteam is currently meeting to work on understanding theresults from the closed sort.

24

Staff: Erica Olmsted-Hawala (x34893)

A.4. Usability Study of the 2010 Web Site

Description: The purpose of this study is to identifyusability problems and successes in an ongoing anditerative way with the 2010 web pages, a domain off ofCensus.gov. This domain is unusual in that as 2010approaches, the audience and content will change.Taking these characteristic into account will be ofprimary importance when designing and modifying theinterface.

Highlights: Staff wrote a quick report with examples ofusability problems. We met with the clients andpresented our findings and discussed with the clientpossible recommendations. An example of a usabilityfinding and the recommendation follows:• The site gave the impression that the content and

main message of the 2010 Census Web site wasdirected towards internal Census Bureau employeesrather than focused on external users, such as thegeneral public. Recommendation:• Refocus content on general and partner users.• Move the important information that is

currently buried under the “About 2010” tabup so that users see this content, or how to getto this content immediately upon entering thesite.

• Keep the left-hand link into the “Recent news”but do not highlight the “Top Story.” Save thisspace for more important information for yourgeneral and partner users.

• Make the FAQs into actual questions.• The wording of the link “2010 Census is

different should be changed to somethingalong the lines of “What is New about the2010 Census?”

Staff: Erica Olmsted-Hawala (34893), Eleanor Gerber

A.5. Usability Study of the Census in the SchoolsWeb Site

Description: The Census in Schools (CIS) programpromotes data literacy and increases awareness ofCensus Bureau products and activities by providingeducators with teaching tools, resource materials,workshops, and other professional developmentopportunities. In addition to targeting teachers, theCensus Bureau’s Public Information Office (PIO) isexpanding the current Web site to include onlineactivities for students in grades Kindergarten (K)-12.PIO is currently developing Web content customized forgrades K-5. The Census in Schools (CIS) design teamwas interested in having usability lab staff evaluate the

developing site with respect to its usability for primaryaged school children. CIS also anticipatedrecommendations for ways to improve the usability forthe users. In this study, usability lab staff recruitedchildren in grades K-6, conducted dry runs to evaluatethe study’s protocol, conducted the study, identifiedareas of the site that worked for the children as well asareas where they encountered usability problems and/orhad low satisfaction. We recommended possiblesolutions for the problem areas.

Highlights: Upon completing the final report on thisproject, staff submitted it to the sponsor. Following arejust a few of the high-priority findings andrecommendations documented in the report, “AUsability Evaluation of the Census in the Schools WebSite”:• For the Counting Way Page: Children wanted

something interactive, something they could playwith. They often went directly to the left-navigation panel and wanted to select anotheroption because they did not immediately seesomething for them to do. We recommended thatthe page be made more interactive and that itprovide a place for the children to enter a numberafter they have counted so that they can check theiranswers and/or get feedback.

• For the Kids Coloring Page: Children struggled toselect the virtual crayons because there was notenough space on or around the crayon image thatwas sensitive to being selected. We recommendedtwo ways to improve selectivity: 1) increaseselection space surrounding the crayon and/or 2)increase the size of the crayon.

• For the Houses Coloring Page: Children hadproblems using the Houses Coloring Page becausethe shapes are too small. We recommended thatthe selection properties on very small items bemade easy to color or that the houses be madelarger.

• For the Memory Game: The rate for resetting cardsis too slow. Some tried to flip to the side of thepage or clicked on another deselected card to try tospeed things up. We recommended that the cardsbe made to flip faster when they reset, or that usersbe allowed to flip the cards back at their own pace.

• For the Word Find: Most kids did not understandhow to highlight words at first. They usually didnot figure it out until they were instructed to readthe instruction. They typically did not look to readthe text though unless told to do so. Werecommended that no reading be required. Providemultiple ways to select to ensure that young userscan figure out how to select. For example, allowthem to select letter by letter if they should chooseto drag the cursor over the word. In addition,

25

provide a visual cue that indicates to them that theyare selecting. For example, the letters might behighlighted as they are selecting.

This Web site was also evaluated for accessibility byusing diagnostic software and a popular screen reader.Accessibility findings and recommendations aredocumented in the final report. The sponsor acceptedmany of the usability and accessibility recommendationsand directed the software contractor to implement them.This project has been completed.

Staff: Erica Olmsted-Hawala (x34893), LarryMalakhoff, Betty Murphy

A.6. Usability Study of the Monthly and AnnualWholesale and Retail Trade Web Site

Description: The Monthly Wholesale Trade and theAnnual Wholesale Trade domains, currently twodifferent domains off of Census.gov are combining intoone primary domain. Staff from the two differentdomains would like to have usability assistance as theyundergo the development and improvement of the newdomain of the Census.gov web site. The WholesaleTrade design team is interested in receiving iterativeusability feedback and recommendations onimprovement as it works on the development of the site.

Highlights: Staff recruited seven users to evaluate theWeb site interface. Staff ran the participants throughthe usability study with at least half of the participantsdoing eye-tracking. Staff gathered and analyzed data onuser accuracy and satisfaction. The average accuracyscore was 73%. The average user satisfaction score was6.5 on a 9 point scale. Staff wrote a quick report andmet with the clients to discuss findings andrecommendations. An example of a usability findingwith a recommendation follows:• Participants struggled to find information while

sifting through dense text. For example, one taskasked participants to identify if historical AnnualWholesale Trade Surveys were available on-line.Only two out of seven participants completed thetask successfully. The low success rate may be dueto the location of the information that provides theanswer which was located at the end of a block oftext.

• We recommend that text be written in a style that isappropriate for the Web. Writing for the Webincludes short concise sentences and bulleted listsinstead of dense paragraphs.

Staff: Erica Olmsted-Hawala (x34893), Jenna Beck,Betty Murphy

A.7. 2007 Economic Census Web Site Redesign

Description: To begin this project, management in theEconomic Planning and Coordination Division (EPCD)asked SRD to conduct an expert review of prototypescreen designs for the 2007 Economic Web site. Oncethe Web site has been designed according to the CensusBureau’s template for corporate look and feel, SRD willevaluate the usability and accessibility of the new site.

Highlights: Staff met with the EPCD team anddiscussed methods to be used in conducting the expertreview.

Staff: Betty Murphy (x34858), Erica Olmsted-Hawala

A.8. Iowa State Research on Spatial Ability

Description: The purpose of this research is to continuethe Census Bureau’s investigations of the role of spatialability in mediating the success of field personnel inperforming computer-based tasks.

Highlights: Staff reviewed Iowa State’s evolvingcognitive model of the address-listing tasks. Wecritiqued screen prototypes to be used in the addresslisting tasks. We reviewed storyboards and scenarios,as well as a detailed software specification developed byIowa State to guide implementation of an experimentaldesign. We reviewed plans for a user study withtreatments to include (1) software that provides step-by-step guidance on what the user should be doing, and (2)software that has the same functionality, but provides noguidance in conducting the address-listing task.

Staff: Betty Murphy (x34858), Kathy Ashenfelter

A.9. Expert Review of ACSD Web Sites

Description: Staff is in the beginning stages ofconducting an expert review on the redesign forAdministrative and Customer Services Division(ACSD) of the web sites for the State and MetropolitanArea Data Book and the County and City Data Book.This project involves providing creative andconstructive guidance for the Local Area Statistics teamthat is redesigning and merging these two web sites.This expert review will result in a report detailing user-friendly features of the web sites, usability issuesidentified by the staff, and recommendations forimprovement of those issues and of the web sites ingeneral.

Highlights: We identified staff to conduct the expertreview and set up a meeting with the Local AreasStatistics team from ACSD to determine the next stepsof the process of this review.

26

Staff: Kathleen Ashenfelter (x34922), ElizabethMurphy, Allison Morgan

B. Questionnaire Pretesting

Description: This project involves coordinating theCensus Bureau’s generic clearance for questionnairepretesting research. Pretesting activities in all areas ofthe Census Bureau may use the clearance if they meetthe eligibility criteria.

Highlights: Six submissions were made for OMBapproval under the generic clearance for questionnairepretesting research: respondent debriefings on theCensus Coverage Measurement Independent ListingBook, usability interviews on the Survey of BusinessOwners, focus groups for the decennial census publicinformation campaign, cognitive interviews for the BeCounted form, cognitive interviews for the ACSSpanish CATI/CAPI form, and a split sample field testfor the CPS Voting and Civic Engagement Supplement.

Future Plans: Staff will consult on generic clearancesubmissions as needed.

Staff: Terry DeMaio (x34894)

C. Questionnaire Design Experimental ResearchSurvey 2006 (QDERS)

Description: QDERS 2006 is an omnibus surveydesigned to facilitate independent research related toquestionnaire design issues and other surveymethodology issues. The QDERS 2006 was conductedfrom the Hagerstown Telephone Center. The focus ofthe 2006 QDERS is an questionnaire design experimentexamining different ways to determine a person’s placeof residency on Census day.

Highlights: Staff completed its analysis of the twodemographic questions within the 2006 QDERS. Staffdid not find the same level of inconsistent relationshipto age data (e.g., reporting “father” rather than “son”) inQDERS as others have found in other censusenvironments. Additionally, the edit to correct forinconsistent data did not prove burdensome. Staff alsofound little overt confusion with a question thatconfirms an age in the past rather than confirmingcurrent age, but the former question takes respondentssignificantly longer to answer. Staff also met withanalysts in the National Processing Center (NPC) for aresidence status coding of QDERS data by trained staff.See Nichols and Childs (2007), SRD Research Report(Survey Methodology #2007-39).

Staff: Jennifer Rothgeb (x34968), Jenny Hunter Childs,Beth Nichols, Rolando Rodríguez, Aref Dajani

D. Language: Interdisciplinary Research on

Language and Sociolinguistic Issues Relevant toSurvey Methodology

Description: There is a need for both qualitative andquantitative interdisciplinary research on how to bestdevelop and successfully use non-English languagecollection instruments and other survey materials.Interdisciplinary research is also needed to determinethe quality of the data that respondents with little or noknowledge of English provide the Census Bureau usingboth non-English and English language data collectioninstruments.

Highlights: Our staff worked collaboratively withresearchers at RTI International and Research SupportServices on cross-cultural issues in survey interviewsand translation methods. Specifically, we studied thefollowing problems: 1) cross-cultural communicationnorms and cognitive interviews, 2) the use ofinterpreters in survey interviews, 3) language andcultural effects on conducting cognitive interviews innon-English languages, and 4) methods to encouragesurvey participation from speakers of languages otherthan English.

Staff completed the first draft of two invited papers oncross-cultural issues in cognitive interviews to bepresented at the 2008 International Conference onSurvey Methods in Multilingual, Multiregional andMulticultural Contexts.

Staff developed a book proposal on Politeness inHistorical and Contemporary Chinese Communication.Based on reviewers’ recommendations, the ContinuumInternational Publishing Group offered a book contractto staff to work on the project.

We are also working closely with an international groupof researchers who are members of the ComparativeSurvey Design and Implementation (CSDI) group, onthe development of interpretation guidelines. Our staffalso participates in the Interagency LanguageRoundtable meetings for discussion of translation,interpretation, and language proficiency testing issues.

Future Plans: We will continue to work with CSDI toconduct research necessary for the development of aCensus Bureau guideline for interpretation. We willalso complete drafting two book chapters, and papersfor the 2008 AAPOR conference.

Staff: Yuling Pan (x34950), Manuel de la Puente, PattiGoerman, Diana Simmons

E. Training for Cognitive Interviewing

Description: Our staff will train members of otherdivisions in the Census Bureau to carry out cognitiveinterviewing and provide consultation and support for

27

projects which they carry out.


Staff: Eleanor Gerber (x33489)

F. Research on Cognitive Testing of Non-EnglishLanguage Survey Instruments

Description: The staff is currently engaged in a studydesigned to test and identify best practices forconducting cognitive interviews with Spanish-speakingrespondents. We are testing both widely accepted andnew techniques and probes (e.g., “What does the termfoster child mean to you in this question?”) withSpanish-speaking respondents of high and loweducational levels. The research is based on a segmentof the CAPI version of the American CommunitySurvey. Future applications of this research shouldextend to cognitive interview techniques for use withrespondents who speak additional non-Englishlanguages.


Staff: Patricia Goerman (x31819), Diana Simmons

G. Interviewer-Respondent Interactions

Description: Survey nonresponse rates have beenincreasing, leading to concerns about the accuracy of(demographic) sample survey estimates. For example,from 1990 to 2004 initial contact nonresponse rateshave approximately doubled for selected householdsample surveys including the Current Population Survey(CPS) (from 5.7% to 10.1%). While mailout/mailbackis a relatively inexpensive data collection methodology,decreases in mailback rates to censuses and samplesurveys mean increased use of methodologies that bringrespondents into direct contact with Census Bureauinterviewers (e.g., field representatives) using CATI(computer assisted telephone interviewing) or CAPI(computer assisted personal interviewing). CAPI caninclude face-to-face or telephone contact. Unsuccessfulinterviewer-respondent interactions can lead toincreased costs, with the need for additional follow-up,and can decrease data quality.

Highlights: During the first quarter of FY 2008, wereceived the mailbacks to the 600 originalquestionnaires sent during FY 2007 to a nationalprobability sample of 600 CPS Field Representatives aspart of an exploratory study on what behaviors CPSinterviewers feel have greatest impact on gainingcooperation. There were two mailings - the original anda follow-up to the nonrespondents. From the firstmailing, we received 529 completed questionnaires.Seventy-one (71) replacement questionnaires were sentto the nonrespondents of which 37 were completed and

returned by November 16, 2007 - yielding an overallunit response rate of 94.3% [=((529+37)/600)x100%].The final unit response rate for the Type Aquestionnaires (mostly open-ended questions) was 90%[=(54/60)x100%], and the final unit response rate forthe Type B questionnaires (mostly multiple-choicequestions) was 94.8% [=(512/540)x100%]. Editing andimputation rules were developed based on a review ofreturned questionnaires; a computerized database forcapturing the responses was completed; and datacapture (i.e., entry of responses) was begun.

Staff: Tommy Wright (x31702), Kathleen Ashenfelter,Jen Beck, Tom Petkunas, Gerri Burt (FIELD), RichardNing (FIELD)

H. Research on Cognitive Testing of HousingQuestions from the American CommunitySurvey (ACS) and the American HousingSurvey (AHS)

Description: As part of a postdoctoral researchfellowship, staff conducted a study designed tounderstand sources of measurement error in number ofrooms and number of bedrooms measures in the ACS,housing quality measures from the AHS, andneighborhood quality measures from the AHS. Themain goal of this research was to determine the extent towhich cognitive difficulties in answering thesequestions may bias indices of dwelling unit density,housing inadequacy, and neighborhood quality.

Highlights: Staff submitted a draft of a postdoctoralresearch report “Measurement of Housing Quality andNeighborhood Quality in the American CommunitySurvey (ACS) and the American Housing Survey(AHS);” reported to OMB on major findings of researchconducted under the generic clearance; and presentedfindings.

Staff: George Carter, III (x31774)

I. Q-Bank: A Database of Pretested Questions

Description: Q-Bank was developed through aninteragency committee, led by the National Center forHealth Statistics (NCHS), in which the Census Bureauis a member. The objective of Q-Bank is to have anonline interagency database of pre-tested surveyquestions and research results obtained primarily fromcognitive interviews. The database is maintained atNCHS and guided and used by other participatingFederal statistical agencies, including the CensusBureau. Q-Bank serves many purposes. When surveyquestions and questionnaires are being developed, Q-Bank can be used by survey methodologists and subjectmatter experts to search through previously testedquestions. Q-Bank provides a forum to catalog ourcognitive testing reports in a manner that is easy to

28

search by content or subject matter. Q-Bank also willallow us to produce meta-data about our pretestingfindings. And, finally, Q-Bank will be an additionalresource for analysts to interpret survey data. Q-Bankhas just reached the production phase and is currentlybeing populated with cognitive test reports which isnecessary before it becomes available to a broaderaudience.

Highlights: Our staff actively participates on theinteragency Q-Bank steering committee, makingdecisions about the continued development of thedatabase. During this quarter, staff worked with NCHSon finalizing the programming of the Q-Bank toincorporate establishment and self-administeredsurveys. Additionally, several reports were coded and/orreviewed for inclusion in the Q-Bank database.

Staff: Jennifer Childs (x34927), Jennifer Rothgeb,Dawn Norris

J. Health Insurance Measurement

Description: The U.S. health care system is apatchwork of public and private programs and plans,thus there are no definitive centralized records on thenumber of individuals without insurance. Researchersmust rely on surveys for this estimate, and the CurrentPopulation Survey (CPS) is the most widely-citedsource for this statistic. It is not without its critics,however, and recent official reports have includedcaveats regarding the data quality. The purpose of thisresearch is to identify particular features of the CPSquestionnaire that are associated with measurementerror, and to explore alternative designs to reduce thaterror.

Highlights: Staff finalized a redesigned set of questionsfor cognitive testing. In keeping with previous researchfindings, the questions represent a departure from thebasic CPS design. In particular they use a current(rather than calendar year) reference period, and follow-up questions determine when the coverage started, andwhether there was any coverage prior to the currentspell within the calendar year. This allows analysts toderive estimates comparable to the CPS (that is,coverage “at any time” throughout the previous calendaryear) but it also establishes current status (generally amore accurate measure than calendar year status) and itproduces estimates of duration of coverage. Theredesigned series also uses a person-level (versushousehold-level) design, as research has demonstratedthis design feature can reduce measurement error.

Staff: Joanne Pascale (x34920)

Research Assistance This staff provides research assistance, technical

assistance, and secretarial support for the variousresearch efforts.

Staff: Tina Arbogast, Alice Bell, Joyce Farmer, JudiNorvell, Barbara Palumbo, Gloria Prout, LorraineRandall, Diana Simmons

29

3. PUBLICATIONS

3.1 JOURNAL ARTICLES, PUBLICATIONS

Ashenfelter, K. T. and Eberhard, K. M. (2007). “Effects of Verb Complexity on Speech Errors,” Memory & Cognition,35(7), 1525-1539.

Callegaro, M. and Moore, J. (In Press). “Seam Effect,” Encyclopedia of Survey Research Methods, (ed. Paul J. Lavrakas),Sage Publications, Newbury Park, CA.

Malec, D. and Maples, J. (In Press). “Small Area Random Effect Models for Capture/Recapture Methods with Applica t io n st oE s t im a t in gCover a g eErrorin theU . S .DecennialCensu s , ”Statis t icsi nMedicine.

Nichols, E. and Childs, J. (In Press). “Respondent Debriefings Conducted by Experts: A Technique for QuestionnaireEvaluation,” Field Methods.

Pan, Y. (In Press). “Review of Kádár, Dániel Z. Terms of (Im) Politeness: A Study of the Communicational Propertiesof Traditional Chinese (Im) Polite Terms of Address.” Journal of Politeness Research.

3.2 BOOKS/BOOK CHAPTERS

3.3 PROCEEDINGS PAPERS

2007 American Association for Public Opinion Research Meeting, Anaheim, CA, May 17-20, 2007.2007 Proceedings of the American Statistical Association.

• Carter, G. and Schwede, L. (2007). “Is This a Mobile Home or a Monastery?: Differentiating Group Quartersfrom Housing Units with a Validation Questionnaire,” 3831-3835.

• Gerber, E. and Landreth, A. (2007). “Respondents’ Understanding of Confidentiality in a Changing PrivacyEnvironment,” 3842-3849.

• Goerman, P. and Caspar, R. (2007). “A New Methodology for the Cognitive Testing of Translated Materials:Testing the Source Version as a Basis for Comparison,” 3949-3956.

• Mulry, M. and Keller, J. (2007). “Investigation of the Relationship Between Census Mail Response and thePartnership and Marketing Program,” 3856-3863.

30

• Nichols, E. and Childs, J. (2007). “Respondent Debriefings Conducted by Experts: A New QualitativeMethodology for Questionnaire Evaluation,” 3809-3816.

• Pan, Y., Landreth, A., Hindsdale, M., Park, H., and Schoua-Glusberg, A. (2007). “Methodology for CognitiveTesting of Translations in Multiple Languages,” 3801-3808.

• Pascale, J., Roemer, M., and Resnick, D. (2007). “Medicaid Underreporting in the CPS: Results from a RecordCheck Study,” 3941-3948.

2007 Joint Statistical Meetings (American Statistical Association), Salt Lake City, Utah, July 29-August 2, 2007.2007 Proceedings of the American Statistical Association.

• Beaghen, M. and Weidman, L. (2007). “Statistical Issues and Interpretation of the American CommunitySurvey’s One-, Three-, and Five-Year Period Estimates,” 3034-3041.

• Childs, J., Nichols, E., Dajani, A., and Rothgeb, J. (2007). “A New Approach to Measuring Residence Status,”2752-2759.

• Hawala, S. (2007). “Partially Synthetic Data to Avoid Disclosure from ACS-GQ Data,” 2976-2978.• McElroy, T. (2007). “Extending X-11 to Handle Finite-Sample Effects and Generate Signal Extraction MSEs,”

934-944.• McElroy, T. (2007). “Model-Based Diagnostic Tests Based on the Log Determinant of the Sample Covariance

Matrix,” 945-952.• Rodriguez, R. (2007). “Synthetic Data Disclosure Control for American Community Survey Group Quarters,”

1439-1450.• Thibaudeau, Y., Mulrow, J., and Shao, J. (2007). “A Study of Basic Calibration Estimates and Their Variance

Estimators in Presence of Nonresponse,” 2744-2751.

Proceedings of the 2006 Annual Conference of the Institute of Electrical and Electronic Engineers (IEEE) ProfessionalCommunication Society, Saratoga, NY, October 22, 2006.

• Olmsted-Hawala, E. (2006). “Card Sorting, Information Architecture and Usability: Adding in our Users’Perspective to Re-design the Census Bureau Web Site.”

Proceedings of the Federal Committee on Statistical Methodology Conference, Washington, DC, November 5-7, 2007.• Hawala, S. and Funk, J. (2007). “Model Based Disclosure Avoidance for Data on Veterans.”• Massell, P. and Funk, J. (2007). “Recent Developments in the Use of Noise for Protecting Magnitude Data

Tables: Balancing to Improve Data Quality and Rounding that Preserves Protection.”• Steel, P. (2006). “Mediated Access for Secure Census Bureau Data.”

Proceedings of the Joint UNECE/Eurostat Work Session on Statistical Data Confidentiality, Manchester, UK, December17-19, 2008.

• Zayatz, L. (2008). “New Implementations of Noise for Tabular Magnitude Data, Synthetic Tabular Frequencyand Microdata, and a Remote Microdata Analysis System.”

3.4 STATISTICAL RESEARCH DIVISION RESEARCH REPORTS(http://www.census.gov/srd/www/byyear.html)

RR (Computing #2007-2), María M. García, Alison Gaicowski, and Andrew Jennings, “Selective Editing Strategies forthe U.S. Census Bureau Foreign Trade Data,” December 10, 2007.

RR (Statistics #2007-13), Scott Holan, Tucker McElroy, and Sounak Chakraborty, “A Bayesian Approach to Estimatingthe Long Memory Parameter,” October 19, 2007.

RR (Statistics #2007-14), John A. D. Aston, David F. Findley, Tucker S. McElroy, Kellie C. Wills, and Donald E.K.Martin, “New ARIMA Models for Seasonal Time Series and Their Application to Seasonal Adjustment and Forecasting,”October 18, 2007.

RR (Statistics #2007-15), Tucker McElroy, “A Nonparametric Method for Asymmetrically Extending Signal ExtractionFilters,” November 13, 2007.

RR (Statistics #2007-16), Tucker McElroy and Scott Holan, “Goodness-of-Fit and Badness-of-Fit Diagnostic Tests forTime Series Models,” November 13, 2007.

31

RR (Statistics #2007-17), Laura Zayatz, “New Implementations of Noise for Tabular Magnitude Data, Synthetic TabularFrequency and Microdata, and a Remote Microdata Analysis System,” November 14, 2007.

RR (Statistics #2007-18), Elizabeth T. Huang and William R. Bell, “An Empirical Study on Using CPS and ACS SurveyData in Bivariate State Poverty Models,” November 16, 2007.

RR (Statistics #2007-19), William E. Winkler, “Analytically Valid Discrete Microdata Files and Re-identification,”December 10, 2007.

RR (Statistics #2007-20), María M. García, Alison Gajcowski, and Andrew Jennings, “Selective Editing Strategies forthe U.S. Census Bureau Foreign Trade Data”, December 10, 2007.

RR (Survey Methodology #2007-38), Laurie Schwede, “A New Focus: Studying Linkages Among Household Structure,Race Ethnicity, and Geographical Levels, with Implications for Census Coverage,” November 5, 2007.

RR (Survey Methodology #2007-39), Elizabeth Nichols and Jennifer Hunter Childs, “Respondent DebriefingsConducted by Experts: A New Qualitative Methodology for Questionnaire Evaluation,” October 3, 2007.

RR (Survey Methodology #2007-40), Joanne Pascale, Marc I. Roemer, and Dean Michael Resnick, "MedicaidUnderreporting in the CPS: Results from a Record Check Study," December 27, 2007

RR (Survey Methodology #2007-41), Anna Y. Chan and Jeffrey C. Moore, "Report on the New "Type 2 PeopleQuestions: A Pretesting Study for the Re-Engineered SIPP," December 18, 2007

3.5 STATISTICAL RESEARCH DIVISION STUDIES(http://www.census.gov/srd/www/byyear.html)

SS (Computing #2007-1), Bor Chung Chen, “CANCEIS Experiments of Edit and Imputation with 2006 Census TestData,” December 10, 2007.

SS (Survey Methodology #2007-27), Patricia Goerman, “Census Bilingual Questionnaire Research Final Round 2Report”, October 21, 2007

SS (Survey Methodology #2007-28), Theresa DeMaio, Jennifer Beck, and Dawn Norris, “Report of Cognitive Researchto Develop the 2008 NCVS Identity Theft Supplement,” December 3, 2007.

3.6 OTHER REPORTS

Schwede, L., Carter III, G., and Jocuns, A. “Results of Cognitive Testing of the Other Living Quarters ValidationQuestionnaire (OLQVQ),” September 29, 2006.

DeMaio, T., Beck, J., and Norris, D. “Report of Cognitive Research to Develop the 2008 NCVS Identity TheftSupplement,” November 5, 2007.

32

4. TALKS AND PRESENTATIONS

Workshop on Ensuring Access and Confidentiality Protection for Highly Sensitive Data, Institute for Social Research,University of Michigan, October 3, 2008.

• Laura Zayatz, “Disclosure: Deciding What Gets Out.”

Washington Statistical Society (WSS), Washington, DC, October 16, 2007.• Paul B. Massell, “Protecting the Confidentiality of Tables by Adding Noise to the Underlying Microdata.”

George Washington University School of Public Health, Graduate Level Methods Class Invited Seminar, Washington,DC, October 22, 2007.

• Laurie Schwede, “How You Can Link Qualitative and Census Data: Using Our Complex Family Types Studyas a Guide.”

Institute of Electrical and Electronic Engineers (IEEE), Saratoga, NY, October 22, 2006.• Erica Olmsted-Hawala, “”Card Sorting, Information Architecture and Usability.”

Federal Committee on Statistical Methodology Research Conference, Arlington, VA, November 5-7, 2007.• Jeremy Funk, “Model Based Disclosure Avoidance for Data on Veterans.”• Paul Massell, “Recent Developments in the Use of Noise for Protecting Magnitude Data Tables: Balancing to

Improve Data Quality and Rounding that Preserves Protection.”• Tucker S. McElroy, "Compatible Trends for ACS Data."• Brian C. Monsell, "The X-13A-S Seasonal Adjustment Program."• Yuling Pan, Ashley Landreth, Alisú Schoua-Glusberg, Marjorie Hinsdale, and Hyunjoo Park, “Effects of

Language and Culture on Interpretation of Translated Confidentiality” and “Mandatory” Survey Messages.”

Conference on the Event History Calendar Method, Washington, DC, December 5-6, 2007.• Jason Fields and Jeff Moore, “Description of Plans for a SIPP Calendar Validation Study: Study Design and

Analysis.”• Jeff Moore, “Seam Bias in the 2004 SIPP Panel: Much Improved, but Much Bias Still Remains.”• Joanne Pascale and Alice McGee, “A Multi-Method Evaluation of the Use of an Event History Calendar.”

Joint UNECE Eurostat Work Session on Statistical Data Confidentiality, Manchester, UK, December 17-19, 2008.• Laura Zayatz, “New Implementations of Noise for Tabular Magnitude Data, Synthetic Tabular Frequency and

Microdata, and a Remote Microdata Analysis System.”

33

5. STATISTICAL RESEARCH DIVISION SEMINAR SERIES

Seminar Series Team: Aref Dajani, Richard Griffin (DSSD), Paul Massell, Barbara Palumbo,Laurie Schwede, Katherine Thompson (ADEP)

Nanda Srinivasan, Cambridge Systematics, Inc., "Partial Data Synthesis for Small Area Tabulations," October 2, 2007.

George Andrews, The Pennsylvania State University, "Introduction to Number Theory and Modelling the AverageRunning Time of Computer Programs," November 8, 2007.

Guillermo Mendez, Arizona State University, "Tree-Based Methods to Model Clustered Data," November 8, 2007.

George Carter III, SRD, U.S. Census Bureau, "Measurement of Housing Quality and Neighborhood Quality in theAmerican Community Survey (ACS) and American Housing Survey (AHS)," November 15, 2007.

Frances Morphy, Fellow, Centre for Aboriginal Economic Policy Research, The Australian National University, "TheIndigenous Enumeration Strategy in the Australian National Census: A Critical Appraisal," November 27, 2007.

Bor-Chung Chen, SRD, U.S. Census Bureau, "Stochastic Simulation of Field Operations in Surveys," November 29,2007.

Joseph Kang, The Pennsylvania State University, "Causal Inference by Semiparametric Imputation," December 4, 2007.

Song X. Chen and Cheng Yong Tang, Iowa State University, "Local Post-Stratification and Estimation in Dual SystemAccuracy and Coverage Evaluation for US Census," December 13, 2007.

34

6. PERSONNEL ITEMS

6.1 HONORS/AWARDS/SPECIAL RECOGNITION

6.2 SIGNIFICANT SERVICE TO PROFESSION

Jen Beck• Reviewed submissions for American Association for Public Opinion Research (AAPOR).

Pat Cantwell• Associate Editor, Journal of Official Statistics.• Associate Editor, Survey Methodology.

Terry DeMaio• Refereed paper for Public Opinion Quarterly.• Reviewed submissions for AAPOR Conference.

Jeremy Funk• Member, Confidentiality and Data Access Committee (CDAC).

Sam Hawala• Member, Confidentiality and Data Access Committee (CDAC).• Member, National Center for Education Statistics Disclosure Review Board.

Paul Massell• Member, Confidentiality and Data Access Committee (CDAC).• Member, Bureau of Transportation Statistics Disclosure Review Board.

Tucker McElroy• Topic Contributed Session organizer, 2008 Joint Statistical Meetings.• Refereed papers for Journal of Multivariate Analysis and Journal of Time Series Analysis.

Brian Monsell• Webmaster and AMSTAT Online Assistant Editor, Business and Economic Statistics Section, American

Statistical Association.

Mary H. Mulry• Chair of the Survey Research Methods Section of ASA.• Associate Editor, The American Statistician.• Associate Editor, Journal of Official Statistics.

Betty Murphy• Reviewed three submissions for workshops proposed for the Usability Professional Association (UPA) 2008

conference.

Yuling Pan• Advisor, Editorial Advisory Board of the Handbook of Business Discourse.• Member, coordinating committee of Chinese Discourse Research group.• Member, AAPOR Multilingual Interest Group.

Joanne Pascale• Refereed paper for the Journal of Official Statistics.

35

Eric Slud• Associate Editor, Journal of Royal Statistical Society, Series B.• Associate Editor, Lifetime Data Analysis.

Phil Steel• Chair, Confidentiality and Data Access Committee (CDAC).• Member of the new FCSM committee being set up to study issues related to privacy.

Bill Winkler• Program committee for Privacy in Statistical Databases 2008.• Refereed one paper for IEEE Transactions on Knowledge and Data Engineering.• Associate Editor, Journal of Privacy Technology.• Associate Editor, Journal of Privacy and Confidentiality.

Tommy Wright• Associate Editor, The American Statistician.• Associate Editor, The American Journal of Mathematical and Management Sciences.• Member, Department of Statistics Advisory Council, George Mason University.• Member, Department of Mathematics Advisory Board for Masters Program, Georgetown University.• Member, 2009 ISI Session Program Committee, International Association of Survey Statisticians.• Member, Morris Hansen Lecture Committee.

Laura Zayatz• Member, Confidentiality and Data Access Committee (CDAC).• Member, Advisory Board of Journal of Privacy Technology.• Member, American Statistical Association’s Committee on Privacy and Confidentiality• As a Member, UK Census Design and Methodology Advisory Committee, Refereed Report on Proposed

Disclosure Avoidance Techniques to be Used by the Office of National Statistics, United Kingdom, for Census2011

6.3 PERSONNEL NOTES

Irma Hindrayanto (Graduate student at Free University of Amsterdam) started an internship with the Time Series ResearchStaff.

Jun Shao (University of Wisconsin) accepted a Schedule A appointment in our Missing Data Methods Research Group.

Allison Morgan (Ph.D. candidate in information sciences and technology at The Pennsylvania State University) joinedour Human Factors & Usability Research Group as an intern.

STATISTICAL RESEARCH DIVISION FY 2008 FIRST QUARTER REPORT Q1.pdf · FY 2008 FIRST QUARTER REPORT...

Documents

Transcript of STATISTICAL RESEARCH DIVISION FY 2008 FIRST QUARTER REPORT Q1.pdf · FY 2008 FIRST QUARTER REPORT...