Designing Interactive Edits for U.S. Electronic Economic Surveys

and Censuses: Issues and Guidelines

Elizabeth M. Nichols, Elizabeth D. Murphy, Amy E. Anderson, Diane K. Willimack and Richard S. Sigman

U.S. Census Bureau

Presented at the UNECE Statistical Data Editing Work Session, May 16-18, 2005, Ottawa, Canada

Outline

Background

Edit design questions

Usability testing

Guidelines

Examples of Edits

Future Directions

Background

Traditional post-collection editing– Some conditions that trigger edit checks:

• Missing value(s)• Inconsistent values• Questionable value(s)• Technically impossible value(s)

Catch edit failures earlier:Move editing to the electronic form

How should we design the edits?

1. When should the electronic questionnaire run the edits?

2. How should the respondent be notified of an edit failure?

3. Where should the electronic questionnaire display the edits?

4. How should we phrase the edit message?5. How should the edits look in terms of font,

color, brightness?

How we answer those questions

Method: Usability testing

at establishments and in the usability lab – Think-aloud procedure– Observation of user behavior– Self-reported ratings of satisfaction

Results: Recommendations for design changes

Preliminary guidelines

1. When should the electronic questionnaire run the edits?

Guidelines: – Perform edit checks immediately.– For inter-item edits, perform those after all the

items in the edit have been entered.– Check for missing data at the end.– Allow edits to be run iteratively

Immediate notification2002 Economic Census

Immediate notification2002 Economic Census

Running the edit before all items are complete

2001 Quarterly Financial Report

Running the edit before all items are complete

2001 Quarterly Financial Report

Preliminary guidelines

2. How should the respondent be notified of an edit failure?

Guidelines: – Give the respondent as much control as

possible. • No unsolicited pop-up messages.

– Avoid violating user expectations for single-purpose functions.

Respondent control: R chooses to run the edits

2004-05 Teacher Follow-up Survey

Unsolicited popup message1998 Field Test Prototype for the Library

Media Center Survey

Preliminary guidelines

3. Where should the electronic questionnaire display the edits?

Guidelines: • Clearly identify which edit failure goes with

which item• Provide easy navigating between an edit-

failure list and the associated items.

Easy navigating between editsSurvey of Industrial Research and

Development (2001)

Easy navigating between edits2004-05 Teacher Follow-up Survey

Clear association between edit and item

2004-05 Teacher Follow-up Survey

Clear association between edit and item

Manufacturer’s Shipments, Inventories, & Orders Survey

Preliminary guidelines

4. How should we phrase the edit message?

Guidelines: • Include the item number, a description of the

problem, and an action to take.• Cognitively test message before fielding.

Examples of poorly worded edit messages

“Check for a typing mistake”

“Enter a number between 0 and 200”

Preliminary Guidelines

5. How should the edits look in terms of font, color, brightness?

Guidelines: • Use standard icons as expected by the

business respondent

What are standard icons for edits?

Future Direction

- Challenge to use guidelines

- Need for experimental results

- Update and add to guidelines

- 508 accessibility compliance

- Edit design for a batch input

Questions?

Contact Elizabeth Nichols

elizabeth.may.nichols@census.gov

Paper:

http://www.census.gov/srd/papers/pdf/rsm2

005-03.pdf