Post on 06-Jan-2016
description
1
Data Quality Standardsat the
U.S. Census Bureau
Pamela D. McGovern and John M. BusheryU.S. Census BureauQuality Program Staff
Washington, DC 20233
European Conference on Quality in Official StatisticsJuly 2008
2
Agenda
• Background• Development Process• Framework and Structure• What Worked Well• Next Steps• Challenges
3
Background
4
Office of Management and Budget (OMB)
• Provides direction on Government-wide budget and management issues
• Issued Information Quality Guidelines (2002)• Issued Standards and Guidelines for
Statistical Surveys (2006)
5
U.S. Census Bureau
Composed of separate program directorates:• Economic• Demographic• Decennial
6
Quality Program
• Initiated in 1998• Purpose
– Advance and maintain the Census Bureau interests in quality of its products and processes
• Key players– Methodology and Standards (M&S) Council– Quality Program Staff
Reference: Landman, et al., (Q2001)
7
Methodology and Standards (M&S) Council
• Steering Committee for the Quality Program• Chiefs of methodology & research divisions
and senior statisticians and methodologists• Ensure sound statistical and methodological
practices
8
Quality Program Staff (QPS)
• Support staff for the M&S Council• Facilitate development and implementation of
standards and guidelines• Evaluate implementation of standards
9
Development Process
10
Original Approach
• Informal process– Team formed based on need– No formal deadlines
• Detailed requirements• Development periods lengthy
– Took from 1 to 4 years
11
New Approach
Structured approach that uses:• Dedicated Staff• Contractor Assistance• Directorate Reviewers• Focus Groups
12
Dedicated Staff (QPS)
Two Mathematical Statisticians and one Survey Statistician
Advantages:• Development times decrease• Cohesion of content improve• Consistency of format improve
13
Contractor Assistance
Advantages:• Provides expertise in conducting focus
groups• Brings outside perspective
Disadvantage: • Lacks familiarity with Census Bureau issues
14
Directorate Reviewers
Focus group participants and program area managers
Advantages:• Allows QPS to obtain directorate input early in
process• Promotes buy-in
Disadvantage:• Risk exists that can lose buy-in
15
Focus GroupsSeven to ten mid-level managers; Two hours in
lengthAdvantages:• Forum for discussing important issues across
directorates• May increase buy-in• Participants more attentive during the review
Disadvantage:• Scheduling difficulties can slow down process
16
Process Steps• QPS sends draft standard to reviewers• QPS compiles comments and produces
discussion issues for focus group• Contractor conducts focus group• QPS revises standard and conducts walk-
throughs with the M&S Council• QPS prepares “final draft” standard and sends to
reviewers along with responses• QPS and quality assurance staff review the
requirements to determine if they are auditable• QPS submits set for final review and approval
17
Framework and Structureof the Standards
18
Framework
• Lists standards and their purpose– Based on the OMB standards – Based on our existing Census Bureau standards
(See our Information Quality website http://www.census.gov/quality/)
• Consists of two types of standards– Process Standards (18)– Supporting Standards (5)
19
Process Standards
Reflect the phases of the life cycle of a data product:
• Planning and Design• Obtaining Data• Processing Data• Analyzing Data• Disseminating Products
20
Supporting Standards
Apply to all phases of the life cycle:• Protecting Confidentiality• Managing Data and Documents• Ensuring Quality• Applying for a Waiver
21
Structure of the Standards
High-level requirements in list form:• State “what” is required; avoid the “how”• Include examples to provide guidance• Provide links to reference documents and to
related standards
22
What Worked Well
23
Accelerated Development
Reduced development times due to:• Early input• Straightforward requirements• Individual walkthroughs with the M&S Council
members
24
Program Area Input
• Interviews with program area experts to get assistance in drafting requirements
• Focus group approach– Structure the discussion based on comments– Overlap participants on related standards– Record the session
25
Focus Group Evaluation
Scale: 1= strongly disagree to 5= strongly agree
Characteristic Rating
The goal of the focus group was clear. 4.2
The participant’s point of view was heard. 4.5
The goal of the focus group was accomplished. 4.0
The length of the focus groups was sufficient. 4.2
The participant’s satisfaction with the outcome. 4.0
26
Next Steps
27
Next Steps
• Review the full set of standards for completeness, consistency, and cohesion
• Submit the standards to the directorates for final review and approval
• Expect completion by September 2009
28
Next Steps
• Implementation of the standards– QPS assists the program directorates– Training
• Evaluations of the implementation– Identify aspects of the standards that need
improvement
29
Challenges
30
Challenges
• Risk of excessive burden– How compliance will affect costs and schedules
• High-level standards– Avoid being too vague– Program areas interpret standards differently
• Verification of compliance– Audit program (Bushery et al., Q2006)
31
Thank you
Contact Information:Pamela D. McGovern4600 Silver Hill RoadU.S. Census Bureau
Washington, DC 20233pamela.d.mcgovern@census.gov