1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S....
-
Upload
eleanore-mitchell -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S....
![Page 1: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/1.jpg)
1
Data Quality Standardsat the
U.S. Census Bureau
Pamela D. McGovern and John M. BusheryU.S. Census BureauQuality Program Staff
Washington, DC 20233
European Conference on Quality in Official StatisticsJuly 2008
![Page 2: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/2.jpg)
2
Agenda
• Background• Development Process• Framework and Structure• What Worked Well• Next Steps• Challenges
![Page 3: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/3.jpg)
3
Background
![Page 4: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/4.jpg)
4
Office of Management and Budget (OMB)
• Provides direction on Government-wide budget and management issues
• Issued Information Quality Guidelines (2002)• Issued Standards and Guidelines for
Statistical Surveys (2006)
![Page 5: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/5.jpg)
5
U.S. Census Bureau
Composed of separate program directorates:• Economic• Demographic• Decennial
![Page 6: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/6.jpg)
6
Quality Program
• Initiated in 1998• Purpose
– Advance and maintain the Census Bureau interests in quality of its products and processes
• Key players– Methodology and Standards (M&S) Council– Quality Program Staff
Reference: Landman, et al., (Q2001)
![Page 7: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/7.jpg)
7
Methodology and Standards (M&S) Council
• Steering Committee for the Quality Program• Chiefs of methodology & research divisions
and senior statisticians and methodologists• Ensure sound statistical and methodological
practices
![Page 8: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/8.jpg)
8
Quality Program Staff (QPS)
• Support staff for the M&S Council• Facilitate development and implementation of
standards and guidelines• Evaluate implementation of standards
![Page 9: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/9.jpg)
9
Development Process
![Page 10: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/10.jpg)
10
Original Approach
• Informal process– Team formed based on need– No formal deadlines
• Detailed requirements• Development periods lengthy
– Took from 1 to 4 years
![Page 11: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/11.jpg)
11
New Approach
Structured approach that uses:• Dedicated Staff• Contractor Assistance• Directorate Reviewers• Focus Groups
![Page 12: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/12.jpg)
12
Dedicated Staff (QPS)
Two Mathematical Statisticians and one Survey Statistician
Advantages:• Development times decrease• Cohesion of content improve• Consistency of format improve
![Page 13: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/13.jpg)
13
Contractor Assistance
Advantages:• Provides expertise in conducting focus
groups• Brings outside perspective
Disadvantage: • Lacks familiarity with Census Bureau issues
![Page 14: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/14.jpg)
14
Directorate Reviewers
Focus group participants and program area managers
Advantages:• Allows QPS to obtain directorate input early in
process• Promotes buy-in
Disadvantage:• Risk exists that can lose buy-in
![Page 15: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/15.jpg)
15
Focus GroupsSeven to ten mid-level managers; Two hours in
lengthAdvantages:• Forum for discussing important issues across
directorates• May increase buy-in• Participants more attentive during the review
Disadvantage:• Scheduling difficulties can slow down process
![Page 16: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/16.jpg)
16
Process Steps• QPS sends draft standard to reviewers• QPS compiles comments and produces
discussion issues for focus group• Contractor conducts focus group• QPS revises standard and conducts walk-
throughs with the M&S Council• QPS prepares “final draft” standard and sends to
reviewers along with responses• QPS and quality assurance staff review the
requirements to determine if they are auditable• QPS submits set for final review and approval
![Page 17: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/17.jpg)
17
Framework and Structureof the Standards
![Page 18: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/18.jpg)
18
Framework
• Lists standards and their purpose– Based on the OMB standards – Based on our existing Census Bureau standards
(See our Information Quality website http://www.census.gov/quality/)
• Consists of two types of standards– Process Standards (18)– Supporting Standards (5)
![Page 19: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/19.jpg)
19
Process Standards
Reflect the phases of the life cycle of a data product:
• Planning and Design• Obtaining Data• Processing Data• Analyzing Data• Disseminating Products
![Page 20: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/20.jpg)
20
Supporting Standards
Apply to all phases of the life cycle:• Protecting Confidentiality• Managing Data and Documents• Ensuring Quality• Applying for a Waiver
![Page 21: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/21.jpg)
21
Structure of the Standards
High-level requirements in list form:• State “what” is required; avoid the “how”• Include examples to provide guidance• Provide links to reference documents and to
related standards
![Page 22: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/22.jpg)
22
What Worked Well
![Page 23: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/23.jpg)
23
Accelerated Development
Reduced development times due to:• Early input• Straightforward requirements• Individual walkthroughs with the M&S Council
members
![Page 24: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/24.jpg)
24
Program Area Input
• Interviews with program area experts to get assistance in drafting requirements
• Focus group approach– Structure the discussion based on comments– Overlap participants on related standards– Record the session
![Page 25: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/25.jpg)
25
Focus Group Evaluation
Scale: 1= strongly disagree to 5= strongly agree
Characteristic Rating
The goal of the focus group was clear. 4.2
The participant’s point of view was heard. 4.5
The goal of the focus group was accomplished. 4.0
The length of the focus groups was sufficient. 4.2
The participant’s satisfaction with the outcome. 4.0
![Page 26: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/26.jpg)
26
Next Steps
![Page 27: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/27.jpg)
27
Next Steps
• Review the full set of standards for completeness, consistency, and cohesion
• Submit the standards to the directorates for final review and approval
• Expect completion by September 2009
![Page 28: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/28.jpg)
28
Next Steps
• Implementation of the standards– QPS assists the program directorates– Training
• Evaluations of the implementation– Identify aspects of the standards that need
improvement
![Page 29: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/29.jpg)
29
Challenges
![Page 30: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/30.jpg)
30
Challenges
• Risk of excessive burden– How compliance will affect costs and schedules
• High-level standards– Avoid being too vague– Program areas interpret standards differently
• Verification of compliance– Audit program (Bushery et al., Q2006)
![Page 31: 1 Data Quality Standards at the U.S. Census Bureau Pamela D. McGovern and John M. Bushery U.S. Census Bureau Quality Program Staff Washington, DC 20233.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649ed35503460f94be30d0/html5/thumbnails/31.jpg)
31
Thank you
Contact Information:Pamela D. McGovern4600 Silver Hill RoadU.S. Census Bureau
Washington, DC [email protected]