On Tap: Developments in Statistical Data Editing at Statistics New Zealand Paper by Allyson Seyb,...
-
Upload
carol-edwards -
Category
Documents
-
view
212 -
download
0
Transcript of On Tap: Developments in Statistical Data Editing at Statistics New Zealand Paper by Allyson Seyb,...
On Tap: Developments in Statistical Data Editing at
Statistics New ZealandPaper by Allyson Seyb, Felibel Zabala and Les
Cochran
Presented by Felibel ZabalaSept 2012
Aim of paper
To describe the latest developments in Statistics New Zealand’s economic and household processing platforms
2
Strategic Developments
Statistics 2020 Te Kāpehu Whetū - Recent actions
Reduction in number of tools
Use of Colectica to centralise storage of all information of Statistics NZ’s outputs
Establishment of processes to research and introduce new standard methods and tools
3
Strategic Developments (cont’d)
International collaboration
Regular bilateral and trilateral meetings with various national statistical offices
Involvement in the Statistical Networks on Confidentiality and the Industrialisation of Editing
Continued investigation on the use of SELEKT
Evaluation of SAS2Argus
4
Platforms
Need for new infrastructure to produce statistics that are fit for purpose in a cost and effective way
A platform is a logical cluster of functionality that enables components to be put together to provide a complete end-to-end system
5
System 1
System 2
System 3
System 4
System 5
System 6
System 7
System 8
System 9
System 10
System 1
System 2
System 3
System 4
System 5
etc
System 6
System 7
System 8
System 9
System 10
System 11
System 12
System 13
System 14
System 80
etc
System 60
System 1
System 2
System 3
System 4
System 5
System 6
System 7
System 8
System 9
System 10
etc
System 50
System 1
System 2
System 3
System 4
System 5
System 6
System 7
etc
System 25
Use standard tools
Use for• Surveys and
censuses• Administrative
sourced data• Mixed sources
Use SAS for processing and analysis
6
Five main platforms
Need Develop & Design Build Collect Process Analyse Disseminate
Collection DisseminationMicro-economic
Household
National Accounts
Household platform
Is a second generation platform with the design informed by an evaluation of the interim platform
Processes and in the future analyse three social surveys and their supplements
Uses standard tools to load, code, micro-edit and finalise a unit record dataset
Processes a Blaise-based survey
7
Uses a mix of shared and specific systemsThe Household Platform
Portal
Process Configure
Setup
View
Code
Search
Edit
View Diary
Workflow
SAS
Execute
SaveLoad
Load Diary
File
Save Diary
Select
Admin
StatisticalToolbox
GREGWt
X12
Core EditsCore DVs
Derivations
CANCEIS
Edits
Screens Tasks Rules
Edit Diary
Info
DataMetadata
DataMetadata
Config.
Config.
Classifications & Stds
Metadata
Survey Specific
Shared across Social
Shared across Stats
Other Data
Configure
Paradata
Treat Estimation
Extract Setup
Surv. Inst.
Paradata
Core Questions
Format of micro-level data
9
The Household platform
Survey cycle code
ID Repeat Variable name
Value
HLFS 107 Person1 11 Age 15
HLFS 107 Person1 11 Sex M
HLFS 107 Person2 11 Sex F
HLFS 107 Household1 11 Household composition
Multi-person
Micro-economic Platform
Previously referred to as BESt platform
Processes and analyses economic surveys and administrative data collections
Also has elements of ‘Develop and Design’, ‘Build’, ‘Collect’ (from general Business Process Model)
Has a user configurable workflow
Allows incremental statistical maintenance and uses standard tools for common processes
10
Uses a mix of shared and specific systemsThe Micro-economic platform
Business Intelligence
View
Configure process
Metadata
Manual edit (generic)
Manual edit
SAS
Load metadataLoad data
Workflow
StatisticalToolbox X12
EditsDerivations
Configuration
Banff
Screens Tasks Rules
Reporting
Info
DataMetadata
DataMetadata
Configuration
Configuration
Classifications & Stds
Metadata
Survey Specific
Shared across Micro Economics
Shared across Stats
Other Data
Process trace
Run process Extract
Trace Reporting
Process trace
System Admin View Respondent
Build cubesGeneric E&I
Sample selectWrite-back
ImputationCode
EstimationTreat
Allocate
Challenges and lessons learnt
Balancing generic and specific needs
Determining process elements suitable to be implemented as common services
Moving from a process culture to a more constructive innovative culture
Key enabler of the culture change - the adoption of an agile project management approach for IT development projects
12
Moving forward
Review of Statistics NZ’s generic Business Process Model
Implementation of a framework to measure and report on the benefits achieved with recent and on-going developments
Transformation of the organisation’s data collection processes
13