The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew...
Transcript of The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew...
![Page 1: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/1.jpg)
The Labour Force Survey – Data Quality Issues
Matthew Steel
![Page 2: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/2.jpg)
• Response rates • Factors • Improving response • Research • Refusal follow-up study • Future work
Data Quality Issues
![Page 3: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/3.jpg)
Response rates
![Page 4: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/4.jpg)
• Falling contact rates • Increasing interview length • Respondents less willing • External shocks • Reduced budgets • Change in survey design
Factors affecting response
![Page 5: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/5.jpg)
• Interviewer training • Incentives • Questionnaire reviews • Varying calling patterns • Survey materials • IT infrastructure
Improving response
![Page 6: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/6.jpg)
• Analysis of attrition bias • Analysis on key estimates • Analysis of paradata • Refusal follow-up study
Research
![Page 7: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/7.jpg)
Refusal follow-up study - Introduction Study:
Eurostat funded project - result of one of the recommendations of the Quality Task Force Review of the LFS, to collect information on the characteristics of non-respondents in order to assess and adjust for non-response bias and to improve fieldwork strategies.
Issue
• Declining response rates:
- August 2011 64% response rate at wave 1 - 10 years ago 75% in wave 1
• Key LFS estimates - sensitive to ethnicity, household type and nationality But – these are not included in the calibration totals
![Page 8: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/8.jpg)
Refusal follow-up study - Introduction
• Therefore, if achieved sample proportions of… - ethnic groups - household groups - national groups… …are not representative of those proportions in… - allocated sample - population… …then the estimates derived from the LFS may be biased.
• RFU survey aimed to:
- identify whether key characteristics of responders and non-responders differentiate - identify if RFU group reasons for refusal differ to general refusing group
![Page 9: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/9.jpg)
Refusal follow-up study – Data Collection
Mainstage RFU exercise: - Main and boost LFS survey - Wave 1 and outright/HQ refusals only - April to June 2011 LFS questionnaire
Data collected in 4 ways… 1. Face-to-face 2. Telephone Unit (wave 1) 3. Survey Enquiry Line (SEL) 4. Self completion
♦ Face-to-face (left at doorstep) ♦ SEL (posted to respondent)
Questionnaire process: - Interviewer/ SEL attempt achieve LFS interview - Not possible, offer RFU as last resort
![Page 10: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/10.jpg)
RFU Letter RFU Questionnaire
![Page 11: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/11.jpg)
Refusal follow-up study – Response Rate
• Total AJ11 outright refusals and HQ refusals = 4,137
• LFS RFU respondents = 787
• LFS RFU response rate = 19 %
• ONS received responses from each of the 4 methods • No interviewer estimate data was allowed, for:
- Ethical reasons - Accuracy/ validity reasons
![Page 12: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/12.jpg)
Refusal follow-up study – Analysis
• LFS collected data on all members of household • LFS RFU collected data:
- Fully on one member of household - Partially on all other members
• Therefore, comparisons between LFS and LFS RFU enabled by… - Selection of one random adult from each LFS responding household - Selection of full data respondent from LFS RFU - Created analysis dataset containing both
![Page 13: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/13.jpg)
Percentage comparison of reasons for RFU and LFS
Refusal follow-up study – Reasons for refusal
![Page 14: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/14.jpg)
Refusal follow-up study - Characteristic comparisons of LFS and RFU respondents
Age
![Page 15: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/15.jpg)
Refusal follow-up study - Characteristic comparisons of LFS and RFU respondents
Sex
![Page 16: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/16.jpg)
Refusal follow-up study - Characteristic comparisons of LFS and RFU respondents
Nationality
![Page 17: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/17.jpg)
Refusal follow-up study - Characteristic comparisons of LFS and RFU respondents
Ethnicity
![Page 18: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/18.jpg)
Refusal follow-up study - Characteristic comparisons of LFS and RFU respondents
Household type
![Page 19: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/19.jpg)
Refusal follow-up study - Characteristic comparisons of LFS and RFU respondents
Workless households
![Page 20: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/20.jpg)
Refusal follow-up study - bias
• Aim? - to determine whether bias exists in LFS data
• How? - Created 1 weight for LFS responders and RFU combined dataset, and - Created 1 weight for LFS responders dataset only (excluding RFU cases)
• Analysis? - Investigated whether proportions in data differed when RFU group included
![Page 21: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/21.jpg)
This indicates: - potential bias in the current estimates from a sample that included these ‘refusers’ in the LFS - or that the current LFS estimates are themselves potentially biased.
Dataset Indicator Status LFSRA only Unemployed No bias
LFSRA plus LFSRFU Employed No bias
LFSRA plus LFSRFU Unemployed Bias
LFSRA plus LFSRFU Inactive Bias
LFSRA only Employed Bias
LFSRA only Inactive Bias
Refusal follow-up study – bias investigations
![Page 22: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/22.jpg)
Refusal follow-up study - Caveats
• LFSRFU data collected from one adult per residence
• LFSRFU sample may not be representative of all ‘refusers’ to LFS
• Not possible to produce standard errors in comparing the two weighted datasets.
![Page 23: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/23.jpg)
• Continue to monitor • Census non-response link study • Regularly review length of survey • Data collection modes • Non-response weights
Future work
![Page 24: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/24.jpg)
Contact details LFS Research Team: [email protected] [email protected] [email protected] Social Survey Data Service: [email protected]
![Page 25: The Labour Force Survey - Data Quality IssuesThe Labour Force Survey – Data Quality Issues Matthew Steel •Response rates •Factors •Improving response •Research •Refusal](https://reader030.fdocuments.in/reader030/viewer/2022040521/5e7adee5435f7224e327546e/html5/thumbnails/25.jpg)
Questions?