2011 Census: Content Workshop

2011 Census: Content Workshop

Office for National Statistics

February 2008

Welcome

Glen Watson

Census

Purpose

• Provide update on funding for fourth page of individual questions

• Allow key users another chance to comment on ONS priorities for topics

• Focus on the new topics/questions that ONS proposes for inclusion

• NB: Presentation focuses on Census in England & Wales

Agenda

• 10.10 Introduction• 10.40 Question testing • 11.00 Second residences and visitors• 11.40 Coffee• 11.55 Migration• 12.35 National identity• 13.00 Lunch• 13.45 Income• 14.15 UK views on content• 14.30 Prioritisation of topics• 16.00 Close

copei

Introduction

Peter Benton

Census

Census provides statistics on:

• Population units: – people and housing &– key demographics (age, sex, marital status, ethnicity)

• Population structures: – households, families

• More detailed characteristics :– eg religion, labour market status, industry, qualifications,

health/disability; etc

• Key requirement to ‘Get the Count Right’– 2011 Census aim : “maximise overall response rates

and minimise non-response in specific areas and among particular population subgroups”

Key lessons from 2001 Census

• Need to gather information on more than just usual residents

– Include visitors

• Need to understand coverage– Comparisons census vs admin data

• Need to improve enumeration of Communal Establishments

Mobile population & complex lifestyles

• More people at more than one address– Weekday residences for work– Holiday / weekend homes– Children of divorced parents

• International migration– More ‘short term visitors to UK’ – resident or not?

• Plus familiar issues of students, armed

forces, prisons, hospitals, hotels, hostels• Risk of undercount, overcount, or wrong location• Need to count right people, in right place

– and be able to demonstrate this

Work done so far – 2005 consultation

• Formal 3 month consultation (May – Aug)• Publication March 2006• Over 2000 responses from 500 users• Scoring of user requirements• Evaluation of ONS considerations & other reasons for collection

• Allocation of topics to one of three categories

Evaluation of topics

Category 1 Topics likely to be included

Question testing to refine questions

Category 2 Topics under consideration

Investigation of alternative sources

Question design & testing

Category 3 Topics not to be included

Topics will not be considered further

Criteria

• Strength of user need– Resource allocation– Policy use– Research

• Small geographies• Alternative sources• Multi-variate analysis• UK comparability• Continuity with 2001

• ONS considerations– Data quality– Public acceptability– Respondent burden– Operational concerns

(eg costs, space)

• Other reasons for collection

– Operational purposes– Use for coding

Key requirement: Robust estimate of population count

Summary of user requirements from 2005

• Most 2001 topics• New topics including:

- income- language- second residences- national identity

• Little expressed requirement for number of employees & hours worked

• More than 3 pages of questions!• Difficult trade-offs to be made

Work done so far – topic groups

• Eight topic groups• Topic experts from within ONS (except DIUS for

qualifications)• UK-wide membership• ONS Harmonisation• Methodology• Census

Topic-specific consultation

• Further specific consultation with users to better understand needs/priorities:

– Ethnicity, identity, language & religion– Population definitions – PDWG– Migration– Demographics– Housing– Income – NS-SEC– Disability

Other consultation and question testing

Cognitive question testing 2005 - 2008

Census Advisory Groups May & Dec 2006, Nov 2007

Statistical Heads of Profession Sept 2006, June & Nov 2007

Open meetings (London, Cardiff, Sheffield)

March 2007

Postal Test April 2007

Focus groups April/May 2007

2007 Census Test May 2007

Census Test Evaluation Survey May 2007

Omnibus Survey testing of new questions Sept - Dec 2007

Fourth page of individual questions

• Would allow inclusion of more topics • Research carried out on effects of longer

questionnaires• Split sample postal test 2007 – 3 vs 4 pages • Comparison with other censuses around world• Cost analysis of fourth page

Changes since 2005 evaluation

• Increased requirement for information on migration– Inter-Departmental Taskforce on Migration 2006– Migration consultation

• Planned Disability Survey• Question testing carried out on category 2 topics• 2007 Test and Postal Test• Priorities for content of 3 & 4 pages discussed:

– Open meetings March 2007– Advisory Groups– Heads of Profession

Questions/Comments

• Any questions or comments?

Question testing overview

Ruth Wallis

Data Collection Methodology

Why pre-test questionnaires?

• Find out if questionnaires meet survey objectives

• Gain an estimate of reliability and validity of answers

• Identify sources of potential non-sampling error

What is cognitive interviewing?

A face-to-face interview to test questions or a questionnaire, which aims:

• to understand how the respondent fulfils the task of answering questions

• to detect any actions or understandings that are not what the designer intended

Cognitive steps in answering a question

• Comprehend• Retrieve• Form judgement• Edit answer

Identifies problems

• Uncovering ‘hidden error’• Understanding the question• Remembering or recalling the information• Selecting a response• Reactions to sensitive questions

Features of cognitive testing

• Observation• Think aloud• Concurrent probes• Retrospective probes• Paraphrasing• Vignettes• Card sorting tasks

Cognitive testing programme

• Phase 1: question development & testing for 2007 Census Test

• Oct 2004 to Apr 2006– The questions tested were developed using:

• 2001 Population Bases and Definitions questions

• ONS’ harmonised questions • new user requirements

• A range of respondents were interviewed to ensure all criteria of interest were covered. In total: – 56 interviews – 2 focus groups

Cognitive testing programme

• Phase 2: development & testing for Rehearsal• Jan 2007 to Aug 2008

– New format testing questions in context of the whole questionnaire

• Five waves of cognitive testing:– Wave 1: 23 respondents (Nov 2006 - Jan 2007) – Wave 2: 69 respondents (Apr & May 2007) – Wave 3: 77 respondents (Jul – Sept 2007)– Wave 4: 27 respondents (Oct & Nov 2007) – Wave 5 is under way

• Whole-of-questionnaire testing (Apr – Jun 2008)

Ethnicity focus groups

• Aims:– explore public opinions surrounding

classifications of ethnic groups– unpack attitudes towards colour labels, in

particular ‘Black’ – understandings of the term ‘ethnicity’– inform development of ethnicity question

• 12 focus groups, 109 participants– Caribbean, African, Any other black

background, mixed background– London, Birmingham, Manchester, April & May

2007• Report due soon

Questionnaire design principles

• Navigation & layout– In West, people start reading at top left, after

that, we guide them with elements of layout– Essential to have a clear path for eye to follow

through the questionnaire– Clear numbering– Consistent even columns & vertical layout– Text set left and ragged right to provide patterns

for eye to grab hold of

Questionnaire design principles

• Minimise clutter– Text, symbols, graphics should be kept to a

minimum, as every mark on the page must be processed by respondent

– Maximise white space to make it easier to read and more appealing

– Keep respondent relaxed

140,000 households 140,000 households

Second Residences & Visitors

Peter Benton

Census – Design Authority

Background

• Information on second residences has not been collected in any previous census

• Information on visitors was collected in the 1991 Census, but not in a structured way in 2001

• Enumeration base for 2011 is ‘usual residents and visitors’ – some information will be collected from everyone at their Census night address.

Second residences consultation

• Over 80 responses received from a range of users– including DCLG, DfT, Welsh Assembly Government, Local

Authorities, Demographics User Group, Market Research Society

• Uses include: – inform service provision

• plan health, waste and public transport services

– resource allocation• provide information on where people are likely to use resources

– understand the housing market • dynamics of housing market and profiling housing demand

– inform housing affordability and development policies• impact of second residences on availability of affordable housing

• Category 2 – more work required to develop suitable questions and assess respondent burden

Second residences in coverage assessment

• Duplicate returns from different locations:– e.g student counted at both term-time and parents’

address, people with second residences for the working week, children of divorced parents

– matching process to search for duplicates– use second residences information to help matching

process

Second residences in processing of outputs

• Census estimate of the population using the Mid-Year Population Estimate base:

– definitional differences between where people are counted

– use second residences information to estimate where people spend the majority of their time and adjust for difference

• Alternative population bases:– e.g. weekday population, out of term population

Second residences in 2007 postal test

• 2.5% of respondents said they had a second residence• A further 1% of respondents said they had a second

residence outside the UK

• Of those who said they had a second residence:– 87% entered the address

• 69% of those entered the full postcode

• 11% of postcodes were half completed

• 20% of postcodes were left blank

• Of those that entered an address, the highest frequency of location was London (13%), followed by West Sussex (9%)

Second residence question

A number of versions of this question have been tested.

An additional question on length of time spent at address was tested, but respondents found it difficult to answer.

The current question has proved successful in testing.

Visitors consultation

• Over 40 responses received from a range of users– including DCLG, DfT, DfES, Welsh Assembly Government, Local

Authorities

• Uses include: – inform service provision

• measure the total pressure on services such as transport and health

– resource allocation• essential for DCLG’s daytime net inflow and foreign visitors indicators

• DCLG still use visitor information from 1991 Census as not collected in 2001

• Category 1

Visitors in communal establishments

• Visitor information will not be collected from visitors in communal establishments

– too complex– could jeopardise the count of usual residents

• This means that there will not be a full visitor count– only visitors in households– no persons present output base

• Primary purpose of collecting visitor information is to get the best usual residents estimate

Visitor information for Quality Assurance

• A sample of visitors will be matched back to their usual residence

– check whether they were missed where they usually live

• In CCS areas, will also be matched to the CCS in the households they were visiting

– check whether they were mis-classified

• This will provide a source of data to assess the quality of, and potentially adjust, the census population estimates

– important when defending the census results

• In non-CCS areas, the mis-classification rate will be used in the QA

• A full match may be done in certain LAs if the QA suggests concerns

Visitors in the 2001 Census

• From a sample of 7 Enumeration Areas:– 9.5% of people recorded as visitors were actually usual

residents– 1.5% were of no fixed abode and should have been

recorded as usual residents

– 17% of visitors were overseas visitors– 67% of visitors were from the UK

• 20% of these UK visitors were missed at their usual residence

– 4.5% of visitors did not have an address recorded

• It is estimated there will be around 2.1 million visitors on Census night in 2011

Visitor questions

In testing, respondents who had visitors generally had a good understanding of who should be included in this question.

When the visitors questions were located in the household section of the form, many respondents filled in the questions for usual residents.

Locating the questions on the back page of the form has proved more successful.

Questions/Comments


Migration

Chris SmithJonathan Smith, Pamela Spicer, Richard Pereira

ONS Centre for Demography

ONS External Consultation; Preliminary Work

• May 2005 ONS published consultation document ‘The 2011 Census: Initial view on content for England and Wales’

• Following on, ONS published an information paper ‘The 2011 Census: Assessment of initial user requirements on content for England and Wales’

Key Migration Requirements Noted

• Address One Year Ago

• Country of Birth

• Clear need for additional migration data

Specific Migration Consultation

• In October 2006 a consultation document on external user needs sent to data users

• 61 responses were received from:

– 12 Central Government Departments

– 34 Local Government

– 10 Academics

– 5 other groups/organisations

October 2006 Consultation: Key Migration Issues Noted

• Address One Year Ago


• Citizenship

• Date of Arrival

Oct 2006 Consultation: Issues

• Address One Year Ago – Identifies internal migrants and international in-migrants

in previous year


– Continuity with previous censuses essential

– To profile the large numbers of migrants from EU ‘Accession’ countries

– House of Commons Library said CoB a ‘poor proxy’ for Citizenship; increased ‘blurring’ between CoB & Citizenship.


• Citizenship

• Has never been included in UK censuses

• Is most easily defined by passport eligibility

• The National Statistics Quality Review (NSQR) on International Migration Statistics recommended inclusion of a Citizenship question in future censuses

Citizenship: Potential Stated Uses

• Defines right of movement, eligibility to vote and access to employment/welfare benefits

• Collection would allow citizens of other EU countries to be measured at local level

• UK is one of only three OECD countries not collecting Census information on citizenship


• Date of Arrival• Date of Arrival for those born overseas not included in past

UK censuses

• NSQR on International Migration Statistics (2003) recommended a question on year of (last) entry;

• Year of entry is included in the LFS, but data at local level are limited.

Oct 2006 Consultation: Issues (cont.)

• Current CoB question makes it impossible to differentiate migrants living in UK for many years from recent arrivals

• Date of Arrival for those born overseas would allow reconciliation of stock/flow estimates of migration

• For those who arrived in UK in previous year, month of arrival & intended length of stay should be collected

• Inter-Departmental Taskforce on Migration highlighted importance of accurate data on short-term migrants for local areas

Task Force on Migration Statistics

• Set up by National Statistician to improve migration estimates

• Necessary due to current importance of migration in population change

• Key recommendations in December 2006 included:– More information about migrants in the country– Use linkage for better information– Provide better indicators of migrant numbers– More coherent reporting

Census 2011: ONSCD Requirements;

• Key Priority is ‘capture’ of the ‘12 month+’ usually resident population

• Enumeration Base of Usual Residents Plus Visitors agreed in February 2005

• In the last two years much public discussion of short-term migration from ‘Accession’ countries

Census 2011: ONSCD Requirements (cont.)

• Short-term migrants not uniformly distributed: many LAs concerned about insufficient data;

• ONSCD concerned to:– meet more recent requirements for information

on short-term migration– ensure Census provides an appropriate base

for mid-year estimation/projection

Specific ONSCD Requirements

Requirements for Flexible Base Estimation

Residence Bases:

– Usually resident population length of stay 12 months+

– Short-term migrants temporarily resident in the UK intending to stay for:

1-3 months 3-12 months 6-12 months

ONSCD: Required Residence Bases

• Priority is headcount of the usually resident population (i.e. those people intending to be resident for 12 months+)

• Headcount estimates required of short-term migrants from the Census

ONSCD: Required Residence Bases (cont.)

• Year/month of arrival and address 12 months ago only provide information on length of stay up to census day

• Intended total length of stay would allow distinctions to be drawn

• Intended length of stay questions are currently being developed

• Implication = all international visitors > 1 month complete all individual questions

Migration – date of entry

This question was based upon the question used in the Labour Force Survey (LFS).

Generally respondents were able to answer this question well and many were quite certain that the date written was correct.

Migration - citizenship

This question was based upon the question used in the General Household Survey (GHS).

Respondents during testing showed a general understanding of the concept of passport entitlement.

Migration – intention to stay

Respondents certain of their circumstances or visa entitlements generally found this question easy to answer

Migration – intention to stay

This question has since been revised during testing to take into account the 1 month usual residence rule; it is therefore still being tested.

National Identity

Hamish MacPherson

Ethnicity, Identity & Inequalities

Background

• The national identity question has never been asked in a census and was first introduced as part of the 2001 Labour Force Survey (LFS)

Numerous groups expressed dissatisfaction with the ethnic group question in the 2001 Census as it did not provide a tick-box for respondents to identify themselves as ‘Welsh’, ‘Scottish’ or ‘English’ but only allowed for ‘British’

As a result of campaigning, ONS have given a public commitment that National Identity will be asked in the 2011 Census

Salience of sub-British national identities varies between places of residence

Majority of most ethnic minority groups identify with one or more GB national identity

Proportion who consider their identity to be British, English, Scottish or Welsh: by ethnic group, 2004, GB

Ethnic Group, National Identity, Religion and Language Consultation

• 2011 Census Stakeholders Consultation 2006/07: Ethnic Group, National Identity, Religion and Language (EILR), England and Wales

• Provided a detailed picture of user needs to help decide which questions should be asked in the 2011 Census.

• 606 responses to the consultation.– central and devolved government– expert, community and special interest groups– Local and regional government– local Service providers

User need

Majority of respondents to the 2006/07 Census Stakeholders Consultation stated they needed a national identity question:

• 58 per cent of respondents in England and Wales

• 75 per cent of respondents in Wales

Meets user requirements by:

• Allowing previously unavailable responses• Improving public and political acceptance of the

questionnaire• Encouraging response rates and improving quality

of information from the ethnic group question• Providing a better understanding of local

populations and communities• Providing a measure of community cohesion• Measuring how people from different areas of the

UK perceive their national identity

Users require the outputs to effect a variety of outcomes

• Grant allocation by central government• Local resource allocation and service provision• Meeting statutory obligations under legislation on

race relations and equal opportunities• Informing policy development and monitoring• Enabling monitoring and regulatory bodies to

function effectively• Product marketing and delivery for private

organisations

Cognitive testing

• Tendency to interpret ‘national identity’ in terms of legal status, often referring to passports and nationality

• Others interpreted “what do you consider your national identity to be” more subjectively referring to how they feel, language, birthplace, residence

• Some overlap with ethnicity in how people understand concept

Combining national identity and ethnicity would not work in England and Wales

A single ethnic group question which included concepts of national identity would not work in England and Wales:

• Operational constraints• Methodology• Clarity• Acceptability• Analysis

National Identity – version 1

• Tested in cognitive testing wave 1-3 question.

• Based on LFS question

• Some respondents do not appear to notice the instruction ‘Tick all boxes that apply’

National Identity – wave 3 conclusions

• Ideas and concepts of national identity centre largely around legal status and country of birth, with some respondents expanding this idea to include affiliation and culture also. – “Proud of being Irish, where you’re born and raised.”– “What I perceive to be my nation, what I affiliate myself

with.”• Evidence suggests respondents are reading all answer

options and are more likely to tick a specific nation i.e. English, Scottish, Welsh or Northern Irish instead of British despite explicitly expressing their ‘Britishness’. – “I do often consider writing British but because the

option for Welsh was there it was easy because I would consider myself Welsh before British and probably not really perceive being British as a national identity.”


• Wording changed to emphasise affiliation

• “Tick all that apply” instruction removed due to space limitations, assuming respondents will infer this from new wording


Wording made clearer

Outstanding issues to improve question clarity:

• Alternative wording: “What do you consider your national identity to be?” allows room for a “Tick all that apply” instruction

• Examples for the “Other” may assist respondents

• Ordering of “British” tick box

Income

Elizabeth McLaren

Census – Questionnaire Design

Income

• Census topic consultation• Topic-specific consultation• Question testing prior to 2007 Test• Findings from 2007 Test• Findings from Test Evaluation Survey• Findings prior to 2001• Why not household income?

Census topic consultation

• Over 100 responses (2005)• Wide range of potential uses

– Measure of deprivation– Indicator of social exclusion– Estimation of local market sizes– Measure of affluence

• User need score – category 1• ONS category 2

– More work required to develop suitable question to collect accurate information & assess public acceptability

• Consultation on priorities (2007)– More demand for other 4th page topics eg carers

Topic-specific consultation

• November 2005• 30 responses• Preference for:

- gross income- closed question (7-10 bands)- individual income to be collected

• Sources of income not supported• Users supported inclusion of income – but not at

expense of poor response rate or poor quality answers

Income question testing prior to 2007 Test

• Pre 2007 cognitive testing• Banded vs exact income• Gross vs net income• Individual vs household• Current vs time period• Sources of income

Findings from 2007 Census Test

• 2.7% significant difference in response rates between income and non-income areas

• Note: Estimated £2 million cost for follow-up per 1% drop in response rate

• Non-response to level of income high (9%) • Non-response higher for unemployed, ethnic

minorities, less educated• No sig. adverse public reaction notified through

contact centre or field staff• No sig. difference in requests for individual forms• No sig. difference in household under-coverage

2007 Census Test – key constraints

Key constraints affecting income evaluation:• Voluntary

– relied on public’s good will to complete a return– 46.1% response rate

• Publicity and media coverage– limited

Findings from Test Evaluation Survey

• 16% refused to answer income questions• 15% unhappy about answering 1 or more questions

– 58% unhappy about income level– 48% unhappy about income sources– 9% unhappy about ethnicity– 2% unhappy about qualifications

• Comments from respondents– “should be left to Revenue, nothing to do with ONS”– “invasion of privacy”– “irrelevant to concept of Census”– “intrusive”

• 64% retest reliability for income level• 77% retest reliability for sources of income

Findings prior to 2001

• Cognitive interviews 1995-97• 1997 Test

– 2.8% difference in response rates but not statistically significant

– 22% item non-response for income

• 1999 Rehearsal

Why not household income?

• Public acceptability– Respondents unhappy about answering on behalf of

others

• Quality of data– Respondents not including everyone in household– Respondents don’t know income of others– Household income better derived from individual

incomes

• Limited space on housing page• International experience

– Other countries collect individual income

Questions/Comments


UK views on content

Elizabeth McLaren

UK views on content

• No final decisions on content from any country• Final consultations taking place• Each country subject to views of respective

parliaments/Ministers• Further question testing to be carried out eg intended

length of stay

Possible UK differences for 4 page content

Topic (new) England & Wales

Scotland NI

Second residences Intended length of stay

National identity *

Month/year of entry into UK

Month/year of entry for those who have lived outside NI for continuous period of 1 year or more

Nature of disability/illness Voluntary work *partially incorporated within ethnicity question

Possible UK differences for 4 page content

Topic (2001) England & Wales

Scotland NI

Hours worked

Part-time/full-time work

Study place address

Transport to study place address

Religion brought up in

Housing questions

• General UK agreement• GROS considering a household income question• NISRA considering accessibility for disabled

question• ONS considering number of bedrooms question

Priorities for content

Glen Watson

Table exercise

• Prioritise questions into low vs medium vs high priority

• Construct questionnaire for four pages• Construct questionnaire for three pages• Feedback views to group

– Where agreement on priority for topics– Where disagreement on priority for topics– What has been fitted on three pages of questions– What has been fitted on four pages of questions

Way forward

Glen Watson

Timetable

Decision on funding for fourth page March 2008

Topics for Census agreed by RGs April 2008

Cognitive question testing April – June 2008

Postal Test and Omnibus Survey April – June 2008

Census Advisory groups April/May 2008

White Paper published with topics Autumn 2008

Questionnaires for 2009 Rehearsal agreed Autumn 2008

Census Advisory groups / roadshows Autumn 2008

Census Rehearsal Autumn 2009

Questionnaires for 2011 Census agreed Autumn 2009

Census questionnaires approved by Parliament Spring 2010

Census day Spring 2011

Contact details

[email protected]

Census Questionnaire Design and Content

Office for National Statistics

Segensworth Rd

Titchfield

Hampshire

PO15 5RR

mailto:[email protected]

2011 Census: Content Workshop

Documents

Transcript of 2011 Census: Content Workshop