Family Resources Survey

Proposals for the treatment of unlinked FRS data*Valerie Christian & Philip Clarke

2

Background to this project

FRS 0809 - first survey year for which we have been able to link* consenting respondents’ data across the entire year.

BUT …. the consent rate achieved in 0809 was just over 60 per cent. **

For the 40 per cent of data not linked, this project aims to assess for any extent of non-consent bias and to adjust non-consenters’ data accordingly.

3

Where we are now and our eventual aim

This work is still in the early stages…..*

Eventual aim – combination of all adjustments (i.e. for both consenters and non-consenters) into one improved dataset.

**

4

What is the case for adjustments to FRS?

CASELOAD capture – all FRS data and administrative data compared

(extract from Table M6 for FRS 0809)

Difference (FRS vs admin)

Benefit received (%)Income Support -37

Pension Credit -43

Retirement Pension -6

Jobseeker's Allowance -20

Incapacity Benefit -12

Attendance Allowance -74Disability Living Allowance (Care) -15Disability Living Allowance (Mobility) -26

1,570

1. Administrative numbers have been rounded to the nearest 10,000 and FRS numbers to the nearest 100,000. Administrative numbers include recipients living in non-private households (apart from Income Support and Pension Credit).

900

11,050

960

2,190

2,4001900

1900

1800

10400

800

1200 1,340

2,570

2007-08 FRS Administrative dataGrossed number ('

000s)

1500

Number1 (' 000s)

2,060

5

What is the case for adjustments to FRS?

MONETARY capture – based on consenters’ raw (ungrossed) FRS 0809 data

Benefit Type/Income source

Change after linking as a proportion of original FRS data (%s)

Attendance Allowance (AA) + 45Disability Living Allowance (DLA) + 4Incapacity Benefit (IB) + 5Retirement Pension (RP) 0Income Support (IS) + 23Job Seekers Allowance (JSA) + 10Pension Credit (PC) + 4452

389766

611922274

636137520

FRS data (in £s 000)After linking to admin sources (in ££s 000)

2693

6

What does all this mean?

While the unadjusted FRS continues to be fit for purpose* it does not always appear fully capture state benefits in household income.

This shortfall – partly reduced by linking consenters* - justifies the work of this project to adjust non-consenters data (in 0809 almost 40 per cent of all FRS respondents).

7

Considerations in assessing non-consenters data

So far, any approach to adjusting non-consenters data* is based on what we know of consenters **

For example, from linking consenters’ data we know the type of adjustments made to selected DWP benefits

8

What adjustments are made to consenters?

Overall net increase to incomes following linking … comprising four distinct patterns of adjustment:

• those which adjust the survey estimate up (correcting for survey underestimation)

• those which adjust the survey estimate down (correcting for survey overestimation)

• those which input a value where none was reported to survey (correcting where survey gives false negatives or hidden recipients)

• those which completely remove a value* reported to survey (correcting survey for false positives)

9

How large are these adjustments?

For pensioner benefits combined (AA, DLA, PC, RP* )

(Adjustments are required for ½ of all cases)

•adjustments which change the survey estimate up or down range from – £315 to + £227 account for 89% of all pensioner adjustment cases

•adjustments to input a value** increase survey estimates between + £31 to + £249 and account for 4% of all pensioner adjustment cases.

•adjustments to remove a value reduce survey estimates by – £330 to – £6 and account for the remaining 7% of all pensioner adjustment cases

10

Our approaches so far:What proportion of cases are adjusted after linking?

The net impact of linking consenters has led to small upward adjustments in their income.

However among recipients of pensioner benefits, adjustments are needed for half* of consenters.

We have therefore assumed a similar proportion** of non-consenters would need an adjustment of some sort.

11

Our approaches so far:Stage 1: Determining which non-consenters’ data to adjust

•A logistic regression model was fitted to pensioner consenters using a range of descriptive FRS variables* to explain the likelihood of requiring an adjustment amongst these consenters

•This regression helps to determine the most significant variables in explaining the need to adjust or not to adjust

•Applying this model to non-consenters gives the likelihood for requiring an adjustment to the FRS amount reported by this group.

12

Our approaches so far:How successful is explanatory power of our regression model?

•Here we assess how successful our model is in predicting a probability of adjustment among consenters

•Predictions align when an adjustment has been made and predicted probability > 0.5 or when no adjustment was made and the predicted probability < 0.5 *

•The pensioner regression gives 59** per cent of consenter cases as correctly predicted.

•Among these cases, factors increasing*** the likelihood of needing an adjustment are martial status, region, age. Those decreasing **** likelihood include material deprivation, and poor health.

13

Our approaches so far:Stage 2: Determining which non-consenters’ data to adjust

Applying this model to non-consenters can give the likelihood for requiring an adjustment to the FRS amount actually reported by this group

However to actually assign the adjustment status to each non-consenter, this probability is combined with the selection of a random number between 0 and 1

Non-consenters would be given a ‘need to adjust’ status if the random number was less than or equal to the model’s predicted probability

14

Our approaches so far:Stage 3: Determining a monetary adjustment for ‘need to adjust’ non-consenters

Once the model has identified the non-consent cases to be adjusted an appropriate monetary adjustment will be assigned to those cases.

The ‘correctness’ of adjustment will then be assessed against appropriate* caseload and monetary administrative totals.

To accommodate any bias (evident if discrepancies are maintained between adjusted FRS and admin data) the model proposes to re-visit the designation of ‘adjust’ / ‘don’t adjust’ to achieve a result which better matches the admin total.

15

Assumptions made so far ……

Work described so far is based on models currently in use across DWP. These assume non-consenters’ behaviours mirror those of consenters.

Apart from the end stage to re-adjust monetary values to satisfy comparisons with similar admin expenditure totals – our basic model does not account for situations where non-consenters may be different from the known group of consenters.

16

Questioning assumptions made so far ……

We recognise that these (i.e. consent / non-consent) groups are self-selected and it may not be appropriate to assume similarity when designing a model.

If bias is detected some attempt should be made to identify the different behaviours and to make different assumptions as to their adjustment.

One regression approach which tests for and estimates any selection bias is the two stage Heckman selection model*.

A further step of this project can consider this model to identify any bias and build this into assumptions in proposing adjustments

17

Current status of project to adjust non-consenters’ data

Work is ongoing – both in terms of agreeing the most suitable approach to modelling, but also with regard to other key issues concerning the level of detail (and complexity) to which we should model

Some key questions……1) is it necessary to apply all four types of adjustments for those non-consenters needing adjustment? ……

2) is it necessary to model adjustments at the level of individual benefits / income streams, or would grouping by client type or benefit type (children / pensioners / working-age adults) suffice? ….

3) will adjustments adequately retain the functionality* of current dataset ?

4) are proposals repeatable on an annual basis?

18

Concluding thoughts ….

Comparisons which show a net FRS undercount are so far best known in relation to DWP benefits*

…. with some evidence of this trend for earnings amongst those with lowest incomes.

How far an undercount is relevant to other income sources (e.g. private pensions data; council tax & housing benefit data; Child Maintenance data) is so far unknown.

19

Concluding thoughts ….

We aim to extend the linking of consenters’ data to as many income sources that are available*. Adjustments for non-consenters will also be considered

Keen to link income strands likely to be received across the income distribution** (i.e. not only benefits – many of which are disproportionately received by lower income households)

We plan to report on the ongoing progress of this work to adjust for non-consenters at other similar public presentations

20

Contacts at DWP:

Our team address:

team.development@dwp.gsi.gov.uk

We invite and welcome your comments / experience in this new area of work