Mandating full New Deal participation for the over-

50s: an experimental analysis

Richard Dorsett & Stefan Speckesser, Policy Studies

Institute

Commissioned by Department for Work and Pensions

Overview

Background Evaluation design Interim results Interpretation and generalisability Conclusion

Background

New Deal 25 plus (ND25+) ND25+ is a mandatory programme for

individuals aged 25+ who have been claiming JSA for 18 months.

ND25+ has three elements: Gateway Intensive Activity Period (IAP) Follow-through.

For those aged 50+, IAP is voluntary. Many choose not to volunteer so in 2002 government announced intention to pilot IAP mandate

ONS population estimates

Evaluation design

Those eligible randomly assigned to action group (mandatory IAP) or control (voluntary IAP) group

Randomisation takes place at start of Gateway - effect possible from that point on

Adviser collects background information then telephones DWP who assign. Customer informed immediately

Randomisation ran from: 5 Apr 2004 – 31 Mar 2006 in 11 areas 10 Jan 2005 – 30 Jun 2006 in 3 (ERA) areas.

Identify effect of mandate rather than IAP itself

Experiment

Data

Data Track outcomes using administrative data 2,305 participants randomised up to 24 June

2005 Merge with New Deal Evaluation Database

(NDED)1. 87 (3.7%) not found in NDED2. 28 (1.2%) appear to start ND25+ after RA3. 129 (6.0%) participants excluded as

duration on Gateway longer than 28 days before RA

Result: 2,061 participants (89%)

Interim results

0%

20%

40%

60%

80%

100%

7 21

35

49

63

77

91

10

5

11

9

13

3

14

7

16

1

17

5

18

9

20

3

21

7

23

1

24

5

25

9

27

3

28

7

30

1

31

5

32

9

34

3

35

7

37

1

38

5

39

9

41

3

Days following random assignment

% o

f A

ctio

n g

rou

p

Pre Gateway/between/missing On Gateway On first/second option

On follow through In unsubsidised employment On alternative benefit

Other destinations* Carried on signing

0%

20%

40%

60%

80%

100%

7 21

35

49

63

77

91

10

5

11

9

13

3

14

7

16

1

17

5

18

9

20

3

21

7

23

1

24

5

25

9

27

3

28

7

30

1

31

5

32

9

34

3

35

7

37

1

38

5

39

9

41

3

Days following random assignment

% o

f C

on

tro

l g

rou

p

Pre Gateway/between/missing On Gateway On first/second option

On follow through In unsubsidised employment On alternative benefit

Other destinations* Carried on signing

-20

0

20

40

60

1 31 61 91 121

151

181

211

241

271

301

331

361

391

days after random assignment

%po

int d

iffe

renc

e in

exi

t to

unsu

bsid

ised

em

ploy

men

t

upper95 Difference lower95

Regression results - exits to employment

-20

0

20

40

60

1 31 61 91 121

151

181

211

241

271

301

331

361

391

days after random assignment

%po

int d

iffe

renc

e in

exi

t to

alte

rnat

ive

bene

fit

upper95 Difference lower95

Regression results - exits to non-JSA benefit

Employment Other benefits Overall 6.3* 5.0* Variation by district - Coventry 20.9* 0.4 - Leicester -7.7 11.7* - Essex 19.5* 6.0 Variation by qualification - No quals 10.3* 1.5 - NVQ1 -9.5 9.7* - GCSE 9.0 9.1* - A-Level 6.5 9.5* - Higher Ed. 6.3 3.0

Duration analysis: predicted effects on status after 1 year (% point differences)

Interpretation and generalisability

substantial number of eligibles excluded substantial number of ineligibles included

Considerable variation across JC+ districts Does this matter? Perhaps it replicates how

policy would operate when rolled out nationally.

1. Are the right people randomised?

Eligible Ineligible Total No random assignment 1,020 1,020 Action 735 262 997 Control 695 233 928 Total 2,450 495 2,945

2. Is randomisation on time?

0

100

200

300

400

Bri

dgen

d

Buc

king

ham

shir

e&

Oxf

Cal

derd

ale

&K

irkl

ees

Cov

entr

y an

dW

arw

icks

Der

bysh

ire

Eas

t Lan

cash

ire

Ess

ex

Gat

eshe

ad &

Sout

h T

yn

Ham

pshi

re

Lei

cest

er

Ren

frew

shir

e,Iv

ercl

y

Shro

pshi

re

Som

erse

t

Suff

olk

Num

ber o

f par

tici

pant

s

Earlier

Same/later/missing

Experiment does not replicate how programme would operate when rolled out nationally

This does matter – do not observe full effect of mandate

3. Are pilot areas representative? Compositional differences between pilot and

non-pilot areas - eg ethnicity, age, length of benefit claim

Treatment differences between pilot areas (action and control groups) and non-pilot areas those in pilot areas take 3 weeks longer

until IAP treatment offered under IAP differs across

pilot and non-pilot areas

Early results appear encouraging but important to note that these are preliminary.

Need to observe individuals beyond first ND25+ exit – final report will use more complete data.

Some deviations from programme design – highlights the importance of monitoring

Some issues relating to generalisability of the results need careful thought: identification of eligible customers timing of randomisation of eligible

individuals pilot representativeness

Conclusion

Interim report available from:

http://www.dwp.gov.uk/asd/asd5/rports2005-2006/

rrep362.pdf