Data Matching Pilot Evaluation

8/2/2019 Data Matching Pilot Evaluation

1/159

Data matching schemesto improve accuracy and

completeness of the electoral

registers evaluation reportMarch 2012


2/159

Translations and other formats

For information on obtaining this

publication in another language or in

a large-print or Braille version, please

contact the Electoral Commission:

Tel: 020 7271 0500

Email: [email protected]

The Electoral Commission 2012


3/159

Contents

Executive summary 1

1 Introduction 13

2 Set-up and coordination 21

3 Databases and the matching process 28

4 Pilot authorities: overview and emerging issues 36

5 Data matching results: Department for Work and Pensions 44

6 Data matching results: Driver and Vehicle Licensing Agency 63

7 Data matching results: Education databases 71

8 Data matching results: Ministry of Defence 75

9 Data matching results: Citizen Account 79

10 Pilot costs 82

11 Conclusions and recommendations 92

Appendices

Appendix A: Local authority pilot profiles 99Appendix B: Data tables 147


4/159

Acknowledgements

The Electoral Commission would like to thank all the staff at both the local

authorities and the data holding organisations for the time and effort they

devoted to these data matching pilot schemes.

We would also like to thank the Cabinet Office for their assistance in the

collection of data from the pilots.


5/159

1

Executive summary

BackgroundAs part of the proposed shift to individual electoral registration (IER), the UK

Government is exploring the extent to which the use of national public

databases can help Electoral Registration Officers (EROs) improve the accuracy

and completeness of their electoral registers.

The Electoral Commission was given a statutory responsibility to report on the

effectiveness of the data matching schemes. The schemes were based on the

piloting of a range of national public databases in 2011 by 22 individual localauthorities in England and Scotland.1 Our statutory evaluation considers the

degree to which data matching schemes assisted EROs in improving the

completeness and accuracy of their registers; resulted in any issues around

administration, time and costs; or prompted objections to the schemes. Our

findings are based on the data and feedback received from local authorities,

data-holders and others during the course of the pilot schemes.

In February 2012 the UK Government published its response to pre-legislative

scrutiny and public consultation on the IER White Paper. In the response, the UKGovernment indicated its intention subject to the results of the evaluation of

pilot schemes and further testing to widen the scope of data matching

simplify the transition to IER for 2 The UK Government

indicated that rather than, as originally intended,

check accuracy and to identify people who may be eligible to register to vote,

and then invite them to apply to register,3 it was now their intention

names and addresses of all individuals currently on an electoral register will be

matched against the data held by public bodies such as the Department for

Work and Pensionsinformation can be matched, the individual will be automatically placed onto the

new IER register and would not need to take any further action to be registered

1 At the outset there was a pilot authority in Wales but they dropped out early in the process.2 HM Government (2012) Government Response to pre-legislative scrutiny and public

consultation on Individual Electoral Registration and amendments to Electoral Administration

law, Cm 8245

3 HM Government (2011) Individual Electoral Registration, Cm 8108


6/159

2

4 Electors whose details could not be matched in this way would be

asked to apply individually and to supply personal identifiers.

This proposal has not been tested by these pilots. Further piloting is needed to

ensure that the advantages and disadvantages of these proposals are

understood. In this report we set out some of the key questions we think need to

be answered to help understand the issues.

Set-up and coordination

The Cabinet Office managed the overall pilot process and was also

involved in the delivery of the pilots. The Commission advised the Cabinet

Office throughout the set-up period, in particular on the need for a clear

common framework for delivering the pilots, but the final decisions on theprocesses were taken by the Cabinet Office.

The open application and selection process used for the pilots, and the

absence of a clear, common framework, contrary to our advice, led to

significant variation in the planned approaches of the pilots. This

introduced challenges for the evaluation in comparing the results of the

different schemes and therefore the ability to draw consistent conclusions.

This also created challenges for local authorities delivering the pilots

because in several cases the methodology they originally planned to use

proved to be based on incorrect assumptions.

The timing of the pilots, which took place alongside the annual canvass,

coupled with delays to the process, put pressure on the capacity of the

local authority teams involved and added to the difficulty in this evaluation

of drawing firm conclusions from the pilot schemes as a whole.

Local authorities reported varying levels of communication with the

Cabinet Office and identified areas for improvement, including a betterunderstanding of what data was going to be shared with them.

The pilots did not follow processes, in terms of the IT systems and

matching arrangements, which would be used for nationwide data

4 HM Government (2012) Government Response to pre-legislative scrutiny and public


law, Cm 8245


7/159

3

matching. The evaluation cannot therefore draw conclusions about how

the costs of these pilots would translate to a national roll-out.

The databases and the matchingprocess

Ten databases were due to be tested as part of the scheme. These were

the:

Department for Work and Pensions (DWP) Centric database

Driver and Vehicle Licensing Agency (DVLA) Driver database

Student Loans Company (SLC) databaseNational Pupil Database (NPD) (through the Department for Education)

Individual Learner Record (ILR) (through the Department for Business,

Innovation and Skills)

Citizens Account (CA) database (through the Improvement Service in

Scotland)

(MoD) Joint Personnel Administration database

and Anite housing database

Higher Education Funding Council for England (HEFCE) student

databaseRoyal Mail change of address database

However, not all were tested to the same extent. In particular, there were

difficulties in accessing HEFCE and Royal Mail data.

A key component of the trials was the matching process between

databases to identify people to invite to register to vote. The processes

employed could not be rolled out nationally but do allow for a greater

understanding of the requirements of any framework for national datamatching.

Two different processes were used for matching the electoral registers with

the DWP data on one hand and the DVLA and education databases on the

other. These different rules make it difficult to compare the results from the

two processes.


8/159

4

Pilot authorities: overview and emerging

issues

There were 22 data matching pilots testing various combinations of

databases.

The complex nature and format of the data supplied to local authorities

highlighted the need for good data management and analysis skills. Many

of the pilots either had these skills available within the local authority or

recruited additional staff using Cabinet Office funding. However, severaldid not and struggled to use the information provided.

The process, as tested in these pilots, was labour intensive with significant

work required to analyse the data. Those involved felt that the level of work

required would not be sustainable in the future.

A number of pilot authorities were able to use locally-held data to

interrogate the data received from the national databases. The results of

this activity suggest that there is scope for more use to be made of localdata both to complement any future national data matching and to

improve accuracy and completeness in general.

Data matching results

Department for Work and Pensions (DWP)

Eighteen pilots accessed the DWP Centric database.

The level of match between the electoral registers and the DWP data

varied significantly between local authorities. For those areas matching the

whole register it ranged from 57.6% to 82.4%. These differences are partly

due to different interpretations across the pilots, in the absence of a

consistent framework, of what constitutes a match but are also likely to

driven by differences between the local authorities in terms of

demographics.


9/159

5

6,573 people were added to the registers as a result of follow-up activity

undertaken using names suggested by the DWP Centric database. This

was 13.2% of all names followed up.

The response rate for the pilot follow up was affected by whether theass.

Where it took place during the canvass, the pilot response was depressed

by the fact that many of the names identified by the match registered

through the canvass.

The pilots highlighted crucial differences in address formats between the

electoral registers and the other national databases. This meant that many

records could not be matched as simple address differences were not

recognised. This problem could have been significantly reduced if time

had been allowed for an address cleansing exercise.

The absence of a unique identifier attached to each address on the public

national databases was a key issue for the pilots. These would have

allowed for a more straightforward matching process for local authorities.

Many of the potential new electors suggested by the match with the DWP

Centric database proved to be based on out of date or incorrect

information. The problems posed by this could have been reduced by the

inclusion of the date when the DWP record changed something whichDWP were willing to provide but were not asked to do so.

The absence of nationality information meant that several pilot authorities

conducted follow up with people ineligible to register. However, the scope

of this issue is not clear from these pilots and will have varied depending

on the demographics of the local authority area.

Driver and Vehicle Licensing Agency (DVLA) Match levels between the DVLA driver database and the electoral registers

were lower than between the registers and the DWP Centric database,

partly because

because the match process used stricter criteria.

The match levels varied from 51.7% to 67.3%.

this was more likely to reflect poor data currency rather than significantunder-registration.


10/159

6

208 were added to the register as a result of pilot follow-up activity. This

was 4.1% of all names followed up.

Many of the responses to follow-up activity indicated the person written to

was not resDVLA Driver database is not current.

The DVLA data was more effective at targeting 16 and 17 year olds as

opposed to the population as a whole.

Education databases

There were very few registrations from data matching with the Student

Loans Company (SLC) database. This, and responses to the follow-up

activity, support the view expressed by the SLC that the data used for

these pilots (at the end of the academic year) was sometimes out of date.

The National Pupil Database (NPD) and Individual Learner Record (ILR)

proved effective at identifying attainers5 in these pilots.

However, while the NPD and ILR identified attainers successfully, the

majority of registrations were achieved through the annual canvass, which

was taking place alongside the pilots, and not in response to follow-upactivity through the pilots. Under IER, unlike in the current household

system, individual attainers might need to complete their own form (rather

than being registered by adults in the household). It is therefore possible

that the number of registered attainers will fall. The ability to use data in

order to target them in this way may therefore be a more useful tool for

EROs in the future.

Ministry of Defence (MoD) The MoD provided limited data for these pilots. They were able to confirm

that existing service voters were still resident but not provide details of

potential new service voters. They also provided details of addresses

occupied by service personnel in the area but this excluded barracks.

5 An attainer is a 16- or 17-year-old who will reach voting age (18 years old) during the life of a

current electoral register.


11/159

7

There was therefore no real prospect of addressing the completeness of

service voter registrations in the pilot areas.

Two pilots were able to use the MoD data to improve the accuracy of their

register and amended or deleted a number of their records (9.6% and13.2%) of the total number of service voters held on the register.

Citizens Account (CA)

The CA database is administered by the Improvement Service in Scotland

and is intended to be a record of all residents within a participating local

authority area.

However, the CA database is not as comprehensive as the pilot authority

originally anticipated the total number of records provided by CA

represented only 27% of the Renfrewshire electorate.

The level of match between the CA data and the electoral register was high

with 88.8% of the CA records also found on the register.

The matching exercise suggested a small number of potential new

electors (1.7% of the size of the register after local matching).

Follow-up activity was still under way at time of publication.

Pilot costs

The overall cost of the pilots is estimated at around 425,910, against an

original budget of 1.2m. These figures exclude staff costs for the Cabinet

Office.

The under-spend is largely explained by initial budgeting over-estimates byboth Cabinet Office and the pilot authorities, due to a lack of clarity about

what the pilot process would entail, and by many local authorities not

completing some of the activities, e.g. follow-up work, which they originally

planned for. Given the size of the over-estimates, it seems unlikely that the

pilots would ever have cost the full amount budgeted.

The main item of expenditure reported by local authorities is the costs of

additional staff, which account for about 50% of the total spent by local

authorities. Local authorities reported that the process was labour intensive


12/159

8

and they needed to incur much of this cost before they could begin the

process of contacting potential new electors.

Staff costs could be reduced by improving the quality of the data matched

and automating more of the process but we cannot conclude, from the

information gathered in these pilots, what the cost would be of any national

data matching roll out.

There was some limited expenditure on databases that were not used in

the pilots and so did not deliver any benefit.

While the costs of these pilots appear high in terms of numbers of people

added to the registers this does not mean that data matching could not be

cost effective if implemented differently.

In order to assess potential scalability of data matching, it would be

necessary to have more consistent information than is available about the

costs incurred, and this information should include the additional internal

costs incurred by both local authorities and data-holding organisations.

Conclusions

Our conclusions broadly follow the statutory evaluation criteria set out in

Sections 35 and 36 of the Political Parties and Elections Act 2009. These criteria

concern the degree to which data matching schemes assisted EROs in

improving the completeness and accuracy of their registers; resulted in any

issues around administration, time and costs; or prompted objections to the

schemes.

The registration objectives: completeness and accuracy

On the whole, these pilots did not prove very effective at getting new electors onto the registers. Despite the efforts invested by authorities in the data pilots, very

few additions (only 7,917) were subsequently made to the registers.

However, better results were achieved where the local authority was able to

begin their pilot follow-up activity before, or at a very early stage, of their annual

canvass. This was largely because, where the follow up did not begin until later,

many people had already registered through the canvass.

In these pilots, the most useful databases in terms of adding people to theregisters were those which targeted specific under-registered groups (e.g. 16-


13/159

9

and 17-year-olds) such as the National Pupil Database (NPD) and the Individual

Learner Record (ILR).

The issues surrounding the currency of address information on some of the

other databases would need to be addressed in order to improve their

effectiveness at finding new electors.

However, the low number of registrations does not mean that the principle of

data matching is not worth pursuing further and many local authorities were

clear that they still see potential in it. Refinements to the matching process

such as improvements to the currency, quality and compatibility of the data

provided would need to be in place before this objective could be fully tested.

In relation to improving the accuracy of the registers, the MoD data was useful,

up to a point, at helping EROs to amend or delete the records of service voters.Two local authorities amended or deleted a number of their records,

representing 9.6% and 13.2% of the total number of service voters held on the

register.

However, there was limited testing of the usefulness of the other databases for

improving accuracy with only one pilot providing information to the Commission

on this aspect.

Finally, not all the public national databases included in the current scheme

were tested to the same degree. As set out earlier in the report, there was no

testing of HEFCE or Royal Mail data by the pilot authorities and we are unable to

draw any conclusions about the usefulness of this data in addressing the

registration objectives.

Objections to the schemes

At the outset there were concerns that the use of public data in this way could

generate objections from the public. However, where data has been provided,

local authorities indicated they received few objections to the schemes. Where

local authorities did receive queries, the vast majority of people were content

with the use of the data when the purposes of the schemes were explained to

them.

This indicates that the data matching pilots did not generate any substantial

level of concern amongst the public. However, any future testing or roll out of

data matching would need to be well implemented in order to ensure there is

continued public support.


14/159

10

Ease of administration

Many pilots raised concerns that in its current format the process of data

matching was too labour intensive for regular use. Additional staff resource was

required by many of the authorities. In the main, this tended to be due to thelarge volumes of data received, issues with data compatibility and the workload

involved in sorting the data for use.

Many authorities also emphasised the need to understand the skill sets required

for this kind of activity, and highlighted that in many cases these skills were not

held by those currently working on registration activities. In the interim,

developing the process of local data matching would not only be useful to EROs

in maintaining the registers but would also help to build skills which could be

used to understand and manipulate data provided from national databases.

Time and costs

The pilot schemes proved to be both time consuming and costly. However, it is

not possible to draw robust conclusions about the long-term cost effectiveness

of data matching from these pilots as the processes used here would not be

repeated in a nationwide system of data matching.

Nevertheless, it is clear that unless the process is made substantially more

straightforward, it is doubtful that many authorities would have had the

resources available to undertake data matching without additional finance

which, in this case, was provided by the Cabinet Office.

Recommendations

This section sets out our recommendations for future data matching activities.

Pilot processesFurther testing of national databases by local authorities would need to beundertaken in order to establish whether data matching is made availablefor use by all local authorities.Any further testing needs to be set up in a way that addresses the limitations set

out in this report in order to ensure that meaningful data can be collated. The

Electoral Commission would encourage the Government to consult us in detail

in order to achieve this.


15/159

11

We recommend that any further piloting (with a focus on improving accuracy

and completeness): takes place outside of the annual canvass period and avoids other

significant electoral events. Piloting data matching alongside the annualcanvass added a layer of complexity to the testing process and meant it

was harder for local authorities to isolate the impact of the data matching

as opposed to canvass response rates. It also had consequences for local

authority capacity to utilise the data when it was available to them. Several

EROs thought that data matching could have more use following the

canvass to pick up new registrants in the run-up to elections.

has a clear framework for the use of data that all participatingauthorities can follow. This current scheme allowed local authorities toadopt varying approaches to piloting the data they received. The differing

methodologies meant it was harder to draw conclusions about the

effectiveness of the data and thus the future of the registration system. A

clear framework would help to ensure comparability between the pilots but

still allow for some local differences for example, targeting particular

groups and making use of local databases.

tests, as closely as possible, the process which would be madeavailable to all local authorities if data matching was to be rolled outnationally.

ensures that participating areas are sufficiently staffed and haveappropriate expertise to complete the pilot and test the data provided.

allows for a better understanding of the benefits of access to nationaldata compared to existing local databases.

allows for a clearer analysis of the cost of data matching through moreinformed budgeting and prescribed reporting of costs incurred.

ensures that good communication between the pilots, the dataholders and the Cabinet Office is maintained throughout the process.

Databases

In relation to the specific databases included in these schemes:


16/159

12

There is merit in re-testing nearly all of the databases included inthese pilots providing the specific issues identified in this evaluationare addressed, namely that:

address format compatibility issues should be mitigated wherepossible. The planned inclusion of Unique Property ReferenceNumbers (a unique identifier for each address held) on the DWP

database will help with this issue,as will plans for a single nationaladdress file. Other mitigating steps could betaken for matches withother databases, for example using address cleansingsoftware.Data currency issues should be tackled by ensuring that, wherepossible, the information shared includes details of the dates onwhich database records are updated.

We would not recommend further testing of the MoD data, unless therange of data which can be shared is increased. While the datasupplied in these schemes was useful for the pilot authorities it is likely to

Proposals for verifying identity

As outlined earlier, the Government is currently considering whether the results

from the data matching exercise could be used to confirm the identity of

individuals captured by the household canvass during the transition to IER. In

relation to this, we recommend that:

There is a need for more evidence to support this proposal, given thatthis is was not an objective of these pilots . Any future piloting thatincludes this as an objective for testing should allow for an analysis of

matched and non-matched records in order to check the accuracy of the

matching process used. It is possible that this analysis could use the

annual canvass process. As a result the timing of these pilots may need tobe slightly different to that of any pilots focused on accuracy and

completeness.

These plans should also stay abreast of developments in the. There are other initiatives

within government on the processes that might be used in the future to

verify identity. Learning lessons and adopting best practice from these

other initiatives is important in order to ensure that the approach to

verification followed under IER, and therefore the security of the registers,

is as robust as possible.


17/159

13

1 Introduction

1.1 statutory

evaluation of the 2011 data matching pilot schemes. The schemes were based

on matching a range of national databases by Electoral Registration Officers

(EROs) in 22 local authorities. This was the first time that EROs have been able

to test the usefulness of national data for improving the quality of their electoral

registers.

1.2 The overall aim of the pilot schemes was for EROs to test whether national

public databases can help to improve the accuracy and completeness of their

electoral registers.

Background

Accuracy and completeness of the electoral registers

1.3 Electoral registers underpin elections by providing the list of those who are

eligible to vote. Those not included on the registers cannot take part in

elections. Registers are also used for other important civic purposes, including

selecting people to undertake jury service, and calculating electorates to informParliamentary and local government boundary reviews, which are the basis for

ensuring representative democracy. People not registered are therefore not

counted for these purposes either.

1.4 In addition, credit references agencies may purchase complete copies of

electoral registers, which they use to confirm addresses supplied by applicants

for bank accounts, credit cards, personal loans and mortgages.

1.5 Great Britain does not have one single electoral register. Rather, each local

authority appoints an ERO who has responsibility for compiling an accurate and

complete electoral register for their local area.

1.6 AccuracyThe accuracy of the electoral registers is therefore a measure of the percentage

of entries on the registers which relate to verified and eligible voters who are

resident at that address. Inaccurate register entries may relate to entries which

have become redundant (for example, due to people moving home), which are

for people who are ineligible and have been included unintentionally, or which

are fraudulent.


18/159

14

1.7 Completenesstherefore refers to the percentage of eligible people who are registered at their

current address. The proportion of eligible people who are not included on the

register at their current address constitutes the rate of under-registration.

1.8 Great Br

20116provided the first national estimates of the completeness of the electoral

registers since estimates of the 2000 England and Wales registers, as well as

the first national estimates of the accuracy of the registers since 1981. This

study was funded by the Cabinet Office in order to inform the development of

the approach to the introduction of individual electoral registration (IER).

1.9 The research estimated the April 2011 Parliamentary registers to be 82.3%

complete; the comparable figure for the local government registers was 82.0%.This equates to approximately 8.5 million unregistered people in Great Britain as

of April 2011.However, this does not mean that these registers should have had

8.5 million more entries, because many, but not all, of those not registered

correctly may still have been represented on the registers by an inaccurate entry

(for example, at a previous address).

1.10 The April 2011 parliamentary registers were 85.5% accurate; thecomparable figure for the local government registers was 85.4%.

1.11 The research also demonstrates the extent to which both the accuracy and

completeness of the registers deteriorate between the publication of the

registers in December each year and the time when elections are usually held in

the following spring. Although in December 2010 the estimated number of

people not registered in Great Britain was at least six million, by April 2011 the

number had grown to around 8.5 million (17.7%).

Current system of updating the electoral registers

1.12 At present, EROs use an annual canvass and rolling registration to update

their registers. Individual electors can register to vote throughout the year by

However,

most updates to the registers take place during the annual canvass, which is

undertaken each autumn. At its simplest, the canvass involves delivering a

6http://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdf
http://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdfhttp://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdfhttp://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdfhttp://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdfhttp://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdfhttp://www.electoralcommission.org.uk/__data/assets/pdf_file/0007/145366/Great-Britains-electoral-registers-2011.pdf


19/159

15

registration form to each household and following up, via postal reminders and

personal visits, those households who do not respond. Revised registers are

then published on 1 December.

1.13 Almost all EROs use locally held data, such as council tax and housing

records, to improve the effectiveness of their registration activity. However,

EROs have not been able to make use of national databases in order to improve

the quality of their local registers.

Data matching and the move to individual electoral

registration

1.14 The previous UK Government, during the passage of the Political Parties

and Elections Act 2009 (PPE Act), introduced legislation providing for thephased introduction of individual electoral registration (IER) in Great Britain. The

PPE Act made provision for IER to be introduced in accordance with a statutory

timetable. The PPE Act also included provisions to allow data matching pilot

schemes to be carried out, with a view to establishing which national public

databases might be useful to EROs in helping maintain electoral registers

during the transition to IER.1.15 Under the PPE Act, data matching schemes approved by the Secretary of

State would require a public or local authority to supply an ERO with data which

they could then use for the purpose of maintaining complete and accurate

registers.1.16 In June 2011 the Coalition Government published a White Paper setting

out its plans to speed up the implementation of IER in Great Britain. The new

system to be implemented from 2014 will require each elector to register

individually (unlike the current system where registration takes place

predominantly by household) and to supply personal information for verification

purposes prior to names being added to the electoral register.

1.17 The IER White Paper explained that the UK Government would explore,

o

identify people eligible to vote but missing from the register so they can invite7 If successful the Government indicated that it would look at

how data matching used in this way could be extended across the country and

support the move to IER.

7 HM Government (2011) Individual Electoral Registration, Cm 8108, p11.


20/159

16

1.18 In February 2012 the UK Government published its response to pre-

legislative scrutiny and public consultation on the IER White Paper.8 In the

response, the UK Government indicated its intention subject to the results of

the evaluation of pilot schemes and further testing to widen the scope of data

The UKGovernment indicated that rather than only using data matching to identify

potential electors, it was now their

individuals currently on an electoral register will be matched against the data

held by public bodies such as the DWP and local authorities themselves

that I be matched, the individual will be

automatically placed onto the new IER register and would not need to take any9 Electors whose details could not be

matched in this way would be asked to apply individually and to supply personal

identifiers.

1.19 The UK Government has acknowledged that this would represent a

significant change to the position set out in the White Paper, which envisaged all

potential electors applying individually and supplying personal identifiers, with

data matching used as a means of identifying potential electors. It stated its

an efficient and effective system ready in time to support the implementation of10

The Electoral Registration Data Schemes Order 2011

1.20 The Electoral Registration Data Schemes Order 2011 (the 2011 Order),

made on 9 June 2011, gave effect to proposals by local authorities to run data-

matching schemes. Under the 2011 Order, an agreement between the data-

holding organisation and the ERO needed to be in place before personal data

could be shared between the two parties. The purpose of the agreement was to

explain governance arrangements for data transfer and matching, explain the

expected outputs and inputs for this process, set out information security

standards, and detail timescales.

1.21 The Cabinet Office was responsible for the selection and coordination of

the schemes. The process for recruiting local authorities to run pilots was by

8 HM Government (2012) Government Response to pre-legislative scrutiny and public


law, Cm 8245.9 Ibid.

10 Ibid.


21/159

17

open application, with the Government wanting to see how people responded to

the idea of using national databases to help maintain the electoral register.

Aims and objectives of pilots

1.22 The overall aim of the pilots was for EROs to test whether public

databases can be useful for improving the accuracy and completeness of their

electoral registers. However, in practice, the majority of pilots were more

focused on completeness (finding people eligible to vote but missing from the

register) than they were on accuracy (finding and removing inaccurate entries

on the register).

1.23 The detailed objectives of the schemes varied due to the open application

process and lack of a common framework. Each authority submitted a proposalon how they would undertake a data matching exercise based on particular

challenges in their area. These proposals varied in terms of both scale and

focus. For example, some pilots matched their whole register with the available

data while others targeted particular wards with historically low response rates to

the annual canvass. Some areas were particularly focused on certain

demographic groups, e.g. attainers or the over-70s, while others looked at all

residents. The objectives of individual pilot schemes are examined in more

detail later in this report.

Role of the Commission

1.24 The Commission was given a statutory responsibility to report on the

effectiveness of the data matching schemes. The approach we have adopted is

based on the requirements for an evaluation set out in Sections 35 and 36 of the

PPE Act.

1.25 The PPE Act

a description of the scheme

an assessment of the extent to which the scheme assists the ERO in

meeting the registration objectives11, which are:

that persons who are entitled to be registered on a register are

registered on it

11 Registration objectives are set out in Section 31.8 of the PPE Act 2009.


22/159

18

that persons who are not entitled to be registered on a register

are not registered on it, and

that none of the information relating to a registered person that

appears on a register or other record kept by a registration

officer is false

whether there was an objection to the scheme, and if so how much

how easy the scheme was to administer

the extent to which the scheme resulted in savings of time and costs, or

the opposite

anything else specified in the order under Section 35. The 2011 Order did

Our approach

1.26 Our approach to the evaluation has been based on our statutory

responsibilities outlined above. We have assessed:

The administration of the pilots: the way the schemes were run, anydifficulties experienced or lessons learned by local authorities, data

holders, other organisations involved and the objections to the scheme.

Data quality: the potential for data matching to improve the registrationprocess.

Resources: resources and skills necessary for administering the pilots,their costs and the extent to which data matching can result in cost and

time savings.

1.27 We worked with the Cabinet Office during the set-up of the schemes withthe aim of allowing for an effective evaluation of each pilot as well as the

schemes as a whole. We emphasised in particular the desirability of consistency

across key components of the schemes (methodology, matching and follow-up

process), in order that the findings could be compared across areas and across

databases. However, the final decisions made by the Cabinet Office did not

always reflect the advice given and no clear, common framework for the pilots

was established.


23/159

19

1.28 Together with the Cabinet Office, we monitored the work of the

participating local authorities throughout the running of the schemes and were in

contact with the authorities to provide assistance and address issues.

1.29 The evaluation is based on a range of qualitative and quantitative data

collected before, during and at the end of the process. Data and other evidence

were collected from:

Questionnaires from local authorities: each authority submitted aproposal before the start of the pilots which outlined their objectives and

their approach to delivering the scheme.

Data from local authorities: we designed a template, with the input of theCabinet Office, for collecting data from the local authorities about the

various databases and the results from the follow-up activities. Localauthorities were asked to submit interim data (between August and

October) and a final return with all results by 14 December 2011. However,

not all authorities met this deadline or provided data in the format

requested.

Evaluation report from local authorities: all authorities were required tosubmit an evaluation of their pilot by 23 December 2011 using a template

designed by us and the Cabinet Office. The report covers the key areas of

the evaluation.

Interviews with local authorities: we conducted individual interviews witheach participating local authority between the end of October 2011 and the

beginning of January 2012.

Interviews with data-holders and software suppliers: we also conductedinterviews with those organisations that hold the datasets being tested in

the pilots and software suppliers who had assisted local authorities with

the data.

Regular contact with the Cabinet Office: we liaised closely with theCabinet Office throughout the project and were part of a Registration

Improvements Board, which monitored the progress of the pilots.

This report

1.30 This report considers the effectiveness of the data matching schemes in

improving the accuracy and completeness of the electoral registers.


24/159

20

1.31 The remainder of this report is divided into the following:

Chapter 2 summarises the set-up and coordination of the pilot schemes

by the Cabinet Office, including details of the selection process and issues

relating to the timing of the pilots.

Chapter 3 summarises the national databases included in the pilot

schemes.

Chapter 4 sets out details of each specific pilot area and issues

encountered by the pilots in delivering the schemes.

Chapters 5, 6, 7, 8 and 9 set out the key data, provided by the local

authority pilots, for each of the national databases accessed. It reviews the

quality of data returned to each local authority and the usefulness of thatdata in meeting the registration objectives set out for the schemes.

Chapter 10 summaries the costs of the data matching schemes.

Chapter 11 summarises the key findings and recommendations.


25/159

21

2 Set-up and coordination

2.1 This chapter sets out how the pilots were set up and coordinated by the

Cabinet Office. It also considers the impact of the approach to the management

of the pilots on the findings of the evaluation.

Key points The Cabinet Office managed the overall pilot process and was also

involved in the delivery of the pilots. The Commission advised the Cabinet

Office throughout the set-up period, in particular on the need for a clear,

common framework for delivering the pilots, but the final decisions on the

processes were taken by the Cabinet Office.

The open application and selection process used for the pilots, and the

absence of a clear, common framework, contrary to our advice, led to

significant variation in the planned approaches of the pilots. This

introduced challenges for the evaluation in comparing the results of the

different schemes and therefore the ability to draw consistent conclusions.

This also created challenges for local authorities delivering the pilots

because in several cases the methodology they originally planned to useproved to be based on incorrect assumptions.

The timing of the pilots, which took place alongside the annual canvass,

coupled with delays to the process, put pressure on the capacity of the

local authority teams involved and added to the difficulty in this evaluation

of drawing firm conclusions from the pilot schemes as a whole.

Local authorities reported varying levels of communication with the

Cabinet Office and identified areas for improvement, including a better

understanding of what data was going to be shared with them.

The pilots did not follow processes, in terms of the IT systems and

matching arrangements, which would be used for nationwide data

matching. The evaluation cannot therefore draw conclusions about how

the costs of these pilots would translate to a national roll-out.


26/159

22

Overview

2.2 The

encompassed:

design of the pilot framework, including drafting of secondary legislation

that set out how the pilots were to operate and when the pilots were to be

undertaken

issuing to all local authorities an invitation to participate, and selecting

which areas were to take part in the scheme

overseeing the delivery of the pilots by local authorities

negotiating with data-holders to allow for matching to take place

ensuring appropriate confidentiality and data security agreements were in

place with participating areas and data-holders

developing the data matching process

for some databases, overseeing the match with the register

providing funding for the scheme and overseeing payments to data-holding organisations and local authorities

2.3 There are several aspects of the set up and management of the schemes

which introduced challenges for local authorities delivering the pilots. They have

also made it more difficult for our evaluation to draw clear conclusions on the

success of the pilots. These are considered below.

Selection of the pilot schemes

2.4 The Cabinet Office issued an invitation, in September 2010, to all local

authorities in Great Britain to pilot data matching. To participate, authorities were

required to submit a proposal outlining their objectives for data matching, how

they would deliver the scheme, and estimated costs. Each of the authorities

then provided further information about the proposed delivery of their pilot in

order to inform the selection process.

2.5 The Cabinet Office assessed the ability of the authorities to meet the

requirements of the scheme and selected participants based on the quality oftheir application, taking into account the demographic groups they wanted to


27/159

23

target, any innovative ideas they proposed and the estimated budget for the

activity. The geographic spread of the final group of selected authorities was

also considered. The final group of pilots was chosen in January 2011 and the

statutory instrument for the schemes was confirmed in June 2011.

Variable methodologies

2.6 As noted, local authorities were encouraged to submit their own proposals

and suggestions as to how data matching might work in their area. The Cabinet

rationale for the open application process was to allow local authorities

to identify ways in which data matching might help them to address the

particular challenges or target audiences relevant to their local area.

2.7 While there are advantages to encouraging ideas and innovative

approaches from local authorities, we consistently stressed, in our advice to the

Cabinet Office, the need for a clear framework for the pilots, which would

provide consistency in delivery and therefore allow for an effective evaluation.

We also formally raised this need as part of our response to the Cabinet Office

consultation on the Electoral Registration Data Schemes Order 2011 and the

Representation of the People (Electoral Registration Data Schemes) Regulations

2011.12

2.8 However, no specific instructions were given to local authorities about

how to implement the schemes, and no clear framework was put in place toensure consistent delivery, although some support was available from the

Cabinet Office.

2.9 The absence of a clear framework for delivery meant that a wide variety of

approaches were adopted for implementing pilots and this wide variation has

made it more difficult to draw clear comparisons in this evaluation. For example,

authorities differed in how they treated the match scores in the data returned to

them (for more information see Chapter 5). A register entry which was matched

against an entry on the Department for Work and Pensions Centric (DWP)database would be scored between 10 and 100 depending on the exact nature

of the match. Some areas chose to treat all scores above 55 as a match while

others chose all scores above 80. The Cabinet Office did not attempt to impose

any standardisation of approach. This has implications when comparing the

quality of the results across local authorities.

12www.electoralcommission.org.uk/__data/assets/pdf_file/0011/117695/Electoral-

Commission-consultation-response-Data-matching-SI.pdf


28/159

24

2.10 The approach adopted for contacting people identified as a result of data

matching varied across authorities, involving either one or more letters to names

identified or one or more visits by canvassers to the addresses associated with

those names, or a combination of both letters and visits. The variety of

approaches taken complicates the analysis of the results, as a high responserate in one pilot may have more to do with the use of canvassers than the quality

of the data for that area.

2.11 Finally, the open nature of the application process meant that the initial

proposals also often made assumptions about certain processes or criteria

being in place for delivering the pilot schemes. Consequently, when some of

these assumptions proved to be incorrect, authorities struggled to deliver the

data matching scheme. As one local authority set out in their evaluation report:

maybe for future work, the Cabinet Office needs to be a little more

prescriptive on the processes and outcomes it requires.

Timing

2.12 The data matching schemes had originally been due to commence in June

2011 with all activities (including evaluations) to be completed by September

2011. During this set-up period, we emphasised the importance of avoiding

significant overlap with the annual canvass.

2.13 However, the Cabinet Office decided to allow pilot activity to continue until

the end of November 2011 with evaluations taking place afterwards. In addition,

there were delays at the outset that compounded the problem. The authorities

had expected to receive the data in late June 2011. However, due to delays in

ensuring all the necessary technical arrangements and data access agreements

were in place, the matching of the registers did not commence until July August

for most authorities. These delays meant that a number of authorities had to

adapt their approach to testing the data because they no longer had the

resources available to manage the process or because they had anticipated

contacting residents in advance of their canvass beginning, but were no longer

able to do so. One local authority commented:

Slipping of the timetable made it impossible to complete the pilot as it

was first intended.

2.14 Running the pilots alongside the annual canvass added a complicating

factor both for the delivery of the pilot schemes and also for assessing the value

of data matching. It also had an impact on the capacity and resources ofauthorities to use the data returned to them (these issues are considered further


29/159

25

in Chapter 4). It was in anticipation of these problems that we raised concerns

during the initial planning phase about the proposed timing of the schemes.

Control groups

2.15 For most areas it was not therefore feasible to contact local residents

before the annual canvass had begun across their area. To address this issue,

we encouraged pilots to create control groups of names identified from the

national data, where no dedicated follow up would take place and the names

would subsequently be tracked in the annual canvass.

2.16 This was intended to determine how many would have been registered

anyway in the absence of the pilot. However, not all the authorities were able to

put in place a clear process for separating out the canvass from the data

matching activities and often people identified to be followed up by letter were

found to have already registered through the canvass.

2.17 For the purposes of this evaluation this means that data on the response

rates for those names followed up by pilot authorities has to be viewed in the

context of how the authority was able to manage the two processes of the pilot

and the annual canvass.

2.18 As the example below shows, in many cases the fact that people were

registering through the canvass depressed the response rate to letters issuedthough the pilot process.


30/159

26

Effect of the canvass on pilot response ratesThe matching process suggests 500 names that appear to be resident in the

area (because they appear on another database) but are not found on the

electoral register.

The canvass has already begun by the time the authority is in a position to write

to these individuals and when the 500 names are checked against canvass

returns 150 are found to have registered already.

The pilot can only therefore write to the remaining 350 names.

From the 350 letters issued, 50 respondents register to vote equating to a 14%

response. There are fewer responses because it is very likely that there will be

proportionately more incorrect names, ineligible people or people less likely toregister among the 350 than among the original 500. This is mainly because 150

people who are resident, eligible and interested have already been removed.

But if all 150 had responded to the letter as they did to the canvass form the

response rate would have been 40% and even if only half (75) had responded it

would still have been notably higher at 25%.

Communication

2.19 The Cabinet Office had intended to run monthly meetings with the pilot

areas. While some meetings took place, they were less frequent and more

sporadic than had originally been anticipated. Notwithstanding this, the Cabinet

Office also made themselves available to local areas to discuss issues and

this was noted by several authorities. For example, one authority reported:

Throughout the project general communication with the Cabinet Office

and the provision of update information was effective.

2.20 However, some areas also commented that in the immediate run-up to the

matching taking place they noticed decreasing contact from the Cabinet Office.

They did not feel fully informed about changes to the process and in some

instances noted that queries went unanswered.

2.21 For example, they had expected that the data returned from the DWP

would include unique property reference numbers to ensure that addresses on

their electoral register could be found on the DWP database. They had also

thought that the data would include dates of record changes. Neither of these


31/159

27

elements was included in the data returned. Some of these things were crucial

to the effectiveness of the pilots and are discussed further below.

2.22 Several pilot authorities also indicated that they did not know what the

format or layout of the data matching results would be before they were sent to

them. Practically, these issues meant there was a period of confusion among

several pilots when they initially received the results of the matching activity.

2.23 Several pilot authorities and the DWP felt that it would have been beneficial

to have had more direct communication, rather than always using the Cabinet

Office as a go-between. This may have helped to ensure the pilots were more

up to date about the process and in a better position to interpret the outputs

from the matching process.

Scalability

2.24 The technical matching processes and the IT systems used in these pilots

could not be scaled up and rolled out across Great Britain. For example, data

files were sent to and from local authorities by email, with some matching

carried out by DWP directly and some by a team within the Cabinet Office (see

Chapter 3 for more details). The approach worked for this limited number of

pilots but would not be sustainable for every local authority in Great Britain.

2.25 This also has an impact on the analysis of the costs of these pilots as the

individual budgets relate to processes which would not be replicated. As a result

this evaluation can make only limited comment on the value for money of data

matching.

2.26 Nonetheless, running the data matching schemes has allowed for a

greater understanding of the requirements of any framework for national data

matching.


32/159

28

3 Databases and the

matching process3.1 This chapter considers the national public databases that were included in

the pilot schemes. As noted above, the Cabinet Office arranged for access to a

range of public databases through discussions with the relevant data-holding

organisations. The databases that could be accessed by each pilot were then

set out in the statutory instrument for the schemes.13

Key points Ten databases were due to be tested as part of the scheme.

However, not all were tested to the same extent. In particular, there were

difficulties in accessing Higher Education Funding Council England

(HEFCE) and Royal Mail data.

A key component of the trials was the matching process between

databases to identify people to invite to register to vote. The processes

employed could not be rolled out nationally but do allow for a greater

understanding of the requirements of any framework for national data

matching.

Two different processes were used for matching the electoral registers with

the Department for Work and Pensions (DWP) data on one hand and the

Driver and Vehicle Licensing Agency (DVLA) and education databases on

the other. These different rules make it difficult to compare the results from

the two processes.

Overview of databases

3.2 Broadly, the databases fall into two groups:

P Centric database (everyone

with a national insurance number), and the DVLA driver database (the

13www.cabinetoffice.gov.uk/sites/default/files/resources/schemes-order-draft.pdf
http://www.cabinetoffice.gov.uk/sites/default/files/resources/schemes-order-draft.pdfhttp://www.cabinetoffice.gov.uk/sites/default/files/resources/schemes-order-draft.pdfhttp://www.cabinetoffice.gov.uk/sites/default/files/resources/schemes-order-draft.pdfhttp://www.cabinetoffice.gov.uk/sites/default/files/resources/schemes-order-draft.pdf


33/159

29

Department for Transport estimated that in 2010 80% of men and 66% of

women had a driving licence)

the education and Ministry of Defence databases

3.3 Table 1 sets out which databases were included in the statutory

instrument. It also sets out the coverage of each database and a brief overview

of how they are updated. While each database contains different information,

the pilots only accessed the specific fields needed to match to the electoral

registers: name, full address and, in some cases, date of birth (so although, for

Centric database includes national insurance numbers this

information was not included in the data supplied to local authorities).

Access to the data

3.4 Between them, the participating authorities were due to test all the

databases included in Table 1. However, there were some difficulties in

accessing some of the databases, which meant that authorities were not able to

use them as had originally been anticipated.

HEFCE data3.5 HEFCE decided not to provide the data directly to local authorities, instead

restricting access to a computer screen at the Cabinet Office. This meant thatlocal authorities could not adequately compare the data against their registers

or locally held data. It also prevented them from testing the quality of the data

through contacting any of the names on the HEFCE database but not on the

register.

Royal Mail data3.6 There were delays in Royal Mail agreeing and signing the Article 4

agreement which was required before data could be transferred. By the time

that the legal agreements were in place only one pilot was still interested in thedata (Colchester). The data was therefore matched but was only available to be

sent to Colchester on 30 November when the staff at the local authority were

participating in strike action. The data was therefore not sent to Colchester

although they would not have been able to make significant use of it at that

point anyway.


34/159

30

Table 1: Databases key informationOrganisation Database Coverage of pilot data UpdatesDepartment for Work and

Pensions (DWP)

Centric All those with either a national

insurance number or a child

reference number

I daily by a

range of sources including benefits offices,

pension providers and employers

Driver and Vehicle

Licensing Agency (DVLA)

Driver database All those holding a provisional or Driver details are updated online or by form

when the driver provides the informationDepartment for Education

(DfE)

National Pupil Database

(NPD)

All pupils in state or partially state-

funded schools in England

Information is collected annually from each

school via the relevant local authorities

Department for Business,

Innovation and Skills (BIS)

Individual Learner

Record (ILR)

All learners at state-funded further

education institutes

Information is collected at set points during

the year

Student Loans Company

(SLC)

Customer database All current students with a loan or

grant

Student initiated: details are updated online,

by phone or by form

Ministry of Defence (MoD) Joint Personnel

Administration

All service voters Ad hoc updates by individual service voters

Anite housing database All addresses classed as service

family accommodation

Centrally managed by Anite


35/159

31

Table 1: Databases key information (continued)Organisation Database Coverage of pilot data UpdatesImprovement service14 Citizens Account All individuals who chose to

maintain an electronic record

Ad hoc updates by individuals and updates

linked from other sources (where consent

has been given)

Higher Education Funding

Council England (HEFCE)

Higher Education

Statistics Agency (HESA)

individualised student

record

All students at state-funded higher

education institutions

Information is collected annually from higher

education institutions

Royal Mail Change of Address All those who register their change

of address with the Royal Mail

Information is provided directly by home

movers close to the time they move house

14 The Improvement Service is a partnership between the Convention of Scottish Local Authorities (COSLA) and the Society of Local Authority Chief Executives

(SOLACE). It is a company limited by guarantee.


36/159

32

The matching process

3.7 The first step in the process was for participating areas to provide their

electoral registers (either to the data-holding organisation or to the Cabinet

Office) for matching. The matched data was then returned to the local

authorities, who used the data to decide who to contact to register. In practice

this meant that, following interrogation, local authorities followed up names

found on the national databases and not on their register. Figure 1 below

illustrates how the process of data matching broadly worked.

Figure 1: The pilot processAll or some of electoral register extracted by pilot authority

produced detailing results of process

results against local data sources for an additional level of check

Sent as an encrypted ZIP file by secure

email to DWP/Cabinet Office or MoD

Sent as an encrypted ZIP file by secure

email to relevant pilot authority

Identify names to be followed up either to encourage registration

or to query validity of existing registration

Follow up activity e.g. issuing letters or sending canvassers

Responses resulting in new registration, deletion, amend or no

action


37/159

33

3.8

the national database records they had been matched against with the

accompanying match score (see below for further information). It also contained

those records which did not match either register entries or national database

entries.

Data transfer arrangements3.9 Throughout these pilot schemes data was transferred as attachments by

secure email. However, one data-holding organisation stressed to us that this

was not their preferred method for sending sensitive data and that their future

involvement in any further piloting would be at least partially dependent on more

robust data transfer processes being put in place.

3.10 In addition, the use of email attachments led, in one instance, to the match

file for one local authority (containing electoral register entries and data from one

national database) being returned to another. In this case the mistake was

swiftly identified and the local authority that wrongly received the data deleted

the file. However, it is clearly important that any future data matching system

(potentially involving hundreds of local authorities) avoids such errors.

Variation in matching processes

3.11 Although Figure 1 provides a generic step-by-step guide to the data

piloting process there were three separate processes in relation to theaccessing and matching of the registers to the different databases:

For the match with data from the DWP Centric database, the matching

process was carried out by the DWP and the results provided to the local

authorities.

For the match with the MoD data, the matching of personnel records to the

register was completed by the MoD and the results provided to local

authorities.

The matching with all other databases was carried out by staff within the

Cabinet Office and the results were returned to authorities in a single

.

Matching process for the Department for Work and Pensions3.12 The process used for matching against the DWP Centric database was a

new, previously untested approach, designed by the Cabinet Office. It used the

first name (F), surname (S), first line of address (A) and the postcode (P) from

register entries in order to match them against the DWP Centric database. Each


38/159

34

examples where there is a small difference in spelling, where one names sounds

like another or where one name includes another, e.g.

3.13 A score was assigned to a match depending on the interaction of the four

variables and whether they were exactly or fuzzily matched. In the list below

fuzzy matches are denoted by an apostrophe. So, for example, a score of 80

would be awarded for a fuzzy match first name and surname and an exact

match postcode and first line of address.

F S P A = 100

= 99

= 95

= 94

= 90

F S A = 85

F P A = 50

F S P = 65

= 60

= 45

= 40

P A = 20

= 10

3.14 The matching process used for all the other databases, apart from those

owned by the MoD, was explained by the Cabinet Office as follows:

The matching process used for all the other databases (apart from

MoD) was based on a complex matching process contained in an IBM

proprietary product (IBM was commissioned to provide the central hub

services). This approach either marked each record as unmatched orgave it a score ranging from 81 to 118. The matching algorithm used in

this process was very sophisticated but (unlike at DWP) tended only to

identify firm matches in the great majority of cases.

Impact of variation3.15 For the purposes of evaluating the comparative strengths and weaknesses

of these databases in updating the electoral registers, the use of several

processes was not ideal. It also added to confusion among the local authorities

over how to interpret the data and hampered attempts to cross referenceinformation provided by the DWP with information from other databases.


39/159

35

3.16 However, the most important difference was that the process used for the

DWP Centric match was less strict than that used for matching against the DVLA

and education databases. As a result matches against the DVLA, for example,

which would have matched (at least partially) through the DWP process werenot counted as matched for DVLA. This has clear implications for the

comparability of the results from the DWP match and the other databases.


40/159

36

4 Pilot authorities: overview

and emerging issues4.1 This chapter sets out details of each of the pilot schemes in terms of the

databases they accessed and the groups or areas they targeted. It goes on to

consider some of the key issues identified by the local authorities in the delivery

of the data matching pilots.

Key points

There were 22 data matching pilots testing various combinations ofdatabases.

The complex nature and format of the data returned to local authorities

highlighted the need for good data management and analysis skills. Many

of the pilots either had these skills available within the local authority or

recruited additional staff using Cabinet Office funding. However, several

did not and struggled to use the information provided.

The process, as tested in these pilots, was labour intensive with significant

work required to analyse the data. Those involved felt that the level of work

required would not be sustainable in the future.

A number of pilot authorities were able to use locally-held data to

interrogate the data received from the national databases. The results of

this activity suggest that there is scope for more use to be made of local

data both to complement any future national data matching and to

improve accuracy and completeness in general.

Overview4.2 Twenty two local authorities were selected by the Cabinet Office to take

part in the data matching schemes. Table 2 provides the full list of each

participating authority and which databases they planned to access. It also

outlines whether or not they matched their full register or part of their register,

and which groups they were targeting as part of the data matching scheme.

4.3 These differences should be remembered when considering the results

from each pilot. In addition, some local authorities opted to conduct a targeted


41/159

37

follow up either in specific areas or with specific groups, while others followed

up with random sample of names from across their area. The results from these

different exercises are not, therefore, always comparable.

4.4 and results is provided in the

profiles in Appendix A.


42/159

38

Table 2: Data matching pilots overviewLocal authority Database(s) requested Target groups AreaBlackpool DWP Centric, NPD, Royal Mail,

ILR, HEFCE, DVLA

Empty properties in low responding areas Six electoral wards

Camden DWP Centric, ILR, SLC, NPD,

HEFCE

Students, young people and the mobile population Whole register

Colchester DWP Centric, SLC, MoD,Royal Mail

General under-registered and service personnel Whole register

Forest Heath DWP Centric Young people and the mobile population Whole register

Forest of Dean DWP Centric, DVLA, ILR, NPD,

HEFCE, DVLA

Attainers Whole register

Glasgow DWP Centric, DVLA, SLC Students, young people and the mobile population Two electoral wards

Greenwich DWP Centric, DVLA, ILR, NPD,

HEFCE, MOD

Young people, BME groups and those under-

registered for financial reasons

Whole register

Lothian DWP Centric General under-registered Whole register

Manchester DWP Centric, DVLA, SLC Empty properties, students and BME groups

Newham DWP Centric Young people and the mobile population Whole register

Peterborough DWP Centric Seasonal workers and those living in houses of

multiple occupation (HMOs)

One electoral ward

Renfrewshire Citizen Account General under-registered Whole register

Rushmoor MoD Service personnel Service voters list

Shropshire MoD Service personnel Service voters list

Southwark DWP Centric, Royal Mail General under-registered Three electoral wards


43/159


44/159

40

Emerging issues

4.5 There are several key evaluation findings which relate to the different skills,

capacity and experience of the local authorities involved in the pilots.

Skills

4.6 There was significant variation between the pilots in terms of the skills

available within the local authority as a whole. Each pilot was generally led by

electoral administrators, who used support available to them within their team,

within the local authority or externally.

4.7 The pilot authorities divided into three groups with regard to how they

managed the data:

Those who managed the pilot within their existing electoral services team

and who had access to existing data management or IT expertise within

the wider local authority

Those who intended to manage the pilot within their existing electoral

services team with no dedicated local authority data handling team

Those who used pilot funding to recruit additional, temporary staff for the

purposes of data analysis and data management

4.8 Those authorities with data management support were able to interrogate

the information provided to a much greater extent, while others were unable to

do so and used the data as it was provided. In the absence of data analysis and

-

volumes of data but with some authorities receiving hundreds of thousands of

lines of data there was a clear need to automate some of the process. In

extreme cases the electoral services team were simply overwhelmed by the

volume of data provided and in the absence of data management skills coulddo little with the information. For example, one area explained:

Because of the unexpectedly large numbers of apparently probable new

identities found in the data matching, and the quantity and difficulty of

dealing with such large volumes of data, it was decided to limit the

number of potential electors


45/159

41

4.9 Another area noted:

Some of the technical issues relating to local management of received

data, and conversion to a form that could be used for our purposes,

needed considerable time to deal with, and suggests there is a need to

develop a range of data management knowledge and skills not available

within the current local electoral services team.

4.10 In a couple of pilots, as a result of delays or a lack of skills and capacity,

no follow-up activity was undertaken even where data matching identified

people potentially eligible but not on the registers.

4.11 Electoral administrators in the authorities with good data management

support were also clear that they could not have coped with this volume of data

in the absence of that support as they do not have the skills themselves.

4.12 Although any future roll-out of data matching across the country would not

follow the process used in these pilots, any process which requires electoral

administrators to manipulate and analyse data would require a change in the

skill sets of many electoral registration teams. The closer any data matching

process gets to an automated provision of lists of potential new electors (which

can be easily integrated into the software used to manage the registers), the

smaller the required change will be.

Capacity

4.13 In addition to the skills required to make full use of this data there was a

more general need for additional capacity within many teams. The volume of

data provided meant that several areas could not do as much with the

information as they had originally intended. This was exacerbated by the delays

which pushed the process further into the canvass period.

4.14 Several pilots raised concerns that in its current format the process of data

matching was too labour intensive for regular use. Many also pointed out that

they are currently facing significant cuts in budgets and as a result they can only

see a future for data matching if it is able to improve registration with no

additional, or a reduced, financial burden for the authority.

4.15 This is significant not just because local authorities are unlikely to be

expanding their electoral services teams in the near future but also because it

calls into question the likelihood of significant resources being devoted to


46/159

42

training existing staff or providing new data management software without clear

evidence that it may lead to cost savings later. For example, one area reported

that:

Currently there are too many records to make this a viable exercise with

the resources available.

Local data matching

4.16 Related to the availability of relevant data management skills is the

variation between the pilots in their existing use of locally-held data to assist with

electoral registration. EROs have powers to inspect other data held by the local

authority for the purposes of maintaining the register and the vast majority of

EROs make some use of information, e.g. from council tax records. However,

there is substantial variation in both the range of data accessed and the

methods by which it is used.

4.17 For example one of the pilots, Newham, has developed a system for use

across the authority, which draws together information from sources including

council tax, housing benefit, libraries and leisure centre records to create a

searchable electronic database of residents in the borough. But another similar

authority only regularly accesses council tax records provided in Excel

spreadsheet format. The different starting points of these two pilot authorities

coming into the pilot process meant that while the first could draw on theexpertise built up during the development of their in-house system, the latter

needed

4.18 This also meant that Newham could check the data provided through the

pilot against the data they already held on their systems, gathered locally. As a

result Newham only issued letters to names which were suggested by DWP

Centric and could be corroborated by local data. As the data in Chapter 5indicates, this did not result in significantly higher registrations than other areas

but, unlike several pilots, they received very few responses indicating that theperson written to was no longer resident.

4.19 This evidence is not conclusive but it does suggest that there is significant

scope for more use to be made of local data.

Conclusion

4.20 This chapter has provided an overview of the pilot schemes, setting out the

databases they accessed and the groups or areas they targeted. It has alsoconsidered the key issues that cut across all the pilot areas the skills and


47/159

43

capacity of electoral services teams as well as the importance of good use of

local data.

4.21 The following chapters go on to consider the results of the matching

exercise and follow up, in turn, for each national database.


48/159


49/159

45

The absence of a unique identifier attached to each address on the

national databases was a key issue for the pilots. These would have

allowed for a more straightforward matching process for local authorities.

Many of the potential new electors suggested by the match with the DWPCentric database proved to be based on out-of-date or incorrect

information. The problems posed by this could have been reduced by the

inclusion of the date when the DWP record changed something which

DWP were willing to provide but were not asked to do so.

The absence of nationality information meant that several pilot authorities

conducted follow up with people ineligible to register. The scope of this

issue is not clear from these pilots and will have varied depending on the

demographic nature of the local authority area.

-legislative

scrutiny of their policy on individual electoral registration, to use national

data sources to verify the identity of electors has not been tested by these

pilots. Further piloting would be required to ensure that the advantages

and disadvantages of these proposals are understood.

Matching results

5.2 Eighteen of the 22 pilot areas accessed data from the DWP Centric

database. The results of the matching process are presented in Table 3 and are

based on the data supplied by local authorities.

5.3 The results vary considerably across different local authorities and this is

partly explained by the different approaches adopted by each pilot area (as set

out in Chapter 2). For example, the Stratford pilot, with the highest reported level

of match, focused its attention on attainers and the over-70s which made it

more likely they would see a higher match level (as both groups are less likely to

change address frequently).

5.4 Also, as mentioned above, the match score at which a pilot accepted a

result as a match varied significantly and this has led to greater variation in the

data returned (see Chapter 3 for a full explanation of the match process and

scores). For example, Peterborough appears to record a relatively low match

level (54.7%) but they accepted only those records which scored 99 and above

as a match. Wigan records a high match rate (82.4%) but accepted all records

scoring 45 and above as a match. If a score of 65, between those two levels


50/159

46

was accepted as a match,

5.5 Those pilots with the ability to fully interrogate the data found that the

likelihood of a match did not necessarily increase with the score. For example,

Colchester found that while those records that scored 65 were very likely to be

real matches, there were many with scores above that which proved to be false

positives, i.e. they received a high match score but, on checking, were found not

to be true matches. Where a local authority had this the data

received from DWP, the match rate cannot easily be compared like-for-like with

thos .

5.6 The data presented in Table 3 should therefore be treated with some

caution. Nonetheless the results show:

There is substantial variation across local authorities regarding the level of

match between the electoral registers and the DWP Centric database

ranging from 45.7% to 85.3%.

In total, 1,925,336 register entries were sent for matching and 1,370,006

were found on the DWP Centric database.15

That equates to a match level of 71.2%.

The percentage of register entries sent for matching but not found on DWP

Centric varied across local authorities from 12.4% to 47.6%.

The number of records foun

Data Matching Pilot Evaluation

Documents

Transcript of Data Matching Pilot Evaluation