Methods Paper Five headline indicators of national success ... · 6 Methods paper: Five headline...

Written by: Karen Jeffrey and Juliet Michaelson Contributions from: Saamah Abdallah, Annie Quick, and Hanna Wheatley New Economics Foundation www.neweconomics.org [email protected] +44 (0)20 7820 6300 @NEF Registered charity number 1055254 © 2015 The New Economics Foundation

Methods Paper Five headline indicators of national success A clearer picture of how the UK is performing

http://www.neweconomics.org/

mailto:[email protected]

2 Methods paper: Five headline indicators of national success:

2

Contents

Contents ..................................................................................................................... 2

Introduction ............................................................................................................... 3

Part 1: Identifying domains for the indicator set .................................................... 4

1.1 How many domains? ........................................................................................ 4

1.2 Identifying five general domains to represent national success ........................ 4

Part 2: Identifying indicators for each domain ..................................................... 21

2.1 Overview of indicator selection process .......................................................... 21

2.2 Living standards indicator ............................................................................... 22

2.3 Employment .................................................................................................... 27

2.4 Subjective wellbeing ....................................................................................... 30

2.5 Environmental impact...................................................................................... 34

2.6 Economic inequality ........................................................................................ 39

2.7 Public goods and services .............................................................................. 42

2.8 Considerations about the indicator set overall ................................................ 49

Part 3: Indicator calculations ................................................................................. 52

3.1 Good Jobs ....................................................................................................... 52

3.2 Wellbeing ........................................................................................................ 56

3.3 Environment .................................................................................................... 56

3.4 Fairness .......................................................................................................... 60

3.5 Health.............................................................................................................. 60

3.6 Trends over time ............................................................................................. 61

Appendix 1: Individuals consulted throughout the process ............................... 62

End notes ................................................................................................................. 63


3

Introduction

The aim of the five headline indicators of national success proposed by the New

Economics Foundation (NEF) is to align political priorities in the UK with the issues

that the British public has said matter most. In the main report, available at

www.neweconomics.org/headlineindicators, we call on the Office for National

Statistics (ONS) to adopt and refine our suggestion for five headline indicators of

national success, in order to encourage policymakers, the media, and the electorate

to reimagine what it means for the UK to be successful in terms that go beyond

narrow economic measures.

In Part 1 of this paper, we describe the process of identifying the broad topics or

‘domains’ to be covered by the set of indicators. In Part 2, we describe the process

of selecting a headline indicator to represent each domain, to give a high-level

summary of general national performance against each domain. In Part 3, we

describe in detail how the absolute values and trends over time presented in the set

of indicators in the main report were calculated.

Throughout the process of developing the set of indicators (summarised in Figure 1),

we consulted a range of organisations from across civil society and the business

sector, as well as academic experts. Engaging this advisory group in the work was a

crucial aspect of the project, which helped to ensure that the final set of indicators

was informed by a range of perspectives and expertise.1 We were also guided by the

results of three UK consultation exercises, which provided evidence on public

priorities: the ONS national debate on Measuring National Well-being;2 UK users’

rankings of the Organisation for Economic Co-operation and Development (OECD)

Better Life Index measurement areas;3 and the public consultation informing the

development of the Oxfam Humankind Index.4

Figure 1. The process of developing the set of five headline indicators of national success.

Refine

indicators in

consultation

with the

advisory

group and

through

user testing

Finalise the

indicators

considering

balance

across the

set

Set out a

conceptual

framework

to underpin

the indicator

set

Review

domains

included in

existing

initiatives

Shortlist

domains using

criteria, public

consultation

results, and in

consultation

with an

advisory

group

Identify

headline

indicators to

represent

each

domain

http://www.neweconomics.org/headlineindicators


4

Part 1: Identifying domains for the

indicator set

In Part 1 of this paper, we detail the steps that we took to identify the domains in our

indicator set, including how we set the maximum number of general indicator topics

or ‘domains’, and how we identified the domains themselves.

1.1 How many domains? In our view, many previous indicator initiatives have failed to gain political traction

because they have included too many indicators to attract sustained public and

media attention. A major study into bringing alternative indicators into policy-making

supports this position, citing the lack of a simple, attractive message as one of the

key barriers to the effective use of alternative indicators in policy-making.5 Work by

psychologists also suggests that anything beyond three to five ‘chunks’ of

information begins to be too much for an individual to hold in working memory and

process at any one time. 6 (Note that three to five is a downwards revision of the

initial theory posited in 1956 which suggests that a maximum of five to nine items

can be processed at once.7) The number of indicators to be included in the set was

a major consideration, which governed a great deal of the decisions made in

developing the indicators. Given that a key aim of the project was to reach a wide

audience, and that this would be best achieved through gaining media attention, we

consulted several journalists on the optimal size of our indicator set in order to

maximise its chances of gaining headline media coverage. The journalists

unanimously agreed that from their perspective, the smaller the set, the easier it

would be to write compelling headlines about it.

Therefore, we decided that our proposal for a small set of resonant measures should

include no more than five indicators.

1.2 Identifying five general domains to represent national success

To reach a set of recommended domains as systematically and as transparently as

possible, we took the following steps:

1. Set out a broad conceptual framework to underpin the selection.

2. Reviewed domains included in existing high-profile headline indicator

initiatives.

3. Created a shortlist of suggested domains.

4. Cross-checked the shortlisted domains against UK public consultation results.


5

5. Identified criteria to inform domain and indicator selection.

6. Assessed the shortlisted domains according to the criteria.

7. Tested the shortlist with an advisory group.

We were aware that the process of selecting the indicators themselves, described in

Part 2, might have implications for the inclusion of the domains. Therefore the

domains identified at the end of Part 1 are not identical to the final domains included

in the indicator set.

1.2.1 Setting out a broad conceptual framework

Setting out a broad conceptual framework is recommended in the literature on

headline indicator development.8 Doing so required us to be explicit about the

approach underpinning our selection of domains, and the rationale for the set as a

whole.

We used NEF’s established framework for measuring progress (Figure 2), which

takes the broadest possible view of societal progress. It regards the three

fundamental spheres of importance to humanity as: natural resources, on which all

human life depends; the ultimate goals which humans aim to achieve; and the

human systems by which those resources are used to achieve these goals.

Figure 2. Conceptual framework for measuring the progress of UK society.

The framework suggests that within the three spheres, the key areas of focus should

be: the sustainable use of resources, the goals of high wellbeing and equality, and

well-functioning economic and social systems.9 We aimed to spread the five

domains of the headline indicator set across these three spheres in order to produce

a coherent set of domains which collectively captures the goals set out in the

conceptual framework as a whole.

Natural resources

Sustainable use of

resources

Goals

High wellbeing

and equality

Human systems

Well-functioning economic &

social systems


6

1.2.2 Review of domains included in existing high-profile headline indicator initiatives

We identified 13 high-profile headline indicators initiatives10 and categorised the

indicators included in each initiative according to general domains. We then counted

how often each domain was included in the 13 initiatives. Although the initiatives

contained several hundred indicators in total, the indicators fell into 18 domains

within our categorisation. The results of this exercise are displayed in Figure 3, and

the detail of the features we categorised each domain into are listed in Table 1. In

some cases, domains appeared more than once within an indicator set, or very

broad domains were used, which encapsulated more than one of the domains in our

categories – in these cases, each time an instance of one of the domains occurred

within a single initiative, it was counted separately.

Figure 3. Domains that emerged from the review of indicator initiatives.

1

4

4

4

4

5

5

6

6

7

8

8

9

10

10

11

11

14

Population growth

Housing

Local environment

Time use

Transport and access

Personal relationships and networks

Crime and safety

Culture

Subjective wellbeing

Employment

Equality

Governance

Economic activities

Social and community

Education

Health

Living standards

Environmental impact

Number of appearances in 13 reviewed initiatives


7

Table 1. Features included within each domain.

Domain Features covered by the domain

Crime and safety Feeling safe, reported levels of crime

Culture Cultural heritage and participation in cultural activities, including arts,

sports, artisan skills

Economic activities Economic growth, public debt, economic productivity, innovation, public and

private investment

Education Educational participation and attainment, skills

Employment Rates of employment, unemployment, long-term unemployment

Environmental impact Climate, energy, ecosystems, natural resources, nature, sustainability,

biodiversity

Equality Equality of income, wealth, gender and age groups

Governance Democracy, trust, safe and peaceful societies, strong institutions

Health Health status, life expectancy, provision of health services

Housing Size, quality and satisfaction with housing

Living standards Income and wealth

Local environment Air quality, green space, noise pollution

Personal relationships

and networks Family and close friends, networks of support

Population growth Growth in the size of the population

Social and community Civic and political engagement, community relationships, trust, belonging

Subjective wellbeing Life satisfaction, positive and negative emotions, spirituality

Time use Leisure time, work-life balance

Transport and access Local services and facilities, transport

1.2.3 Creating a shortlist of suggested domains

Using the domains that emerged from the indicator initiatives as our starting point,

and guided by our conceptual framework, we next created a shortlist of 9 domains

for further consideration from the 18 identified, summarised in Table 2.

At this stage, we excluded ‘Economic activities’ and ‘Employment’ from the shortlist

on the basis that GDP, the inflation rate, and the unemployment rate currently

receive a huge amount of attention in the media and in political debate, and therefore

did not need to be included within our set of headline indicators. In Part 1.2.8, we

describe how we reconsidered these decisions at a later stage of the selection

process. We also excluded ‘Population growth’, on the grounds that it only appeared

once in the review of existing initiatives, and because no consensus has been

reached regarding whether or not population growth is a desired outcome.

We also grouped the following seven domains under a new umbrella domain, titled

‘Public goods and services’: health, education, culture11, local environment, transport

and access12, housing, crime and safety.13 Although many of these domains ranked

highly on their own terms, we felt there was a clear justification for grouping them,


8

given that they sit together within our conceptual framework as the elements of

social systems which cannot be adequately provided by the market alone, and for

which there is therefore a strong role for state provision, support, and/or regulation.

We were aware that the headline indicator we would eventually select to represent

this domain might not directly reflect all the different aspects within the domain, but

might instead be selected to stand for the domain as a whole by relating to key

exemplar aspects of it.

Equality-related domains were among the most frequently occurring in our review of

headline indicator initiatives. Among the instances of equality-related domains, the

most prevalent issue, with strong links to all the other issues, was that of economic

inequality. For this reason, we decided to focus on economic inequality, rather than

other types of inequality.

Table 2. The rationale for including or excluding domains from our shortlist.

Domain Rationale for inclusion in or exclusion from the shortlist

Crime and safety Grouped: Under the new ‘Public goods and services’ domain

Culture Grouped: Under the new ‘Public goods and services’ domain

Economic activities Excluded: Already receives a huge amount of attention

Education Grouped: Under the new ‘Public goods and services’ domain

Employment Excluded: Already receives a huge amount of attention

Environmental impact Included

Equality (economic

inequality) Included

Governance Included

Health Grouped: Under the new ‘Public goods and services’ domain

Housing Grouped: Under the new ‘Public goods and services’ domain

Living standards Included

Local environment Grouped: Under the new ‘Public goods and services’ domain


and networks Included

Population growth Excluded: Appeared only once in our review of existing initiatives, and consensus has not been reached on whether population growth should be regarded as a desired outcome or not.

Social and community Included

Subjective wellbeing Included

Time use Included

Transport and access Grouped: Under the new ‘Public goods and services’ domain


9

1.2.4 Cross-checking the shortlisted domains against UK public consultation results

Having compiled a shortlist of nine domains, we cross-checked the domains against

the results of the ONS, OECD, and Oxfam public consultations to guide our domain

selection towards what the UK public has said matters most to them. At the time of

writing, these consultation results are four, two, and three years old, respectively. We

would have liked to carry out a more up-to-date consultation with a representative

UK sample (none of the previous consultations were with fully representative

samples), but we were unable to acquire funding that would allow us to do so.

However, the broadly similar results of the three consultations provide reassurance

that the results reflect genuine public priorities.

Cross-checking our shortlist of domains against the results of the public

consultations, we found that the shortlist broadly reflected the results of the

consultations. To do this, we used the parts of the consultation results that produced

a numerical ranking of domains, hence we used the ONS national debate results

from the survey element of that consultation. The full comparison between our

shortlist of domains and the domains which emerged from the public consultations is

detailed in Table 3. It shows all of our shortlisted domains represented among the

public consultation results, apart from the ‘Equality (economic inequality)’ domain.

But we also took account of the fact that the ONS consultation used a number of

other methods of collecting public views, which produced important domains not

reflected in the survey results. In particular, the National Well-being Framework

produced by the ONS drew on the national debate results to describe

‘equality/fairness’ and ‘sustainability issues over time’ as cross-cutting issues which

‘are part of national well-being measures’.14 This provided strong support for our

inclusion of the ‘Equality (economic inequality)’ domain on the shortlist.

1.2.5 Identifying criteria to inform domain and indicator selection

We developed a set of evidence-based criteria, against which possible domains –

and later, candidate indicators – could be assessed in order to test their suitability,

and potential contribution to the aims of the indicator set. We developed the criteria

by drawing on previous research by NEF and others, including a major pan-

European study on bringing alternative indicators into policy-making.15,16,17

The criteria for domain selection state that domains must:

Matter to people: In a set of indicators designed to reconnect politics to what

matters most to people, the key domains should be those which have been

stated to be of greatest importance to considerable proportions of the public,

and to resonate with their concerns and aspirations.


10

Table 3. Rankings from UK indicator consultations and mapping to shortlist domains (longlist domains in brackets)

ONS National Debate Survey ranking

Mapping to shortlist domains

OECD Better Life Index UK ranking


Oxfam Humankind Index ranking based on domain weightings*


1 Health Public goods & services 1 Life Satisfaction


1=Physical and mental health Public goods & services

2 Relationships with friends and relatives

Personal relationships 2 Health Public goods & services

1=Affordable, decent, and safe home

Public goods & services

3 Economic security Living standards 3 Work Life Balance

Time use 3 Clean and healthy local environment


4 Education and training

Public goods & services 4 Education Public goods & services

4=Satisfying work (Employment)

5 Environment Environmental impact 5 Environment Environmental impact

4=Relationships with friends and relatives


6 Job satisfaction (Employment) 6 Jobs (Employment) 6=Safety Public goods & services

7 Crime Public goods & services 7 Housing Public goods & services

6=Access to green and communal spaces


8 Ability to have a say on issues

Governance 8 Income Living standards 8=Secure work and suitable work (Employment)

9 Relationship with a spouse or partner

Personal relationships 9 Safety Public goods & services

8=Having enough money Living standards

10 Income and wealth

Living standards 10 Community Social & community 8=Having a secure source of money Living standards

11 Personal activities, including volunteering

Time use/Social & community

11 Civic Engagement

Governance 8=Access to arts, hobbies and leisure activities


12 Cultural activities Public goods & services 12=Having facilities you need locally Public goods & services

13 Unpaid caring Indirect to Time use, and Experience of Life

12=Education and skills Public goods & services

14 Spirituality or religion

Indirect to Subjective wellbeing

12=Transport Public goods & services

12=Being part of a community Social & community

16 Access to high-quality services Public goods & services

17=Feeling good Subjective wellbeing

17=Human rights Governance


11

Be clear and easy to communicate: Domains, and what is signified by a

change in an indicator representing the domain, should be capable of being

discussed clearly and simply by non-experts, and the set as a whole should

not exceed five indicators in order to be clear and memorable. Outcomes

which cannot be easily communicated and understood are less likely to

achieve the media attention and the public awareness needed to stimulate

political action.

Be far reaching: Individual domains should be far-reaching, meaningfully

representing as much of what is set out in one or more spheres of the

conceptual framework as possible. Issues which are important but narrow will

not have a place in our set of five. Each domain should also be

complementary to the others in the set, drawing attention to previously

unaddressed elements of the conceptual framework. This way, the set as a

whole will also be far-reaching.

Be able to be influenced by policy: The domains should cover issues for

which people feel able to blame or praise politicians. As such, this criterion

also requires selection of domains for which the changes resulting from

politicians’ actions are perceptible within a relatively short timescale – even if

the changes reflect goals which will only be achieved in the long term (such

as environmental sustainability).

Be measurable: In order to be included in this set, each domain – and

meaningful changes within it – should be able to be broadly represented by a

single headline indicator.

The criterion which states that domains should be measurable using a single,

headline indicator, necessitates some consideration of possible indicators at this

stage.

The criteria for indicator selection state that indicators must:

Be representative of the overall outcome: Indicators should, as far as

possible, be representative of the overall outcome being measured, and avoid

having an overly narrow focus, which might not be representative of the

domain as a whole.

Be easy to interpret: Indicators should be avoided if it is not possible to

explain briefly and in broad terms what the direction of movement of an

indicator signifies, or how a measure has been constructed. Where indicators

bring various components together in a single measure, they should not allow

an increase in one component to act as a substitute for a reduction in another,

non-substitutable component.


12

Be comparable: Indicators should be comparable over time, and ideally,

between countries. This way, the indicators can be used to identify strong or

weak relative performance.

Be accurate: Indicators should capture what is intended by their use, and

should be able to be supported by a robust and impartial methodology.

Be timely: It should be possible to collect data for an indicator without too

much lag in availability. This is required so that the effects of policy are

reflected by the indicator without a long delay. This should be possible to

achieve in the future, assuming that investment in data collection and

indicator development is made available, even if existing data availability

means that this criterion is not currently being met.

Tensions exist between some of these criteria. For example, identifying an indicator

that is broad enough to be ‘representative of the overall outcome’, and at the same

time, simple enough to be ‘easy to interpret’ may be challenging. Similarly, one

criterion might point to a certain domain when considered in isolation, but quite a

different domain when considered alongside other criteria. For example, the criteria

that domains should be ‘far-reaching’ and cover as much of what ‘matters to people’

might be interpreted to necessitate the inclusion of personal relationships as a

domain. However, consideration of the ‘influenced by policy’ criterion might then rule

out inclusion of personal relationships as one of the five domains, as an area

typically considered to be less amenable to direct policy influence.

We took the position that failure to satisfy one criterion need not necessarily rule out

consideration of a domain or indicator. Instead, the criteria should be used to guide

selection, helping to ensure that important characteristics are considered, and

strengths and weaknesses of different options are reasonably weighed up.

1.2.6 Applying the criteria to the shortlisted domains

We assessed each of the shortlisted domains against the domain criteria to produce

a first suggestion of five headline domains for discussion with the project’s advisory

group. The results of this exercise are summarised in Table 4. Our assessment of

whether or not each domain fulfilled, partially fulfilled, or did not fulfil each selection

criterion was based on judgement, and was intended to be a prompt to stimulate

discussion with the advisory group, rather than a definitive assessment.

Based on performance against the selection criteria, we identified ‘Living standards’,

‘Equality (economic inequality)’, ‘Environmental impact’, ‘Subjective wellbeing’ and

‘Public goods and services’ as the five domains to suggest to the advisory group for

inclusion in our set.


13

Table 4. Summary of assessment of domains against selection criteria.

Ma

tte

rs to

pe

op

le

Cle

ar

an

d e

asy

to c

om

mu

nic

ate

Far-

rea

ch

ing

Influ

en

ce

d b

y

po

licy

Me

asu

rab

le

Living standards

Equality (economic inequality)



Public goods and services

Governance (quality of democracy)



Time use

= The domain was judged to meet the criteria

= The domain was judged to partially meet the criteria

= The domain was judged not to meet the criteria

The key considerations we presented to the advisory group are set out for each of

the nine shortlisted domains:

Living standards

Matters to people because it relates to people’s ‘economic capacity’ to choose

the kind of life they most value.18

The relevance and importance of living standards to so many aspects of our

daily lives, and the growing political importance of the concept, means that it

is a familiar idea which is clear and easy to communicate.

Living standards can be influenced by policy that focuses on taxation of

individuals and companies, social security and benefits, spending on public

services, how we regulate employment and pay, among others.


14

Equality (economic inequality)

Data from the British Social Attitudes survey shows that more than 80% of

people think the income gap between rich and poor is too large – and thus

that inequality matters to people.

Messages about economic inequality, such as ‘over the past 30 years the top

1% have seen a 50% increase in their share of every pound [of GDP

created]’19 are clear to communicate, and highly resonant.

Economic inequality has been found to be associated with lower social

mobility, less cohesive communities, lower wellbeing, reduced voter turnout

amongst the poorest members of society, and greater economic

instability,20,21 with other evidence suggesting that far from high economic

inequality being necessary for a competitive, well-functioning economy as is

sometimes suggested, it can in fact be harmful to this goal.22

Policy can influence economic inequality by focusing on tax reforms,

legislating on tax evasion and avoidance, and reducing the gap in incomes

between the highest and the lowest earners in society, or by focusing on

security of employment, or provision of affordable childcare.


Environmental impact is a domain which seems to matter to a majority of the

public, although not to everyone. In a survey of a representative sample of

British people, 68% said they were fairly or very concerned about climate

change.23 Environmental impact came within the top five domains of both the

ONS and the OECD consultation results.

Many terms which describe aspects related to environmental impact have

entered everyday language, including ‘climate change’, ‘sustainability’,

‘conservation’, and general use of the term ‘green’ to denote environmental

sustainability, making it a domain which, at least at a high level, is clear and

easy to communicate to a wide audience.

Environmental impact is an extremely far-reaching domain, posing a serious

threat to future economic activity,24 as well as to the sustainability of human

life on Earth.25

Environmental impact is strongly influenced by policy, with serious climate

change mitigation and adaptation efforts requiring large-scale investment in

low-carbon technologies and infrastructure, as well as policies to correct

market failures which contribute to the depletion of natural resources.

Robust and detailed measures relating to carbon and greenhouse gas

emissions have emerged during the past decade, which allow for detailed

analysis of national environmental impact in terms of climate change;


15

however, a reliable headline measure of the state of natural capital does not

yet exist.26,27


Human experience – the focus of subjective wellbeing indicators – matters to

people at a fundamental level. If we do not experience our lives as going well,

then however good they might look from the outside, it is hard to describe

those lives as really going well.

Subjective wellbeing measures are sensitive to a whole range of factors that

can be influenced by policy, such as living standards and material conditions,

employment and working conditions, economic inequality, democracy, social

relationships and networks, and time use, among many others.28,29 There is a

strong case that a subjective wellbeing indicator would make the set as a

whole more far-reaching by helping to assess the success of this wide range

of policy factors, not all of which will be directly represented in the headline

set.

Subjective wellbeing is strongly and robustly measurable, with considerable

social scientific and policy attention on how best to do this over recent years.

While there remain some public doubts over the measurability and

meaningfulness of wellbeing measures, the case for the robustness of the

measures has been supported by the ONS’s official use of them since 2011.


Most of the constituent domains grouped under this domain featured

prominently in our review of domain types and ranked highly in the results of

the UK consultations.

Using an aggregate domain to represent many different features is more

challenging to communicate than when discussing each of the constituent

elements on their own; however, the phrases ‘public goods’, and especially

‘public services’, are still very common in the media and in public debate.

Creating this aggregate indicator will require that, in order to be measurable,

either a new index will need to be created covering the set of constituent

domains, or a single headline indicator used that can be seen as representing

the full public goods and services domain. Neither is entirely straightforward,

but both are possible.


16

Governance (quality of democracy)

Governance-related domains appeared several times in our review of

indicator initiatives, but were generally not rated as very important by

members of the public in the UK consultations. However, as the ONS’s

domain of ‘ability to have a say on issues’ was ranked relatively highly (8 out

of 14), we therefore focused our attention on the quality of democracy.

The quality of democracy arguably affects the quality of outcomes across all

the other shortlisted domains.

There is a sense that democratic processes are quite distant from people’s

day-to-day lives: in 2011, 71% of people agreed that ‘[i]t doesn’t really matter

which party is in power, in the end things go on just the same.’30

There are challenges in measuring the quality of democracy – particularly

around separating people’s political preferences from assessments of the

quality of democracy.

The quality of democracy can be influenced by policy, through changes to

democratic systems such as elections, devolution, participatory mechanisms,

and party funding rules, as well as whether policies are responsive to public

sentiment and whether politicians behave in ways that build trust.


We used the domain ‘social and community’ to capture domains in our review

that were described as being about: ‘cohesive communities’, ‘being part of a

community’, ‘community vitality’, and ‘volunteering’.

‘Community’ is a familiar idea that would be clear to communicate, particularly

in terms such as ‘the strength of our communities’.

It is measurable in a variety of ways, including measures of frequency of

behaviours such as volunteering and involvement in political activity, and

subjective measures of concepts such as social trust, social norms (e.g.

willingness to help others), and sense of belonging. This creates the

possibility of a composite index to cover the various issues within the domain.

The domain did not receive high rankings within UK consultations – coming

towards the bottom of the rankings in all three consultations. Our assessment

is therefore that this domain does not matter to people as much as many of

the other domains.


17


This domain includes domains from our review which were described in terms

of ‘relationships with family and friends’, ‘social networks’, and ‘social support’.

Relationships are perhaps the most prominent issue in popular culture and

people’s own narratives about their lives, and the relationships domain was

rated highly in the ONS and the Oxfam consultations.

The prominence of relationships in our culture means that the domain is likely

to be clear and easy to communicate, being based on the familiar and highly

salient issue of how people relate to those closest to them.

Well-developed indicators of frequency of social contact, close relationships

which provide support, the extent of help and support received from wider

networks, and satisfaction with relationships can be used to measure this

domain.

This domain is less far reaching than other domains, given that it is focused

relatively tightly on immediate personal relationships. Furthermore, as we

know that such relationships are strong drivers of people’s experiences of

their lives, it is likely to be captured to a fairly large extent by the broader

‘Experience of life’ domain.

While a number of policy-relevant factors affect people’s personal

relationships, at least indirectly – for example, factors which affect the number

of hours people need to spend doing paid work, such as levels of pay, living

costs, and commuting times – this is a domain that is commonly perceived as

less amenable to direct policy intervention, and many of the direct drivers of

personal relationships are people’s own decisions and behaviour.

Time use

‘Time use’ relates to work-life balance, time pressure, and time use more

generally.

While work-life balance ranked highly in the results of the OECD consultation,

it was not present in the other two consultations, and time domains featured

only moderately in our review (4 instances across 13 initiatives).

How we use our time affects all aspects of our lives, but this means that there

are strong overlaps with many other domains.

Time use can clearly be influenced by policy – for example, through

regulations on working time, the distances and transport quality of commutes

to work, and the relationship between living costs and earnings.

The ideas of work-life balance and time pressures are relatively familiar and

thus potentially clear to communicate.


18

Time use is not straightforward to measure as a headline indicator. It requires

top-down judgements to be made about what level of time spent on a

particular activity is desirable, such as exceeding a certain number of working

hours, or having a minimum amount of leisure time. A measure based on

attitudes to work-life balance would not be relevant to the significant non-

working proportion of the population, including students, unemployed people,

full-time carers, and retired people.

1.2.7 Testing the suggested domains with the advisory group

We invited the project advisory group to discuss and critique our initial suggestion of

five domains to ensure that the final set would be informed by a range of

perspectives and expertise.

At a roundtable event hosted in London on 9 March 2015, 25 advisory group

members, mostly from civil society organisations, but also including academics and

businesses, came together to learn about the project and discuss the shortlisted

domains.

After they had read a background paper, listened to a presentation, and had some

discussion about the domains, we invited advisory group members to participate in

an informal voting exercise. Each participant was given five votes and asked to vote

for their preferred domains, selecting from among the nine shortlisted domains as

well as any ‘other’ domains they wished to suggest. The results, shown in Figure 4,

revealed general consensus in favour of the five domains suggested by NEF.

Figure 4. The results of stakeholder voting on the shortlist of domains.

0

1

1

1

1

2

3

7

12

12.5

14

17.5

0 2 4 6 8 10 12 14 16 18 20

Time use

Biodiversity

Equality (non economic)

Employment

Governance - quality of democracy





Living standards

Economic inequality


Number of votes


19

Although there was a high level of agreement with our suggested domains at this

stage, several of the environmental organisations present made the case that

biodiversity should either be represented by the ‘Environmental impact’ domain, or

included as an additional indicator. The advisory group also suggested that we

should be clearer on whether the set of indicators would replace, or complement

existing headline indicators in the economic sphere, including GDP, the employment

rate, and the government deficit.

We agreed to consider these issues during the next phase of research, which would

identify indicators to represent each domain.

1.2.8 Finalising the domains to take forward to indicator selection

Before identifying an indicator to represent each domain, we considered whether the

set of indicators should aim to replace or complement existing headline indicators

(such as GDP). We had previously struck ‘Economic activities’ and ‘Employment’ off

the shortlist of domains on the basis that our set was intended to highlight those

issues which did not currently receive enough focus at headline level.

The advisory group persuaded us that a stronger approach would be to include all

indicators worthy of headline consideration in a single set which we could then

describe as capturing all the things that really matter to people.

Considering ‘Economic activities’ and ‘Employment’ against our domain selection

criteria, we noted that a strong case could be made that both economic activity, as

represented by GDP growth, and employment, already meet all of these criteria. But

we were mindful of the path-dependency of indicators – that is, the way that those

indicators already occupying ‘premier league’ positions in political and media

attention shape people’s views about what is important.

To check the degree to which these domains really matter to people, we revisited the

results of the three UK consultations on headline measurement priorities, discussed

in Section 1.2.4. Employment featured highly in the results of all three consultations.

However, we thought it very revealing that none included overall economic activity, in

the sense represented by GDP, separate from ‘income or having enough money’

and ‘economic security’. We judged that these priorities were better represented by

our living standards domain than by an economic activity domain.


20

We therefore decided to include employment as a sixth domain to take forward to

indicator selection, but not to include economic activities (i.e., the domain most

commonly represented by GDP). The extended set of domains therefore included:

Living standards

Employment



Economic equality


Selecting these domains to take forward represented a compromise between the

desire for a very small indicator set – ideally of no more than five indicators – and the

need to cover the key priorities arising from the UK public consultations on

measuring progress. However, we were aware that the process of selecting

indicators to represent the domains might affect the final choice of domains included

in the set, as the assessment of whether each domain was truly ‘measurable’ would

not be made until its candidate indicators had been considered in detail.


21

Part 2: Identifying indicators for each

domain

2.1 Overview of indicator selection process

In this section of the paper, we outline the steps we took to identify a headline

indicator to represent each domain:

1. We created a list of indicator options for each domain, including indicators used

in existing initiatives and statistics used commonly in policy discussions, which

we then assessed according to the indicator selection criteria described in

Section1.2.5. Where necessary, we also referred back to the domain selection

criteria, where they helped distinguish between indicator candidates. At this

stage, indicator options were not specified precisely, given the number of options

considered, with precise details of indicators worked out at a later stage.

2. Informed by our assessment, we identified a working suggestion for an indicator

to represent each domain, with the exception of ‘Economic inequality’ and ‘Public

goods and services’.

3. At a meeting with the advisory group held in London on 2 June 2015, we

presented suggested working suggestions for an indicator to represent each

domain, alongside alternative indicator options, and discussed the strengths and

weaknesses of the different options.

4. We carried out semi-structured interviews with a convenience sample of 13

individuals (acquaintances or friends and family members of the research team).

Altogether, these respondents constituted a diverse range of ages, educational

levels, and nationalities, living in England and Scotland in both large cities and

small towns. The interviews were carried out on a one-to-one basis, or in groups

of two-to-three respondents to one interviewer. In addition, a group of

approximately 25 individuals (NEF supporters) also provided feedback in a

workshop setting. During the interviews and workshop, these ‘testing

respondents’ were presented with between one and three indicator options to

represent each domain, including preliminary data for each indicator, and asked

specific questions to check interpretation and understanding of what the indicator

represented. We also invited respondents to comment on the balance achieved

across the set of indicators.

5. We used feedback from the advisory group and the testing respondents,

considerations about the composition of the indicator set as a whole, and further

discussions with advisory group members to refine our final selection of

indicators.


22

We now describe the outcomes of these steps, one domain at a time.

Each domain subsection includes

a table giving a summary assessment of the candidate indicators against the

indicator selection.

details of the key points considered for each candidate indicator.

summaries of the advisory group and testing respondents’ feedback on

candidate indicators.

a final discussion showing how we responded to the feedback and the final

decision made about which indicator, if any, would be used to represent each

domain.

2.2 Living standards indicator

Table 5 summarises our assessment of candidate indicators for the ‘Living

standards’ domain against our indicator criteria.

Table 5. Summary assessment of headline indicators to represent the ‘Living standards’ domain.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

Real median equivalised household disposable income

Real median household income after housing costs

An indicator of the proportion of the population on low incomes or living in poverty

Housing affordability

Basket of goods

Housing quality

An adjusted GDP figure




= The indicator was not assessed against this criterion, given that strong failure to meet other

criteria ruled out further consideration of the indicator


23

2.2.1 Candidate living standards indicator assessments

The key points considered for each of the candidate ‘Living standards’ indicators

were as follows.

Real median equivalised household disposable income (working suggestion for

headline indicator)

Many notable economists have endorsed the use of median income measures,

(most notably, the Stiglitz-Sen-Fitoussi Commission) which avoids the distortive

effects of very high incomes that statistics based on the mean are subject to.

The adjustments made to this income measure are designed to produce an

accurate representation of people’s lived experiences: equivalisation helps to

account for the differences in cost of living for different household compositions.

At the same time, inflation adjustments enable us to make meaningful

comparisons over time. Further, because the measure counts income net of

taxes and including state benefits, policy changes to income taxes and benefits

will both be reflected in this measure.

Presented as pounds of income, this indicator has seemingly clear meaning.

However it is not entirely straightforward to interpret – as many people are likely

to think about their income in terms of their personal pay-packet, which does not

equate to income for the whole household, and others in terms of their gross

salary, which has the same problem and in addition does not relate to the post-

tax figure in this indicator.

Robust data for this indicator is released annually by the ONS, though at the time

of writing, the most recent data available was more than two years out of date.

There is a particular issue for this indicator (and other potential indicators based

largely on income) in terms of how it relates to others in the set. Although

increasing living standards is a desirable target in the short-term, striving for

infinitely increasing living standards is likely to imply higher consumption of

natural resources, and may not be compatible with environmental sustainability in

the long term. As such, if this indicator is used, it will be important to emphasise

that progress should be understood in terms of achieving balance across the full

set of indicators.

Real median household income after housing costs

This might be desirable as a means by which to capture the escalating house

prices and cost of renting, particularly in London.

However, measures of income after housing costs treat all expenditure on

housing as an essential cost, and do not capture the aspect of accumulation


24

of an asset, which mortgage-payers gain, and thereby somewhat artificially

deflate the living standards of mortgage-payers compared to renters.

Variation between individuals’ preferences regarding the type, location, and

therefore price of housing, further challenge interpretation and comparability

of this indicator, particularly given the huge variation of housing costs around

the country.

An indicator of the proportion of the population on low incomes or living in

poverty

While clearly an important issue, these and similar indicators would exclude

much of the experiences of the full income distribution, and therefore, would

not be representative of the overall domain.

Housing affordability

We considered using a measure which shows the difference between typical

incomes and the affordability of buying a typical starter home. An indicator

such as this would have the benefit of having a clear target, for example, that

housing should be affordable, in line with incomes. But being focused on

housing means it would only illuminate one aspect of living standards. Given

this, and the difficulty in capturing the quite different financial implications of

renting versus owning a home, as well as the wide variance in house prices

across the country, and differing individuals’ preferences regarding housing,

we decided against the use of such a measure.

Basket of goods

Living standards indicators based on basket of goods measures, which show

the cost of a bundle of certain goods which people are likely to buy have been

used by other initiatives. However, we ruled this type of measure out because

research which has produced these sorts of measures, such as the Minimum

Income Standard, has done so by producing a number of different baskets for

different household compositions.31 We judged that it would be difficult to

translate this into a headline indicator to which different people from different

household types could easily relate, unless done in terms similar to that of the

Living Wage, which is based on a weighted average of household types.32

However, this then moves the measure away from a basket of goods measure

and becomes an earnings measure, relevant only to those in work.

Housing quality

Differing expectations about what constitutes a good quality home are difficult

to address, and using an objective measure, such as overcrowding, means


25

ignoring many other manifestations of poor housing conditions, as well as

being skewed by under-occupancy figures.

In addition, housing represents just one aspect of living standards, whereas

income provides a better guide to living standards overall.

An adjusted GDP figure

Although this could be an interesting and eye-catching measure, adjusting a

standard GDP measure upwards and downwards for social and

environmental impacts, we ruled it out based on our experience working on

measures of this type such as the ISEW,33 as it is likely to be fairly

complicated to calculate and therefore explain.

2.2.2. Advisory group input

The advisory group expressed an interest in using an income measure calculated

after housing costs. However, after some discussion of the challenges described,

and the communication issue it could present given that people tend not to think

about their income in these terms, we ruled this option out.

The group were then broadly convinced of our working suggestion to use ‘Real

median equivalised household disposable income’ as the headline indicator of the

‘Living standards’ domain, with the following caveats:

We should consider how best to address the environmental trade-off of

highlighting an increase in incomes as positive performance.

We should consider how meaningful the absolute number presented would be

to people.

We should attempt to find an alternative official data source, which is timelier.

2.2.3 Testing respondents’ input

To test the indicator of real median equivalised household disposable income with

the testing respondents, we presented it using our preliminary calculations as: ‘Real

incomes: The typical after-tax income in the UK is £23,240 per person. 1.6%

deterioration.’

Testing revealed that:

Overall, respondents felt that living standards was an important issue to

cover.

The technical meaning of ‘real’ (i.e., inflation-adjusted) was not widely

understood by those interviewed.

Respondents generally felt that, without capturing the context of what it costs

to live, the absolute value was quite meaningless.


26

Some respondents noted that the deterioration in incomes we reported was at

odds with recent news coverage they had seen (a consequence of having

used an untimely data source for a topic that receives frequent and timely

news coverage).

We also tested an international comparison of the measure. It quickly became

apparent that making an international comparison would require a strong explanation

of the theory behind ‘purchasing power parity’, which could be quite challenging to

do effectively.

2.2.4 Deciding on a living standards indicator

In considering how to make an absolute income figure meaningful, we were drawn

back to considering basket of goods measures given their basis in real living costs –

but as noted above, the only viable way of doing so as a measure applicable to

multiple household types was via a Living Wage type measure, which applied only to

employees. Among candidate indicators for the ‘Employment’ domain, the concerns

about the overlap between job satisfaction and the satisfaction with life Wellbeing

indicator, were leading us to prioritise an indicator of employment and job quality, to

include earnings.

The final decision was therefore not to include a stand-alone indicator of living

standards in the set, but to ensure that the issue of living standards was covered

by the employment indicator, which, by incorporating an indicator of pay into it, would

become an indicator of both employment and living standards.


27

2.3 Employment

Table 6 summarises our assessment of candidate indicators for the ‘Employment’

domain against our indicator criteria.

Table 6. Summary assessment of candidate headline indicators to represent the ‘Employment’

domain.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

The proportion of the labour force in satisfying employment

An indicator combining the employment rate

with objective indicators of job quality

A measure of earnings

Percentage of the labour force without satisfying employment

Employment rate multiplied by underemployment rate

The unemployment rate or employment rate




2.3.1 Candidate employment indicator assessments

The key points considered for each of the candidate employment indicators were as

follows.

The proportion of the labour force in satisfying employment (working

suggestion for headline indicator)

The satisfaction element of this indicator asks people to rate how satisfied

they are with their job, allowing the individual to weigh up the different

elements of their job that matter most to them. The individual is free to decide

how important different factors such as salary, hours, conditions, relationships

at work, etc., are to them. By combining this judgement with objective data on

employment, the indicator manages to represent a broad domain, including

quality not just quantity of employment, without having to create a complex

index.


28

Job satisfaction is one of the headline measures included in the ONS National

Well-being Programme. As such, it is gaining credibility as a measure, and

updated data is released each year. However, there are potential difficulties

with the overlap between job satisfaction and overall life satisfaction as the

likely indicator for the ‘Subjective wellbeing’ domain, and a sense that using a

measure which included job satisfaction would not create enough balance

across the indicator set as a whole.

Although the time series for job satisfaction goes back to 2002, a change in

methodology in 2009 resulted in a significant difference in responses before

and after the methodological change. As such, using data from 2008 or earlier

would be inconsistent, which means having a fairly short time series.

We have not yet been able to identify equivalent international data with which

to make comparisons.

An indicator combining the employment rate with objective indicators of job

quality

For example, an index might include the extent of underemployment,

earnings, hours worked, temporary contracts in addition to overall

employment or unemployment. This would be likely to produce a more

nuanced indicator of employment quality than the standard employment and

unemployment rates.

However, it could be challenging to identify which elements should be

included and how to weight their importance appropriately.

It might also be challenging for users to interpret, given the added complexity.

If the measure included earnings data, there would be a risk of overlap with

an indicator of living standards using income data.

A measure of earnings

Pay is an important feature of the quality of employment, but it does not

represent everything that it is important about a job. Given that we were also

considering including a measure of living standards based on income in the

set, which would provide information about income for a much broader section

of the population than just those in employment, and an employment indicator

that would link earnings data to employment/unemployment data, as well as

other aspects of job quality, there did not seem to be a strong case to

prioritise an earnings measure on its own.

Percentage of the labour force without satisfying employment

As an alternative to the indicator of percentage of the labour force in satisfying

employment, it would be possible to report the proportion of people in the

labour market who are unemployed, or not in satisfying employment.

However, this would not make a clear distinction between those in non-


29

satisfying employment and those who are unemployed. and would therefore

be extremely difficult to interpret.

Percentage of the labour force employed at full capacity

This indicator would multiply the percentage of the labour force in employment

by the percentage of employees who are not underemployed. Underemployed

workers are those who are employed but who either wish to work more hours

in their current role or who are looking for an additional job or for a

replacement job which offers more hours. Although a fitting indicator of one

aspect of job quality given concerns about zero hours contracts, initial

investigations revealed that despite underemployment figures currently about

3% higher than their pre-crash level, when combined with the climbing

employment rate, this trend is masked.

Our assessment was that this indicator would represent a relatively narrow

aspect of job quality, in comparison to an indicator which combined a number

of elements of job quality.

The unemployment rate or employment rate

We considered using the standard unemployment rate, or employment rate,

which are the most commonly cited employment statistics. However, our

judgement is that it is crucial that policy is oriented towards good quality jobs,

and not just ‘jobs at any cost’.

2.3.2 Advisory group input

The advisory group acknowledged the merits of using the working suggestion of an

indicator of the proportion of the labour force in satisfying employment as the

headline indicator over the other measures we presented, though they were

somewhat sceptical of using the subjective job satisfaction measure as part of the

indicator. They were, however, strongly supportive of the idea of using an indicator

which combined information about the employment rate with aspects of job quality.


To test the indicator of proportion of the labour force in satisfying employment with

the non-expert group we presented it (based on initial calculations) as: ‘Satisfying

jobs: 49.8% of people in ‘good jobs’. 3.2% deterioration.

On the whole, respondents felt that it was an important feature of life to cover in the

set. Some of the respondents commented that it would represent an improvement on

the employment rate, because it would be more difficult to manipulate this statistic by

including people in highly precarious employment, or working very few hours.

Several respondents pointed out that it was not obvious from the text supplied how

satisfaction had been measured, and that this should be clearer.


30

2.3.4 Deciding on an employment indicator

Several concerns with the job satisfaction element of the ‘proportion of the labour

force in satisfying employment’ indicator led us to decide against using it: the

scepticism of the advisory group, the lack of clarity in what it meant to testing

respondents, and the concerns about overlap with the subjective wellbeing indicator.

Based on the advisory group’s strong support for an indicator which combined

information about levels of employment with quality of employment, our final

decision was to use an indicator which combined employment level with two

fundamental and measurable aspects of job quality: pay and job security.

We decided not to include further elements of job quality in the indicator to ensure

that the indicator retained a readily-understandable meaning – we felt that ‘people in

secure, decently-paid jobs’ was a relatable concept, but that adding more elements

could compromise communicability.

2.4 Subjective wellbeing

Table 7 summarises our assessment of candidate indicators for the ‘Subjective

wellbeing’ domain against our indicator criteria.

Table 7. Summary assessment of candidate headline indicators to represent the ‘Subjective

wellbeing’ domain.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

Percentage of the population reporting 7+ on overall satisfaction with life (on a 0‒10 scale)

The proportion of the population with medium-high wellbeing, i.e., those reporting 5 and above on life satisfaction

Average life satisfaction score of the population

An aggregate of the four ONS wellbeing measures

Satisfied life years




31

2.4.1 Candidate subjective wellbeing indicator assessments

The key points considered for each of the candidate subjective wellbeing indicators

were as follows.

Percentage of the population reporting 7+ on overall satisfaction with life (on a

0‒10 scale) (working suggestion for headline indicator)

Two recent high-profile examinations of how subjective wellbeing should be

measured at national level – by the OECD and the UK’s Expert Commission

on Wellbeing and Policy – suggested that life satisfaction represents a good

overall proxy for more detailed aspects of wellbeing. They both endorse life

satisfaction as the primary core measure of subjective wellbeing. 34,35 We

concur that it is a good choice of headline indicator to represent the domain of

subjective wellbeing.

As part of the Annual Population Survey, the ONS asks people the life

satisfaction question: ‘Overall, how satisfied are you with your life nowadays?’

Respondents give their answers on a scale of 0 to 10 where 0 means ‘not at

all’ and 10 means ‘completely’. Respondents are then categorised by the

ONS as having ‘low’ (0‒4), ‘medium’ (5‒6), ‘high’ (7‒8), or ‘very high’ (9‒10)

wellbeing. The proportion reporting ‘high’ or ‘very high’ wellbeing is the key

statistic that the ONS presents in its summary of the Measuring National

Wellbeing programme, and is therefore fairly well established, and has

relatively intuitive interpretation.

Data are published annually by the ONS with a one-year lag, allowing for

timely comparisons to be made.

However, the ONS only began collecting data on life satisfaction in 2011,

which means only a short time series of data is available.

Similar life satisfaction questions and response scales are now included in

numerous surveys, allowing international comparisons to be made.

The proportion of the population with medium-high wellbeing, i.e., those

reporting 5 and above on life satisfaction

This indicator is like the previous candidate considered, with a lower reporting

threshold so that everyone with life satisfaction of 5 and above is included.

Using a fairly low threshold such as this could be a useful way to pre-empt

concerns that the set is overly ambitious. However, the percentage of the

population reporting 5+ in terms of life satisfaction is very high, at 94.4%, and

as such, there is little room for improvement on this measure, making it less

policy relevant.


32

Average life satisfaction score of the population

This indicator has all the advantages of a life-satisfaction-based measure,

discussed in relation to the first candidate indicator, but uses an average

statistic rather than a threshold-based proportion.

Average (mean) life satisfaction is the standard way in which wellbeing scores

have tended to be reported throughout their history, and this indicator has the

advantage of being expressed simply, with a clear frame of reference for what

the number means provided when the 0‒10 scale is mentioned.

There were some concerns that the indicator may not have been meaningful

at a glance, that we were keen to investigate through testing.

An aggregate of the four ONS wellbeing measures

This indicator would combine the four ONS subjective wellbeing measures of

life satisfaction, happiness yesterday, anxiety yesterday, and the sense that

what people do in their lives is worthwhile.

While this would offer a more rounded picture of wellbeing, it would not be

possible to make international comparisons and it would have a much less

clear interpretation.

An aggregate indicator of this type would require judgements to be made by

experts on how to combine and weight different elements.


Satisfied life years is a measure which multiplies life satisfaction by life

expectancy to produce a measure of the ‘average years of happy life’ which

can be expected by a country’s population. It brings in a health-related aspect

of objective wellbeing together with a subjective measure of wellbeing.

There were concerns that the ‘health’ component would be lost in the

interpretation of the measure, and that the overall interpretation of the

measure would be difficult.


The advisory group was fairly divided over our working suggestion to use the

‘Percentage of the population reporting 7+ on overall satisfaction with life (on a 0‒10

scale)’ as the headline indicator of the ‘Subjective wellbeing’ domain.

Some members of the group expressed the following concerns:

That the indicator does not seem to change much over time.

That the increasing trend in wellbeing we had found, in contrast to

deterioration on several of the other indicators, seemed to send a conflicting

message.


33

However, the group’s previous voting exercise had made ‘Subjective wellbeing’ the

top-ranked domain – a position reflected by the UK users’ rankings of the

components of the OECD’S Better Life Index. The advisory group agreed that

wellbeing reflected an ultimate goal of this initiative, and were eager to find a solution

to the concerns described. They suggested that we consider an alternative

presentation of the indicator, such as:

Focusing on another aspect of wellbeing, such as the distribution between

population segments – though many felt that this would not be easily

interpretable as a headline indicator.

Investigating a longer time series of data, to better understand life satisfaction

in the UK before the economic crisis

Consider using the absolute value rather than the threshold which would be

easier to compare internationally and give context to the indicator.


Given the lack of a strong candidate indicator to represent the ‘Subjective wellbeing’

domain at headline level, we tested three options with the non-expert group:

‘Satisfied life years’ formulated as ‘Satisfying years of life: 60.9 years of

satisfying life. 0.9% improvement.’

The ‘Percentage of the population reporting 7+ on overall satisfaction with life

(on a 0-10 scale)’ indicator, presented as: ‘High wellbeing: 78.5% have high

wellbeing. 1.9% improvement.’

‘Average life satisfaction score of the population’ formulated as: ‘Wellbeing:

Average wellbeing is 7.51 on a scale of 0‒10. 0.8% improvement.

Overall, respondents generally felt that wellbeing was an important aspect to include

in the set. Some of the respondents also commented that ‘happiness’ and ‘wellbeing’

are two quite distinct concepts, but that the more sober terminology of ‘wellbeing’

was a better representation of what was being measured, and a more appropriate

tone for this exercise.

There were mixed responses to the ‘Satisfying life years’ indicator, which combined

life expectancy with wellbeing. Some respondents felt that health and wellbeing sat

together well, and as a result, this made for a good measure. Others found it

confusing as a concept, or did not regard the measure as conveying information

about health. One respondent commented that it seemed illogical to combine health

and wellbeing in this way, given that the two are non-substitutable, i.e., 100 years of

half-satisfied life is not the same as 50 years of fully satisfied life.

There were also mixed views and levels of understanding over the ‘High wellbeing’

indicator (or proportion of the population reporting 7+ in terms of life satisfaction).


34

Overall, respondents were slightly in favour using average life satisfaction on its own

as the clearest and simplest measure of ‘Subjective wellbeing’.

2.4.4 Deciding on a subjective wellbeing indicator

The advisory group and testing respondents’ input both pointed us towards our final

decision to use average life satisfaction as the headline measure of the

‘Subjective wellbeing’ domain, as easiest to interpret, and most comparable and

therefore meaningful. This decision was confirmed when we looked across the

indicator set as a whole, where a number of other indicators were expressed in

terms of proportions, and therefore we felt it would be an advantage to balance this

by using an average figure for the ‘Subjective wellbeing’ domain.

2.5 Environmental impact

Table 8 summarises our assessment of candidate indicators for the ‘Environmental

impact’ domain against our indicator criteria.

Table 8. Summary assessment of candidate headline indicators to represent the ‘Environmental

impact’ domain.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

Consumption-based carbon emissions

Consumption-based greenhouse gas emissions, in tonnes of CO2 equivalent

Raw Material Consumption (RMC) / Material Footprint

An indicator of biodiversity/species extinctions

Air quality/air pollution

Human Appropriation of Net Primary Production







35

2.5.1 Candidate environmental impact indicator assessments

The key points considered for each of the candidate environmental impact indicators

were as follows.

Consumption-based carbon emissions (working suggestion for headline indicator)

Using this indicator to represent the broad ‘Environmental impact’ domain

does not include other greenhouse gases and leaves many important

environmental considerations entirely unaddressed, such as the sustainability

of natural resource use, or destruction of habitats and loss of biodiversity.

However, we have not been able to find a strong headline indicator that

adequately captures more than one of these issues. We therefore believe that

a carbon emissions indicator is the best proxy for the overall ‘Environmental

impact’ domain, because climate change is the single biggest and most

pressing environmental threat to human civilisation, and because, as a result,

it has come to be the flag-bearer issue in the public consciousness for

environmental concerns in a policy and political context.

The indicator can be seen as an improvement on the current territorial-based

carbon emissions indicator used by the Department of Energy and Climate

Change (DECC), as it allocates responsibility for the carbon emissions

associated with lifestyles in the UK to the UK, rather than just the emissions

created in the UK. Formulation as a footprint measure (i.e., carbon footprint)

would make this indicator somewhat easy to interpret, as it uses familiar

terms. However, without a meaningful budget against which to compare levels

of absolute emissions, the indicator does not allow interpretation in terms of

how carbon emissions relate to environmental limits, particularly risks of

climate change.

An academic study which compares the different methodologies for

calculating consumption-based emissions has found that, although incredibly

complex, the data on consumption-based emissions is robust and reliable.36

The Department for Environment, Food and Rural Affairs (Defra) has

committed to collecting the consumption-based carbon emissions data until

2016, ensuring that official data will continue to be available in the short term.

However, due to the global nature of this indicator, and the complexities of

calculating it, it is unlikely that it will be possible to update the data more

frequently than annually, and without considerable lag ‒ even if there was

political will and funding to do so.


36

Consumption-based greenhouse gas emissions, in tonnes of CO2 equivalent

This would offer a more complete picture of the harmful emissions associated

with consumption in the UK, and is ultimately what needs to be reduced.

However, the data on greenhouse gas emissions is more experimental and

less robust, and the conversion to units of CO2e would require an extra layer

of explanation, and would therefore be harder to interpret.

At the same time, using an indicator of carbon emissions captures

approximately 80% of total greenhouse gas emissions, so using the CO2

figure is a relatively good proxy for, though not entirely representative of, the

total amount of greenhouse gas emissions.

Raw Material Consumption (RMC) also known as Material Footprint

The measure captures the quantity of raw materials used in the production of

goods consumed in the UK each year, including upstream 'hidden' material

flows related to imports and exports of raw materials and products. As such, it

offers a wide overview of environmental impact within a single headline

indicator.

However, RMC does not distinguish between the very different environmental

impacts of different resources, such that one kilogramme of uranium, or fossil

fuels is treated similarly to one kilogramme of a relatively more benign

substance, like gold (though of course the extraction of nearly all materials

does generate some negative environmental impact).

Measures of environmentally weighted RMC exist, but the data available for

this complex measure is experimental.

This indicator poses challenges for interpretation, as public awareness of

environmental issues is not generally thought of in terms of total material

consumption, and it would therefore be very difficult to link more familiar

environmental issues such as climate change, biodiversity, water and air

quality to this measure.

An indicator of biodiversity/species extinctions

Whilst clearly an important and evocative issue, we believe that an indicator of

biodiversity would not provide a representative overview of the entire

environmental sphere, and would not be seen to address the issues at the

heart of environmental impact, unless supplemented with a second indicator

of environmental impact that related in broad terms to resource use and

depletion.

In addition, given the anthropocentric focus of politics, and the aim of this

project to change politics and policy-making with a concise set of indicators,


37

we find it difficult to justify inclusion of an indicator of biodiversity within the

set.

Air quality/air pollution

Air pollution is an increasingly relevant and important indicator, with real

implications in terms of human health, as well as being highly amenable to

policy intervention. However it represents a narrow an aspect of the

‘Environmental impact’ domain.

Data on air quality is frequently updated and highly accurate measurements

are widely available, both within the UK and internationally – allowing detailed

comparisons to be made.

Human Appropriation of Net Primary Production

A measure of the total photosynthetic energy appropriated by humans. Whilst

this indicator is well-respected and has a robust methodology, there is not an

established methodology around how to attribute fair shares of appropriation

between different countries

In our view, this indicator would be extremely difficult to communicate

effectively as ‘total photosynthetic energy’ is a concept very far from public

awareness.


The advisory group agreed that consumption-based carbon emissions was not a

perfect measure, given that it captures just one element of the ‘Environmental

impact’ domain. However, the group broadly agreed that, given that climate change

represents the most pressing environmental issue we’re currently faced with, and

given the limitations of the other indicators presented, if we could position the

indicator in relation to a carbon budget to make it meaningful in terms of

environmental limits and climate change, it would represent the best option with

which to assess the ‘Environmental impact’ domain at headline level.

However, many members of the group also noted that environmental issues seemed

somewhat underrepresented in the set, and advocated for the introduction of a

biodiversity or habitat indicator if we would be comfortable with extending the set of

indicators to seven.


To test the consumption-based carbon emissions indicator with the testing

respondents, we presented it in relation to a carbon budget (as the advisory group


38

had suggested), using placeholder data, in the form: ‘Carbon footprint: 95% of

budget used. 1.0% improvement.’ The group expressed the following views:

There were mixed responses regarding inclusion of an environmental domain

within the set. Some respondents pointed out that it was not an issue that

everyone would prioritise. Others echoed the advisory group’s concern that

such a wide and important issue as the environment was not adequately

captured using a single indicator.

In general, respondents struggled to make sense of what was intended by the

statement ‘95% of budget used’. Several understood the term ‘budget’ to

represent a financial budget, rather than a carbon budget as we had intended.

Using dummy data, we also presented the indicator in the format ‘50% chance

of avoiding dangerous climate change’. This formulation of the indicator

produced a much more emotional reaction and was generally, better

understood. Some respondents preferred this presentation, but others said

that it sounded overly biased.

We also tested an international comparison of consumption-based carbon emissions

with the group members. One member stated that the international comparisons

were not comparing like-for-like, because of different climatic conditions and different

types of industry in the different countries. This critique would be overcome to an

extent through use of a consumption-based emissions indicators, though not entirely.

2.5.4. Deciding on an environmental impact indicator

The advisory group’s feedback convinced us that a consumption-based carbon

emissions indicator, related to a carbon budget, was the best indicator option for the

‘Environmental impact’ domain. The outstanding issue was therefore whether to

include a second indicator of biodiversity. The key factor that led us to decide against

this was our assessment that an indicator of this sort would have difficulty gaining

purchase in the extremely anthropocentric context of our politics and policy-making,

and therefore in meeting the ‘matters to people’ criterion for a sizeable section of the

public. Its inclusion would also have pushed the indicator set to six or seven

(depending on decisions in other domains), moving us away from the goal of a set of

no more than five indicators. The final decision was therefore to include a single

indicator of environmental impact, using consumption-based carbon

emissions in relation to a carbon budget.


39

2.6 Economic inequality

Table 9 summarises our assessment of candidate indicators for the ‘Economic

inequality’ domain against our indicator criteria.

Table 9. Summary assessment of candidate headline indicators to represent the ‘Economic inequality’

domain.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

Various income ratios between the top and bottom ends of the income spectrum

GINI coefficient

Inequality ratios after housing costs

Wealth measures




2.6.1 Candidate economic inequality indicator assessments

The key points considered for each of the candidate economic inequality indicators

were as follows.

We considered several options to represent the ‘Economic inequality’ domain without

making a working suggestion to the advisory group for a headline indicator as we

were undecided about how the strengths and weaknesses of the options we had

considered weighed up overall.

Various income ratios between the top and bottom ends of the income

spectrum

We considered a selection of income ratios, each with its own benefits and

limitations in terms of the coverage of the income distribution that it could

offer. Key ratios considered were 10:90, 20:80, 1:99, and the Palma ratio.

Strengths and weaknesses variously include the lower accuracy of ratios that

use the incomes of the top 1% or bottom 10%; audiences’ potential ability to

relate to different portions of the income distribution; and the capability of

different measures to capture the interesting changes in inequality, by

focusing on more or less extreme ends of the income spectrum.


40

Overall, we felt that income ratios would be easy to interpret, making use of

the familiar ratio construction, as well as presenting an opportunity to make

highly resonant comparisons between different income groups.

GINI coefficient

The GINI coefficient is a widely known indicator of inequality, which has the

benefit of capturing information about the full income distribution.

However, the GINI coefficient is much more sensitive to changes around the

middle of the income distribution than the other measures we considered.

Whilst acknowledging that the middle of the income distribution is important,

expert advice was that in a UK context, the key policy-relevant issue

regarding economic inequality related to movement at either end of the

distribution.

In addition, the GINI coefficient is much more difficult to interpret and explain

than ratio measures which tell clear stories about differences between the top

and bottom ends of the income spectrum. Despite these concerns, we felt that

the GINI coefficient was a well-established, robust measure, and worthy of

further consideration.

Inequality ratios after housing costs

Calculating inequality ratios using income after housing costs was another

option, but would be subject to the same strengths and limitations described

in relation to after housing costs data already discussed in relation to living

standards candidate indicators.

Wealth measures

Wealth is an extremely important factor in terms of the underlying structures

of power inequality that perpetuate economic inequality.

However, collecting reliable data on the wealth distribution is difficult given the

under sampling of the top end of the distribution, and the tendency of wealthy

individuals not to declare all of their assets.

Wealth measures, as they are based on accumulations rather than flows, are

considerably less reactive to policy in the short term than income, and data is

not produced in a timely fashion.


When the advisory group met in June 2015 it spent some time discussing the

strengths and weaknesses of different ratios of income inequality. It was quite

divided over which formulation of a ratio would be best. Key comments and concerns

included that:

A great deal of inequality lies within the top 1%.


41

Ratios which use more extreme ends of the income distribution reveal a

starker level of inequality. Perhaps the 10:90 ratio is a good compromise

between the more vague 20:80 and extremely specific 1:99 ratios.

Despite this discussion, there was general agreement within the group that, given

the inclusion of an income-focused measure to represent the ‘Living standards’

domain, and given the importance of wealth in terms of inequality, we should

consider using a measure of wealth inequality to represent this domain.

However, later, members of the advisory group with particular expertise in inequality

measures expressed strong concerns about lack of policy-reactivity of wealth-based

measures, which would make it very difficult to argue for the use of a wealth-based

measure as a headline indicator.


We presented the following two indicator options to the testing respondents:

‘Wealth inequality: The wealthiest 20% has 105 times more wealth than the

least wealthy 20%. 14.6% deterioration.’

‘Income inequality: The top 20% earns 5.3 times what the bottom 20%

earns. 3.9% deterioration.’

There was some agreement across responses that economic inequality was a good

measure to include in the dashboard, though one respondent pointed out that not

everyone would consider prioritising a reduction in economic inequality to be a good

thing. In general, respondents tended not to have a strong preference for either

measure.

2.6.4 Deciding on an economic inequality indicator

The expert input we received on the lack of policy-responsiveness of wealth-based

indicators led to our preference for using an income-based indicator. The decision to

use an indicator of employment and earnings to cover both the ‘Living standards’

and ‘Employment’ domains, and therefore not to have an absolute income measure

in the headline set, confirmed our decision to use an income-based inequality

indicator.

We decided to use an income ratio as the clearest way of communicating the issue

of inequality. Revisiting the discussion at the advisory group meeting led us to

identify a ratio of the richest 10% to the poorest 10% as focusing attention on either

end of the income distribution, where inequality is starkest, while avoiding focusing

on extremes such as the top 1%, which are likely to have been perceived as too

niche to represent the state of inequality overall.


42

Researchers distinguish between the P90/10 ratio which compares the incomes of

those at the 90% percentile with those at the 10% percentile, and the S90/10 ratio,

which compares the average income of the top 10% to the average income of the

bottom 10%. We preferred the S90/10 ratio as a headline indicator as it was more

representative of the experience of a larger proportion of the population, being based

on averages of the top and bottom, rather than specific percentiles. Our final

decision was therefore to use the ratio of the average income of the top 10%

and the bottom 10% of the income distribution as the headline indicator of the

‘Economic inequality’ domain.

2.7 Public goods and services

The process for selecting the public goods and services indicator took part in two

stages. At the first stage, we aimed to develop an indicator to represent various

different aspects of the domain, and these were the options which we discussed with

the advisory group. Our assessments of these original candidate indicators are

summarised in Table 10. When it became clear that it would not be possible to meet

this aim, we reconsidered further candidate indicators to represent health, our

assessments of which are summarised in Table 11.

Table 10. Summary assessment of initial candidate headline indicators to represent the ‘Public goods

and services’ domain overall.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

An index combining public service outcomes

An index of focusing on certain public service

outcomes within a theme

A health outcome measure such as life

expectancy








43

Table 11. Summary assessment of second-stage candidate headline indicators to represent the

health aspect of the ‘Public goods and services’ domain.

Repre

sen

tative o

f

overa

ll ou

tcom

e

Ea

sy t

o

inte

rpre

t

Com

pa

rab

le

Accu

rate

Tim

ely

Proportion of deaths considered avoidable

Life expectancy or an adjusted life

expectancy measure

Patient satisfaction

NHS staff satisfaction

Proportion of the population with private

health insurance






2.7.1 Initial candidate public goods and services indicator assessments

We considered several options to represent the ‘Public goods and services’ domain

as a whole, but were initially unable to find a satisfactory working suggestion to make

to the advisory group.

The options we considered were:

An index combining measures of public service outcomes, such as health,

education, crime, local environment (e.g. air pollution) and housing outcomes

This would measure public service outcomes, rather than public services

directly, whereas our reading of the evidence on public priorities is that people

care about both the outcomes and the quality of public services.

Some of these outcomes are difficult to accurately capture using indicators, or

rely on normative assumptions about what it means to be doing well. For

example, education is most easily measured based on the number of students

gaining or not gaining certain qualifications, which is a fairly narrow measure

of the quality of education overall.


44

Similarly, performance against many health measures, such as life

expectancy, seems to be driven by longer-term trends, which do not have

clear relationships to standard policy-levers.

It is not clear that all outcomes should be weighted equally; judgement would

be required regarding how best to determine weightings for the different

outcomes.

There were strong concerns about the difficulties of interpreting a composite

measure of this sort.

An index which focuses on a small number of outcomes where patterns are

less linear and the aspects fit together well

For example, crime, air pollution, and housing could be combined into a

‘where we live’ indicator, where the elements may be combined more easily

given a shared context. However, this would lose education and broader

health aspects, which were regarded as higher priorities in public

consultations and therefore the measure would not succeed in representing

the core of what matters most to the public.

There were still likely challenges in interpreting the index and in the

methodology for combining the different elements.

A health outcome measure, such as life expectancy

Health is a key outcome, regarded as highly important in the public

consultations, and health policy and services are also extremely important in

public regard, so there is a case for focusing the ‘Public goods and services’

domain on a health measure.

However, our initial assessment was that as described in relation to a possible

public services index, the most obvious measures of health such as life

expectancy tended to steadily increase, and therefore not show great

sensitivity to policy change. This means the indicator could not be used for

useful comparisons over time, and would risk failing the domain criterion of

measuring something amenable to policy intervention.

As discussed in relation to the ‘Subjective wellbeing’ domain, we also considered

adopting satisfied life years as the headline indicator of wellbeing, as a way to

capture both health and wellbeing within a single indicator. This would create the

benefit of reducing the number of domains in the set, but, as we discussed, testing

revealed that combining these two important measures risks obfuscating one or both

elements.


45

2.7.2 Advisory group input on initial candidate indicators

Despite lengthy discussion, there was a general sense that the proposed

measurement options did not succeed in capturing the intended concept. However,

the discussion did reveal that many members of the group considered health to be

the most important aspect of the ‘Public goods and services’ domain.


With no clear candidate public services indicator at the point of testing, we combined

life expectancy with life satisfaction, to create ‘Satisfying years of life’ in an attempt to

represent both the health element of ‘Public goods and services’ and the ‘Subjective

wellbeing’ domain within a single indicator. The testing respondents’ feedback on

this, detailed in Section 2.4 on ‘Subjective wellbeing’, broadly suggested that this

was interpreted favourably by some, but as confusing, illogical, or not clearly

capturing information about health, by others.

However, testing respondents did make clear that they felt it was very important that

the indicator set included an indicator focused on health.

2.7.4 Responding to initial input

The advisory group and testing respondents’ input convinced us that there was no

viable measure of public services and that health emerged as the strong priority

issue from among those we had categorised together as ‘public services’. Our final

decision was therefore not to include an indicator of public services as a whole, but

to try again to identify a viable measure of health. Re-visiting the public consultation

evidence made clear that health was the top priority within public goods and services

across all three consultations, which confirmed this decision.

While timescales meant that we could not go through the full advisory group and

testing respondent process with this indicator, we obtained the advice of leading

health researchers to guide our selection of an indicator, and received written

advisory group input on our final recommendation.

The key insight we used to guide indicator selection was that there was clear public

interest both in health outcomes and in the public service aspect of health provision

(through the NHS and broader health policy), and that as far as possible, a headline

indicator should aim to represent both of these elements.


46

2.7.5 Candidate indicators for the health aspect of public goods and services

The options for health indicators we assessed were:

Proportion of deaths considered avoidable (working suggestion for health

indicator)

The ONS has produced statistics since 2001 which identify deaths considered

avoidable, based on their cause, as either ‘amenable’ to good-quality

healthcare or ‘preventable’ via public health interventions.37

This measure has the advantage for our purposes of relating both to ultimate

health outcomes, and linking these to the quality of health provision, and is

therefore the only health measure of those we considered to be fully

representative of public priorities in relation to health. As with all the health

candidate indicators, it will be seen to represent health rather than public

services more broadly. This makes the inclusion of the health provision

element of the measure particularly important to ensure the link back to the

original ‘Public services’ domain.

The trend in avoidable deaths reflects the same long-term improvement in

other mortality-based statistics, which, in relation to the life-expectancy-based

measures we have noted, creates a challenging sense of unresponsiveness

to policy action. However, in the case of avoidable deaths, we feel the framing

of the statistic in terms of deaths linked to the quality of health provision

creates a tangible link to policy action, and that highlighting the negative

outcome of a considerable proportion of avoidable deaths creates a strong

incentive for action.

The process which led to the production of the ONS statistics involved a

detailed expert consultation so that there is good evidence of the robustness

and accuracy of the statistics, and they are now produced on an annual basis

with a relatively long back-cast time series to 2001.

Deaths considered amenable to healthcare exclude the deaths of those aged

75 and over, as this age group is regarded as having health needs too

complex to reliably assign to a single cause. This means that the measure is

not sensitive to the quality of healthcare provision for this age group.

‘Avoidable deaths’ is an unfamiliar concept to the public, and requires careful

explanation. However, our view is that it is understandable with a relatively

brief explanation that the indicator measures deaths which experts consider

avoidable with better health provision, and if used as a headline indicator the

concept would quickly become more familiar over time.

The indicator is produced by the ONS for England and Wales only, with no

equivalent statistic for Scotland or Northern Ireland. This reflects the very


47

separate healthcare systems in the different countries of the UK and applies

to many statistics relating to healthcare, but is far from ideal in the context of

our initiative, which aims to reflect the state of the UK as a whole.

Life expectancy or an adjusted life expectancy measure

Life expectancy was originally considered as a candidate indicator to

represent public services overall, but we reconsidered it as a candidate

indicator of health. We considered life expectancy, as the most commonly

used summary statistic of health outcomes, as well as related measures

which adjust for expected healthy years of life (based on people’s subjective

assessments of their health) and for expected ‘disability-free’ years of life

(based on whether people report living with a disability or a long-term health

condition). Examining the trends in all these measures revealed the long-term

increasing trends which were the basis of our original concern about lack of

sensitivity to policy change.

In addition, these measures relate to the health outcomes aspect of the

public’s concern with health, but not the public service element of health

provision.

Patient satisfaction

The British Social Attitudes survey has asked a question for over 30 years on

people’s satisfaction with ‘the way in which the National Health Service runs

nowadays’. This initially appears a very clear, useful measure of public

perceptions of the NHS, a key component of public services relating to health.

In March 2015, however, analysis released of the data from this measure has

shown that people do not respond to the question in a straightforward way. It

found that the recent increase in satisfaction on the measure ‘has largely

been confined to those who support the Labour party, who as a result are just

as likely to express satisfaction as those who back either of the parties in the

Coalition. This suggests that rather than expressing satisfaction some of our

respondents were instead expressing support for the NHS at a time when

they may see it as under threat’.38 This suggests that the indicator is being

used to express political preferences regarding the NHS and that it therefore

does not provide an accurate representation of public satisfaction with NHS

services.

By focusing on the NHS only, not only does the measure focus on health

provision to the exclusion of outcomes, it also relates only to NHS services,

whereas we know that policy and public service provision which are relevant

to health outcomes are considerably broader than just the NHS.


48

NHS staff satisfaction

There is some evidence to suggest that measures of NHS staff satisfaction

provide a good overall guide to the quality of health services, with a 2013

study finding correlations between staff agreement that patient care was their

trust’s top priority, and that they would be happy with the care provided for a

friend or relative, and NHS trust hospital standardised mortality ratios (a

commonly used measure of the healthcare outcomes).39

Input from the communications experts we consulted about this measure

suggested that it would suffer from interpretation problems, because it

appeared too much like simply asking the providers of a service to rate their

own performance, without any credibility as a robust performance measure.

While the evidence-based link to standardised mortality means these

measures could be argued to relate both to health service quality and health

outcomes, it retains the problem of the patient satisfaction measure in

focusing only on NHS provision, rather than broader policy and public

services which are relevant to health.

Proportion of the population with private health insurance

This measure would assess the quality, and perceived quality, of health

service provision by monitoring the proportion of the population who feel

motivated to ‘opt-out’ of this provision with private health insurance.

The measure would have a fundamental interpretation problem used as a

headline indicator in that a growing proportion of people with private health

insurance would be a negative outcome from the point of view of the quality of

public health provision, but may not be viewed generally as a negative

outcome.

The indicator is very narrow in focus, in that it relates only to perceived public

health service quality, not actual quality or outcomes. In addition, many

individuals may be covered by private health insurance as an automatic

benefit of their employment benefit, without having deliberately sought out the

insurance, while the measure would regard them as in some sense

‘dissatisfied’ with public health provision, which would be inaccurate.

2.7.6 Advisory group input on health indicator

We set out how our working suggestion for a headline health indicator – proportion of

avoidable deaths – could be presented and received written feedback from advisory

group members. Most members were happy with the suggested indicator. One

member raised concerns about the fact that causes of death amenable to healthcare

provision exclude deaths of those aged 75 and over, suggesting that this created a


49

policy incentive to de-prioritise healthcare for this age group. Another was concerned

with what they perceived as negative framing of health in terms of deaths.

2.7.7 Deciding on a health indicator

We considered the concerns raised by the advisory group members in relation to the

avoidable deaths indicator.

We felt that the risk of the avoidable deaths indicator incentivising poorer healthcare

for older people was relatively low, because most medical specialisms deal with

adults of all ages, and therefore raising standards of healthcare in general would be

likely to improve healthcare for older people rather than lead to its deterioration. In

addition, older people’s deaths from causes preventable through public health

provision are included in the indicator. To mitigate the risk, however, we decided it

would be important to recommend a supplementary indicator focused on healthcare

for the 75-and-over age group.

Regarding the potential negative framing of a deaths-based indicator, we judged that

the top-level explanation in terms of deaths being avoidable through good-quality

health provision would make clear that the real focus of the indicator was on

improving health provision and outcomes, which is an essentially aspirational tone.

We also considered the issue of the indicator relating only to England and Wales, a

big drawback for our initiative which aims to reflect the performance of the UK as a

whole. However, because the indicator was otherwise a very good fit with the aim of

reflecting both health provision and health outcomes, we decided we should use it

but include a point of advocacy to agree a common statistical framework to allow the

measure to be produced for the UK as a whole.

Our final decision was therefore to use the proportion of avoidable deaths as

the headline indicator of health, and therefore of the ‘Public goods and

services’ domain.

2.8 Considerations about the indicator set overall

The indicator selection process led us to the following final domains and indicators in

the set:

Living standards and employment (combined) : An indicator of employment,

decent pay and job security

Subjective wellbeing: Average life satisfaction

Environmental impact: Consumption-based carbon emissions in relation to a

carbon budget


50

Economic inequality: The S90/10 ratio – the ratio of the average income of the

top and bottom 10% of the income distribution

Health (the decided focus of the ‘Public goods and services’ domain):

Proportion of deaths considered avoidable through good-quality healthcare or

public health interventions

In addition to the wealth of domain-specific feedback we received from the advisory

group and testing respondents, the two groups also raised a number of more general

points which influenced the final form of our set of headline indicators.

Data availability and timeliness

Some members of the advisory group were particularly concerned about how up to

date the data we used to report performance against each indicator were. This is

clearly important, not only in terms of holding government to account on current

performance, but also to avoid presenting a trend that is at odds with media

coverage on the same issue. Meanwhile, others expressed that it would be important

not to dilute the message of the project by selecting ‘second-best’ indicators on the

basis of data availability.

As a compromise, we have allowed, where necessary, a data lag of less than three

full years (i.e., 2012 data for publication in October 2015) as acceptable for our

purposes, but have also made the issue of untimely data a point of advocacy for the

project. In the report, we point out that the reason national accounts figures are

available so much more frequently than most other statistics is because of decisions

that have been taken to prioritise and better resource them, and advocate for

increased regularity in the availability of data for all of our indicators.

Accounting for fluctuations in trends

For some indicators, and particularly for the environment indicator, annual changes

seem to contain a relatively high amount of ‘noise’. Therefore, in consultation with

the advisory group, as well as taking advice from professional statisticians, we

elected to present trend data as an average annual change over the previous four

years (which allows us to use three data points on year-to-year changes). The

precise method of calculation is explained in Part 3.6 of this paper.

Communicating what is expressed by the indicators clearly and consistently

Testing the indicators with the testing respondents made it extremely evident that the

wording of labels and descriptions included in the dashboard needed to be made

clearer. We therefore adopted the approach of using clear, succinct labels for the

five selected headline indicators: Good Jobs, Wellbeing, Environment, Fairness, and

Health.


51

The testing also revealed low familiarity with the concepts of ‘GDP’ and ‘GDP per

head’, which we have tended to refer to as ‘economic growth’ throughout the report

as a result.

The advisory group also emphasised the importance of adopting an aspirational

tone. We bore this in mind when developing our succinct indicator labels, in

particular eschewing ‘Income inequality’ in favour of ‘Fairness’. And while it proved

impossible to accommodate the suggestion of the advisory group that all increasing

trends for the indicators should denote something positive, we have adopted the

language of ‘improvement’ and ‘deterioration’ rather than ‘increase’ and ‘decrease’ to

achieve consistency in the description of the trends in indicator values. This means

that while an increase in ‘Good Jobs’, ‘Wellbeing’, and ‘Health’ represents an

improvement, a reduction in the carbon emissions associated with ‘Environment’ and

the inequality associated with ‘Fairness’ would also represent an improvement.

Stressing the importance of achieving balance across the dashboard

During discussions with the advisory group, and testing with the non-expert

respondents, both groups made it apparent that there was a need to stress the

importance of achieving balance across the indicator set. For example, to address

tensions between indicators – such as possible conflict between increasing wages

and reducing carbon emissions. As a result, in the report, we stress the usefulness

of the indicator set as a policy measure in bringing to light such tensions and trade-

offs, and requiring political decision-making to achieve balance between the different

goals represented by the indicators.

A single index to rival GDP?

Several individuals also expressed enthusiasm over indexing the five domains into a

single headline figure, to rival the dominance and impact of GDP. While we can

certainly see the appeal of having a single indicator with which to rival GDP, the

need for balance across the set of indicators, as described above, is a key reason

why we have not advocated for the creation of an overarching index to provide a

single number that sums up all five. Doing so would obscure the tensions between

the different components. We were also concerned that it would be very difficult to

understand without reference to the constituent parts.


52

Part 3: Indicator calculations

This section describes the calculations made to compute the absolute values for

each indicator presented in the final indicator set (Figure 5). It also describes how

the trends in the indicators were calculated.

Figure 5. Five headline indicators of national success.

Note: Four-year trends based on the most recent data available as of 23 September 2015.

Source: Good Jobs figures calculated using ONS, Labour Force Survey data 2011–2014 (UK),

Department for Work and Pensions, Family Resources Survey 2010–2013 and Living Wage rates (UK

& London), 2011–2014. Wellbeing figures published in Office for National Statistics, Measuring

National Well-being, 2011–2014 (UK). Environmental impact figures calculated in collaboration with

the Tyndall Centre for Climate Change Research, using Department for Environment, Food and Rural

Affairs data, 2009–2012 (UK). Inequality figures calculated using ONS, The Effects of Taxes and

Benefits on Household Income data, 2010–2013 (UK); Health figures calculated using ONS,

Avoidable Mortality, 2010–2013 (England and Wales only).

3.1 Good Jobs

The statistic we have used as our headline indicator of ‘Good Jobs’ makes use of

established official data, but compiles it in an innovative way, to allow us to capture

the proportion of the labour force in good jobs, that is, in secure employment that

pays at least the Living Wage.

We began by using data from the ONS Labour Force Survey, which is conducted

throughout the UK, with new results released every three months. We combined the

quarterly Labour Force Survey results from Q1 of 2010 to Q4 of 2014, selecting only


53

the responses from Waves 1 and 5 in order to ensure that each respondent was

included in our analysis only once (this was necessary, given the five-wave panel

design of the survey).40 We weighted the data to allow us to make inferences from

the survey sample to the entire eligible population. From this dataset, we calculated

the following statistics for each year from 2011 to 2014:

1. The proportion of employed respondents (excluding self-employed

respondents) that meet our criteria of being in a good job

To do this we calculated each of the following, in a sequence of steps:

i. The number of individuals classified as in employment.

ii. The number of individuals classified as self-employed (a subset of those

classified as being in employment). We used this information to calculate the

proportion of employed individuals who are not self-employed.

iii. Of this subset of employed, but not self-employed individuals, we calculated

the proportion of individuals who met both of the following criteria, which we

had defined as being necessary for an individuals to be considered as having

a good job:

Being in a job that pays at least the research-based Living Wage:

To meet this criterion, individuals are required to have gross hourly pay

that is equal to or greater than the hourly Living Wage for the relevant

year. Individuals working in London are required to have gross hourly

pay equal or greater than the higher hourly rate set for the London Living

Wage (see Table 12 for all rates). It should be noted that the Living

Wage is set at a level that assumes the individual works 35 hours per

week, and receives tax credits, if eligible. Using the available data, we

were unable to distinguish between individuals working less than 35

hours per week out of choice, and those working less than 35 hours per

week because they were unable to find more work. Therefore, we

counted everyone earning the hourly Living Wage rate as meeting this

criteria, irrespective of whether they worked 35 hours per week or not.

At this stage of the analysis, self-employed individuals automatically

dropped out of the results, due to the fact that self-employed Labour

Force Survey respondents are not asked to report their income, and

therefore systematically fail to meet this criterion.

Being in secure employment

To meet this criterion, individuals are required to either have a

permanent employment contract, or have a temporary employment


54

contract and have cited the reason for having a temporary contract as

‘Did not want a permanent contract’.

With these criteria applied, the valid response rate dropped to just 29‒

30% of the sample. It is common to see response rates drop when

sensitive questions, such as questions about income, are asked.

However, this is a particularly high non-response rate, suggesting a

possible issue with the design of the survey. The implication of this high

non-response rate is that the results can no longer strictly be said to be

representative of the population of the UK, as there may be a systematic

bias in the group that chose not to respond to these questions. However,

this is currently the best available data allowing investigating of

employment and the quality of jobs.

iv. We multiplied the proportion of employed (but not self-employed) individuals

that meet the two criteria above, then multiplied that proportion by the

proportion of all employed individuals (employed and self-employed) who are

employed to give us the proportion of all employed individuals who are

employed and in good jobs.

Table 12. The research-based Living Wage, 2011‒2014.

Year UK Living Wage (£/hour) London Living Wage (£/hour)

2011 6.89 7.85

2012 7.20 8.30

2013 7.45 8.55

2014 7.65 8.80

2. The proportion of self-employed respondents that meet our criteria of being

in a good job

We calculated this statistic by performing the following in a sequence of steps.

i. Having previously calculated the number of individuals that are self-employed

in the steps outlined, we calculated the proportion of self-employed individuals

working within and outside of London.

ii. As previously mentioned, the Labour Force Survey does not ask self-

employed respondents to report their income. Therefore, to assess whether

self-employed respondents earn the Living Wage, we used data on self-

employed earnings from the Department for Work and Pensions’ Family

Resources Survey.41


55

iii. The Family Resources Survey responses were adjusted for non-response

bias using multi-purpose grossing factors that control for tenure type, Council

Tax Band, and a number of demographic variables.

iv. Using the Family Resources Survey, we obtained the proportion of individuals

identified as self-employed (according to the International Labour

Organization’s (ILO’s) definition of self-employed), who were earning a gross

weekly wage equivalent to being paid for 35 hours at the hourly Living Wage

rate working inside and outside of London, using the values for the Living

Wage and London Living Wage, respectively. At the time of writing, the results

of the Family Resources Survey for 2014 were not available. Therefore, we

imputed the 2014 values, by extending the linear trend from 2010 to 2013 by

one more year using the Microsoft Excel =TREND(…) function.

v. We multiplied the proportion of self-employed individuals working within and

outside of London who are earning the Living Wage (using data from the

Family Resources Survey) by the proportion of all employed individuals (i.e.

employed and self-employed) who are self-employed working within and

outside of London respectively (using data from the Labour Force Survey) to

calculate the proportion of employed individuals who are self-employed and in

good jobs.

vi. It should be noted that the Family Resources Survey is carried out in Great

Britain rather than the whole of the UK. For our purposes, we therefore

assumed that the proportion of self-employed individuals who are earning the

Living Wage in Northern Ireland would reflect the average proportion of self-

employed individuals earning the Living Wage in the rest of the UK.

vii. Further, the results of the Family Resources Survey represents the financial

year (April until March), while Labour Force Survey results are representative

of the calendar year. For the purposes of this calculation, we treated the data

from both surveys as relating to the earlier calendar year of the financial year

e.g. where the Family Resources Survey data was for 2010-11, we treated it

as 2010 data, and correspondingly used 2010 Labour Force Survey data.

This small inconsistency is likely to have had very minor effects on the overall

indicator results, as the affected segment of the data relates to a minority of

the labour force (the self-employed) for a minor portion of the time period (3

out of 12 months), but it would ideally be addressed in future calculations.

viii. Given that there is no standard approach to assessing whether self-employed

respondents are in secure employment or not, all self-employed respondents

were assumed to be in ‘secure employment’. This probably leads to over-

estimating the percentage of self-employed people who are categorised as

having ‘Good jobs’.


56

3. The proportion of the labour force that is employed or self-employed and in

a good job

i. Having calculated the proportion of all employed individuals that are either

employed (but not self-employed) and in good jobs, and the proportion of all

employed individuals that are self-employed and in good jobs, we added the

two together to give the proportion of all employed individuals in good jobs.

ii. Finally, we multiplied the proportion of those employed who are in good jobs

by the proportion of the labour force (all those employed or unemployed -

using the ILO’s definition of unemployment - 42 according to the Labour Force

Survey) that is employed to give us the number of individuals who are

employed in good jobs, as a proportion of the total labour force.

3.2 Wellbeing

The four years of wellbeing data presented in the report, including the 2014/2015

figure highlighted in the headline indicator set, are the average (mean) responses

reported by the ONS, in response to the question: 'Overall, how satisfied are you

with your life nowadays?' where 0 is 'not at all' and 10 is 'completely'.

The question was asked of 165,000–166,000 individuals, aged 16 and over

throughout the UK, as part of the Annual Population Survey, with responses

weighted to achieve a representative sample of the UK population.43

3.3 Environment

The statistic we’ve used for our headline indicator of ‘Environment’ draws on existing

official data from Defra on consumption-based carbon emissions, 44 set in relation to

a limit in the form of an annual consumption-based carbon emissions budget. The

limit was set using a peer-reviewed method developed by the academics, Raupach

et al. 45 However, the calculations presented here, which bring Defra’s data together

with Raupach et al.’s methodology, in a model developed by the Tyndall Centre for

Climate Change Research, 46 are novel. The process required us to first calculate the

UK’s share of global carbon emissions, then apply a formula to allocate those

emissions over time in order to create annual emissions budgets, before assessing

actual emissions released against the annual budgets calculated.

1. Calculating the UK’s share of global carbon emissions

Using Raupach et al.’s method,47 we calculated the share of global emissions to be

allocated to the UK, using Equation (1).

sj(w) = (1 – w) fj

F + w

pj

P (1)


57

The feature that each of the components in the equation represents, the values

used, and the source data we drew on are detailed in Table 13.

Table 13. The components of Equation (1) used to calculate our consumption-based carbon

emissions budget for the UK.

Component What it represents Value used Data source

sj The UK’s share of a global

carbon emissions quota

No value input, this was

the output of the equation -

w

A sharing index used to weight

between the endpoints of

sharing by inertia (w = 0) and

by equity (w = 1).

0.5

Based on a normative

assessment by NEF,

informed by an

approach advocated

by a number of

experts (see main

text)

fj UK consumption-based carbon

emissions released in 2008 816.63 tonnes of CO2 Defra

48

F Global carbon emissions

released in 2008

32,181,000,000 tonnes of

CO2

Oak Ridge National

Laboratory: Carbon

Dioxide Information

Analysis Center49

pj UK population in 2008 61,823,772 people ONS50

P Global population in 2008 6,758,911,727 people World Bank51

Setting component w, which represents a sharing index used to weight between the

endpoints of sharing by inertia (w = 0) and by equity (w = 1), required us to make a

normative judgement about how global emissions budgets should be distributed

between countries. Sharing by inertia (w=0) represents dividing the global emissions

budget amongst countries based on existing levels of national emissions. This would

imply that countries currently releasing a lot of emissions would be given a bigger

emissions allocation than those countries releasing fewer emissions. Sharing by

equity (w=1) represents dividing the global emissions budget amongst countries in

accordance with the size of each country’s population. In practice, this population-

based method of allocation would benefit countries like India, who currently have

very low emissions per head of population, and severely penalise countries like the

USA, whose emissions per head are more than ten times that of India’s.52 As noted


58

in the main report, problems exist with both positions, and an agreed method for

distributing global emissions between countries has not yet been reached. In light of

this lack of agreement, and given the importance of taking a balanced approach in

order to reach international agreement, we based our emissions allocation on a

blended approach (w=0.5), as advocated by several experts in the field. 53,54,55,56 This

approach considers both the UK’s share of global population, as well as the its share

of current level of emissions when setting the UK’s share of the global emissions

budget.

The most recent available consumption-based carbon emissions data available from

Defra details emissions attributed in the UK in 2012. We therefore used 2008 figures

for components fj, F, pj, and P as the base year, and then calculated budgets for

2009‒2012, to give us a trend over four years.

Using the formula and inputs described, we calculated the UK’s share of global

carbon emissions as 1.7% of total allowable emissions. To translate this into an

absolute level of emissions, we multiplied it by the global emissions allowance of

1,000 Gt CO2. This global allowance has been set by the Intergovernmental Panel on

Climate Change (IPCC)57 as the maximum allowable level of emissions that may be

released from 2011 onwards to offer a 66% chance of avoiding an average global

temperature increase of 2oC relative to pre-industrial levels (the temperature at

which dangerous climate change is expected to be triggered). The discrepancy

should be noted between our budget beginning in 2008, and the use of a global

emissions target which begins in 2011. We used the 2011 figure from the IPCC as

we believe it to be the most accurate available, and we expect the effect of this

discrepancy to be small.

This gave a total UK emissions budget of 17,262 Mt CO2.

2. Distributing the consumption-based carbon emissions budget for the UK

between 2008 and 2100

Having calculated an emissions allowance for the UK, we then distributed the

allowance between 2008 (the base year used to calculate the remaining global

emissions allowance) and 2100, to give an annual emissions allocation which

decreases year-on-year, until stabilising at a sustainable level. To calculate the

annual emissions allocations, we used Raupach et al.’s 58 equation (Equation (2)).

f(t) = f0 (1 + (r + m) t) exp (-mt) (2)

The feature that each of the components in Equation (2) represents, the values

used, and the source data we drew on are detailed in Table 14.


59

Table 14. The components of Equation (2) used to calculate our consumption-based carbon

emissions budget for the UK.59

Component What it represents Value used Data source

f(t) The UK’s consumption-based

carbon emissions at time t

No value input, this was

the output of the equation -

f0

The UK’s consumption-based

carbon emissions at the start

of mitigation (2008)

816.63 tonnes of CO2 Defra60

r The initial growth rate in

emissions -0.023 % per year

Calculated using

Defra’s data on

consumption-based

carbon emissions61

m

The rate at which emissions

exponentially decline after

having peaked

0.081 % per year Equation 3 (see

below)

𝑚 = 1+√1 + 𝑟𝑇

𝑇 (3)

Where r in Equation 3 is as detailed in Table 14, and T in Equation 3 is the time until the quota is

exhausted if current emissions are held steady, calculated by dividing the UK emissions budget of

17,262 Mt CO2 by f0. 62

Component r, the initial growth rate in emissions is used to take account of

persistence in emissions growth, to allow for the time needed to implement the

changes required for emissions mitigation to begin. We calculated r using Defra’s

data from 2005 and 2008 on consumption-based carbon emissions in the UK, to give

a three-year growth rate. We selected a three-year growth rate as we felt that it was

the fairest approximation of the recent trend in emissions reductions. As can be seen

from Table 14, this produced a negative initial growth rate in emissions, which

implies that emissions in the UK had already begun falling during this period.

Using Equation (2), we were able to calculate an annual, consumption-based carbon

emissions budget for the UK for each year between 2009 and 2100.

3. Calculating the proportion of annual, consumption-based carbon emissions

budgets for the UK used, 2009‒2012

Finally, we used Defra’s figures on actual consumption-based carbon emissions data

to calculate the proportion of the annual, consumption-based carbon emissions

budgets used in the UK each year. We chose not to carry over any unused

emissions allowance, given that in the initial years making emissions reductions will


60

be easiest, while there is maximum potential for emissions reductions, but will

become increasingly more difficult as more and more mitigation is achieved.

Therefore, we have treated any unused budget as contingency which may be drawn

on in future years when making emissions reductions is likely to be more

challenging.

4. Accounting for changes in population

The UK and global population figures used in our calculations are not a projection;

however, global population is expected to change considerably between now and

2100. To account for this, sharing by population could be based on a future

population forecast. However, when modelling for this, Raupach et al. found only a

small sensitivity of required mitigation rates given a global population of nine billion,

versus a global population of seven billion.63

3.4 Fairness

The values for our ‘Fairness’ indicator were calculated using data from the ONS

release: The Effects of Taxes and Benefits on Household Income.64 Data in the

release is calculated from responses to the Living Costs and Food survey.

We calculated the income ratio between the top and bottom ends of the income

spectrum by dividing the average (mean) equivalised disposable income per

household (£) of the tenth decile by the first decile.

3.5 Health

The ONS reported the percentage of all deaths considered avoidable in 2013 as the

headline statistic within its Avoidable Mortality in England and Wales bulletin, and we

used this as the absolute value of our headline ‘Health’ indicator. However the ONS

does not publish time-series data in this form. 65 We therefore calculated the number

of avoidable deaths as a percentage of all deaths for years before 2013 to produce

the time series values for our headline ‘Health’ measure.

We used data on the number of deaths in England and Wales considered avoidable

from the ONS release on avoidable mortality.66 We used data on the total number of

deaths in England and Wales from the ONS Vital Statistics release.67

We divided the number of deaths from all avoidable causes registered in England

and Wales in each calendar year by the total number of deaths in England and

Wales, to give us the proportion of all deaths considered avoidable.


61

3.6 Trends over time

To calculate the improvement or deterioration for each indicator over time, we used

the most recent four years of available data, as of October 2015. Table 15 shows the

four years which were used for each indicator. We used four years of data to provide

a balance between capturing recent performance, to avoidrandom fluctuations overly

biasing results, and to have adequate data availability.

We used a log linear regression model implemented via the Microsoft Excel

=EXP(LINEST(LN…)) function to give us the average percentage rate of change

from year to year over the four most recent years of available data.68 This enabled us

to report an average year-to-year trend for each indicator.

Table 15. The four most recent years of available data used to calculate the trends over time for each

indicator.

Indicator Four years of data used to calculate the average year-to-

year trend for the indicator

Good Jobs 2011–2014

Wellbeing 2011–2014

Environment 2009‒2012

Fairness 2010‒2013

Health 2010‒2013

For simplicity and consistency, in the main report we refer to data for a financial year (which runs from

April to March) by the earlier calendar year; for example, 2012–2013 is referred to as 2012.


62

Appendix 1: Individuals consulted

throughout the process

Throughout the process of developing the headline indicators set, we consulted with

the following individuals, referred to throughout this paper as the advisory group.

Although the input we received from this group has greatly shaped our final set of

indicators, the views expressed in this report remain those of the authors.

Izzy McRae, Action for Happiness

Mark Williamson, Action for Happiness

Mervyn Kohler, Age UK

Nicholas Schoon, Bioregional

Richard Lewney, Cambridge

Econometrics

Lydia Elliott, Carnstone

Nick Sammons, Carnstone

Robin McAlpine, Common Weal

Richard Dyer, Friends of the Earth

Adam Davidson, Happiness Works

Paul Allin, Imperial College

Dax Lovegrove, Kingfisher

Angus Middleton, Landmark

Information Group

Leila Woodhouse, Mind

Katherine Trebeck, Oxfam

Philomena Cullen, Oxfam

Adam Dutton, RSPB

Shannon Harvey, Shelter

Dorothea Mueller, The Children's

Society

Duncan Exley, The Equality Trust

Tim Stacey, The Equality Trust

Geoff Tily, TUC

William Lamb, Tyndall Center for

Climate Change Research

Dan O'Neill, University of Leeds

Andrew Oswald, University of Warwick

Charles Seaford, World Future Council

Lucy Young, WWF


63

End notes

1 See Appendix 1 for a list of organisations involved in this process.

2 Evans, J. (2011). Findings from the National Well-being Debate. London: Office for National

Statistics.

3 Data on UK responses to the OECD Better Life Index collected 2011–2013 kindly supplied by the

OECD.

4 Dunlop, S., Swales, K. & Trebeck, K. (2012). Oxfam Humankind Index – The new measure of

Scotland’s prosperity. Oxford: Oxfam GB.

5 Whitby, A. et al. (2014). BRAINPOoL Project Final Report: Beyond GDP ‒ From Measurement to

Politics and Policy. BRAINPOoL deliverable 5.2. A collaborative programme funded by the European

Union’s Seventh Programme for research, technological development and demonstration under grant

agreement No. 283024. World Future Council. Retrieved from

http://www.worldfuturecouncil.org/fileadmin/user_upload/Future_Justice/BRAINPOoL_Project_Final_

Report.pdf

6 Cowan, N. (2010). The magical mystery four: How is working memory capacity limited, and why?

Current Directions in Psychological Science, 19, 51–57.

7 Miller, G.A. (1956). ‘The magical number seven, plus or minus two: Some limits on our capacity for

processing information. Psychological Review, 63, 81‒97.

8 Allin, P. & Hand, D.J. (2014). The Wellbeing of Nations: Meaning, Motive and Measurement.

Chichester: John Wiley & Sons Ltd. Ch. 5.

9 By ‘social systems’ we mean societal systems such as the justice system, the electoral system, the

education system, etc.

10 The European Commission ‒ Europe 2020; The OECD ‒ How's life?; The Centre for Bhutan

Studies and Gross National Happiness Research ‒ Gross National Happiness; The United Nations

Development Programme - Human Development Index; University of Waterloo ‒ Canadian Index of

Wellbeing; Vermuri and Costanza ‒ National Well-being Index; Sustainable Society Foundation ‒

Sustainable Society Index; Roland Zieschank and Hans Diefenbacher ‒ German National Welfare

Index; Welsh government – Wales’ Wellbeing of Future Generations Bill; Scottish Government ‒

http://www.worldfuturecouncil.org/fileadmin/user_upload/Future_Justice/BRAINPOoL_Project_Final_Report.pdf

http://www.worldfuturecouncil.org/fileadmin/user_upload/Future_Justice/BRAINPOoL_Project_Final_Report.pdf


64

Scotland Performs; Office for National Statistics ‒ Measuring National Wellbeing; Oxfam Scotland ‒

Humankind Index.

11 Culture may appear to be a domain which less clearly addresses basic human needs than others in

this domain. However, examining the UK consultation rankings convinced us that culture was not

seen as clearly being of lesser importance than the other domains we grouped in the public goods

and services domain. For example, it featured in the ONS results unlike transport, and was weighted

higher in the Oxfam Humankind Index than education and transport.

12 This domain type includes domains from our review which referred explicitly to transport, as well as

those which focused on the ability to access services and facilities. We regard access as a matter

partly, but not entirely, relating to transport and therefore classified these domain types together.

13 Our review suggests that crime and safety are very closely related issues, with safety usually being

defined in terms of the absence of crime, at least implicitly (e.g. in survey questions which ask about

the feeling of safety when walking alone after dark).

14 Beaumont, J. (2011). Measuring national well-being—Discussion paper on domains and measures.

Newport: Office for National Statistics.

15 Abdalla, S., Michaelson, J., Seaford, C., & Stoll, S. (2011). Measuring our progress. London: NEF.

16 Michaelson, J., Seaford, C., Abdallah, S., & Marks, N. (2014). ‘Measuring what matters’ in Huppert,

F.A. & Cooper, C.L. (Eds.). Interventions and Policies to Enhance Well-Being. Volume VI. Chichester:

Wiley-Blackwell.

17 Bringing Alternative Indicators Into Policy, BRAINPOOL. Retrieved from

http://www.brainpoolproject.eu/

18 Sen, A. (1985). Commodities and Capabilities. Amsterdam: North-Holland.

19 Whittaker, M., & Savage, L. (2011). Missing out: Why ordinary workers are experiencing growth

without gain. London: Resolution Foundation.

20 Nolan, B., Salverda, W., Checchi, D., Marx, I., McKnight, A., Toth, I. & van de Werfhorst, H. (Eds.)

(2014). Changing Inequalities and Societal Impacts in Rich Countries: Thirty Countries’ Experiences.

Oxford: Oxford University Press.

21 Wilkinson, R. G., & Pickett, K. (2011). The spirit level. Old Saybrook, CT, USA: Tantor Media

Incorporated.

http://www.brainpoolproject.eu/


65

22 Cingano, F. (2014). Trends in income inequality and its impact on economic growth. OECD Social,

Employment and Migration Working Papers, No. 163, OECD Publishing. Retrieved from

http://dx.doi.org/10.1787/5jxrjncwxv6j-en

24 Natural Capital Committee. (2015). The state of natural capital: Protecting and improving natural

capital for prosperity and wellbeing. Third report to the Economic Affairs Committee. Retrieved from

http://www.naturalcapitalcommittee.org/state-of-natural-capital-reports.html

25 Rockstrom, J., Steffen, W., Noone, K., Persson, A., Chapin, F.S. III, Lambin, E., Lenton T.M.,

Scheffer M., Folke, C., Schellnhuber, H., Nykvist, B., De Wit, C.A., Hughes, T., van der Leeuw, S.,

Rodhe, H., Sorlin, S., Snyder, P.K., Costanza, R., Svedin, U., Falkenmark, M., Karlberg, L., Corell,

R.W., Fabry, V.J., Hansen, J., Walker, B., Liverman, D., Richardson, K., Crutzen, P., & Foley, J.

(2009). Planetary boundaries:exploring the safe operating space for humanity. Ecology and Society

14(2), 32. Retrieved from:http://www.ecologyandsociety.org/vol14/iss2/art32/

26 Natural Capital Committee. (2014). Towards a framework for defining and measuring changes in

natural capital. Working Paper 1. Retrieved from http://www.naturalcapitalcommittee.org/state-of-

natural-capital-reports.html

27 Giljum, S., Lutter, S., Bruckner, M. & Aparcana, S. (2013). State of play of national consumption-

based indicators. Vienna, Austria: Sustainable Europe Research Institute.

28 Stoll, L., Michaelson, J. & Seaford, C. (2012). Well-being evidence for policy: A review. London:

NEF.

29 MacKerron, G. (2011). Happiness economics from 35,000 feet. Journal of Economic Surveys

26(4), 705–735.

30 Lee, L. & Young, P. (n.d.). Do fewer people think that they can make a difference? NATCEN.

Retrieved from http://bsa-30.natcen.ac.uk/read-the-report/politics/can-people-make-a-difference.aspx

31 Bradshaw, J., Middleton, S., Davis, A., Oldfield, N., Smith, N., Cusworth, L. & Williams, J. (2008). A

minimum income standard for Britain: What people think. York: Joseph Rowntree Foundation.

32 Details on how the Living Wage is calculated. Retrieved from

http://www.lboro.ac.uk/research/crsp/mis/thelivingwage/

http://dx.doi.org/10.1787/5jxrjncwxv6j-en



http://bsa-30.natcen.ac.uk/read-the-report/politics/can-people-make-a-difference.aspx

http://www.lboro.ac.uk/research/crsp/mis/thelivingwage/


66

33 Daly, H.E., Cobb, J.B., & Cobb, C.W. (1994). For the common good: Redirecting the economy

toward community, the environment, and a sustainable future (No. 73). Boston: Beacon Press.

34 OECD. (2013). OECD Guidelines on Measuring Subjective Well-being. Paris: Organisation for

Economic Co-operation and Development.

35 O’Donnell, G., Deaton, A., Durand, M., Halpern, D., & Layard, R. (2014). Wellbeing and policy.

London: Legatum Institute.

36 Barret, J., Peter, G., Wiedmann, T., Scott, K., Lenzen, M., Roelich, K., & Le Quere, C. (2013).

Consumption-based GHG emission accounting: a UK case study. Climate Policy, 13(4), 451‒470.

37 ONS. (n.d.). Definition of avoidable mortality. Retrieved from http://www.ons.gov.uk/ons/about-

ons/get-involved/consultations/archived-consultations/2011/definitions-of-avoidable-

mortality/definition-of-avoidable-mortality.pdf

38 Appleby, J., Robertson, R., & Taylor, E. (2015). Health: Public attitudes towards the NHS in austere

times” in Ormston, R., & Curtice, J. (Eds.). British Social Attitudes: the 32nd Report, London: NatCen

Social Research. Retrieved from www.bsa.natcen.ac.uk

39 Pinder, R.J., Greaves, F.E., Aylin, P.P., Jarman, B., & Bottle, A. (2013). Staff perceptions of quality

of care: an observational study of the NHS Staff Survey in hospitals in England. BMJ Quality and

Safety, DOI:10.1136/bmjqs-2012-001540

40 As per guidance in ONS. (2011). User Guide: Volume 1 – LFS Background and methodology.

Version 1.0 London: Office for National Statistics.

41 Department for Work and Pensions. (2014). Family Resources Survey, 2010–2013.

42 Under ILO guidelines, anybody who is without work, available for work and seeking work is

unemployed. ONS. (n.d.). Unemployment and The Claimant Count [online]. Retrieved from

https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CC4QFjACahUKEwiy

v-uKiMLIAhWCPxQKHSCvCqg&url=http%3A%2F%2Fwww.ons.gov.uk%2Fons%2Fguide-

method%2Fmethod-quality%2Fspecific%2Flabour-market%2Fclaimant-count%2Funemployment-and-

the-claimant-

count.pdf&usg=AFQjCNEGBeshge7UpsWcjszLUz0FzI3TcQ&bvm=bv.104819420,d.d24&cad=rja

43 Data taken from the reference table: Comparison over time of estimates of personal well-being from

the Annual Population Survey (APS) Personal Well-being dataset: by UK, country and region,

http://www.ons.gov.uk/ons/about-ons/get-involved/consultations/archived-consultations/2011/definitions-of-avoidable-mortality/definition-of-avoidable-mortality.pdf



http://www.bsa.natcen.ac.uk/

https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CC4QFjACahUKEwiyv-uKiMLIAhWCPxQKHSCvCqg&url=http%3A%2F%2Fwww.ons.gov.uk%2Fons%2Fguide-method%2Fmethod-quality%2Fspecific%2Flabour-market%2Fclaimant-count%2Funemployment-and-the-claimant-count.pdf&usg=AFQjCNEGBeshge7UpsWcjszLUz0FzI3TcQ&bvm=bv.104819420,d.d24&cad=rja






67

2011/12 to 2014/15. Retrieved from www.ons.gov.uk/ons/rel/wellbeing/measuring-national-well-

being/personal-well-being-in-the-uk--2014-15/rft-table-2.xls

44 Defra. (2015). UK’s Carbon Footprint 1997 to 2012. Retrieved from

https://www.gov.uk/government/statistics/uks-carbon-footprint

45 Raupach, M.R., Davis, S.J., Peters, G.P., Andrew, R.M., Canadell, J.G., Ciais, P., & Le Quéré, C.

(2014). Sharing a quota on cumulative carbon emissions. Nature Climate Change, 4(10), 873–879.

46 We were assisted in particular by William F. Lamb from the Tyndall Centre for Climate Change

Research at the University of Manchester.





49 Boden, T.A., Marland, G., & Andres, R.J. (2015). Global, Regional, and National Fossil-Fuel CO2

Emissions. Retrieved from http://cdiac.ornl.gov/GCP/

50 The mid-year population estimates for the UK, relating to the usually resident population as at 30

June of the reference year. They account for long-term international migrants (people who change

their country of usual residence for a period of 12 months or more) but do not account for short-term

migrants (people who come to or leave the country for a period of less than 12 months). ONS. (n.d.).

MYEDE Population Estimates for High Level Areas [webpage]. Retrieved from

http://www.ons.gov.uk/ons/data/dataset-finder/-

/q/datasetView/Social/MYEDE?p_auth=MndqNqt0&p_p_auth=pZRE1F8w&p_p_lifecycle=1&_FOFlow

1_WAR_FOFlow1portlet_geoTypeId=2011STATH&_FOFlow1_WAR_FOFlow1portlet_UUID=0#

51 Mid-year population estimates counting all residents regardless of legal status or citizenship ‒

except for refugees not permanently settled in the country of asylum, who are generally considered

part of the population of their country of origin. World Bank (2015) World Development Indicators

[webpage]. Retrieved from http://wdi.worldbank.org/table/2.1



http://www.ons.gov.uk/ons/rel/wellbeing/measuring-national-well-being/personal-well-being-in-the-uk--2014-15/rft-table-2.xls

http://www.ons.gov.uk/ons/rel/wellbeing/measuring-national-well-being/personal-well-being-in-the-uk--2014-15/rft-table-2.xls



http://cdiac.ornl.gov/GCP/

http://www.ons.gov.uk/ons/data/dataset-finder/-/q/datasetView/Social/MYEDE?p_auth=MndqNqt0&p_p_auth=pZRE1F8w&p_p_lifecycle=1&_FOFlow1_WAR_FOFlow1portlet_geoTypeId=2011STATH&_FOFlow1_WAR_FOFlow1portlet_UUID=0



http://wdi.worldbank.org/table/2.1


68

53 Meyer, A. (2000). Contraction and Convergence. The Global Solution to Climate Change.

Schumacher Briefings 5. Cambridge, UK: Green Books.

54 Hohmeyer, O., & Rennings, K. (1999). Man-made Climate Change: Economic Aspects and Policy

Considerations. Mannheim, Germany: ZEW Economic Studies, Centre for European Economic

Research.

55 Bows, A., & Anderson, K. (2008). Contraction and convergence: an assessment of the CCOptions

model. Climatic Change, 91, 275–290.



57 Table 2.2: Cumulative carbon dioxide (CO2) emission consistent with limiting warming to less than

stated temperature limits at different levels of probability, based on different lines of evidence in IPCC.

(2014). Climate Change 2014: Synthesis Report. Contribution of Working Groups I, II and III to the

Fifth Assessment Report of the Intergovernmental Panel on Climate Change [Core Writing Team,

R.K. Pachauri and L.A. Meyer (Eds.)]. Geneva, Switzerland: IPCC, 151 pp.64.

58 For detailed methodology, see Raupach, M.R., Davis, S.J., Peters, G.P., Andrew, R.M., Canadell,

J.G., Ciais, P., & Le Quéré, C. (2014). Sharing a quota on cumulative carbon emissions:

Supplementary text. Nature Climate Change, 4(10).

59 ibid.



61 ibid.


(2014). Sharing a quota on cumulative carbon emissions: Supplementary text. Nature Climate

Change, 4(10).

63 For detailed methodology, see Raupach, M.R., Davis, S.J., Peters, G.P., Andrew, R.M., Canadell,

J.G., Ciais, P., & Le Quéré, C. (2014). Sharing a quota on cumulative carbon emissions:

Supplementary text. Nature Climate Change, 4(10), 873‒879.

64 Data taken from the reference table: TABLE 14: Average Incomes, Taxes and Benefits by Decile

Groups of ALL Households, 1977 to 2013/14. Retrieved from www.ons.gov.uk/ons/rel/household-


http://www.ons.gov.uk/ons/rel/household-income/the-effects-of-taxes-and-benefits-on-household-income/historical-data--1977-2013-14/ref--table-14-oecd.xls


69

income/the-effects-of-taxes-and-benefits-on-household-income/historical-data--1977-2013-14/ref--

table-14-oecd.xls

65 ONS. (2015). ONS Avoidable Mortality in England and Wales, 2013. London: Office for National

Statistics.

66 Data taken from the reference table: Age-standardised mortality rates (with 95 per cent confidence

intervals) for causes of death considered avoidable, England and Wales, 2001-2013. Retrieved from

http://www.ons.gov.uk/ons/publications/re-reference-tables.html?edition=tcm%3A77-404329

67 Data taken from the reference table: Annual data: Deaths (numbers and rates: total, infant,

neonatal). Retrieved from http://www.ons.gov.uk/ons/rel/vsob1/vital-statistics--population-and-health-

reference-tables/summer-2014-update/annual-table.xls

68 With thanks to Dr. Dan O’Neill for his advice in this area. Dr O’Neill is Lecturer in Environmental and

Ecological Economics at the University of Leeds, and the Chief Economist at the Center for the

Advancement of the Steady State Economy.



http://www.ons.gov.uk/ons/publications/re-reference-tables.html?edition=tcm%3A77-404329

http://www.ons.gov.uk/ons/rel/vsob1/vital-statistics--population-and-health-reference-tables/summer-2014-update/annual-table.xls

http://www.ons.gov.uk/ons/rel/vsob1/vital-statistics--population-and-health-reference-tables/summer-2014-update/annual-table.xls

Methods Paper Five headline indicators of national success ... · 6 Methods paper: Five headline...

Documents

Transcript of Methods Paper Five headline indicators of national success ... · 6 Methods paper: Five headline...