TPK Measure Performance 2013 - Te Puni Kōkiri

MÄ

TAU

RA

NG

A K

NO

WLED

GE

Ka möhio, ka mätau, ka ora: He ia körero

Measuring performance and effectiveness for Mäori: Key themes from the literature

2

4 1 Executive Summary

4 1.1 Introduction

5 1.2 Nature of the literature

6 1.3 Key distinctions

6 1.4 Methodology

6 1.5 Developing and reporting against measures of effectiveness for Mäori

7 1.6 A key introductory question: why measure and report performance by ethnicity?

9 2 No one-size-fits-all – performance measurement is selective

10 3 Guidance on measuring for Mäori

11 4 Measuring performance for Mäori – what and whose benchmarks?

11 4.1 Different priorities and preferences of Mäori service users

16 5 Involve citizens in the development of measures and reporting on performance

17 5.1 Measurement as part of performance improvement

18 5.2 Comparative measures – good practice for Mäori

22 5.3 The importance of comparative analysis for Mäori

24 5.4 Achieving meaningful comparisons between ethnic groups

25 6 Capturing variation within Mäori

25 6.1 Comparative impact measures or results

29 6.2 Comparative analysis in particular types of measures

30 6.3 Mäori-specific measures and their characteristics

32 7 The ability of Mäori to succeed as Mäori

32 7.1 Mäori-specific measures are attuned to Mäori realities and Mäori world views

33 7.2 Wellbeing for the individual and the group

34 8 Measures capturing whänau advancement

34 8.1 Strength-based measures of advancement in contemporary Te Ao Mäori

34 8.2 Measures for Mäori as part of a measurement framework

35 8.3 Te Hoe Nuku Roa and Hua Oranga frameworks

38 9 Key implementation issues for measuring performance for Mäori

38 9.1 Good quality data and robust data analysis

38 9.2 Organisations need the right capability

40 9.3 Staff need to feel confident of the reasons for measuring performance for Mäori

41 10 The need for forethought

41 10.1 The need for long-term commitment

43 10.2 The need for collaboration

44 10.3 Conclusion

47 10.4 Glossary

48 10.5 Appendix One

Mä te rongo, ka möhio; Mä te möhio, ka märama; Mä te märama, ka mätau; Mä te mätau, ka ora.

Through resonance comes awareness; through awareness comes understanding; through understanding comes knowledge; through knowledge comes life and wellbeing.

The framework above identifies three key

enablers that are fundamental to Mäori achieving

Te Ira Tangata (improved life quality) and realising

their potential. All our written information has

been organised within these three key enablers

or Te Ira Tangata.

1

Mätauranga – Building of knowledge and skills. This area acknowledges the importance of knowledge to building confidence and identity, growing skills and talents and generating innovation and creativity. Knowledge and skills are considered as a key enabler of Mäori potential as they underpin choice and the power to act to improve life quality.

2 Whakamana – Strengthening of leadership and decision-making.

3 Rawa – Development and use of resources.

4 Te Ira Tangata – The quality of life to realise potential.

TE I

RA

TA

NG

ATA

LIF

E Q

UA

LITY

RA

WA

RES

OU

RC

ES

MÄ

TAU

RA

NG

A K

NO

WLE

DG

E

WH

AK

AM

AN

A I

NFL

UEN

CE

R E A L I S I N G M Ä O R I P OT E N T I A L

DISCLAIMER This publication is intended to provide information on the matters contained herein. It has been written, edited and published and made available to all persons and entities strictly on the basis that its authors, editors and publishers are fully excludedfrom any liability or responsibility by all or any of them in any way to any person or entity for anything done or omitted to be done by any person or entity in reliance, whether totally or partially, on the contents of this publication for any purposes whatsoever.

© Te Puni Kökiri 2013 ISBN 0-478-34528-5 March 2013

1 2 3 4

+ + =

This work is licensed under the Creative Commons Attribution 4.0 New Zealand License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/4.0/

1

CO

NT

EN

TS

2 EXECUTIVE SUMMARY

5 INTRODUCTION

5 Nature of the literature

5 Key distinctions

5 Methodology

8 DEVELOPING AND REPORTING AGAINST MEASURES OF EFFECTIVENESS FOR MÄORI

8 A key introductory question: why measure and report performance by ethnicity?

9 No one-size-fits-all – performance measurement is selective

9 Guidance on measuring for Mäori

10 Measuring performance for Mäori – what and whose benchmarks?

10 Different priorities and preferences of Mäori service users

10 Involve citizens in the development of measures and reporting on performance

12 Measurement as part of performance improvement

13 COMPARATIVE MEASURES – GOOD PRACTICE FOR MÄORI

13 The importance of comparative analysis for Mäori

13 Achieving meaningful comparisons between ethnic groups

15 Capturing variation within Mäori

16 Comparative impact measures or results

17 Comparative analysis in particular types of measures

20 MÄORI-SPECIFIC MEASURES AND THEIR CHARACTERISTICS

20 The ability of Mäori to succeed as Mäori

20 Mäori-specific measures are attuned to Mäori realities and Mäori world views

21 Wellbeing for the individual and the group

21 Measures capturing whänau advancement

22 Strength-based measures of advancement in contemporary Te Ao Mäori

23 Measures for Mäori as part of a measurement framework

23 Te Hoe Nuku Roa and Hua Oranga frameworks

25 KEY IMPLEMENTATION ISSUES FOR MEASURING PERFORMANCE FOR MÄORI

25 Good quality data and robust data analysis

25 Organisations need the right capability

26 Staff need to feel confident of the reasons for measuring performance for Mäori

27 The need for forethought

27 The need for long-term commitment

27 The need for collaboration

28 CONCLUSION

29 GLOSSARY

32 APPENDIX ONE

34 APPENDIX TWO

36 APPENDIX THREE

38 BIBLIOGRAPHY

2

T E P U N I K Ö K I R I M E A S U R I N G P E R F O R M A N C E A N D E F F E C T I V E N E S S F O R M Ä O R I : K E Y T H E M E S F R O M T H E L I T E R A T U R E

EXECUTIVE SUMMARY

INTRODUCTION

This report highlights the key themes of the

literature on measuring and reporting on

the performance of government services in

relation to Mäori (or other indigenous people).

It provides illustrative examples particularly in

relation to employment services, but will be of

interest to practitioners involved in measuring

the performance of any government services

that are provided to Mäori.

The literature reviewed falls into three main

categories: international academic literature,

domestic academic literature, and domestic

practitioner literature (particularly ‘official

guidance’ on performance measurement from

central agencies).

Overall, there is relatively little literature about

measuring or reporting on the performance

of government services in relation to Mäori or

indigenous peoples. However, there is a larger

body of domestic literature addressing the

problem of measuring overall Mäori wellbeing

(as opposed to the performance of particular

services). This strand of the literature is included

in this review, as it holds some lessons that can

be applied to performance measurement.

There is a clear opportunity to strengthen

New Zealand’s own official guidance, which

contains little current advice about how to

undertake performance measurement and

reporting in a way that is responsive to the

needs of Mäori (or other ethnic groups). An

opportunity exists to build on documents

such as The Treasury’s 2012 guidance for the

preparation of departmental annual reports.

MEASURING PERFORMANCE FOR MÄORI IN ORDER TO IMPROVE SERVICES

At present the government is implementing

its Better Public Services reforms, which

aim towards better results achieved more

efficiently. The corollary of this emphasis on

results is that public service agencies need

performance measurement systems that can

track results and provide information to drive

service improvement.

The domestic literature emphasised that there is

a legitimate need to measure performance and

wellbeing for Mäori, who may have different

needs – and different views of what makes

for an effective service – from the majority

population. Mäori authors argue strongly that it

would be a mistake for government to behave

as if ethnicity did not exist, as this would

compromise the achievement of best outcomes

(e.g. Durie, 2004).

NEXUS BETWEEN INTERNATIONAL AND DOMESTIC ACADEMIC LITERATURE

Who needs to know, and for what purpose?

There is a nexus of views between the domestic

and international academic literature:

performance measurement is selective. What

gets measured and how it gets measured

depends on the interests of those engaged in

commissioning and doing the measurement.

Given the need to select between a wide

range of possible measures, a key question in

performance measurement should be “Who

needs the information, and for what purpose?”

3

E X E C U T I V E S U M M A R Y

(Gill and Russell, 2011, p.8). As Durie expresses

the question from a Mäori perspective: against

what and whose benchmarks should Mäori

gauge progress?

MÄORI PARTICIPATION IN THE PERFORMANCE MEASUREMENT PROCESS

The international literature makes a very

strong case for engaging citizens and service

users in the design and development of

performance measures. The more citizens

are involved, the more public buy-in and

the more the information will be used and

seen as credible. Similarly, Mäori writers

emphasise the importance of engaging

Mäori consumers to better understand Mäori

consumer needs and aspirations. Measuring

performance for Mäori rests on ascertaining

what Mäori value and consider to be an

important or a positive outcome.

Two main approaches to performance measurement for Mäori

The literature reflects two main approaches

in measuring the effectiveness of government

services for Mäori:

• comparative measurement, which compares

ethnic differences in universal or generic

performance (e.g. the total rate of people

exiting from the benefit system into

employment, compared with the exit rate

for Mäori clients); and

• Mäori-specific measurement, which is

attuned to Mäori realities and world views,

and which aims to assess performance

against Mäori norms.

Discussion of comparative measurement

Mäori academic literature affirms that

comparative measures, particularly in

combination with Mäori-specific measures,

can provide useful information. Other strands

of the domestic literature tend to concentrate

only on comparative measurement.

Common practice is to provide measurement

information that compares Mäori with

non-Mäori. Some of the academic literature

questions the validity of these comparisons,

as ‘non-Mäori’ is not an ethnic group. A

more valid comparison may be achieved by

providing a full breakdown to compare results

with those of other specific ethnic groups (i.e.

Mäori, Pacific, and New Zealand European).

Characteristics of Mäori-specific measurement

Mäori-specific measures might seek to capture

results at the level of collectives rather than

individuals. Capturing the perspectives of the

individual Mäori client, the client’s whänau,

and the service provider gives a robust

perspective of what the service has achieved.

There are many other ways in which the

‘norms’ of Te Ao Mäori might be reflected

in agencies’ performance measurement. For

example, a measure of adequate housing

might take into account the provision for

extended families and manuhiri.

This is not to say, there is such a thing as a

single, monolithic ‘Mäori view’ of what is

important in service performance or overall

Mäori wellbeing. Durie (2003a, p. 312)

4


suggests that there are some commonly-

held views on these matters within Mäori

society, but at the same time he and other

Mäori academics emphasise diversity. As

noted above, the engagement of Mäori in the

development of performance measures will

help to determine what is most important.

Implementing Mäori-specific measurement

Most agencies are not used to measuring

performance in this way. Mäori-specific

measures will require new data collection

mechanisms or a reinterpretation of existing

data. There may be some trial and error

involved before the measures are introduced.

OTHER USEFUL INFORMATION

Overcoming barriers to performance measurement for Mäori

The domestic academic sources noted that

performance measurement for Mäori can be

highly politicised. It can also be hampered by a

lack of good quality data on Mäori customers

and robust analysis of ethnicity data.

Overcoming these barriers requires agencies

to: build the right technical and policy

capability; exercise strong leadership; and

build an organisational culture that fosters

and supports Mäori perspectives.

Opportunities to use more sophisticated analysis

It is important that agencies understand the

diversity of the Mäori population, and what

that means for the delivery of their services.

It should be possible to identify particular

cohorts within the Mäori population that

hold a particular stake in the services being

delivered. Particular cohorts may be at greater

risk of poor outcomes or lessened access

to the services. One source suggests that

agencies have the opportunity to employ

a regression analysis of performance data

to identify and address differences in the

effectiveness of their own local offices.

Collaboration

Some of the literature notes that agencies

who are serving the same clients should

collaborate to improve data and share

performance information. This message of

collaboration is entirely consistent with the

government’s Better Public Services reforms.

5

I N T R O D U C T I O N

PART 1 : INTRODUCTION

1.1 PURPOSE

This review is intended to furnish key themes

and findings that will assist practitioners

involved in measuring and reporting on the

performance of any government services that

are provided to Mäori. It offers illustrative

examples in relation to employment services,

but is intended to be widely applicable across

the government sector.

1.2 NATURE OF THE LITERATURE

The review covers the international and

domestic literature on measuring and reporting

on the effectiveness of government services

for Mäori. The literature reviewed falls into

three main categories: international academic

literature, domestic academic literature, and

practitioner literature (e.g. ‘official guidance’

on performance measurement issued by central

agencies and the Office of the Controller and

Auditor-General).

1.3 KEY DISTINCTIONS

Performance measurement refers to the

process of measuring progress towards goals

or outcome targets that are expected from

a service or the activities of an agency. This

entails assessing the cost-effectiveness with

which ‘inputs’ (e.g. monetary and human

resources) are turned into ‘outputs’ (e.g. goods

and services provided to customers), and the

effectiveness with which ‘outputs’ produce

the desired impacts or outcomes. Performance

measurement should concentrate on impacts

and outcomes that are achievable and

reasonably within the delivery agency’s control.

Table One on page 7 provides live examples of

these kinds of performance measures.

Population-level outcomes, such as changes

in the national employment rate, are useful

indicators of societal wellbeing, but are

usually beyond the reach of performance

measurement (see Figure One, below). For

example, it is reasonable to assume that Work

and Income’s activities have some effect on

the national employment rate, but impossible

to measure or quantify that effect given the

myriad of other influences on that indicator.

Thus, the 2008 spike in unemployment

was influenced more by global economic

conditions than a change in the quality of

any particular government service. Table

One below provides a few examples of input,

output and impact quality better practice

measures for Mäori from agencies' recent

accountability documents.

Having made this distinction explicit, it is

important to acknowledge that a significant

amount of the domestic literature reviewed

deals with the theory of measuring Mäori

wellbeing, with many of the outcomes

discussed being at a population level. This

domestic literature is included in the review

because many of its themes and lessons

are also applicable to agency performance

measurement.

1.4 METHODOLOGY

This report represents a summary of five

different strands of literature review,

undertaken simultaneously. This resulted in the

following internal working papers, the findings

of which have been brought together and

summarised in this report:

• Aninternationalliteraturereview(Gilland

Russell, 2011);

Other contributing factors ( e.g.: world economy; contributions of other services and policies; etc ).

6


• Areviewofwhattheofficialrequirements

and primary guidance say about good

practice in developing and reporting

against effectiveness measures for outputs

for Mäori (Te Puni Kökiri, 2011a);

• Asummaryofgoodpracticeguidanceand

requirements (Te Puni Kökiri, 2011b);

• Adesktopanalysisofperformance

reporting by the Ministry of Social

Development (MSD) in relation to its Job

Search Service 2002/03 to 2011/12 (Te Puni

Kökiri, 2011c); and

• TowardsaMäoriperspectiveofgood

practice in developing and reporting

against measure of performance

effectiveness for a front-line employment

service for Mäori on the unemployment

benefit (Te Puni Kökiri, 2012).

More detail on each of the working papers,

and the methodologies used, is presented in

Appendix One.

Figure One – dimensions of a performance story

Inputs Outputs Impacts / Intermediate Outcomes

Population-level Outcomes

Resources that go into a service or programme

Goods and services supplied to stakeholders

Immediate effect of outputs on stakeholders

Broader states or conditions of society, the economy or environment

Example:

Increased capability to deliver a quality service for Mäori

Relevant measures:

•costs

•stafftrained and qualified

• proprtionof Mäori staff

• staffculturalcompetencies

Example:

x service provided to n number of Mäori customers

Relevant service quality measures:

•timeliness

• accuracy

• entryandcompletion rates

• qualitystandards

• volume

• customersatisfaction

• coverageand%oftarget population reached

Example:

The service made x difference to n number of Mäori clients

Relevant quality impact measures:

•%ofclientsdemonstrating positive gains over time

• %oftarget group reached

• customerandstakeholder satisfaction

Example:

Improvement in employment outcomes for Mäori

Relevant measures:

•%ofMäoriworking - age population in the labour force / not in the labour force

• employmentandunemployment rates

• NEETrates

• incomelevels

D I M E N S I O N S O F M E A S U R A B L E P E R F O R M A N C E

DIF

FIC

UL

TY

IN

AT

TR

IBU

TIO

N

7

I N T R O D U C T I O N

Table One – examples of performance measures from agencies’ public reporting

Inputs

Input Result Source

Ministry of Social Development’s Aka Matua programme which supports Mäori and Pacific managers to complete a master’s degree in Public Policy

Six Mäori and Pacific managers supported

MSD Annual Report 2010/11

Developed a Mäori cultural competency for their staff to enhance their capability to deliver for Mäori. To be embedded in performance management, recruitment and personal development processes

Not reported Ministry of Education Annual Report 2010/11

Outputs

Output target Budget standard Source

90%ofclientsexpresssatisfaction with Victim Support Services1

90% Vote Justice Information Supporting the Estimates 2012/13

Impacts

Impact target Budget standard Source

Reduce truancy rates for frequent truants in Years 9and10by1.0%,withseparate targets for Mäori and Pacific.

Mäori2.5%

Pacific1.5%

Vote Education

Information Supporting the Estimates 2012/13

Impact target Reported result Source

The proportion of UB jobseekers who exit a main benefit into employment and achieve six months employment

61.8%(notpubliclyreported by ethnicity)

MSD Annual Report 2011

1 Agency reporting does not generally provide ethnic breakdowns of output quality measures, such as customer satisfaction. See State Services Commission (2011a) for an analysis of Mäori customer satisfaction from the cross-Government Kiwis Count 2009 survey.

8


2.1 A KEY INTRODUCTORY QUESTION: WHY MEASURE AND REPORT PERFORMANCE BY ETHNICITY?

It cannot be assumed the need to measure

and report performance by ethnicity will be

intuitively accepted. Much of the domestic

academic literature notes that performance

measurement by ethnicity can be controversial,

especially where Mäori are concerned. The

official guidance material is thin on this

question, with nothing comprehensive since

guidance released by the Controller and

Auditor-General in 1998. It is therefore

worthwhile to recap some of the main

arguments in the literature pertaining to the

necessity of measuring performance for Mäori.

The importance to New Zealand of Mäori success

A number of authors refer to the relatively

young age structure of the Mäori population,

meaning that Mäori will make up a greater

proportion of the potential workforce.

Business New Zealand’s Chief Executive has

been quoted as saying: “If Mäori and Pasifika

don't succeed in the next twenty years; New

Zealand will fail as a nation. It's that simple”

(Human Rights Commission, 2011a, p.32). This

being the case, it would be worth the effort

for government agencies to ensure that their

services are equally effective for Mäori.

The realities of contemporary New Zealand

Durie (2005b) suggests that policies,

programmes and practices that purport to be

“blind” to race and ethnicity mask diversity,

PART 2 : DEVELOPING AND REPORTING AGAINST MEASURES OF EFFECTIVENESS FOR MÄORI

compromise best outcomes and foster an

assimilatory approach. Elsewhere (2004) he

suggests that it is illusory to act as if race and

ethnicity do not play a large and increasing

part in New Zealand life.

The drive for service improvement

The government’s Better Public Services

reforms emphasise agencies working together

towards the efficient achievement of

enhanced results. Almost all of the ten “result

areas” established by the government relate to

social services of which Mäori are significant

consumers. This creates further momentum

for the public service to develop measurement

systems that can: track results; provide

information to drive service improvement; and

assure taxpayers that government services are

as effective as possible for all consumers.

Mäori interest in performance measurement

Performance information will also be of interest

to members of ethnic groups themselves,

who might like to know how they faring

from government services, and who ought to

be part of the conversation about ongoing

improvements to those services. Several

leading Mäori academics note this type of

measurement is very important for Mäori.

Rights-based arguments

Collecting and analyzing data by ethnicity is

supported by the recommendations of several

UN Human Rights bodies (Callister, 2007;

Human Rights Commission, 2011a, p.9). Mäori

academic writers tend to see this approach as

a legitimate equity endeavour, consistent with

the Treaty of Waitangi.

9

D E V E L O P I N G A N D R E P O R T I N G A G A I N S T M E A S U R E S O F E F F E C T I V E N E S S F O R M Ä O R I

2.2 NO ONE-SIZE-FITS-ALL – PERFORMANCE MEASUREMENT IS SELECTIVE

Academic literature in agreement

The international and domestic academic

literature is mostly in agreement that there is

no one-size-fits-all ‘best practice’ performance

measurement and management system. The

international academic literature and some

domestic academics emphasise that measuring

performance is a social activity; it depends

upon who wants to know the performance

story and for what purposes (Dormer, 2010;

Durie and Kingi, 2010; Gill and Russell, 2011).

Much of the international academic literature

tends to emphasise that “performance

information itself is not comprehensive or

objective but incomplete and ambiguous. It is

subject to selective measurement, presentation

and interpretation based on the interests of

the actors involved” (Moynihan (2008, p.24)

discussed in Gill and Russell (2011, p.8).

De Bruijn (2006, p.56) uses the metaphor of

floodlighting a statue: “Illuminating the statue

from several different perspectives creates

a different image each time. Each image is

correct but a single perspective always gives a

distorted image.”

The quest for best practice

“This is in marked contrast with the

practitioner literature which tends to

emphasise ‘best practice’ and the associated

notions of the golden thread or one integrated

set of performances information” (Gill and

Russell, 2011, p.8). Commenting on best

practice in the international literature review

Gill and Russell (2011, p.8) noted that “Pollitt

and Bouckaert, (2003) suggest: What we are

dealing with here is best described as a kind

of a religion ... a system of belief founded on

faith (sourced from OECD, 2009, p.36)”.

Official performance measurement

requirements and guidance in the New

Zealand state sector can be regarded as

statements of ‘practitioner best practice’

(see, State Services Commission and Treasury,

2008; The Treasury, 2008; Controller and

Auditor- General, 2008, 2009a, 2009b, 2011).

However, that guidance is not consistent.

See, for example the FABRIC performance

measurement principles in State Services

Commission and Treasury (2008); revised

audit requirements in Controller and Auditor-

General (2009a); and Controller and Auditor-

General (1998) guidance, discussed below.

There is no current or up-to-date

comprehensive guidance for measuring

the effectiveness of mainstream services

or outputs for Mäori within the corpus of

technical requirements that agencies are

required to use for measuring performance.

2.3 GUIDANCE ON MEASURING FOR MÄORI

The most relevant guidance in this area was

released by the Controller and Auditor-General

in 1998. It advised agencies to:

• estimatetotalservicedemandtrends(isit

growing or shrinking), how many customers

are repeat users and the characteristics of

major customer groups;

• identifytheproportionoftheircustomer

base that is Mäori (by age and gender)

in different regions or districts. This

information is important for ascertaining

coverage measures;

10


• monitorandevaluate:proposalsfor

improvements to systems, service design

and policy; service uptake and impact on

Mäori by age and gender and region, or

urban/rural; the extent to which the service

has met Mäori needs and expectations; and

capability to improve outcomes for Mäori;

• produceperformancemeasurestoassess

progress towards objectives and outputs

and changes in outcomes for Mäori; and

• includeappropriateservicedelivery

standards for Mäori in contractual

arrangements with providers.

Guidance to departments on preparing annual

reports states that: “Departments should also

consider how significant achieving outcomes

for different population groups is in achieving

Ministerial priorities and whether the annual

report should specifically cover progress for

different population groups” (The Treasury,

2010, 2012a, p.8).

2.4 MEASURING PERFORMANCE FOR MÄORI – WHAT AND WHOSE BENCHMARKS?

Several domestic academic writers, including

Humpage (2002) and Durie and Kingi (2010),

concur with the international literature that

measuring performance is highly subjective.

Measuring outcomes and performance for

Mäori from a Mäori perspective in turn rests

on ascertaining and measuring what Mäori

consider to be important and not anyone else

(Humpage, 2002).

Measuring the performance of a mainstream

service for Mäori starts from the question:

Who wants to know and why? And what do

or would Mäori (consumers) regard as good

performance? Mäori have their own views on

what constitutes good performance including

outcomes from mainstream services and an

agency for Mäori. This is supported in the

international literature which found – “what

managers’ think is important in terms of

performance can often differ from what

citizen and service users think is important.

This can generate what Ho (2008, p.206) terms

as an undesirable “performance perception

gap” (Gill and Russell, 2011, p.9). Similarly,

Gill and Russell (2011, p.13) suggest “what

is ‘good’ will in turn depend on whose

perspective is being privileged”.

2.5 DIFFERENT PRIORITIES AND PREFERENCES OF MÄORI SERVICE USERS

The State Services Commission’s analysis of

its Kiwis Count 2009 survey confirmed that

Mäori held a slight difference in expectations

of public services. “Whilst the drivers of

satisfaction for Mäori are the same as for the

New Zealand population overall, except that

‘the service experience met your expectations’

is not a driver for Mäori. Being treated fairly

is the most important driver for Mäori” (State

Services Commission, June 2011a, p.12).

2.6 INVOLVE CITIZENS IN THE DEVELOPMENT OF MEASURES AND REPORTING ON PERFORMANCE

The case for engaging with citizens

There are common threads across the

literature reviewed for engaging Mäori as

citizens and significant users of services in

decisions about what to measure and what is

good performance. The international literature

lends support to the view that performance

measurement systems need to include the

perspectives of everyone involved, including

citizens and clients (de Lancer Julnes 2001;

Ho, 2008; OECD, 2009 discussed in Gill and

Russell, 2011).

11

D E V E L O P I N G A N D R E P O R T I N G A G A I N S T M E A S U R E S O F E F F E C T I V E N E S S F O R M Ä O R I

“In brief there is a prima facie case for

engaging citizens and service users in the

design and development of performance

measures. The more citizens are involved the

more buy-in and the more likely it will be used

and credible” (Ho, 2008). This engagement

may also build the public capital of the

organisation concerned by reducing public

apathy about the public service” (Gill and

Russell, 2011, p.9).

This point is stated in another way by Thomas

(2006, p.66) “performance measurement …

should involve consultation with the key

stakeholders and the public at large, so

that the results of the PMM [Performance

Measurement Management] system have more

legitimacy and support, especially among the

people most directly affected by programs”.

Although silent on specific user groups,

official guidance emphasises the importance

of agencies engaging with stakeholders and/

or significant user groups of services to agree

on the appropriate measures, including direct

measures of quality (Controller and Auditor-

General, 2002, 2009; The Treasury, 2008).

External measures of output quality that focus

on the purchaser or consumers of the output

are a better test of real performance rather

than the standards set and evaluated by the

provider itself (The Treasury, 2008).

The case for engaging Mäori

Kingi (2003) argues that: measures need to

consider what is important or relevant to

the Mäori individuals receiving the service;

and outcomes for Mäori derive from a Mäori

world view. (Durie, Kingi and Graham 2012,

p.42) recently advised “that effectiveness

was in part linked to an agency’s ability to

engage successfully with Mäori……It was

further stressed that engagement must be

meaningful and viewed as an opportunity to

better understand the needs of Mäori and

to ensure that outcomes for Mäori are fully

maximised”. They also emphasise, at page 32,

that “consumer satisfaction measures [are]

based on the simple premise that consumer

needs and preferences should be met. To

improve access to, and use of public services,

Mäori preferences, choices and decisions must

be fully considered”.

Williams (2000) and Kawharu (2001, p.2)

suggest at the very least, any discussion about

outcomes and government effectiveness for

Mäori should allow Mäori to participate in the

decision about what kind of measure is used

Humpage (2002, p.193).

Reporting to Mäori and the wider public on performance

Both the international literature generally (de

Lancer, 2001; de Bruijn, 2006, p.59; Thomas,

2006, p.63; OECD, 2009), and domestic

literature emphasise the importance of

using performance information to reflect

on performance and improve the service

delivery effectiveness and reporting results

to stakeholders. The 1998 Controller and

Auditor-General guidance required agencies

to provide feedback to Mäori. However, it does

not specify which Mäori.

Durie and Kingi (2010) suggest for reporting to

Mäori at least against Mäori-specific impact

measures, it is important that Mäori clients and

participants receive reported information on

their overall progress, or lack of it.

“If outcome measures are to be useful,

information concerning their use and

application must be made available

to Mäori….[service providers]. Such

information dissemination mechanisms

will necessarily require a broad range of

strategies. The costs associated with this

process, and training in particular, will

also need to be considered” (Durie and

Kingi, 2010, p.32).

12


2.7 MEASUREMENT AS PART OF PERFORMANCE IMPROVEMENT

Under official requirements agencies

are expected to make full and good use

of performance information in their

accountability documents, as part of a cycle

of continuous improvement and review.

This is consistent with the expectation that

departments will take a systematic approach

to continuous improvement and actively seek

opportunities to learn from benchmarking

studies (Controller and Auditor-General, 1999,

p.107; State Services Commission, 2010b).

Sources in the international and domestic

literature including from Mäori academics

concur that the process for good performance

measurement is an iterative journey. This

means periodically assessing whether the data

being collected remains the most relevant to

collect (Controller and Auditor-General, 2002,

p.51), and whether the measures are capturing

the most significant aspects of performance

for their Minister and stakeholders, and for

their own continuous improvement (Controller

and Auditor-General, 2002, 2008, 2009b, 2011;

The Treasury, 2008).

The Treasury (2008, p.16) suggests that

perfect measures are rare. Kingi (2003, p.9)

suggests that for most outputs “the possibility

of an optimal outcome is also unrealistic. A

questionnaire cannot for example ask ‘as a

result of an intervention are you now cured’.

That is of course unless cure is the most

probable or expected outcome”. He adds: “We

must remember the measurement is not the

endpoint rather an indicator of whether or not

the prime objective … has been satisfied.”

13

C O M P A R A T I V E M E A S U R E S – G O O D P R A C T I C E F O R M Ä O R I

3.1 THE IMPORTANCE OF COMPARATIVE ANALYSIS FOR MÄORI

Mäori academics Durie and Kingi (2010)

emphasise that comparative measures

for Mäori are important for measuring

effectiveness and impacts for Mäori. Universal

indicators (e.g. the total employment rate)

without comparative measures are insufficient

measures of need and outcome (Durie, 2005b).

The purposes of comparative measurement can

include ensuring:

• theeffectivenessandefficiencyof

government services in achieving the aims of

those services, such as meeting Ministerial

priorities (The Treasury, 2012a, p.8); and

• equitableoutcomesbetweendifferent

population groups (OECD, 2009; Human

Rights Commission, 2010; 2011a, p.12).

3.2 ACHIEVING MEANINGFUL COMPARISONS BETWEEN ETHNIC GROUPS

Mäori vs. non-Mäori comparisons

Mäori compared to non-Mäori is the most

commonly used comparative measure. The

Department of Corrections, for example, gives

such a measure in its annual report in relation

to the successful completion of community-

based sentences and orders.

However, Mäori academic authors have

criticised this approach because:

• Duriesuggests:“Sometimesmorerelevant

benchmarks may be found with other iwi,

or in other indigenous communities. It is

misleading to use crude comparisons with

non-Mäori as a type of shorthand for best

outcomes, or to assume that Mäori/non-

Mäori comparisons always provide useful

information about Mäori progress” (Durie,

2003a, p.202);

• ‘non-Mäori’isnotanethnicgroup,but

a collection of specific ethnic groups

experiencing disparate outcomes; and

• itmaymaskdifferingresultsachievedfor

different ethnic groups such as between

Europeans and Pacific peoples, the latter

of whom are much closer to Mäori in socio-

economic status than other groups included

in the non-Mäori category (Statistics New

Zealand, 2001a, p.90, 2001b, pp.63-71;

Humpage, 2002, pp.178-9).

Full analysis by ethnicity as better practice

Comparing Mäori against other specific ethnic

groups is more consistent with ensuring equity

between those groups. Such an approach

is endorsed by the OECD (2009) and UN

Committee for the Elimination of Racial

Discrimination (Human Rights Commission,

2010, p.3-4; 2011a, p.12). “Another function of

the outcome indicators may be as a check on

the crucial social issue of equity. Are individual

clients in similar situations being treated

equally? .... Such measures might reasonably

cover the distribution of key services by socio-

economic groups” (OECD, 2009, p.83).

Equity and discrimination

The concern about equity is all the more

important in view of the need to counter

discrimination, which does occur in New

PART 3 : COMPARATIVE MEASURES – GOOD PRACTICE FOR MÄORI

14


Zealand according to a number of the

sources. Several sources state that Mäori

experience racial discrimination in the labour

market (Alexander, Genc and Jaforullah,

2003; Durie, 2005b; Robinson, Cormack and

Cram, 2007, p.33; NZIER, 2009), and that

perceptions of ethnicity and skin colour are

linked with this disadvantage (see Chang, and

Dodd, 2001; Dannette et al., 2011; Carter and

Gunasekara, 2012).

The Human Rights Commission (2010) has

issued guidance emphasising that measures

must be:

• necessarytoaddressdisadvantageor

ensure equality with other members of the

community for groups against whom it is

unlawful to discriminate;

• carriedoutingoodfaith,andtailoredto

reduce the actual disadvantage of the

group it is aimed at;

• proportionaltothedegreeofunder-

representation or disadvantage; and

• temporary–“lastingonlyuntiltheissue

it is designed to address is substantively

resolved”.

Illustrative examples: measures that would lend themselves to ethnic comparisons

A number of employment outcome measures currently in use would be suitable for comparison

between Mäori and other specific ethnic groups (i.e. New Zealand European and Pacific peoples):

•thelabourforceparticipationrate;

•theemploymentandunemploymentrate;and

•thenotinemploymentoreducationortraining(includescaregivers)rate(NEET).

NEET rates can be employed not only as an outcome measure but also as a coverage measure, for

example the percentage of NEET accessing a particular service.

Table Two – NEET type measures from MSD’s 2012/15 Statement of Intent (SOI)

Ministry outcome - More young people are in education, training and work

Intermediate Outcome

Measure Target Comment / Current Result

More young people are in education, training or employment without needing a benefit

The proportion of Youth Services (Youth Payment) recipients or Youth Services (NEET) participants, who gained a main benefit within three months of exiting either service

Decreasing proportion

New measures for 2012/13

The proportion of Youth Services (NEET) participants who are in full-time education, approved training or work-based learning, leading to at least an NCEA Level 1 qualification

Decreasing proportion

New measures for 2012/13

15


3.3 CAPTURING VARIATION WITHIN MÄORI

Identifying consumers most likely to benefit

There is considerable diversity and

heterogeneity within the Mäori population.

Understanding this diversity is essential

for policy making. For example Leggatt-

Cook (2008) found that gender differences

in pathways and types of employment are

particularly significant for women who

identify as Mäori only. Policies may have

different effects on Mäori living in different

circumstances.2 “Services need to take into

account different types of variation within the

Mäori population”, such as geographic and

gender differences (Ringold, 2005, p.7).

While it may be easy to assume that Mäori

ethnicity is a proxy for lower socio-economic

status, Durie (2005b, pp.24-5) urges caution. He

argues that ethnicity and socio-economic status

are both relevant considerations in performance

measurement, but are not the same thing:

“The strong relationship between ethnicity

and adverse socio-economic circumstances

has sometimes led to an assumption that

one is a proxy measure for the other. Being

Mäori, for example, is often seen as a

synonym for being poor and being poor

is sometimes seen as the distinguishing

characteristics of Mäori and Pacific peoples.

While there is a significant correlation

between the two measures – ethnicity

and socio-economic status – they do not

measure the same phenomena… Recent

research, however, has demonstrated

that not only is class distinguishable from

ethnicity, but that universal indicators by

themselves are insufficient measures of need

and outcome”.

The literature variously identifies specific

cohorts within the Mäori population who

may be at particular risk of adverse outcomes.

These include:

• thosewithloworverylowfamily

or household income levels (Durie,

2005b; Danette et al., 2011; Carter and

Gunasekara, 2012);

• thosewithnoqualificationsorqualifications

less than Level 4 on the New Zealand

Qualification Framework (MSD, 2003a,

2007, 2008a, 2008b, 2008c; Dixon and

Crichton, 2007; NZIER, 2009);

• singlemenwiththeabovevariablesand/or

with drug and alcohol problems or a prison

conviction (The Salvation Army, 2006); and

• Mäorilivinginruralareaswithoutaccess

to employment or education opportunities

accessible to Mäori living in cities (Ringold

(2005, p.7) drawing upon Maani, 2002).

Targets for Mäori including for those with ‘high’ need, at the service level

The Treasury (2004) has suggested that

if services to reduce ethnic gaps are not

accompanied by services to the ‘needy’ in

other population groups, there could be

arguments that people with similar needs are

not receiving the same treatment. It suggests

the best option is to capture performance for

Mäori (and other ethnic groups), by ethnicity

and another indicator of need. The rationale

for such targets need to be fully understood

by the agency and sector. The targets need to

be supported with leadership, and adequate

resources to monitor, evaluate and reflect on

performance (The Treasury, 2004, 2006).

2 See also Cunningham et al. (2006).

16


‘Sole’ vs. ‘mixed’ Mäori – different shades of Mäori ethnicity?

There is debate in the literature on the merit

of capturing outcome performance within

ethnic groups by intra-ethnic variation to get

a fuller picture of the complexity of outcomes

across and within ethnic groups.

In favour of such an analysis, Humpage

(2002, pp.180-1) suggests:

“Because ‘sole’ Mäori are more likely to be

distributed among the deprived deciles than

‘mixed’ Mäori. Yet when the ‘sole’ Mäori

category is used alone, figures are under-

estimated. Ideally, ‘sole; ‘mixed’ and ‘total’

Mäori categories should be combined to

show the ‘bigger picture, starting with the

simplest (‘total’) and gradually moving on

to more complex and detailed (‘sole’ and

‘mixed’) analysis, because reliance on either

external or internal indicators can favour

one interpretation of the disparity issue…

MOH (2001:24) asserted that combining

these categories would avoid very simple ‘on

average’ conclusions being made in an area

where considerable complexity exists”.3

However, Kukutai (2004) says such an approach

is limited. In differentiating needs within

the Mäori population, a critical variable is

orientation towards the Mäori ethnic group

(e.g. the group a person identifies most strongly

with). This dynamic cannot be captured in

a crude ‘sole’ Mäori versus ‘mixed’ Mäori

dichotomy. Durie et al. (2003b, p.100) suggest

“the only significant differences lie between

Mäori who identify as Mäori (either solely or in

combination with other ethnicities) and those

who are descended from Mäori but identify as

non-Mäori (Kukutaki, 2003)”. Te Puni Kökiri

concludes therefore that any analysis involving

disaggregation of this nature should be

approached with caution.

3.4 COMPARATIVE IMPACT MEASURES OR RESULTS

Multiple sources, including official

requirements, MSD’s Performance

Improvement Framework Review, and

writings by Mäori academics, emphasise the

importance of impact measures. Impacts are

to be directly linked or attributable to the

entity’s activities.

Examples of impact measures for employment services

There is a range of generic impact type

measures in MSD’s 20012/15 and 2011/14

SOIs for their employment service Job Search

Service and its Appropriation Tailored Sets of

Services to help people into work or achieve

independence (see Appendix Three).

It is also notable that these measures are

capable of disaggregation by ethnicity, but

such an analysis is not provided. MSD’s

Performance Improvement Framework

(PIF) review noted difficulties in outcome

monitoring and evaluation, and suggested

some improvements in that regard (State

Services Commission, May 2011b, pp.34-5).

However, that review did not address MSD’s

performance for Mäori.

From an academic perspective, Gill and Dormer

(2011, p. 297) suggest Work and Income

staff generally perceive that paying benefit

entitlements and placing people in jobs is

not complex or difficult to measure, but the

organisation’s role in social development is.

Although “social development outcomes are

3 Sole Mäori are those who are identified in the data as having only Mäori ethnicity, ‘mixed’ Mäori are those identified in the data as having Mäori and at least one other ethnicity), and total Mäori (the total of all those with Mäori ethnicity).

17


quantified in the annual Social report, sense

is made of this information via subjective

evaluation (“gut feeling”) rather than by the

application of a formal logic of cause and

effect” (Dormer, 2010, p.55).

Illustrative examples: potential impact measures for employment services for Mäori

The literature identifies several

meaningful impact measures for

Mäori receiving employment services

including measuring:

• incrementalstepstoemployment

such as addressing health or training

barriers and increasing participation

and having this reflected in outcome

measures and contracting guidelines

(MSD, 2003a, 2008a, 2008b, 2008c);

• theeffectivenessofsequential

interventions (Perry, 2007);

• whetherexitingabenefitmarks

a move to employment, and if so,

the impact of that employment

on individual and whänau/family

wellbeing (see Englert, 2001;

Stolte, 2006; MSD, 2007; Dixon

and Crichton, 2007) and long-term

employment and income prospects.

Best way to measure the impact of an intervention for Mäori is through multiple perspectives

Durie and Kingi (2010) emphasise the best

way to measure the impact of an intervention

for Mäori is through data from multiple

perspectives, such as the Mäori client, the

service provider, and family/whänau member,

with quantified weightings for their responses.

By obtaining the perspectives of these three

key stakeholders a more balanced impression

of outcome could be obtained. It is possible to

measure impact status at several points in the

service delivery process: at assessment, while in

receipt of the service and on exiting the service.

3.5 COMPARATIVE ANALYSIS IN PARTICULAR TYPES OF MEASURES

The whole range of performance measures

included in figure one (see above, Part 1) lend

themselves to comparative measurement.

It is worth discussing briefly the literature

that relates to measures of coverage and

accessibility; customer satisfaction; and the

comparative performance of different service

delivery sites.

Coverage and service accessibility

Coverage or reach are important measures of

service adequacy for Mäori, as they focus on

customer accessibility or uptake of services.

“Coverage really matters. Outputs must reach

the groups or area where need exist”, and

“poor ability to target effective outputs on

areas of need will limit value for money” (The

Treasury, 2008, p.11 and p.5).

The domestic literature on public services

shows that Mäori are more likely to experience

greater barriers to accessing state services

than some other population groups (State

Services Commission, 2011a, pp.6, 12)4.

Examples of coverage measures include:

percentage of population in need of receiving

the output; percentage of target group who

meet entry criteria; percentage of target group

who did not access or use the service; and

transit time (or other barrier to use).

4 See also: Ministerial Advisory Committee on a Mäori Perspective for the Department of Social Welfare, 1988; Te Puni Kökiri, 1996a, 1996b; Ministerial Inquiry into Department of Work and Income, 2000.

18


Customer satisfaction measures

Whilst the international literature identified

a range of common customer satisfaction

measures of effectiveness of services it “is not

clear whether these expectations are equally

shared by indigenous people in New Zealand

when utilising public services” (Gill and

Russell, 2011, p.15). Mäori may have different

priorities and perceptions of service quality.

The State Services Commission’s 2009 Kiwis

Count Survey (also discussed above at section

2.5) found the drivers of satisfaction for

service quality were slightly different for

Mäori, relative to non-Mäori. Mäori were a

little less satisfied with the quality of services

than non-Mäori. Mäori rated 19 out of 26

frequently used services as low quality (SSC,

2011a, p 11), and reported lower levels of trust

in public services than non-Mäori.

The report recommended to improve the

experience of Mäori customers, agencies

need to ensure “staff are competent, keep

their promises and treat Mäori fairly. Earlier

research has shown that for Mäori; staff being

non-judgemental and treating people with

respect are key elements of fair treatment”

(State Services Commission, 2011a, p.17).

Durie et al. (2012, p.32) drew upon He Taura

Tieke, a framework for effective health services

for Mäori, to identify the following aspects of

service delivery that are important to Mäori

consumers:

• access;

• information;

• informedchoice;

• trustandrespect;and

• participationandseamlessness.

Other literature (Donnell and Social Programme

Evaluation Unit, Department of Social Welfare,

1985; Te Puni Kökiri, 1996a, p.18, 1996b, 2009;

Ministerial Inquiry into the Department of Work

and Income, 2000; Englert, 2001; The Salvation

Army, 2006, p.64; State Services Commission,

2009a; Whänau Ora Taskforce, 2009) added

the following characteristics:

• privacy;

• fullunderstandableinformationdisclosure;

• consistentinformationacrossphone

channels and sites;

• staffandinformationthatissensitiveto

Mäori world views and cultural practices;

• staffadequatelytrainedandknowledgeable

about other services; and

• anextendedfocusnotjustonthe

individual Mäori consumer but if possible

so that whänau can use and strengthen

their own resources and expertise.

The State Services Commission encourages

agencies to use the Common Measurements

Tool5 for ascertaining customer satisfaction

(State Services Commission, 2010b).

5 The tool allows state sector agencies to measure client satisfaction and identify service delivery improvements for service users in a format that can allow international benchmarking with Canada and other jurisdictions that adopt it.

19


Raising performance between sites/offices

Drawing upon Molsely and Moller’s (2007)

study of employment services in Germany the

international literature review highlighted a

standard finding that there is considerable

variation in the relative efficiency of individual

sites or local offices (Gill and Russell, 2011,

p.18). Regression analysis may be able to

unpack the relative contributions of different

variables in explaining differences in results

achieved. Hence there is scope for agencies

that deliver front-line services to Mäori to

raise the average performance of individual

office sites by increasing the performance

of the lowest quartile of local offices to the

national average.

20


4.1 THE ABILITY OF MÄORI TO SUCCEED AS MÄORI

A number of Mäori authors argue that Mäori

wellbeing should be measured against Mäori-

specific outcome indicators that are derived

from their own culture and resources.

In Te Puni Kökiri’s view, this argument can be

equally valid for the measurement of agency

or service performance. If only comparative

measures are used, this assumes that Mäori

must become ‘as good as’ other ethnic

groups, when Mäori may aspire to be better

or different. This has been called at times a

deficit approach (Durie, 2003a, p.202; Durie

et al., 2012, p.45).

4.2 MÄORI-SPECIFIC MEASURES ARE ATTUNED TO MÄORI REALITIES AND MÄORI WORLD VIEWS

Durie (2003a, 2006) argues the Mäori-specific

approach moves away from a comparative

approach. The aim is not to compare Mäori

with non-Mäori (or any other specific ethnic

group), but to assess norms that are attuned

to Mäori realities and Mäori world views

(Durie, 2006). Durie argues that although

universal indicators and measures can

be applied to Mäori, as they can to other

populations, there are additional unique

characteristics of Mäori that require special

measurement. Although Mäori are far

from homogenous: “There are nonetheless

sufficient commonalities to warrant

treatment as a distinctive population…..”

(Durie, 2003a, p. 312).

Capturing participation as Mäori requires the

use of Mäori-specific measures related to

personal wellbeing. Mäori-specific measures

are attuned to contemporary Mäori realities

and Mäori world views (Durie, 2004). It

is important to note the emphasis on

contemporary Mäori society, as indigenous

cultures are dynamic and responsive to the

realities and needs of ‘their time’ (Henriksen,

2008, p.36).

Examples of Mäori-specific measurement

For instance, a Mäori-specific measure of

adequate housing might take into account

the level of provision for extended families

and manuhiri. A measure of educational

attainment might include measures that

relate to the use and knowledge of Mäori

language. Similarly Mäori-specific indicators

for employment might include employment in

Mäori-designated positions, participation in

affirmative action programmes or participation

in employment that supports Mäori cultural

development (Durie, 2003a, p.315). It may

also include ascertaining if the employment

supported Mäori staff to be Mäori (e.g.

whether the employment supports te reo

Mäori in the workplace, and other Mäori

cultural practices, such as leave or flexible

working hours for tangi and other cultural

responsibilities).

The challenges posed by existing data collection methods

In a similar vein, and in the context of

measuring performance for Mäori, Statistics

PART 4 : MÄORI-SPECIF IC MEASURES AND THEIR CHARACTERISTICS : THE LESSONS FROM LITERATURE ON MÄORI WELLBEING

21

M Ä O R I - S P E C I F I C M E A S U R E S A N D T H E I R C H A R A C T E R I S T I C S : T H E L E S S O N S F R O M L I T E R A T U R E O N M Ä O R I W E L L B E I N G

New Zealand (2002) explains that historically

governments have had their own reasons

to collect statistics on Mäori, and that

this has meant that Mäori statistics have

tended to meet specific government needs

(concurring with Durie’s view), rather than

overall development aspirations of Mäori.

Mäori statistics tend to represent non-Mäori

analytical frameworks and philosophical

approaches, fail to represent Mäori realities,

and are based on the underlying assumptions

that some Mäori have questioned the

relevance of that data.

4.3 WELLBEING FOR THE INDIVIDUAL AND THE GROUP

A key difference between Mäori writings and

the domestic and international literature is

their emphasis on measuring wellbeing for not

just the individual but the group. Durie has

consistently argued that indicators should be

able to capture both the individual and the

group (see Durie, 2003c; Durie et al., 2005c).

Durie (2003a) suggests that in order to promote

positive Mäori participation in society as

Mäori6 and in Mäori society, Mäori-specific

measures must include some measurement at

a level wider than the individual; otherwise

they will not adequately reflect a Mäori

world view. Humpage (2002, pp.192-3) notes:

“Key indicators of economic status such

as…..employment……do not account for the

collective economic status of whänau (extended

family), hapu and iwi. Nor do they take note of

the culturally significant relationships between

individuals and hapu/iwi or other Mäori groups

(Kawharu, 2001, pp.3-4)”.

4.4 MEASURES CAPTURING WHÄNAU ADVANCEMENT

Whänau – the foundation Mäori institution and most effective change agent

Durie and several other Mäori academics

(Durie, 2003a, 2003c, 2006; Durie, Fitzgerald,

Kingi and Stevenson, 2003b; Durie et al.,

2005c; Durie and Kingi, 2010) say that

measuring the impact of a service on a client’s

whänau is important because whänau is the

foundation Mäori institution. Whänau has the

potential to point its own members towards

good outcomes in both the generic and Mäori

sense. Whänau have also been identified as

the most sustainable and effective agents of

change for Mäori.

The strength of using both individual and whänau measures

Client and whänau impact measures (for

service delivery) should include three different

perspectives of the impact or outcome

achieved, those of: the client and service

provider; and a member of the client’s

whänau. This would involve ascertaining their

views of progress or lack thereof, against

pretested questions at practical intervals

(Durie and Kingi, 2010, p.36).

The theory of measuring whänau impacts

Therefore what is needed are measures that

ascertain: how an intervention impacts

on Mäori customers’ wellbeing as Mäori,

and their relationships with their whänau

6 Mäori participation in society is about being able to participate while retaining a Mäori identity (Durie, 2003a, p.309).

22


including their capacity to perform those tasks

expected of whänau (Durie and Kingi, 2010);

and the status of whänau by assessing their

capacities to perform those tasks expected of

whänau (Durie, 2003c). Key characteristics of

whänau relationships are: collective identity,

interdependence, mutuality, reciprocity and

shared responsibility within a Mäori context.

Whänau is more than the socio-economic

status of individual members. A whänau’s

strengths or weaknesses cannot be fully

measured by adding together the sum total of

indicators based on the circumstances of its

individual members.

Durie et al. (2012, p.39) suggest Mäori-specific

wellbeing measures (whänau or otherwise)

are likely to have maximum utility if they are

able to be applied across a range of agencies,

sectors, and organisations, regardless of the

size or operational design.

4.5 STRENGTH-BASED MEASURES OF ADVANCEMENT IN CONTEMPORARY TE AO MÄORI

A measure of Mäori wellbeing and other

Mäori specific measures should emphasise

progressive advancement rather than the

management of adversity. Several Mäori

authors see this as relevant to aspirations

towards self-determination. O’Sullivan (2006,

p.7) asserts that “Mäori perceive development

as development as Mäori, not advancement

by assimilation”. In other words, development

for Mäori “includes their desire to take charge

of their own development [as Mäori, not just

as citizens]; an ongoing interest in self-

determination, autonomy, and involvement

in policies and programmes that affect them”

(Ringold, 2005, p.8). Mäori have their own

perception of their world and how it should

evolve. Durie (2003a, p.312) points out that

under the Mäori-specific approach the aim

is to assess outcomes for Mäori against

contemporary norms common amongst Mäori,

not to compare Mäori with any other ethnic

population group or collective of groups.

Care is needed to ensure the measures reflect

the diverse cultural realities of Mäori and are

not shaped by views which are outdated, too

constrained, or out of alignment with the

contemporary environment (Durie and Kingi,

2012, p.40).

Integrated approach

Models of Mäori development and social

advancement have all stressed the need for an

integrated approach to Te Ao Mäori outcome

states. “A deficit in any of the domains would

constitute an unhealthy position (for Mäori

at least) and irrespective of how favourable

the other outcome domains are. To this end,

Mäori measures of effectiveness are likely to

be mutually reliant – that is a positive outcome

might only be achieved when all constructs

show positive gains” (Durie et al., 2012,

p.41). Agencies need to do more to: recognise

Mäori potential and capacity and strengths,

particularly within the community; and use

that information to address Mäori concerns

(Durie et al., 2012, p.42). Durie et al. (2012,

p.50) argue that agencies should explore the

“opportunities to exploit the positive aspects of

Mäori society, Mäori endeavour, Mäori culture

or Mäori success…” Their work and Hua Oranga

also suggests that agencies with a provider arm

might be best placed to trail such measures.

23

M Ä O R I - S P E C I F I C M E A S U R E S A N D T H E I R C H A R A C T E R I S T I C S : T H E L E S S O N S F R O M L I T E R A T U R E O N M Ä O R I W E L L B E I N G

4.6 MEASURES FOR MÄORI AS PART OF A MEASUREMENT FRAMEWORK

Mäori outcomes require multiple dimensions

and indicators/measures of performance

reflecting the holistic nature and concepts of

Mäori wellbeing. Mäori-specific or cultural

measures might best be employed as part

of a framework to measure improvement

in Mäori outcomes. The framework should

capture wellbeing for the individual and Mäori

groups, such as whänau (as discussed above),

and the Mäori population as a whole. Ideally

the framework would capture indicators and

measures of advancement for other significant

Mäori groups such as hapu and iwi.

Developing and trialling a framework

In building such a framework, Durie and Kingi

(2010, p.31) suggest practitioners need to start

from an analysis of what the literature says

are important outcomes or impacts from an

intervention for, and according to, Mäori clients.

As with mainstream universal and

comparative measures, such a framework of

Mäori-specific measures would need to be

trialled prior to implementation. Importantly,

this would help to refine the outcome or

impact dimensions that are important to

Mäori clients and the key aspects of whänau

advancement to be captured.

Durie et al. (2012, p.41) also point out that

either new data collection mechanisms will be

needed or the reinterpretation or analysis of

existing information will be required.

4.7 TE HOE NUKU ROA AND HUA ORANGA FRAMEWORKS

The literature review found a limited number

of Mäori-measurement framework examples

in the public domain. Significant frameworks

were Te Hoe Nuku Roa and Hua Oranga,

the groundbreaking indigenous health

measurement framework in the health sector.

The Hua Oranga framework has impact

measures which capture Mäori customer cultural

enhancement (or not) during, and at the end of

an intervention (Durie and Kingi, 2010).

Te Hoe Nuku Roa was developed from a

longitudinal Massey University study involving

550 households and is in use in the health

and housing sectors. It is a study built on

a relational framework made up of four

interacting axes: paihere tangata (human

relationships); Te Ao Mäori (Mäori culture

and identity); ngä ähuatanga noho-ä-tangata

(social-economic circumstances); and ngä

whakanekeneke (change over time). Each axis

forms a pütake (root), from which subsets,

ngä peka (branches), take form, resulting in

ngä rau (leaves), the areas of inquiry that will

provide essential information both to quantify

and qualify ngä peka and ngä pütake.

The benefit of this framework is one axis

can be linked with items on the other. This

provides for the creation of a more complete

profile of Mäori than has been possible in

the past. Most descriptions of Mäori have

suffered from cross-sectional limitations

and a single sectoral interest (Te Pümanawa

Hauora Research Centre for Mäori Health and

Development, undated).7

7 In 2009 the study applied the Healthy Housing Index to 80 homes in the Nelson/Marlborough region, gathering information on the structure and conditions of various homes.

24


Illustrative example: applying elements of Te Hoe Nuku Roa to employment

By way of illustration as to how the framework could be used in a multitude of settings,

key points of connection between measuring the effectiveness of a mainstream employment

service for, and with, Mäori, and the Te Hoe Nuku Roa framework are underlined in

Table Three below.

Table Three - Te Hoe Nuku Roa framework

Ngä Pütake Axes

Ngä Peka Subsets

Ngä Rau Focused units of inquiry

Axis 1 Paihere Tangata

Human Relationships

Individual

Family

Household

Whänau

Household Roles and relationships Whänau cohesion

Interdependence

Axis 2 Te Ao Mäori

Mäori Identity

Mana ake (personal identity)

Taonga tuku iho (cultural heritage)

Ngä rawe a Rangi räua kö Papa (natural resources)

Whakanohohanga Mäori (Mäori institutions)

Ethnic affiliation Language

Tikanga Land Fisheries Forests Environment Marae Hapu activities Iwi links

Axis 3

Ngä ähuatanga noho ä tangata

Socio-economic circumstances

Oranga tangata (wellbeing)

Whai taonga (societal standing)

Whai huanga (economic position)

Health Education Housing Employment

Lifestyles Income

Axis 4 Ngä Whaka-nekeneketanga

Change over time

Changing household dynamics

Wider interactions

Shift in cultural identity

Altered circumstances

MobilityStability Realisation of aspirations Vulnerability Impact of external factors New groupings

25

K E Y I M P L E M E N T A T I O N I S S U E S F O R M E A S U R I N G P E R F O R M A N C E F O R M Ä O R I

5.1 GOOD QUALITY DATA AND ROBUST DATA ANALYSIS

The availability of reliable data will be a

key challenge for agencies in carrying out

performance measurement by ethnicity (Durie

et al, 2012; Ringold, 2005).

Ethnicity data

The fluid nature of ethnicity poses one of the

main challenges. People in New Zealand, as

in other countries, may change the ways in

which they identify themselves over time, or

they may identify themselves differently in

different environments, such as at work and at

home (Statistics New Zealand, 2005).

Ethnicity data is based on self-identification,

but that self-identification, for a variety of

reasons, is not always possible. When self-

identification is not possible, proxy responses

are often collected. This creates inaccuracies

because in these cases the individual has

their ethnicity identified by a third party

on their behalf, based on the third party’s

perception of the individual’s ethnic identity.

Nonetheless, agencies are expected to comply

with Statistics New Zealand’s classification

standards on ethnicity.

Kukutai (2004) suggests agencies ought

to be more forthcoming on how Mäori is

defined, and the assumptions about why and

how it is deployed. There are guidelines for

the state sector on how to use the non-

Mäori group in ethnicity statistics (Statistics

New Zealand, 2005).

Many mainstream services would benefit from

Mäori-inclusive evaluations (Thomas, 2002),

which would provide information that informs

PART 5: KEY IMPLEMENTATION ISSUES FOR MEASURING PERFORMANCE FOR MÄORI

improvement. There would also be benefit

in agencies sharing data sets and the results

of evaluations to build an evidence base

about progress towards desired longer term

outcomes (State Services Commission, 2011b).

5.2 ORGANISATIONS NEED THE RIGHT CAPABILITY

In order to maintain ethnicity data quality,

agencies need the right leadership, and technical

and policy expertise to drive demand for it.

Leadership

When agencies set ethnicity targets they

need to be supported by a champion at senior

management level (The Treasury, 2004, 2006).

Mäori under-represented in key public sector roles

One of the main themes that emerges from

the literature is the under-representation of

Mäori in the public service in many influential

roles: chief executive; senior management;

middle management; policy; professional; and

technical roles (State Services Commission,

2003a, 2010a, 2010c, 2011c; Human Rights

Commission, 2011a). A number of sources note

that diversity is important for effectiveness

of public services (OECD, 2009 discussed in

State Services Commission, 2010c, and Human

Rights Commission (2011a, p.32).

Durie (2004, p.6) suggests that: “Increasing

the level of direct involvement of Mäori in

the State is important and can be justified

on several grounds but unless the effort

leads to demonstrable benefits for Mäori the

exercise becomes one of primarily creating

equal employment opportunities rather than

26


specifically advancing Mäori people. The

equal employment goal is not insignificant

but it is a different aim from achieving best

outcomes for Mäori.” One impediment is the

“innate tendency for a dominant group to

tend to appoint people like themselves and

listen more to people like themselves – often

unaware of the bias involved” (State Services

Commission (2004), discussed in Human

Rights Commission (2011a, p.33).

Durie et al. (2012, p.44) suggest that for

agencies that have high numbers of Mäori

clients, one measure of an effective Mäori

workforce is the extent to which the agency

has a corresponding high number of Mäori

staff employed at various levels, particularly at

managerial or senior levels.

The responsibility of all public servants

It is also worth noting that effectiveness for

Mäori is not just the responsibility of Mäori

staff, but a Crown responsibility for all public

servants (Ministerial Advisory Committee on a

Mäori Perspective for the Department of Social

Welfare, 1988, pp.78-9; Love, 2001, discussed

in Humpage, 2002, p.186; The Families

Commission, 2012). Durie et al. (2012, p.44)

place importance not just on the numbers

of Mäori staff in key positions, but also the

extent to which all staff throughout an agency

understand Mäori perspectives.

Systems and structures

Systems and structures can also play a

part. For example, policy advice is often not

developed with the implications of Mäori and

Pacific peoples in mind, and there is currently

no requirement to show Cabinet that policy

advice has considered the direct or indirect

implications of that advice for Mäori (Human

Rights Commission, 2011a, p.31).

Organisational culture needs to foster and support Mäori perspectives

Durie et al. (2012, p.44) suggest what is needed

is an organisational culture that fosters and

supports Mäori perspectives, and that Mäori

in senior positions are able to maintain their

“Mäori-ness” without compromise.

5.3 STAFF NEED TO FEEL CONFIDENT OF THE REASONS FOR MEASURING PERFORMANCE IN RELATION TO MÄORI (OR ANY OTHER ETHNIC GROUP)

Focusing on Mäori can be politically contentious

The arguments in favour of measuring

performance in relation to Mäori have been

stated above in Part One. Nevertheless, it can

be difficult for the state sector and the wider

public to accept the legitimacy of difference

and indigenous aspirations (O’Sullivan,

2006, p.8). Humpage (2008) suggests this

is because focusing on Mäori or addressing

ethnic inequity to improve the socio-economic

status of Mäori tends to be perceived by the

press and much of the state sector generally,

as privileging Mäori on the basis of race

over others. Humpage’s views are shared

by O’Sullivan (2006) and the Human Rights

Commission (2011a).

In view of the above factors, government

agencies can experience pressure to report

on what is safe and easily measurable, with

performance indicators for improving outcomes

for Mäori selected to ensure they do not

embarrass the chief executive or minister

(Humpage (2008, pp 418-9). See also Gill

and Russell, 2011). This also creates tensions

between the purposes of collecting information

for accountability and for service improvement.

The former implies some risk averseness, while

the latter implies a willingness to innovate (Gill

and Schmidt, 2011, p.16).

27

K E Y I M P L E M E N T A T I O N I S S U E S F O R M E A S U R I N G P E R F O R M A N C E F O R M Ä O R I

5.4 THE NEED FOR FORETHOUGHT

Much of the international literature notes that

developing good performance measures and

reporting requires a lot of forethought around

the behavioural implications that might result

from particular indicators (Gill and Russell,

2011, pp.10-12, 36-42). The international

literature clearly identified examples of

gaming and cheating in the delivery of

employment services, and associated with

performance measures of ethnicity based on

administration data.8

More broadly the international literature noted

the dynamics established by performance

measurement of performance management

means that four perverse laws operate, namely

the laws of:

• decreasingeffectiveness(thesystem

is perverted);

• mushrooming(thesystemisbloated);

• collectiveblindness(myopiabasedon

short-sighted putting too much weight on

production figures); and

• preservingpervertedsystems(i.e.

insufficient incentives for abandoning the

system) (de Bruijn (2006, p.37) discussed in

Gill and Russell, 2011, p.11).

While it is not the topic of this literature

review, international sources provided some

useful advice about reducing perverse strategic

behaviour (see, for example, de Bruijun, 2006).

5.5 THE NEED FOR LONG-TERM COMMITMENT

Successfully measuring and improving

effectiveness for Mäori requires agencies to

have a consistent and long-term commitment

to measuring and improving outcomes for

Mäori. This commitment is needed because

changes in high level outcome states can

take a long time to materialise. Other

contributing factors include the challenging

nature of improving high level outcomes for

Mäori (Humpage, 2002, 2008; Human Rights

Commission, 2011a). Developing robust Mäori-

specific performance measures for the benefit of

Mäori consumers of the service can take years.

The Te Hua Oranga measurement framework tool

took 10 years of trialling to develop.

5.6 THE NEED FOR COLLABORATION

Strong leadership is also required in order to

ensue “a whole of government approach if

the needs of diverse population groups are

to be addressed effectively” (Human Rights

Commission, 2011a, p.33).

“The ability of an agency to work

collaboratively with other agencies was

considered essential and a critical measure of

effectiveness (for measuring effectiveness from

a Te Ao Mäori perspective), as was the ability

of an agency to put in place interventions that

were more holistic in nature and addressed

multiple needs” (Durie et al., 2012, p.45). Many

of the social issues experienced by Mäori

consumers of social services are complex (as

with other clients), often linked to a broad

range of concerns, not able to be addressed

by a single agency or even single intervention

and therefore require integrated and

comprehensive solutions. This is particularly

true for agencies that seek to facilitate a

whänau approach to resolve problems, and

measure effectiveness for Mäori consumers

and their whänau (Durie, et al., 2012).

8 One example was the use of unnecessary interventions targeted at the easy-to-place job seekers in order to improve the recorded performance of the measures. The list of hazards relevant to employment services includes:

•cheatingandgaming(Radnor,2008);•creatingunintendedconsequences(seeNun,BickkerstaffeandMitchell,2009,onnegativeexternalities);•excessivecostsrelativetothebenefits(deBruijn,2006);and•displacementleadingtoworthwhileactivitiesbeingde-prioritised(discussedinGillandRussell,2011,p.17).

28


In summarising the domestic academic

opinion, three main reasons emerge for

measuring agency and service performance

in relation to Mäori:

• Thisperformanceinformationcanfeed

continuous improvement to government

services, increasing the transparency,

efficiency and effectiveness of government

spending.

• Mäori,asindigenouspeopleandTreaty

partners, may have different interests and

preferences, meaning that universal and

comparative performance measurement

may not always provide the best

information.

• FocusingonMäoritoachieveequityis

consistent with our international human

rights obligations and is endorsed by the

UN Committee for the Elimination of Racial

Discrimination and the OECD.

Notwithstanding these arguments, a focus

on Mäori can be politically contentious and

requires some determination and leadership. In

some cases, it also requires more sophisticated

data capture and analysis, meaning that

agencies would have to invest in their systems,

capability, and organisational culture.

In a significant nexus between the

international and domestic academic

literature, both sets of opinion conclude that

there is no ‘one-size-fits-all’ performance

measurement framework. Designing a

successful framework should take into account

the needs of the stakeholders, including

citizens and service users. Engaging those

stakeholders can add legitimacy and public

and political buy-in to the measures.

PART 6 : CONCLUSION

Engaging with Mäori communities on these

matters may bring into focus the need for

Mäori-specific measures, which measure

progress against Mäori-determined priorities

and cultural preferences.

It is clear that comparative measurement of

ethnic differences in universal outcomes (e.g.

the national employment rate) remains a valid

tool. Some authors sound a caution against

using traditional methods of comparative

measurement in a way that would produce

a ‘disparity’ analysis. Some also recommend

providing full ethnic comparison (i.e. Mäori;

New Zealand Europeans; and Pacific people),

which offers a richer picture than the

traditional Mäori/non-Mäori analysis.

If there is a serious disconnection within

the literature, it is between the academic

literature and the ‘official requirements.’ The

official requirements lack up-to-date advice

on collecting and analysing performance

information about Mäori or other ethnicities.

The last comprehensive set of guidance from

one of the central agencies was issued in

1998, by the Controller and Auditor-General.

This points the way to further work. The

government is implementing its Better Public

Services reforms, which aim towards better

results achieved more efficiently. It has

set specific targets for improved outcomes

from services of which Mäori are significant

consumers. It would be timely, then, to

consider the findings of this literature review

in the development of guidance and advice to

the state sector.

29

G L O S S A R Y

Aspirational goals

Expected performance that will not be

achieved in the next year.

Attribution

The extent to which an impact or outcome

can be directly assigned to the activities

undertaken by an agency or agencies.

Capability

What an organisation needs in terms of access

to leadership, people, culture, relationships,

processes and technology, physical assets, and

structures to efficiently deliver the goods and

services required to achieve the results sought

by the entity - whether those results are set by

reference to government policy or by statute.

Cost-effectiveness

Describes the relationship between the level

of resources used (costs) and progress toward

a predetermined outcome (effect). The terms

‘cost-effectiveness’ and ‘interventions’ are

not formally defined in legislation, so a broad

interpretation should be taken consistent

with the expectation that the major activities

that entities carry out add real value at a

reasonable cost. Assessing cost-effectiveness,

and identifying cost-effectiveness intentions,

involves entities tracking performance over

time to allow readers to understand whether

the costs of services are justified by the

impact and outcome results produced.

Dimensions of performance

Are the aspects or properties of performance

that a particular performance measure can

address. They include, but are not limited to,

quantity, quality, timeliness, location, and cost.

GLOSSARY

Disaggregation

The process of deconstructing an output

or outcome into their component parts, or

reporting performance separately for different

groups, e.g. population groups or product lines.

Effectiveness

The difference agencies make through the

services they provide. Effectiveness focuses on

the impact that has been achieved through

the delivery of one or more outputs.

Efficiency

The price of producing a unit of output

(‘technical efficiency’). Alternatively, it can

mean the proportion of output reaching target

groups (‘allocative efficiency’). To make valid

comparisons the outputs or output mixes

being compared must be homogenous.

Elements of non-financial performance reporting

Include inputs, outputs, impacts, and

outcomes, which can be measured for the

purpose of reporting and assessing the entity’s

performance.

Equal employment opportunity

For the purposes of section 58 of the State

Sector Act an equal opportunities programme

means a programme that is aimed at the

identification and elimination of all aspects

of policies, procedures, and other institutional

barriers that cause or perpetuate, or tend to

cause or perpetuate, inequality in respect to the

employment of any persons or group of persons.

Intervention

Interventions include legislation, policies,

transfers, programmes and service delivery

arrangements.

30


Impact

The contribution made to an outcome by a

specified set of outputs, or actions, or both.

Impacts are the contributions made to an

outcome by a specified set of outputs. Often

referred to as “intermediate outcomes”, they

represent the relatively immediate or direct

effect on stakeholders of the entity’s outputs.

Intermediate outcome

Articulates the effect that an agency’s

services and interventions are having on New

Zealanders. Intermediate outcomes allow an

agency or sector to determine what difference

it is making through the services it is providing

with its outputs, and to discern progress

towards the achievement of outcomes.

Intervention logic

The strategic and/or operational articulation

of how one or more interventions will produce

desirable outcomes for New Zealanders,

including valid measures of success at the

output, intermediate outcome and outcome

levels of performance.

Main measures

Are the medium-term (at least three-

year) measures the entity uses to assess

and report on its impacts, outcomes, or

objectives, and the cost-effectiveness of its

interventions. Main measures can cover the

entity’s organisational health and capability

to perform its functions and conduct its

operations effectively, and any other matters

necessary to understand its operating

intentions and, for government departments,

direction specified by their minister.

Objectives

The term “objectives” recognises that not

all outputs and activities and departmental

functions are intended to achieve “outcomes”

as that term is defined above. Some

outputs and activities and departmental

functions – such as ministerial servicing,

or managing certain core Parliamentary

or government processes like drafting

legislation or appointing departmental chief

executives – do not target a direct societal,

economic or environmental effect, and their

effectiveness should not be judged on that

basis. Nevertheless, they can be important in

the context of departmental performance, and

if so should be reported on.

Outcome

A state or condition of society, the economy or

the environment and includes a change in that

state or condition.

Outputs

Outputs are final goods and services – that

is, they are supplied to someone outside the

entity. They should not be confused with

goods and services produced entirely for

consumption within the department. Output

classes are groups of similar outputs.

Standard

An intended level of performance within

a stated timeframe.

Whänau

Can mean a whakapapa whänau (a whänau

tied together by kinship) or a kaupapa whänau

(a group tied together by other kinds of shared

interests). It is possible to use similar outcome

sets and measures for both.

A P P E N D I C E S

32


INFORMATION ON THE WORKING/BACKGROUND RESEARCH PAPERS FOR THIS REVIEW

The international literature review

This review was commissioned by Te Puni

Kökiri from the Institute of Policy Studies (IPS)

at Victoria University; IPS literature analysis

for Te Puni Kökiri on performance measures to

improve service effectiveness for Mäori (Gill

and Russell, 2011). Te Puni Kökiri supplied IPS

with 48 recent contributions and 20 older

sources from the international literature on

performance measurement and management.

• Internetsearcheswerecompletedusinga

variety of search engines.

• Thereferencesectionsofallthemost

pertinent literature were reviewed, and any

relevant leads were pursued.

• Severaloftheleadingauthorsin

the performance management and

measurement field were approached for

any references they were aware of that

might be of relevance.9

• TheIPSliteraturereviewsupplementedTe

Puni Kökiri’s literature review with a further

30 articles or monographs.

The official requirements review

That report analysed canvassed 25 official

primary guidance and requirement reports.

There is no one up to-date “master document”

to guide performance measurement on

the public sector intranet in this area. The

number of official requirements and guidance

documents add to subject matter complexity.

The summary requirements report

The summary report summarised the above

review and elaborated slightly with a few

additional sources. The summary report

covered: development requirements;

principles; working out what to measure;

hierarchy of measures; standards and targets;

testing; performance management systems;

data collection; reporting requirements for

2011/12 accountability documents; leadership

expectations; issues; and barriers.

Employment-specific examples (“the desktop report”)

This report analysed MSD’s performance

reporting of its front-line services: for the

appropriations:10

• OutputClassD2.OutputExpense:Services

to minimise the duration of unemployment

and move people into work (used from

2002/03 to 2007/8); and

• Tailoredsetsofservicestohelppeopleinto

work and achieve independence (used from

2008/09 onwards).

APPENDIX ONE

9 The leading authors approached included Professor Patria de Lancer Julnes from the University of Baltimore, Associate Professor Alfred Tat-Kei Ho from Kansas University. Associate Professor Alfred Tat-Kei Ho confirmed this was internationally groundbreaking work and were keen to find out how Te Puni Kökiri got on (Ho, personal communication, 2011). Professor de Lancer Julnes, one of the world’s leading academics in this area, confirmed: “[She had] not come across any research that deals with performance measurement and monitoring for indigenous people. In fact, research on the inclusion of monitories in any kind of government endeavours tends to be scant” (de Lancer, personal communication, 2011).

10 The Appropriation averaged just under $450 million per annum.

33

A P P E N D I X O N E

The “Mäori Perspective” Report

This report was prepared by Kim Aiomanu,

Monitoring Senior Analyst at Te Puni Kökiri. It

consulted over 100 academic and practitioner

reports and publications, sourced by Te Puni

Kökiri library and internet searches. Dr Te

Kani Kingi was asked to provide a comment

on the report. He found that it “captures the

main issues very well and it’s an excellent

examination of the key issues for Mäori”

(Kingi, personal communication, 2012).

34


HUA ORANGA

This framework/tool was developed in a

mental health context, but upon examination

Te Puni Kökiri believes it has potential to be

adopted and adapted to other contexts, such

as employment services the latter being an

illustrative service examined in this

literature review.

ELEMENTS

The tool proposes:

• Amoreaccurateimpressionofoutcome

could be obtained by capturing the client’s

and clinician’s (or the person doing the

intervention) impressions of outcome

performance, along with the views of

whänau. Key stakeholders are: clients;

the service provider; and a member of the

client’s whänau.

• Fourdomainsofoutcomereflectingthe

holistic nature and concepts of Mäori

health and wellbeing.

• Fiveendpointsinthetreatmentprocessat

which it makes sense to measure outcomes

(assessment, inpatient, outpatient treatment,

community care and discharge). The end

points align with contemporary service

structures and appreciate the need for a

dynamic tool, capable of responding to a

number of consumer treatment variables.

• Asetofquestionsthatcouldbeadoptedfor

measuring the effectiveness of a mainstream

service like Job Search Service for Mäori

from a holistic Mäori view of wellbeing.

HUA ORANGA MEASUREMENT QUESTIONS

Particularly useful base-line questions from

Hua Oranga that could be adopted for

measuring the effectiveness of a mainstream

employment service (like Job Search Service)

for Mäori, from a holistic Mäori view of

wellbeing are presented in the following table.

APPENDIX TWO

35

A P P E N D I X T W O

Illustrative examples of Hua Oranga base-line questions that could be trialled for employment services

Table Four - Hua Oranga measurement questions

Dimensions Questions for the dimensions

Wairua Client –

As a result of the

intervention do

you feel

Service provider –

As a result of the

intervention does the

client feel

Whänau –

As a result of the

intervention does

your relative/whänau

member feel

1 Dignity, respect More valued as a person

2 Cultural identity Stronger in yourself as Mäori

3 Personal contentment More content within yourself

4 Spiritually

(non-physical experience)

Healthier from a spiritual point of view

Dimensions Questions for the dimensions

Whänau Client -

As a result of the

intervention do

you feel

Service provider -

As a result of the

intervention does the

client feel

Whänau–

As a result of the

intervention does your

relative feel

1 Communication More able to communicate with your whänau

2 Relationships More confident in your relationships with other people

3 Mutuality Clearer about the relationships with your whänau

4 Social participation More able to participate in your community

36


Illustrative examples of generic impact measures for employment services

Examples of generic impact measures for employment services from MSD’s recent SOIs are

set out in Tables Five and Six below.

Table Five - Measures for employment services from MSD’s 2012/15 SOI

APPENDIX THREE

Ministry outcome - More people get into work and out of welfare dependency

Intermediate Outcome Measure Target Comment/current Result

Fewer clients are reliant

on welfare

The proportion of clients

who get work before they

require a benefit

Increasing proportion New measures for 2012/13


who are working part time



who cancel their benefit and

exit into employment


Fewer clients require a

benefit long term


that are work ready



who remain on a working

age benefit for longer than

12 months

Decreasing proportion New measures for 2012/13

37

A P P E N D I X T H R E E

Table Six - Measures for employment services from MSD’s 2011/14 SOI

Ministry outcome - More people get into work and stay in work

Intermediate Outcome

Measure Current Historic states and trends

More clients get work before they require a benefit

Proportion of people who do not require benefits within 28 days of attending a Work for You seminar:

•Youth

•General

48.5%

43.1%

Increasing

Increasing

More clients are assisted to be work-ready

Proportion of people who participated in the Job Search Service who do not remain on Unemployment Benefit for longer than 13 weeks

45.2% Increasing

More clients are preparing for work

Proportion of people who participated in the Job Search Service who do not remain on Unemployment Benefit for longer than 13 weeks

45.2% Increasing

More clients are preparing for work

Average cumulative time spent in employment (over a 12-month period) by people who exit:

Unemployment Benefit

Work-ready Domestic Purposes Benefit and Sickness Benefit

38.6 weeks

36.4 weeks

Increasing

Increasing

More employers employ our clients

Number of Unemployment benefit job seekers who get work through employer and industry partnership programmes and services and cancel their benefit

New measure for 2011/2012

New measure - no trend available

38


Alexander, W., M. Genc and M. Jaforullah (2003) Mäori Disadvantage in the Labour Market, Dunedin: Department of Economics, School of Business, University of Otago of New Zealand

Biddulph, F., J. Biddulph and C. Biddulph (2003) Best evidence Synthesis: The Complexity of Community and Family Influences on Children’s Achievement in New Zealand, Wellington: Ministry of Education

Callister, P. (2007) Special Measures to Reduce Ethnic Disadvantage in New Zealand, Wellington: Institute of Policy Studies, Victoria University of Wellington

Callister, P. (2008a) ‘Skin colour: Does it matter in New Zealand?’, Policy Quarterly, 4, (1), pp.18-25

Callister, P. (2008b) Skin colour: Does it matter in New Zealand? Working Paper, Updated March 2008, Wellington: http://callister.co.nz/skin-colour.pdf

Carter, K. and F. Gunasekara (2012) Dynamics of Income and Deprivation in New Zealand, 2002-2009 A descriptive analysis of the Survey of Family, Income and Employment (SoIFE), Public Health Monologue series No 24, Wellington: Department of Public Health, University of Otago of New Zealand

Chang, H. and T. Dodd (2001) ‘International Perspectives on Race and Ethnicity: An Annotated Bibliography’, Electronic Magazine of Multicultural Education [online journal].

Carter, K. and F. Gunasekara (2012) Dynamics of Income and Deprivation in New Zealand, 2002-2009 A descriptive analysis of the Survey of Family, Income and Employment (SoIFE), Public Health Monologue series No 24, Wellington: Department of Public Health, University of Otago of New Zealand

Cunningham, C., B. Stevenson, E. Fitzgerald and R. Rolls (2006) Mäori Women in Aotearoa A Report for the Ministry of Women’s Affairs Analysing Data From Best outcomes for Mäori – Te Hoe Nuku Roa Executive Summary, Palmerston North: Research Centre for Mäori Health & Development, Massey University of New Zealand

Dannette, M., D. Fergusson and J. Boden (2011) ‘Ethnicity and Pathways to Welfare Dependence in a New Zealand Birth Cohort’, Policy Quarterly, 7 (2), pp.14-22

de Bruijn, H. (2006) “Managing performance in the public sector”, 2nd ed. New York: Routledge

de Lancer Julnes, P. (2001) ‘Does Participation Increase Perceptions of Usefulness? An Evaluation of a Participatory Approach to the Development of Performance Measures’, Public Productivity and Management Review, 24 (4), pp. 403-418

de Lancer Julnes, P. (2011) Professor School of Public and International Affairs, University of Baltamore, email 1 June 2011 to Derek Gill for Te Puni Kökiri

Department of Corrections (2011) Department of Corrections Annual Report 2010/11, Wellington: Department of Corrections

Dixon, S. and S. Crichton (2007) ‘The Longer-Term Employment Outcomes of People Who Move From a Benefit to Work’, Social Policy Journal of New Zealand, 31, http://www.msd.govt.nz/about-msd-and-our-work/publications-resources/journals-and-magazines/social-policy-journal/spj31/31-the-longer-term-employment-outcomes-of-people-who-move-from-a-benefit-to-work-pages137-157.html

B IBLIOGRAPHY

39

B I B L I O G R A P H Y

Donnell, A., and Social Programme Evaluation Unit, Department of Social Welfare (1985) Quality of Service to Unemployment Benefit Clients at Department of Social Welfare Reception Areas, Wellington: Department of Social Welfare

Dormer, R. (2010) Missing Links Doctoral Thesis submitted to Victoria University. Unpublished PhD thesis, School of Accounting and Commercial Law, Victoria University of Wellington

Durie, M. (1995) ‘Te Hoe Nuku Roa Frameworks: A Mäori Identity Measure’, Journal of Polynesian Society, 104, pp. 461-470

Durie, M. (1998) Te mana, Te käwanatanga: the politics of Mäori self-determination, Auckland: Oxford University Press

Durie, M. (2002) ‘Universal Provision, Indigeneity and the Treaty of Waitangi’ Victoria University of Wellington Law Review, 33, (3/4), pp. 591-602 http://www.victoria.ac.nz/law/research/publications/vuwlr/prev-issues/pdf/vol-33-2002/issues-3-4/durie.pdf

Duire, M. (2003a) Ngä Kahui Pou Launching Mäori Futures, Wellington: Huia Publishers

Durie, M., E. Fitzgerald, Te K. Kingi, S. McKinley and B. Stevenson (2003b) ‘Monitoring Mäori Progress Te Ngahuru: A Mäori Outcome Framework’, New Zealand Population Review, 29 (1), pp.99-109

Durie, M. (2003c) ‘Whänau Capacities: A Framework for Measuring Whänau Outcomes’, in Te Pa Harakeke 2004, 1, Otaki, Te Wänanga o Raukawa

Durie, M. (2004) ‘Public Sector Reform, Indignity, and the Goals of Mäori Development Development’, Public Sector, 27 (2), pp.2-7

Durie, M. (2005a) Mäori Development Trends and Indicators, paper presented at IAOS Satellite Meeting on Small and Indigenous Populations, Wellington: http://www.massey.ac.nz/massey/fms/Te%20Mata%20O%20Te%20Tau/Publications%20-%20Mason/M%20Durie%20Mäori%20Development%20trends%20&%20indicators.pdf

Durie, M. (2005b) ‘Race and Ethnicity in Public Policy: Does it Matter?’, Social Policy Journal of New Zealand, 24, pp.1-11

Durie, M., T. Black, C. Cunningham, A. Durie, F. Palmer and C. Hawkins (2005c) The Parameters of Whänau Wellbeing: A report for Te Puni Kökiri, Palmerston North: Te Mata o Te Tau the Academy for Mäori Research and Scholarship, Massey University of New Zealand

Durie, M. (2006) Measuring Mäori Wellbeing, Treasury Guest Lecture Series, Wellington: Massey University of New Zealand, http://www.treasury.govt.nz/publications/media speeches/guest lectures/pdfs/tglsdurie.pdf Education Workforce

Durie, M. and Te K. Kingi (2010) ‘Hua Oranga’ A Mäori Measure of Mental Health Outcomes, Palmerston North: School of Mäori Studies, Te Pümanawa Hauora, Massey University of New Zealand, http://www.earlypsychosis.org.nz/PDF/NZ/HuaOranga.pdf

Durie, M., Te K. Kingi and M. Graham (2012) A framework for Measuring Te Ao Mäori Perspectives of Effectiveness prepared for Te Puni Kökiri, Palmerston North: Te Mata o Te Tau the Academy for Mäori Research and Scholarship, Massey University of New Zealand

40


Englert, J. (2001) The Application of Self Theory to the Case Management of the Young

Unemployed. Unpublished PhD thesis, Department of Psychology,

Victoria University of Wellington

Gill, D. and R. Dormer (2011) ‘Case Study – Work and Income: How staff define their role and

the performance information they use’, in D. Gill (ed), The Iron Cage Recreated, Wellington,

Institute of Policy Studies, Victoria University of Wellington

Gill, D. and N. Russell (2011) IPS literature analysis for TPK on performance measures to improve service effectiveness for Mäori, Wellington: Institute of Policy Studies, School of Government,

Victoria University of Wellington

Gill, D. and T. Schmidt (2011) ‘Organisational Performance Management: Concepts and Themes’

in D. Gill (ed), The Iron Cage Recreated, Wellington: Institute of Policy Studies, School of

Government, Victoria University of Wellington

Gill, D. (ed). (2011) The Iron Cage Recreated, Wellington: Institute of Policy Studies, School of

Government, Victoria University of Wellington

Henriksen, J. (2008) Research on Best Practices for the Implementation of the Principles of ILO Convention No. 169 Case Study 7: Key Principles in Implementing ILO Convention No. 169, International Labour Office http://www.ilo.org/indigenous/Resources/Publications/

WCMS_118120/lang--en/index.htm

Ho, A-Tat-Kei (2008) ‘Reporting Public Performance Information: The Promise and Challenges of Citizen Involvement' in W. Van Dooren and S. Van de Walle Performance information in the

public sector: how it is used, Basingstoke [England]: New York: Palgrave Macmillan

Ho, A-Tat-Kei (2011) Associate Professor and Director of Public Affairs Programs Iowa State

University, email 22 June 2011 to Derek Gill for Te Puni Kökiri

Human Rights Commission (2010) Guidelines on Special Measures to Ensure Equality, Wellington:

Human Rights Commission

Human Rights Commission (2011a) A fair go for all? Structural Discrimination and Systemic Barriers to Ethnic Equality A discussion paper by the Human Rights Commission, Wellington:


Human Rights Commission (2011b) Tüi Tüi Tuituiä Race Relations in 2010, Wellington:


Humpage, L. (2002) Closing the Gaps? The Politics of Mäori Affairs Policy. Unpublished PhD

thesis, Department of Sociology, Massey University of New Zealand

Humpage, L. (2008) ‘Relegitimating Neoliberalism? Performance Management and Indigenous

Affairs Policy’, Policy & Politics, 36 (3), pp. 413-429

Kawharu, M. (2001) ‘Local Mäori Development and Government Policies’, Social Policy Journal of New Zealand Te Puna Whakaaro, 16, pp 1-15

Kingi, Te K. (2003) ‘Developing Measures to Monitor the Impact of Outcomes on a Specific

Cultural Group Such as Mäori’, paper presented at the Second Annual Measuring, Monitoring

and Managing Outcomes in Health Conference, Auckland: New Zealand

Kingi, Te K. (2012) Director of Te Mata o Te Tau the Academy for Mäori Research and Scholarship,

Massey University of New Zealand, March email to Te Puni Kökiri

41


Kukutai, T. (2004) ‘The Problem of Defining an Ethnic Group for Public Policy: Who is Mäori and

Why Does It Matter?’ in Social Policy Journal of New Zealand, 23, pp. 86-108

Leggatt-Cook, C. (2008) Gendered Pathways to Employment: An Analysis of the Experiences of Mäori, Mixed Mäori and Non-Mäori Young People, Research Report 2/2008 Auckland: Labour

Market Dynamics Research Programme, Massey University of New Zealand

Love, N. (2000) ‘TPKS role to empower Mäori communities’ in Kökiri Paetae, 29 (September): 3,

Wellington: Te Puni Kökiri

Maani, S. (2002) Education and Mäori Relative Income Levels Over Time: The Mediating Effect of Occupation, Industry, Hours of Work and Locality Working paper 02/17,

Wellington: The Treasury

Ministerial Advisory Committee on a Mäori Perspective for the Department of Social Welfare

(1988) Puao-Te-Ata-Tu The report of the Ministerial Advisory Committee on a Mäori Perspective for the Department of Social Welfare, Wellington: Ministry of Social Development

Ministerial Inquiry into the Department of Work and Income (2000) Report of the Ministerial Inquiry into the Department of Work and Income, Wellington: The Minister

Ministry of Health (2001) Monitoring Ethnic Inequalities in Heath, Public Health Intelligence Occasional Bulletin, no 4, Wellington: Ministry of Health

Ministry of Social Development (2003a) Barriers to Employment Among Long-Term Beneficiaries: A Review of Recent International Evidence: Summary, Wellington: Ministry of Social Development

Ministry of Social Development (2003b) Mosaics, Whakaahu Papariki: Key Findings and Good Practice Guide for Regional Co-ordination and Integrated Service Delivery, Wellington:

Ministry of Social Development

Ministry of Social Development (2007) Pockets of Significant Hardship and Poverty, Wellington:

Ministry of Social Development

Ministry of Social Development (2008a) Briefing to the Incoming Government: Social Outcomes Briefing, Wellington: Ministry of Social Development

Ministry of Social Development (2008b) Briefing to the Incoming Government: Policy and Practice Briefing, Wellington: Ministry of Social Development

Ministry of Social Development (2008c) Briefing to the Incoming Minister Policy and Practice Briefing Part 2 Income and Employment for Adults, Wellington: Ministry of Social Development

Ministry of Social Development (2011) Ministry of Social Development Statement of Intent 2011-

2014, Wellington: Ministry of Social Development

Ministry of Social Development (2012) Ministry of Social Development Statement of Intent 2012-2015, Wellington: Ministry of Social Development

Mosley, H. and K. Muller (2007) ‘Benchmarking employment services in Germany’, in J. de Koning,

The evaluation of active labour market policies: measures, public private partnerships and benchmarking, Cheltenham, UK: Northampton, MA: Edward Elga

Moynihan, D. (2008) ‘Advocacy and Learning: An Interactive Dialogue Approach to Performance

Information Use’, in W. Van Dooren and S. Van de Walle, Performance information in the public sector: how it is used, Basingstoke [England]; New York: Palgrave Macmillan

42


New Zealand Institute of Economic Research (2009) Trends in Mäori Employment: A Historical

Perspective and Look Ahead: Report for Te Puni Kökiri, Wellington: New Zealand Institute of

Economic Research

Nunn, A., T. Bickerstaffe and B. Mitchell (2009) International review of performance management systems in Public Employment Services, Norwich: Department for Work and Pensions

OECD (2009) OECD Measuring Government Activity Report, http://www.oecd.org/document/19/0,

3746,en_2649_33735_42858259_1_1_1_1,00.html

Office of the Controller and Auditor-General (1998) Part 4: Delivering Effective Outputs for Mäori, Wellington: Office of the Controller and Auditor-General

Office of the Controller and Auditor-General (1999) Towards Service Excellence: The Responsiveness of Government Agencies to their Clients, Wellington: Office of the Controller and Auditor-General

Office of the Controller and Auditor-General (2002) Reporting Public Sector Performance,

Wellington: Office of the Controller and Auditor-General

Office of the Controller and Auditor-General (2008) The Auditor-General’s observations on the quality of performance reporting, Wellington: Office of the Controller and Auditor-General

Office of the Controller and Auditor-General (2009a) AG-4 (revised) (updated at March 2011) The Audit of Service Performance Reports, Wellington: Office of the Controller and Auditor-General

Office of the Controller and Auditor-General (2009b) The Auditor-General’s Overview Statements of intent Examples of reporting, Wellington: Office of the Controller and Auditor-General

Office of the Controller Auditor-General (2011) Central government: Cost-effectiveness and improving annual reports, Wellington: Office of the Controller and Auditor-General

Office of the Controller and Auditor-General (2012) Education for Mäori: Context for our proposed audit work until 2017, Wellington: Office of the Controller and Auditor-General

O’Sullivan, D. (2006) ‘Needs, rights, nationhood and the politics of indigeneity’, MAI Review 2006

1, Target Article 1, http://www.review.mai.ac.nz/index.php/MR/article/viewFile/1/1

Perry, G. E. (2007) Economic Evaluation of Active Labour Market Policy in New Zealand 1989 to

1997. Unpublished PhD thesis. Auckland: Department of Economics, University of Auckland

Digital Doctoral Thesis https://researchspace.auckland.ac.nz/handle/2292/525

Radnor, Z. (2008) ‘Hitting the Target and Missing the Point? Developing and Understanding of

Organizational Gaming’, in W. Van Dooren and S. Van de Walle, Performance information in the public sector: how it is used, Basingstoke [England]; New York: Palgrave Macmillan

Ringold, D. (2005) Accounting for Diversity: Policy Design and Mäori Development in Aotearoa New Zealand, Wellington: Fulbright New Zealand, http://www.fulbright.org.nz/wp-content/

uploads/2011/12/axford2005_ringold.pdf

Robson B., Cormack, D., Cram, F. (2007) ‘Social and Economic Indicators’, in B. Robson and R.

Harris (eds), Hauora: Mäori Standards of Health IV, Wellington: Te Röpü Rangahau Hauora

a Eru Pömare (the Eru Pömare Mäori Health Research Centre), University of Otago of New

Zealand, http://www.hauora.Mäori.nz/hauora/

State Services Commission (June 2003) EEO Progress in the Public Service 2003 Report with special focus on Mäori, Wellington: State Services Commission

43


State Service Commission (2003a) Public Service Policy Staff: including a focus on Mäori,

unpublished presentation to Public Service Human Resource Managers, Wellington: State

ServicesCommissionhttps://psi.govt.nz/polnet/Shared%20Documents/Forms/Mäori%20

Responsiveness.aspx

State Services Commission (2004) International evidence on appropriateness of EEO group targets, Wellington: State Services Commission

State Services Commission (2009a) Understanding the Drivers Fact Sheets: You Were Treated Fairly Fact Sheet, Wellington: State Services Commission

State Services Commission (2009b) Understanding the Drivers Individual Circumstances Fact Sheet, Wellington: State Services Commission

State Services Commission (2009c) Understanding the Drivers ‘Staff Kept their Promises’ Fact Sheet, Wellington: State Services Commission

State Services Commission (2010a) 2010 Equity and Diversity Report - Diversity in the Senior Management in the Public Service, Wellington: State Services Commission

State Services Commission (2010b) A guide to using the Common Measurements Tool, Wellington: State Services Commission

State Services Commission (2010c) Human Resource Capability (HRC) Survey of Public Service Departments as at 30 June 2010, Wellington: State Services Commission,

State Services Commission (2011a) How Different Groups of New Zealanders Experience Public Services, Wellington: State Services Commission

State Services Commission (2011b) Performance Improvement Framework Formal Review of the Ministry of Social Development, Wellington: State Services Commission

State Services Commission (2011c) Human Resource Capability (HRC) Survey of Public Service Departments as at 30 June 2011, Wellington: State Services Commission

State Services Commission & Treasury (2008) Performance measurement: Advice and examples on how to develop effective frameworks, Wellington: State Services Commission

Statistics New Zealand (2001a) Labour Market Statistics 2001, Wellington: Statistics New Zealand

Statistics New Zealand (2001b) Looking Past the Twentieth Century, Wellington: Statistics New Zealand

Statistics New Zealand (2002) Towards a Mäori Statistics Framework: A Discussion Document, Wellington: Statistics New Zealand, http://www.stats.govt.nz/browse_for_stats/people_and_

communities/Mäori/towards-a-Mäori-stats-framework.aspx

Statistics New Zealand (2005) Understanding and working with ethnicity data, a technical paper, Wellington: Statistics New Zealand, http://www.stats.govt.nz/browse_for_stats/population/

census_counts/review-measurement-of-ethnicity/papers.aspx

Stolte, O. (2006) Training the Disadvantaged Unemployed Policy frameworks and community

responses to unemployment. Unpublished PhD thesis, Department of Geography, Tourism and

Environmental Planning, University of Waikato.

Te Pümanawa Hauora Research Centre for Mäori Health and Development (undated). Best outcomes for Mäori Te Hoe Nuku Roa, Palmerston North: Massey University of New Zealand,

http://www.tehoenukuroa.org.nz/about_us.htm

44


Te Puni Kökiri (1996a) NZ Income Support Service: Review of Service Delivery to Mäori, Wellington: Te Puni Kökiri

Te Puni Kökiri (1996b) NZ Income Support Service: A Qualitative Report on Mäori Customer Satisfaction, Wellington, Te Puni Kökiri

Te Puni Kökiri (2009) The Dynamics of Mäori Jobseekers: Understanding the Dynamics that Affect the Labour Market Attachment of Mäori Jobseeker, Wellington: Te Puni Kökiri

Te Puni Kökiri (2011a) A review of what the official requirements and primary guidance say about good practice in developing and reporting against effectiveness measures for outputs for Mäori, Wellington: Te Puni Kökiri

Te Puni Kökiri (2011b) A Summary of good practice requirements as per official guidance and requirements, Wellington: Te Puni Kökiri

Te Puni Kökiri (2011c) Desktop analysis of trends in Ministry of Social Development’s performance measurement reporting of its employment services 2002/03 to 2011/12, report prepared by McKendry, C., Wellington: Te Puni Kökiri

Te Puni Kökiri (2012) Towards a critical analytical Mäori perspective of good practice in developing and reporting against measures of performance and effectiveness for a front line mainstream employment service for Mäori on the unemployment benefit, Wellington: Te Puni Kökiri

The Families Commission (2012) Safety of Subsequent Children Mäori Children and Whanäu A review of selected literature, Wellington: The Families Commission, http://www.familiescommission.org.nz/publications/research-reports/safety-of-subsequent-children

The Treasury (2004) The Use of Ethnicity in Targeting, Wellington: The Treasury

The Treasury (2006) Use of targets to Improve Health System Performance: English NHS Experience and Implications for New Zealand Working Paper 06/06, Wellington: The Treasury, http://www.treasury.govt.nz/publications/research-policy/wp/2006/06-06/

The Treasury (2008) Demonstrating Performance A Primer for Expenditure Review, Wellington: The Treasury, http://www.treasury.govt.nz/publications/guidance/reporting/demonstrating/

The Treasury (2010) Preparing the Annual Report Technical and Process Guide for Departments, Wellington: The Treasury

The Treasury (2012a) Preparing the Annual Report Technical and Process Guide for Departments, Wellington: The Treasury

The Treasury (2012b) Treasury Report: A descriptive analysis of income and deprivation in New Zealand, Briefing to the Minister of Finance, Wellington: The Treasury, http://www.treasury.govt.nz/publications/informationreleases/income-deprivation

The Salvation Army (2006) Forgotten People: Men on their own, Manukau: The Salvation Army Policy Unit and Parliamentary Unit, http://salvationarmy.org.nz/uploads/ForgottenPeople.pdf

Thomas, P. (2006) Performance measurement, reporting, obstacles and accountability, ANU Press: Canberra

Whänau Ora Taskforce (2009) Whänau Ora: A Whänau-Centred Approach to Mäori Wellbeing: A Discussion Paper, Wellington: Ministry of Social Development

Williams, J. Justice (2000) ‘Building the Constitution Conference’ in C. James (ed), Building the Constitution, Wellington: Institute of Policy Studies, Victoria University of Wellington

www.tpk.govt.nzTe Puni Kökiri, Te Puni Kökiri House

143 Lambton Quay, Wellington 6011, New Zealand

PO Box 3943, Wellington 6140, New Zealand

PHN Waea 04 819 6000 FAX Waea Whakaahua 04 819 6299

TPK Measure Performance 2013 - Te Puni Kōkiri

Documents

Transcript of TPK Measure Performance 2013 - Te Puni Kōkiri