Audit Scotland

8/8/2019 Audit Scotland

1/13

Management Case Study

Building a Strategy Map to Drive

Performance Accountability and

Improvement: The Case of Audit Scotland

For more information please visit: www.ap-institute.com
http://www.ap-institute.com/http://www.ap-institute.com/


2/13

2010 Advanced Performance Institute, BWMC Ltd. (All rights reserved) www.ap-institute.com Page 2

Building a Strategy Map to Drive External and Internal Performance Accountability and Improvement API Case Study

Building a Strategy Map to Drive Performance Accountability and


By

Bernard Marr* and James Creelman

Abstract: This case study describes the best-in-class implementation of a

performance management framework in form of a strategy map with supporting Key

Performance Questions and Key Performance Indicators. It shows how AuditScotland, the nations independent auditor of public bodies, was able to distill the

long stakeholder wish-list down to a simple and focused strategy map. Outlining the

strategic priorities on just one page using simple language has completely

transformed the way Audit Scotland is managing its performance to ensure external

accountability and internal performance improvements.

Version: 18th May 2010 *corresponding author

Bernard Marris the Chief Executive and Director of Research at the Advanced Performance Institute. E-mail:

[email protected]

James Creelman is a Fellow of the Advanced Performance Institute

The Advanced Performance Institute (API) is a world-leading independent research and advisory

organisation specialising in organisational performance. It provides expert knowledge, research, consulting

and training to performance orientated companies, governments and not-for-profit organisations across the

globe. For more reading material or information on how the API might be able to help your organisation please

visit:www.ap-institute.com

How to reference this case study:

Marr, B. and Creelman, J. (2009) Building a Strategy Map to Drive Performance Accountability and

Improvement: The Case of Audit Scotland, Management Case Study, The Advanced Performance Institute

(www.ap-institute.com).
mailto:[email protected]://www.ap-institute.com/http://www.ap-institute.com/http://www.ap-institute.com/mailto:[email protected]


3/13



Building a Strategy Map to Drive Performance Accountability and


Introduction

Having amassed a wealth of useful data and

information regarding stakeholder priorities over the

years, Audit Scotland faced the not uncommon

challenge of distilling this down into something that

was focused and manageable.

To achieve this, Audit Scotland crafted a strategy

map that comprises just two core activities and nine

supporting performance enablers. Each of these,

which are collectively aligned to the organizations

vision, is supported by Key Performance Questions

(KPQs) and Key Performance Indicators (KPIs)

which together provide rich insights into how the

organization is progressing toward its strategic

goals.

The strategy map is designed to help Audit Scotland

make step-change improvements in performance

and to provide a more streamlined and focused

corporate plan.

An Impact framework is also being used to assess

and report the longer term benefits of Audit

Scotlands work, which will serve to provide a more

complete view of the organizations performance.

About Audit Scotland

Headquartered in Edinburgh, Audit Scotland was

created in 2000 to provide the Auditor General and

the Accounts Commission with the services thatthey need to check that public money is spent

properly, efficiently and effectively. With about 300

full-time equivalent employees and an operating

budget of 27.5 million, the organization arranges

the audits of about 200 public bodies, such as the

Scottish Government, 32 councils and 44 joint

boards and committees (including police and fire,

health boards and rescue services). Importantly,

Audit Scotland is independent of the bodies that it

audits.

Three types of audit

Audit Scotland conducts three types of audits:

1. An annual audit provides an independent

opinion of the public bodies financialarrangements and governance practices.

2. Best value audits provide an independent

opinion of an organizations outcomes focus as

well as the effectiveness of its corporate

performance management and use of

resources.

3. Performance audits provide an independent

assessment of key areas of parliamentary or

public interest, often spanning traditional sector

boundaries. Performance audit reports include

Scotland Public Finances,The role of Boards,

Efficient Justice and Public Sector Pension

Schemes.

Together, these three audit types provide a detailed

and systematic evaluation of the efficiency and


4/13


5/13



Stakeholder priorities

The priorities identified by the stakeholders groups,which have therefore become the key priorities of

Audit Scotland, are:

Deliver more streamlined audits in partnership

with other scrutiny bodies.

Maximize our contribution to the improvement of

public services.

Increase the impact of our work.

Become a centre of excellence for public audit.

Improve the transparency of our costs and

governance arrangements.

The priorities are based on the following issues that

stakeholders are grappling with, that emerged

through the consultation process:

increasing pressure on public spending

Increasing emphasis on more efficient working

Figure 1:Audit Scotlands Corporate Strategy Map (source: Audit Scotland Corporate Plan 2009 -12)


6/13



The continuing need for high standards of

corporate governance, financial management

and control

The need to deliver more efficient and effectivescrutiny and accountability.

Creating the Corporate Strategy Map

The first step in creating the strategy map that

would deliver to the vision and stakeholder priorities

involved API conducting focused and individual

interviews with each member of the executive

management team and the assistant directors as

well as the organizations chair John Baillie and

another board member.

A core benefit of one-to-one interviews is that it

enabled the external facilitator to gain a clear

picture as to how individual senior managers viewed

the drivers of organizational strategic success. Any

differences were debated within the workshops

conducted to create the strategy map, with the

ultimate goal of reaching consensual agreement as

to the critical few drivers of success.

Strategy mapping workshops

Audit Scotlands management group created a draft

strategy map in two dedicated workshops. A key

role of the external facilitator was to ensure the

selection of truly strategic objectives, rather thangoals that are essentially operational.

A further role was to facilitate the creation of cross-

organizational objectives instead of ones that could

be siloed that are of relevance and interest to just

one part of the organization. Audit Scotlands

management team wanted to ensure that the

strategy map would galvanize the whole of the

organization in unison toward step change

performance improvement.

In-house consultationWide-ranging staff engagement was seen as crucial

for securing buy-in to the strategy map concept.

Through focus groups and discussions with small

groups of staff we gained feedback on what they

thought of the map, what they thought were its

strengths and weaknesses and where they saw

areas for improvement, recalls McGiffen. This

showed us how closely the management teams

views were aligned with those of their colleagues

and the staff feedback was used to inform the final

map.

According to McGiffen, the clarity and simplicity of

the map helped most staff to understand and talk

about what it means to them Within the group

discussions it became evident that colleagues could

see how it all fits together, what Audit Scotland is

trying to achieve, how it will be achieved and what

they must do in their own jobs to make it happen.

This, she adds, was largely due to the maps

simplicity and accessibility.

Ensuring the map is simple and focused

As an illustration of simplicity, the map comprises

just eleven elements: two core activities (which Audit Scotland calls objectives) and nine

performance enablers (which Audit Scotland calls

supporting activities). A prime driver of formulating

the strategy map was to distil and make

manageable a mass of data and information, Audit

Scotland rightly realized that there would be little

benefit in creating an elaborate and complex map.

Indeed the experience of other companies shows


7/13



that complicated maps do not provide sufficient

focus onto the key drivers of success and rarely

survive over the longer term due to being too

unwieldy and unmanageable a fate Audit Scotlandwas determined to avoid.

Although simple, the map is far from simplistic.

Rather, each objective and supporting activity

serves as a doorway to a host of performance

management and measurement focal points

primarily captured through Key Performance

Questions (KPQs) and Key Performance Indicators

(KPIs) - that work together to provide an overall

picture of how well Audit Scotland is performing

and, ultimately, to steer the organization toward the

delivery of its vision and priorities (For more

information on KPQs and KPIs see the API white

paper What are KPQs and the API management

overview Key Performance Indicators1).

Although KPIs are commonly used withinorganizations, KPQs are substantially less so. So

as an explanation, a KPQ focuses on and highlights

what the organization needs to know in terms of

executing existing strategic objectives. KPQs

enable a full and focused discussion on how well

the organization is delivering to these objectives

and serve as an important bridge between

organizational goals and KPIs. Indeed KPQs are

used to provide a performance context to KPIs and

help to more effectively prioritize the indicators

chosen, and this was certainly true of Audit

Scotland.

1http://www.ap-institute.com/resources.htm

People in most organizations, and we are no

exception, have quite rightly developed strong

disciplines in KPIs as a way to monitor and improve

performance, says McGiffen. KPQs do not replaceKPIs but are an enhancement. KPQs help us to

capture a much richer set of information than cannot

be gained solely from performance data.

Today, KPQs are used to discuss performance at

many organizational levels within Audit Scotland

and Audit Scotland is refining and developing its

collection and reporting of data to address the

KPQs. A key initiative was a Performance

Management Festival held over two days to bring

focus and attention from across the business to the

performance reports.

We will now explore how the objectives/activities on

the map, KPQs and KPIs work together with Audit

Scotlands strategy management framework.

Two strategic objectives

As cited, Audit Scotlands strategy map includes just

two core activities (called objectives in their case),

these being: we will conduct excellent risk based

audits of the public sector and report them in

public, and we will systematically identify and

promote good practice to help public bodies to

improve.

Although both objectives are equally important, a

key message that emerged from the extensive

stakeholder consultation was that they were looking

to Audit Scotland to do more to help them to

improve performance. The KPQs to support the

improve objective (and note that KPQs and indeed

KPIs are still work in progress within Audit Scotland,

while they test which are the most powerful
http://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htm


8/13



questions and indicators to focus on) presently

include: to what extent has our work led to

improvement and to what extent are we identifying

and actively promoting good practice (or betterways of doing things) to help public bodies improve.

Note that the KPQs focus on both lagging (that is

how we have performed) and leading (how we will

improve performance) dimensions, enabling a fuller

overview of past, present and likely future

performance.

KPIs for these KPQs include impact reports (see

later in this case study for details of Audit Scotlands

impact framework); client feedback; number of

presentation at meetings, events and conferences

which include best practice guidance, and

performance audit reports with good practice

recommendations (see Figure 2 for examples).

Supporting Activities

How Audit Scotland achieves its objectives is

largely through the delivery of enabling objectives

(called supporting activities in their case). In

creating the map, the senior team strived to ensure

that there was a balance of focus on what matters

to stakeholders, what matters to staff and improving

some of the key business support processes to

support that work.

Stakeholder-facing performance improvement

For a stakeholder-facing improvement, consider the

activity: We will closely engage and communicate

with our key stakeholders, clients and partners, and

other scrutiny bodies.

Audit Scotland recognizes that it is only through

close engagement and communication that it can

effectively identify appropriate topics for its audit

work. Closely engaging and partnering with other

scrutiny bodies and improvement agencies also

helps to avoid duplication of effort and maximizesAudit Scotlands contribution to the improvement of

public services.

To support the improvement objective it is also

recognized that it must clearly communicate its

findings and recommendations to ensure that its

messages are heard and its recommendations are

followed up.

KPQs used for this activity are to what extent are

we keeping our stakeholders informed about our

programmes of work and their role/implications and

how well do we engage the people we work with.

KPIs include positive stakeholder feedback

(engagement levels, clarity about and acceptance of

roles), website statistics (downloads, click

throughs) and the number of shared riskassessment frameworks in place.

Internal process improvement

Step-change internal business process

improvements are largely captured by the

supporting activity: We will deliver our work and

manage our resources efficiently and effectively and

will be clear where we need to improve. We willhave excellent governance procedures and will

monitor and report on the impact of our work.

This activity sets out to ensure that Audit Scotland

has strong internal governance processes in place

and that it is deploying best practice public

management principles and procedures. To this

end, Audit Scotland carries out a programme of


9/13



internal best value reviews to improve its efficiency

and effectiveness and manage its risks.

The KPQs for this activity include to what extenthave we effectively and efficiently managed our

resources to deliver our outputs and how effective

are our internal governance arrangements for

delivering our objectives. KPIs include: percentage

of audit reports delivered on-time and to budget,

reducing our carbon footprint by one per cent,

percentage of invoices paid within 30 days and

internal best value reports published.

Staff performance improvementMoving onto improving staff performance this is

captured by several supporting activities, including

We will maintain and develop our professional skills

and competencies.

As with any high-performing organization, Audit

Figure 2: Sample KPIs and Evidence of Performance (source: Audit Scotland Corporate Plan 2009-12)


10/13



Scotland recognizes that it can only deliver high-

quality work if its staff have the right skills and

competencies to do their job well. To achieve this,

the organization strives to embrace an ethos ofcontinuous improvement and continues to invest in

the development its people. The organization has

recently revised its competency framework and has

embedded this into its Performance Development

System. Regular individual performance reviews are

used to assess and follow-up on training and

development needs.

The KPQs for this activity include to what extent are

our employees engaged and motivated and to

what extent do we have strong and effective

leadership in the organization. KPIs include staff

survey results, staff focus group feedback and the

development of 360 degree feedback (through

which employees are appraised by a range of

people, including their managers, colleagues and

direct reports).

Knowledge management

A particularly interesting supporting activity, in that it

clearly demonstrates how Audit Scotland is working

to improve its own internal capabilities in order to

help its stakeholders improve is We will gather,

organize and share knowledge and intelligence.

The organization recognizes that the basis of its

work is having the right knowledge and intelligence.

Therefore it is working diligently to ensure that the

right information is systematically gathered, that it is

organized in the most appropriate way, and shared

among staff and externally with stakeholders.

KPQs for this activity are to what extent do we hold

the information that we need ' to what extent are we

sharing our key findings internally, and to what

extent are we able to find and share the information

that we find. KPIs include focus group feedback,

intelligence reports, publication of technicalbulletins, number of briefing reports produced and

shared internally.

Creating Strategy Maps for Specific Activities

Audit Scotland is presently reviewing and

developing how it gathers, organizes and shares

knowledge and intelligence. This is leading to some

interesting applications of the strategy map concept,

as McGiffen explains. Our strategy map identifies

the key support activities where we need to focus

attention and commit resources, she says. What

we are doing now is creating strategy maps for

specific areas where we want to prioritize major

performance improvements, such as information

and knowledge management and shared learning.

She continues: Our stakeholders are increasingly

looking to Audit Scotland to share knowledge and

information quickly from the full range of our work

and advanced knowledge sharing and collaborative

capabilities. However, our technologies and

methodologies dont enable us to do what we need

to do to meet stakeholder requirements. We have

started to invest in this area and a strategy map

should prove a powerful mechanism for prioritizing

our investments against identified stakeholderneeds. Environmental performance is another area

for which a dedicated strategy map will be created.

Performance Reporting

With the strategy map now the core tool for driving

performance improvement, Audit Scotland ensures

that it is closely managed on a day-to-day basis,

and this is the responsibility of McGiffen and a small


11/13



team from corporate reporting that is skilled in data

analysis. The team reports performance to the

management team and board on a quarterly basis,

who particularly look to see how objectives andactivities are improving and to focus on areas that

are proving challenging and where corrective action

might be required.

From a reporting standpoint, the strategy map and

related KPIs have also enabled McGiffen and her

team to cut the corporate plan down from a lengthy

document to something concise and focused and

that is more accessible to external stakeholders and

internal staff respectively.

Performance Management Festival

One innovative approach to engage people in

performance reporting was to stage a Performance

Management Festival. In late 2009 Audit Scotland

invited key staff for a two-day festival of

performance. The aim was to bring together KPI

owners, performance analysts, and directors and

board members to agree performance reporting

formats. Using an off-site location with music, guest

speakers and performances this innovative event,

facilitated by the API, was designed to make the

topic of measurement and performance reporting

interesting, engaging and fun.

During the two-day event staff re-designed

performance reports using the latest techniques of

information communication and data visualization.

Within these reports, headlines and narratives

(essentially a performance story) were used to

capture the key messages, using ideas and

graphic/textual formatting from magazines and

newspapers. These new performance reports were

then trialed and improved during a staged mock

performance review meeting with the directors.

Impact FrameworkPerformance improvement is also being galvanized

through the deployment of an impact framework to

assess and report on the longer term impact of

Audit Scotlands work - in particular its performance

audits - and to capture evidence of that impact. The

organization has identified four areas where it

expects its work to have an impact and contribute to

improvements:

Assurance and accountability

Planning and management

Economy and efficiency

Effectiveness and quality.

The impact framework describes the types of

measurable change expected to result from Audit

Scotlands work within the different categories ofimpact and includes examples of how individual

reports may contribute to these changes.

The impact framework is expected to dovetail well

with the strategy map in providing a complete view

of how Audit Scotland is performing now and is

likely to perform in the future.

Conclusion

In conclusion, McGiffen summarizes the role that

the strategy map has played in improving the

performance of Audit Scotland. From the beginning

we knew that we wanted our Corporate Plan to be

simpler and more relevant and meaningful for

colleagues. she says, From an external

perspective we also knew that as an organization


12/13



we wanted a framework that would help our

stakeholders understand what we do and what we

have to be best at.

The strategy map is helping us to meet our external

and internal goals, she continues. But in the final

analysis, it is perhaps most important that the map

has better positioned Audit Scotland to be able to

demonstrate to stakeholders and clients that we are

doing what we promised that we would do.

Endnotes, References & Further Reading

Marr, Bernard (2010), The Intelligent Company: Five Steps to Success with Evidence-Based

Management, Wiley, Oxford.

Marr, Bernard (2009), Managing and Delivering Performance: How Government, Public Sector

and Not-for-profit Organizations can Measure and Manage what Really Matters, Butterworth-

Heinemann, Oxford.

Marr, Bernard (2006), Strategic Performance Management, Butterworth-Heinemann, Oxford

www.ap-institute.comFor more case studies, reports andarticles visit www.ap-institute.com

Working with the Advanced Performance Institute to develop a performance framework in form of astrategic map, KPQs and KPIs has exceeded my expectations. It has helped us clarify our strategic

objectives quickly and easilyin a way we couldnt have done on our own. The strategy map has

completely transformed the way we now communicate about our goals and deliver our strategy and it

has been an extremely powerful tool to engage everybody in managing and delivering performance.

Diane McGiffen, Director of Corporate Services, Audit Scotland


13/13



Highlights from the API resources library:

Our Resource and Research Library offers a selection of relevant downloads and links to books, articles and

case studies. These have been selected as useful information sources for further reading and to illustrate best

practice and leading thinking.

Some key ideas explained in simple terms:

What isPerformance Management?

What is theBalanced Scorecard?

What is aKey Performance Indicator?

To read more just click:http://www.ap-institute.com/resources.htm
http://www.ap-institute.com/Performance%20Management.htmlhttp://www.ap-institute.com/Performance%20Management.htmlhttp://www.ap-institute.com/Balanced%20Scorecard.htmlhttp://www.ap-institute.com/Balanced%20Scorecard.htmlhttp://www.ap-institute.com/Key%20Performance%20Indicators.htmlhttp://www.ap-institute.com/Key%20Performance%20Indicators.htmlhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/resources.htmhttp://www.ap-institute.com/Key%20Performance%20Indicators.htmlhttp://www.ap-institute.com/Balanced%20Scorecard.htmlhttp://www.ap-institute.com/Performance%20Management.html

Audit Scotland

Documents

Transcript of Audit Scotland