Project benefits / impacts and their measurement

Paul Mayes16th April 2010

http://betterprojects.jiscinvolve.org/

Working definition is that a benefit is a positive type of impact

There can be negative impacts from a project, some predicted in a risk analysis (but offset against greater positive impacts) but some unexpected

What types of benefit are there?

‘Hard’ benefits:

H1 Production of material, software, services, processes, etc

H2 Improved collaboration

H3 Improved awareness of the ideas, concepts, materials, etc within the institution, the sector and other wider areas eg government or the general public

H4 Improved skills and attitudes of users, project team, institution staff, stakeholders and staff in sector

‘Soft’ / indirect benefits:

S1 Catalyst for change both in institution and wider

S2 Impact on professional or governmental views, actions or policies

S3 Improvement of image, standing or effectiveness of the funding body of the project

S4 Improvement of image, standing or effectiveness of the institution

S5 Improvement of image, standing or effectiveness of the project team

This approach assumes a large commitment by the project team to benefits / impact and their measurement

This will also be over a long period of time:

• When a project idea originally occurs (this of course assumes that the project bid is not a kneejerk response to a funding call)

• During the bid writing process

• During the project implementation itself

• For at least 5 years after the project finishes

Why are benefits (and their tracking or measurement) important?

In advance of the project:

List of possible benefits strengthens first draft of user engagement plan and helps form an evaluation/measurement approachList of possible benefits strengthens the bid document by giving the funding body more assurance about the value to them of the projectList of possible benefits may strengthen buy-in within institution and encourage more/ better cooperation with partners/stakeholders outside the institution

During the project:

Part of good project management practice to achieve targets, milestones, etc

At end of project:

Clear approach to benefits/impacts strengthens final report (being slightly cynical here)Underpins better identification / ‘measurement of value’ as a framework for further areas of research and developmentProvides a long-term and binding structure for sustaining the project outcomesWill help develop an understanding of how the balance of investment can be adjusted to improve impact

Some practical steps for ‘benefits realisation’

• Ensure that the project team is really aware of the commitment that will be needed before, during and after the project

• Develop a community of practice inside and outside of the institution

• Foster a climate of sustainability of the project outcomes

• Really adopt a user engagement approach

Some examples of methods & tools for tracking or measurement

For

H1 User satisfaction measuresUsage data

H2 Before and after focus groups or other user engagement tools

H3 Awareness surveysTracking the use of project outcomes or mention of the project in

conventional media (eg journals, newspapers), newer media (eg Twitter, blogs), conference mentions, etc

H4 Skills and attitude audits of users, staff, etc throughout the project and afterwards using a learning impact measurement framework such as that of Bersin & Associates (see article at http://www.cedma-europe.org/newsletter%20articles/Clomedia/The%20Learning%20Impact%20Measurement%20Framework%20%28Nov%2006%29.pdf )

For

S1 A change assessment timeline started right from when the original idea of the for the project arose

S2 Tracking of evidence of awareness, use or mention of the project ideas

Soft surveying of professional or government staff

S3 Some quantitative data to give to funding body about the return on their investment in terms of mention of their name in the media, benefits to their other projects and evidence of change resulting from the project

S4 Some quantitative data to give to the institution in terms of enhancementof reputation, benefits to learning & teaching/research/BCE resulting from

the project, etc

S5 Soft investigation of change of image / standing of project team

Hard measures such as increases in conference paper and publication acceptance, invitations of participate in activities or further funding obtained