xxxxxx
Measuring Economic Impact- Overview -
January 2012
Rupert Greenhalgh
Director: NES Consulting
xxxxxx
About us…
Economic policy & strategy
Local economic & skills assessment
Project evaluation & cost benefit analysis
Consultancy
ProBono Economics
Think Pieces & Thought Leadership
New Economic Strategies
xxxxxxPerformance framework
Two key components:
A ‘top-down’ component:
monitors the changing state of conditions in the functional economic area / administrative area
requires the use of ‘Headline Indicators’ & ‘Condition Indicators’
A ‘bottom-up’ component:
enables the Strategy’s activities, outputs & outcomes/impacts to be assessed
uses appropriate ‘Response Indicators’
xxxxxxHeadline indicators (1)
Should be embedded within the strategic process:
Plan > Do > Act > Check
Clear & robust insight into strategy’s efficiency & effectiveness
Outputs & outcomes aligned to overall achievements
Use & development of the framework will be iterative
Devised to yield additional information
What actions are more effective & efficient than others?
xxxxxxHeadline indicators (2)
Focused on the added value of the Economic Strategy
Reflect the totality of value added rather than individual priorities
Provide an outward facing assessment of progress
Enable comparison of performance against other areas
Widely understood by Gov & business
Provide a common evidence base to share local economic performance with all key stakeholders
xxxxxxHeadline indicators (3)
Delivering growth & prosperity:
1.GVA per capita (as a measure of resident prosperity)
2.GVA per worker (as a measure of employee productivity)
3.Total jobs (employee jobs located within workplaces)
4.Growth in private sector employment (representing & additional measure of re-balancing)
5.Business density (measured by the total number of businesses per 1000 population)
6.Other measures? Discuss
xxxxxxExample 1: Setting Targets and Monitoring
Indicators
xxxxxxCondition indicators (1)
Business support
Condition Indicator Data / definitions
GVA per employment in transformational sectors
£GVA in constant prices
Stock of VAT registrations per 10,000 working age residents
VAT registered enterprises
% of small businesses in area showing GVA growth
VAT registered enterprises
Workforce median weekly wagesAll employees working within region
xxxxxxCondition indicators (2)
Condition Indicator Data / definitionsTotal GVA from Advanced Manufacturing
£GVA in constant prices
Proportion of enterprise turnover due to new or significantly improved products
Local enterprise survey
Perception of region as a place of innovation & business opportunity
Local, national & international perceptions from commissioned surveys
Number of patents granted to enterprises in the region
Patents granted (not applications)
Innovation
xxxxxxCondition indicators (3)
Condition Indicator Data / definitions
New VAT registrations per 10,000 working age residents
Working age population aged 16-64 yrs
Stock of VAT registrations - cohort surviving 36 months
VAT registered as % of original cohort of start-ups 36 months previous
% of small businesses in an area showing employment growth
% of total SME’s growing in last 12 months
% of 16 to 24 year olds not in employment, education or training (NEET)
National definition for NEET
Entrepreneurship
xxxxxxCondition indicators (4)
Condition Indicator Data / definitionsNumber of new graduate level jobs created through inward investment each year
Assisted by regional stakeholders in last 12 months
Number of Foreign Direct Investment (FDI) projects in region each year
New FDI projects locating within year
Annual perception survey with key national (public & private sector) stakeholders
Commissioned survey supported by range of qualitative measures
Number of trade enquiries through inward investment agencies
Monitoring information of key partners
Number of targeted communications published aimed at key stakeholders
Monitoring information of key partners
Inward investment & place marketing
xxxxxxCondition indicators (5)
Condition Indicator Data / definitions
% of working age residents qualified to Level 2+
Working age population aged 16-64 yrs
% of working age residents qualified to Level 4+
Working age population aged 16-64 yrs
% of working age employees in LCR qualified to Level 4+
Working age population aged 16-64 yrs
% of 15 year olds getting 5 or more GCSE/GNVQs at grades A*-C
Pupils in public & private education
% of working age residents undertaking job related training
Working age 16 to 64 years within last 13 weeks of survey
Labour market & workforce skills
xxxxxxCondition indicators (6)
Major projects
Condition Indicator Data / definitions
Amount of accommodation floor-space brought forward (sq m)
Commissioned survey supported by range of qualitative measures
Value of Local Economic Multipliers (LM3) secured through major developments
Commissioned survey supported by range of qualitative measures
xxxxxx`Strategic Added Value (SAV)
leadership & catalyst: demonstrating the benefits / communicating effectively development needs to stakeholders
strategic influence: affecting the behaviour & allocation of funds &/or activities by stakeholders
leverage: securing funds to contribute to wider objectives
collaboration: improving exchange & synergy between stakeholders, encouraging co-ordinated stakeholder activity
engagement: openness & involving stakeholders in the design & delivery of activities
empowerment: empowering responsibility to people & organisations best placed to deliver effective & efficient outcomes
xxxxxx
Consultation on the condition indicators will be important to highlight that valid issues are being measured. These should be reviewed periodically to ensure they represent priorities
The framework should be used as a tool with local partners to negotiate and agree the outputs required to successfully contribute to the outcomes required by the strategy
Time must be allocated upfront to support stakeholder’s understanding of what is required to support the frameworks implementation
Performance monitoring will/should already be taking place. It is important that collaborative work between organisations takes place at the start to reduce costly duplication
Robust monitoring and evaluation processes must underpin the Economic Strategy. Systems should be put in place at the start of strategic planning to capture the outcomes of activity
Some other considerations: Implementation
xxxxxx
Other approaches:
Evaluation
Social accounting and audit
Logic models
Theory of change
Balanced score card (for local enterprise)
Local Multipliers
Social Return on Investment
CBA…
And finally…
xxxxxx
Linking project to impacts & addressing ‘gross to net’
xxxxxx
What is evaluation:
“a process of measuring or assessing success”
“learning lessons for the future”
Looks critically at what has been achieved through a project, programme or other intervention
Other similar activities:
monitoring
performance management
Valuing costs and benefits (1)
xxxxxx
Why is evaluation important:
To understand impact & assess whether objectives have been met
To identify learning, informing strategy, resource allocation, delivery
To demonstrate accountability, value for money, & contribution to wider strategic objectives
To develop evidence for funding bids, promotions, target resources
Provides a rigorous framework - or ‘Logic Model’ that links project aims, inputs, outputs, outcomes, & lessons learnt
Valuing costs and benefits (2)
xxxxxxContextual conditions
Policy conditions
Rationale Inputs Activities
Intended Outcomes
Outputs
Intended Impacts
Programme Objectives
Project:
xxxxxx
1. Deadweight – questions about what would have happened in any case?
consider baseline trends, map impacts of activities, interviews with partners
2. Leakage – proportion of outputs benefiting people from outside target
monitoring data (e.g. jobs for local residents) or beneficiary (e.g. JSA)
3. Displacement – degree to which the project reduces activity elsewhere
qualitative questioning & changes in secondary data
4. Multiplier effects – the positive knock-on effects on the wider economy
specialist studies/methodologies (e.g. LM3), or use of benchmarks
Valuing costs and benefits (3)
xxxxxx
HMT Green Book (HMG Appraisal and evaluation): http://www.hm-treasury.gov.uk/data_greenbook_index.htm
Magenta Book (HMG Policy evaluation): http://www.hm-treasury.gov.uk/data_magentabook_index.htm
White Book (DCMS Creative Industries): http://www.compass.unibo.it/Materiali/Documenti/2007/Documenti_Mazzanti_whitebookDCMS%20UK.pdf
Historic Environment (Appraisal and evaluation):http://www.hlf.org.uk/aboutus/howwework/Documents/Valuingthehistoricenvironment_eftecFinal_Report.pdf
Assessing the Impacts of Spatial Interventions (3Rs):http://www.communities.gov.uk/documents/regeneration/pdf/156906.pdf
Orange Book (Appraisal and risk management):http://www.hm-treasury.gov.uk/media/A/5/managingrisks_appraisal220705.pdf
SROI Network and supporting training materials:http://www.sroi-uk.org/publications-uk/cat_view/29-the-sroi-guide-2009
Resources
xxxxxx
Friendly, free advice/information
Collaboration – research projects
NES Consulting
info@nesconsultingcouk
Contact us at New Economic Strategies
Top Related