BAM IBNW ref impact workshop
51
Transcript of BAM IBNW ref impact workshop
Research engagement with users and Impact in International Business and
International Management
Frank McDonaldUniversity of Liverpool Management School
BAM/IBNW Manchester 19 Sept 2016
1
Welcome � Welcome to our three speakers
� Ian Drummond, Principal Research Officer, Department for Business Energy and Industrial Strategy
� Nigel Driffield, Professor of Strategy and International Business Warwick Business School
� Richard Hindle, Director of SQW Limited
� Thanks to MMUBS for hosting and especially to Heinz Tuselmann
BAM/IBNW Manchester 19 Sept 2016 2
Stern Report� Three recommendations on Impact in Stern Report
1. Institutions should be given more flexibility to showcase their interdisciplinary and collaborative impacts by submitting ‘institutional’ level impact case studies, part of a new institutional level assessment.
2. Impact should be based on research of demonstrable quality. However, case studies could be linked to a research activity and a body of work as well as to a broad range of research outputs.
3. Guidance on the REF should make it clear that impact case studies should not be narrowly interpreted, need not solely focus on socioeconomic impacts but should also include impact on government policy, on public engagement and understanding, on cultural life, on academic impacts outside the field, and impacts on teaching.
� www.gov.uk/government/uploads/system/uploads/attachment_data/file/541338/ind-16-9-ref-stern-review.pdf
BAM/IBNW Manchester 19 Sept 2016 3
Some tentative questions � What are the main characteristics of high quality
Impact?
� How do we more effectively engage with users to produce high quality Impact?
� Who are good users to target?
� How do we record Impact to justify high quality Impact?
� What type of research activities and whole body of work (Stern)is likely to lead to high quality Impact?
� What impact on teaching will count (Stern)?
BAM/IBNW Manchester 19 Sept 2016 4
REF ImpactImpact: 20 per cent of the overall results
Definition for the REF ‘Impact’ is any effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.
Each submission included:
• Impact case studies. These four-page documents described impacts that had occurred between January 2008 and July 2013. The submitting university must have produced high quality research since 1993 that contributed to the impacts. Each submission could include one case study, plus an additional case study for every 10 staff.
• An impact template. This document explained how the submitted unit had enabled impact from its research during the period from 2008 to 2013, and its future strategy for impact.
REF ImpactThe criteria for assessing impacts were ‘reach’ and ‘significance’:
In assessing the impact described within a case study, the panel formed an overall view about its ‘reach and significance’ taken as a whole, rather than assess ‘reach and significance’ separately.
• Four star - Outstanding impacts in terms of their reach and significance.
• Three star - Very considerable impacts in terms of their reach and significance.
• Two star - Considerable impacts in terms of their reach and significance.
• One star - Recognised but modest impacts in terms of their reach and significance.
• Unclassified -The impact is of little or no reach and significance; or the impact was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit.
REF ImpactSub Panel report “The impact cases covered a very wide range of impact types,
including impacts within organisations, on national policy, on international
agreements and on the public. The high scoring impact cases typically provided
clear evidence of the reach and significance of the impact, the underpinning
research which was cited clearly met the threshold requirement of international
excellence and there was a strong narrative that described how the research led
to the impact”.
Profile type 4* (%) 3* (%) 2* (%) 1* (%) UC (%)
Impact 37.7 42.5 17.0 2.2 0.6
The weighted average impact sub-profile for the sub panel was:
REF ImpactCase studies
Criteria – reach and significance
Perhaps the key issue was evidence. Convincing evidence included:
• Citations in policy documents. Eg World Bank, OECD, UN etc. As well as national government and private sector organisations, businesses.
• Actual changes in policy and practice that were clearly attributable to the research. But problem here is demonstrating that this research was the key driver of change.
• Appointments to high level boards, etc.
• Public engagement/Press coverage.
• International reach clearly positive.
• Testimonials – probably not the most convincing evidence.
REF ImpactExample comments on impact case studies:
• Not totally convincing in terms of significance – impacts largely limited to influencing debates.
• Some convincing details of how worked with XXXXXXXXX and XXXXX. Membership of Advisory Group also positive. But not clear how original this research was.
• Significant impact in one large firm, but nothing wider.
• Active dissemination – practical guidance materials for UK manufacturers, workshops etc. But weak on actual impacts.
• Clear strategy for engaging non-academics and clear pathways to impact.
• Some of the projects cited are very small scale.
• Citations in policy documents, Advisory group membership, ongoing funding stream. Relatively strong in terms of both reach and significance.
• May be more actual concrete impacts to come, but to date both reach and significance limited.
• No apparent strategy for wider dissemination.
• Impacts may well be significant, but the case isn’t well made.
• Impacts exaggerated.
• Significance probably overstated – cited strategy document drew on wide ranging sources.
• At face value has clear impacts, but lacks evidence.
REF ImpactImpact templates
Sub panel report … “In general, the impact templates were rated lower than the impact cases and it seemed that some institutions were uncertain what to say in this section of their submission. A particular problem was that some submissions merely used the final section of the template (Relationship to case studies) to summarise each impact case study rather than to relate them to the historic or current strategy of the unit”.
“The extent to which the unit’s approach described in the template was conducive to achieving impacts of ‘reach and significance”’
Not really sufficient to say we have a research services office or an ‘engagement officer’.
Dissemination isn’t impact. But relatively few case studies demonstrated that the research had had an integral dissemination strategy.
Need to demonstrate strategy, support and actual links to research activity/case studies.
Culture of securing impacts.
REF Impact1. Read the exam question
ensure research is eligible, ensure impact is eligible
explain any anomalies (e.g. researchers moving during REF period)
make it easy for the panel!
2. Tell a good story
be clear on contribution of submitting HEI to research (e.g. in collaborations)
explain research clearly
link impact to research
do not overstate the impact – be realistic
be clear on contribution of submitting HEI to impact
3. Provide appropriate and compelling evidence
evidence should offer diverse mix of independent qualitative and quantitative sources that directly support claims being made
follow through the ‘stories of change’ (i.e. what changed as a result of the research)
4. Communicate reach and significance of impact
describe context in which the impact takes place to give an indication of its significance
set out the original objectives of the research e.g. what was the intended reach? If you maximised the possible reach of your
research then you were likely to score higher
5. Appropriate use of language and presentation
make narrative coherent and present a linear, chronological story of development
case study should be written so it is understood by a reader without specialist knowledge
clear presentation makes it easy to read: sub-headings, adequate spacing, pictures or diagrams were well received.
make sure the case study is self contained
http://www.bulletin.co.uk/31441/research-impact-case-studies-tips-success-assessors/
REF Impact
Securing impact from research conducted for government
Government research is almost always commissioned to inform policy development so
should be some impact. However, can be very difficult to establish a direct link with
policy development in practice. There is normally a body of evidence and internal
documentation may well not be available.
• Set out the policy relevance of the research in the report.
• Offer to present the findings to senior officials/ministers.
• Ask whether and how the research findings are being used.
• Track citations, etc. in official reports.
• If appropriate, work with departmental Press Offices.
IMPACT (and how to achieve it)
– a personal perspective
Franks questions
What is high quality impact
How do we more effectively engage with users
Who are good users to target?
How do we record impact
What type of work leads to impact?
What type of impact leads to teaching ?
Main message
VCs like impact and engagement more than you like they do
WHY ?
Disclaimer – I first wrote the next slide in 2010!
Be instrumental for a minute
Research relies on funding
Universities are under pressure “to show what they are for”
So are business schools !
So are Govt Departments, HEFCE etc
So are RCUK
People who do this will be paid more, promoted faster and be
in demand in the job market (and not just in the UK!)
And try writing an ESRC bid without an impact plan – its
impossible !
IMPACT, ENGAGEMENT AND INTEREST
It has opened doors for me, locally and nationally, in terms of
policy engagement
I find it interesting
It has led to other funding opportunities
(most importantly ?) My VC likes it …..
Local Engagement
Universities feel the need to show local engagement :
More than at any time in the last 20 years, more policy makers seem interested in
universities:Plus : interventions lead to evaluation and
there is money in evaluation !
The Private Returns
There is research money in this but it is a long journey
However – being instrumental, it is a good story to tell at
interviews – it impresses VCs
So there is a tradeoff – how many more papers could I have
written (and does it matter ?)
I went for a job interview…..
More generally…
Local engagement now needs to be seen in the context of localism, and the contribution that universities can make to
economic growth.
Universities are keen to show this, directly and indirectly (future funding ?)
The LEP’s have some money but this has to fit their agenda (smart specialization)?
How to start this process
Matched funding - potential sources ?
How to start out ?
All city councils have no money, and oodles of great data, on all
sorts of things ?
Are you in an area where you can write papers on local data –
perhaps not for JIBS, but Economic Geography for eg ?
Industry – parliamentary trust – does your HEI work with them
?
Do you want to do media ?
ESRC and REF define impact differently
REF “reach and significance”
ESRC – engagement and influence
http://www.esrc.ac.uk/research/evaluation-and-impact/what-is-impact/
NB PATHWAYS to impact – ie who are your stakeholders and are they
really engaged ?
How I think about impact
Lancet :Brownell & Roberto http://dx.doi.org/10.1016/S0140-6736(14)62397-7
My strategy: My impact is local
Parochial ?
Area of WMCA is about £180bn – so not so tiny to be of no
interest
I stick to what I know – inward investment and strategies for
growth
I always have some IB theory at the back of my mind (motive?)
YOU define this – so stick to your area of interest
Can I publish from this work ?
To be honest probably not –
Or to be honest not in a journal that I am interested in for REF
– but I do send stuff to people like Ian and they are interested.
Equally, a paper that is a 2nd round R&R at JIBS started off as a
piece of work on local issues – in terms of the ideas.
Media ?
Do you want to be on the telly all the time ?
Or spend your life on twitter ?
I did this over Brexit (basically pushing a paper we had written
on Brexit and FDI)
Was picked up by various cabinet members (but we lost and
its exhausting)
I now have a set of MPs / cabinet ministers / BIS people whom
if I write something of interest I send a 300 word summary to –
who knows……
Routes within your university
Media office always after stories
TheConversation
Impact Accelerator Accounts ?
Associate Deans / PVCs for engagement more open to junior people than you think – they have jobs to justify too
VCs want regional engagement
Finally … impact and teaching
I teach MBAS
MBAs have non-satiation of examples…..
Research and analysis to inform policy - the Northern Powerhouse Independent Economic ReviewBAM/IBNW, 19 September 2016 – Richard Hindle
Content
1. CONTEXT� Focus of the Independent Economic Review� Approach to the work, and process
2. OUTPUTS: specific findings on� The North’s ‘Performance Gap’� ‘Capabilities’ of the North� Growth scenarios for a future Northern Powerhouse (NPh)
Timing� SQW team began work in October 2015� Interim outputs, briefings, presentations from December 2015� Draft and Final Reports, March-April 2016� Launch (Liverpool) – July 2016
1
CONTEXT (i)
� Aim/objective> An Independent, & robust Economic Review of the North’s economy> Commissioned by Transport for the North (TfN) on behalf of partners
� Three drivers…> Data & evidence to support specific NPh transport proposals (Mar ‘16)> Starting to frame the NPh ‘narrative’, based on this evidence> Analysis to provide a bedrock on which to build further work
� And three aims, to > Characterise NPh’s economic position, & underpinning causes> Take an holistic spatial perspective: transport links, cities, other areas> Identify the ‘peaks through the clouds’, opportunities for the North
� It was not about> NPh strategy, action planning or governance> Ruling in/out local thinking/activity
2
CONTEXT (ii)
� Starting point - this is not understudied territory!� Economic scale, functions and geography well-attested� Many economic actors; wide range of functional links
� Footprint for the North of England� 16m population, 7.2m jobs, £300bn GVA� Eleven LEP areas, spanning conurbations, cities, old industrial
areas, market towns, extensive rural areas
� & c.8,000 pages of evidence!
3
4
Start-Ups & Spin-Outs
In-/out-movers
Firms & sectors
GVA
Large firms
• Congestion• Travel to Work• Quality of life• Housing• Job satisfaction
• Location• Flexibility• Cost• Transport &
Connectivity• Environment
• Culture/Ambition• Education• Training• Entrepreneurship• Access to work
Land, property, infrastructures
KnowledgeMoney & External Business Expertise
PeopleSustainability?
• HE/FE• Research institutions• Strategic alliances• Informed networks
• Banks /Angels• Venture capital• Accountants• Lawyers• Marketing Experts
Process – 5 workstreams 5
WS 1 Northern performance gaps:
Prosperity & Productivity
WS 2 Bottom-up place analyses
(x11 LEPs)
WS 3 Northern sectoral competitive
advantages . . . & ‘Capabilities’
WS 4 A transformed NPh: growth
scenarios
WS 5Proposals for Independent Panel
Nov 15
Feb 16
May 16 Final Reporting
Testing & calibration with:• Leaders & LEP Chairs• TfN’s Partnership Board• TfN’s Executive Board• Economic Reference
Group (Ec Dev Leads from all 11 LEPs)
• TfN’s Project Team
. . . & wider stakeholders, e.g. HMT, BIS, UKTI, DfT, NIC, CabOff, IPPR, N8, NetRail, HS2 etc.
WS1: What is the ‘performance gap’…What should policy-makers be targeting?
� ‘Prosperity gap’� Measured as GVA (incomes – ‘profits & wages’ - generated in the
North) per capita� Can be broken down into
> ‘productivity gap’: GVA per worker> ‘employment rate gap’: Share of workers in working-age population> ‘age structure gap’: Share of working-age population in whole popn.
� ‘Growth gap’� Measured as growth in GVA, jobs, &/or population
� For infrastructure planning, it’s growth in GVA, jobs & population that ultimately matters� But supporting argument for faster growth may be based on
closing the prosperity gap
6
A persistent prosperity gap (GVA/capita) over time
� Marked improvement in the decade to 2007, partly reversed since the recession
� Gap to European comparators is 30-35%
7
Rest of England (excl. London)
WS1: In summary…
8
North’s ‘performance gap’ (GVA per capita) ⇨ persistent & entrenched25% vs rest of England, or 10-15% excl. London.
Productivity: key factor explaining gap, widening post-recession.
Contribution to gap (2009/13): 17% vs rest of England
Employment rate : gap persistent, but largely stable over past decade
Contribution to gap (2009/13): 5% vs rest of England
What is driving the North’s performance gap …?
… & what is driving the
productivity gap?
North’s skills gap ⇨ most important factor driving the ‘performance gap’, influences productivity & employment
rate.
Other important causes of the productivity gap include under-
investment (widened notably since 2008), low enterprise rates ,
agglomeration & poor connectivity .
Sectoral mix less important.
Also, jobs per worker - contribution to gap (2009/13): 4% vs rest of England
WS2&3: What could we research and assess –sectors, capabilities and competitive advantage?...
� Build understanding through� Analysis of historic prosperity & productivity gaps (WS1)� Bottom-up mapping of sectoral priorities for 11 LEPs (WS2)� Top-down Northern GVA, employment & productivity forecasts -
for 45 sectors (WS3)
9
…and where were the main Challenges?
1. Data provide only a partial picture � Imperfect view of sectoral activity � Little intelligence on relationships, synergies & connections…� …Even less about underpinning drivers of change (market & tech)
2. Little will be gained by ‘us too’ sectoral approaches� Instead, look to Smart Specialisation thinking
> ‘Embeddedness’, ‘relatedness’, ‘connectedness’ > ‘Staying ahead’ & ‘Routeways to excellence’
3. NPh’s approach is not about ‘one sector per area’ thinking� Aim to understand ‘Peaks through the clouds’ at the level of the
North
10
Pan-Northern Capabilities: basis for assessment
� ‘Capabilities’ assessment for NPh based on clear evidence for� Sector-based specialisation – GVA and employment� ‘Hard’ assets eg international class R&D facilities & Centres of
Excellence)� ‘Soft’ assets eg nationally/internationally recognised
collaborations, networks, & interrelationships
11
Pan-Northern Capabilities: Prime and Enabling
� Four distinct ‘Prime’ Capabilities at level of the North� Advanced Manufacturing – materials & processes� Energy – technologies underpinning generation, storage, low carbon� Health Innovation – Life Sciences, Med Tech/Devices, e-health &
service delivery processes� Digital – computation, software design/tools, data analytics,
simulation/modelling
� Three ‘Enabling’ Capabilities, supporting the growth and development of the ‘prime’ Capabilities…� Financial & Professional Services (FPS)� Logistics� Education (primarily HE)
� . . . all underpinned with a unique offer� Quality of Life
12
WS2&3: In summary13
Advanced Manufacturing – materials &
processes
HealthInnovation –Life Sciences,
Med Tech/Devs, & service delivery
processes
Energy –generation,
storage, & low carbon
technologies
Digital – computation, software design/tools, data analytics, simulation/modelling
Financial & Professional ServicesLogistics
Education (primarily HE). . . & Quality of Life
WS4: Scenarios for future growth
� Key questions� Based on productivity, sector, & capability findings
> What could growth look like in the North?> What transport & other factors will be needed to realise this growth?> What options are realistic, on the balance between GVA &/or jobs?
� 3 scenarios1. The future is like the past (‘business as usual’)2. The sum of the ambitions set out in LEPs’ Strategic Economic
Plans, as collated by TfN (‘SEPs’ expectations’)3. Transformational trajectory for the North’s future performance,
relative to the past (‘transformational’)
14
WS4&5: In summary
� Under the Transformational Scenario:� At 2050, potential for…
> GVA to be 15% higher than ‘business as usual’ projection– In 2050, GVA is £97bn higher (in 2015 prices) in the ‘transformational’
scenario than in the ‘business as usual’ case> 850k additional jobs compared with ‘business as usu al’ (& 1.56m
more than in 2015)> Productivity to be 4% higher, with progress towards closing
productivity gap cf rest of England.
� Growth led by ‘Prime’ & ‘Enabling’ capabilities, including but not limited to agglomeration effects for FPS � Jobs growth mostly in Services
� Requires improved inter-city rail links, global connectivity (air & ports), intra-city public transport, & smart ticketing
� With concomitant improvements in Education & Skills, Graduate Retention, Innovation, Inward Investment
15
Next steps for the Northern Powerhouse?
� Successful launch – well-received on the day, afterwards� IER part of wider work on policy development, priorities
� Earlier ‘One North’ report, 2014� Publication of The Northern Transport Strategy, Spring 2016 � On-going TfN work - Integrated Transport Strategy
� Key question: Commitment of all parties going forward� Momentum maintained with new Minister� Last week’s announcement: Northern Ports Association… � …and George Osborne’s Northern Powerhouse Partnership
� Agenda for NPh Manchester conference, February 2017� Includes Productivity, Infrastructure: also, housing, ‘big data’� With focus on implications Post-Brexit - international positioning
16
