Defining and Measuring Impact Professor Andy Neely Deputy Director, AIM Research.
-
Upload
gwendolyn-park -
Category
Documents
-
view
213 -
download
0
Transcript of Defining and Measuring Impact Professor Andy Neely Deputy Director, AIM Research.
Defining and Measuring Impact
Professor Andy NeelyDeputy Director, AIM Research
Three questions…
1. What is impact?
2. Why does impact matter?
3. How can impact be measured?
And then to a more important question: how can impact be enabled?
What is impact?
Why does impact matter?
The James Ladyman position…
“Those of us who oppose the research council’s emphasis on impact do so
because we do not believe its policy is in the best interest of taxpayers.
Accusing us of wishing to promote our own interests as researchers is
a cheap shot… The impact agenda is distorting research priorities
and distracting academics from their core activity, which is to
produce scholarship for the consumption of their peers, who are
usually the only people equipped to understand it and interested
enough in it to bother trying”.
Ladyman, J. (2009) letter in the Times Higher Education Supplement
The UK budget deficit!
Source: http://www.statistics.gov.uk/
Three questions…
1. What is impact?
2. Why does impact matter?
3. How can impact be measured?
Measurement is a mess…
A =
QASASjs x Wjs + QATASjL x WjL + QATASkL x WkLjs jL kL
S T T
Wjs + WjL+ WkL
S T T
js jL kL
“Often we measure the wrong things, in thewrong ways and frequently we measure too
much”
What is AIM?
• AIM: The UK’s Research Initiative on Management.
• Budget of £35+ million invested by the Economic and Social Research Council and the Engineering and Physical Sciences Research Council.
• Used to fund over 250 Fellows and Scholars – all leading academics in their fields…
• Working in cooperation with leading international academics and specialists as well as UK policy makers and business leaders…
• Undertaking a wide range of collaborative research projects…
• Disseminating ideas and shared learning through publications, reports, workshops and events…
• Fostering new ways of working more effectively with managers and policy makers…
• To enhance UK competitiveness and policy…
AIM’s mission and objectives
Mission – to significantly increase the contribution of and future capacity for world class UK research on management.
Objectives1. Conduct research that will identify actions to enhance the UK’s international
competitiveness.
2. Raise the scientific quality and international standing of UK research on international competitiveness.
3. Expand the size and capacity of the active research base for UK research on management.
4. Develop the engagement of that capacity with world class research outside the UK and with practitioners as co-producers of knowledge about management and other users of research within the UK.
Three questions…
1. What is impact?
2. Why does impact matter?
3. How can impact be measured?
And then to a more important question: how can impact be enabled?
Institutional context that enables research and impact
Research centre/group
Who enables impact?
Individual research activity
Individual research activity
CBP Team
CBP Community
Activity / Internal Processes:
How value is
created and sustained
Capabilities: Intangible value drivers e.g. people, systems, climate and culture
Best People
Generate Funding
International Thought Leaders in Organisational
Performance
Relationships In Press Community
Goal:
International Recognition for Thought Leadership
Press Practitioners Academics
RadioNewspapersTelevision
Target Community:
Training / Conferences /
Education
Books / Magazine Articles
Top JournalsAcademic Conferences
PMA
Industrial Collaboration
Building Relationships
Thought-Leading Research:• Best Practice
•Theory Building & Testing•Research Agendas
•Tools and Techniques
Technology & DataBases
Culture / Fun Place to Work /Mutual Support / Knowledge Sharing
Output:
Value proposition
AcademicCollaboratio
n
CBP Web Page
Policy Makers
Policy Makers
Reflections on impact
• Impact is not simply defined – its not just economic impact.
• We need to think about the routes to impact:• Patents and spin-outs are a minority activity…• Other routes - e.g. press and media, books and practitioner articles are
important…• Don’t ignore consultancy – a hidden market…• The most undervalued route to impact - teaching…
• We need to understand what is involved in having impact and then create the infrastructure to help:• Explore the academic value chain and the institutional infrastructure…• Bear in mind an academic’s intellectual curiosity – many faculty move on to the
next problem once they have “solved” the current one...
Questions for research directors – [i] do you do enough to support the multiple routes to impact – especially the creation of teaching materials; [ii] which activities in the academic value chain should you support?
Reflections on evaluation
• The dominant form of evaluation (as I perceive it) – publications, publications, publications…
• HEFCE proposals for REF – the implication of measuring citations…
• We need counter-balancing measures and incentives (but remember we are dealing with a global system)…
• We should consider the hidden routes to impact in our evaluation (including promotion) models.
Question for research directors – do you do enough in evaluation to counter balance the system’s natural focus on scholarly publication?