A story about research policy and numbersOn research assessment

Per M. KochDirector for Analysis and Strategic DevelopmentThe Research Council of Norway

6th Elsevier Nordic Librarian Forum, Helsinki, Nov 6th 2008

Research and innovation policy assessment for librarians

Universities, colleges and research institutes are policy organizations in their own right

Politicians, civil servants, industrialists and NGO staff ask for such information

A lesson in how to wake up a policy maker

Lesson 1: Give her a simple story that anybody can understand

The linear model works perfectly in a policy setting Research and

technological development in universities, RTOs and companies gives birth to an idea and relevant new knowledge

Companies make use of these ideas in the development of new products and processes

The company brings the new product to the market

Why?

It is a good story:

The unselfish scientist striving for the common good

It is simple: Give us the money and we will give you the results

By black boxing the whole process it communicates easily.

You don’t need to know how the computer works to make use of it

It works well with many of the traditional macroeconomists

The systemic model of learning and innovation is not easily reduced to a one minute sound bite

Knowledge of customer and market needs

In-house learning

market pull

New or improved products, processes or services

User input

Marked knowledge

Tacit knowledge

Acquired technology

Literature

Conferences and fairs

New employees

Commissioned R&D

In-house R&D

Rogers 1995

Klein

Havelock/Przybylinski

…because social reality is very complex

Lesson number 2: Policy makers are number fetishists

The policy game is based on indicators

1. Numbers make things true

There are limits to how much info policy makers can absorb, and they have to make a leap of faith

Since Newton, mathematics is understood as the best tool for describing the world. Numbers give legitimacy

Economists are powerful in this field, and their science ideal is physics

The number game part 2

2. You can choose the numbers that proves your point

(and disregard the rest)

Compare government budget meeting memos from various ministries

3. You can use the lack of indicators to kill a competitor

“It has not been documented that this policy has an effect…”

The numbers game part 3

Politicians and policy makers know the system. To get something done, you need a concrete commitment, and guess what?

You need a number!

3% objective

Policy makers are not stupid

But they know that they will have to make decisions based on less that perfect information.

They have to persuade!

Numbers make you look good.

There is a whole advisory industry that thrives on numbers.

Experts inside government

Consultants and researchers

Cynical? Not me!

This is the setting in which indicators are used.

There is no point in denying it.

There remains more than enough policy-makers and politicians that are genuinely interested in how the world really works!

How to Thomas Kuhn STI indicators

As long as the indicators do not diverge to much from “the facts on the ground” a “story” might survive.

When the epicycles become too complex the story becomes a political burden instead of a useful tool.

Then there is a need for a new “story” – a new interpretation of the numbers.

The Norwegian puzzle, a case study

The Norwegian Paradox

NO SE DK FI EU25

R&D/GDP

2005

1.5% 3.9% 2.4% 3.5% 1,9%

GDP per capita 2005

172 115 122 110 100

Unem-ployment 2006

3.5% 7,0% 3,9% 7,7% 7,9%

Low R&D InvestementsExtreme wealth creation

The Swedish and Norwegian paradoxes Sweden

Huge investments in R&D and high tech companies do not lead to more innovation, productivity or profit

Norway

Seemingly low R&D investments

Nevertheless the richest and most productive country in Europe

EUs innovasjonsindeks 2004

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

EU Innovation Scoreboard indicates a poor innovation capacity

Kilde: EU Innovation Scoreboard, 2004

Summary Innovation Index, 2004

But Norway has an extremely high productivity

EU 25

FinlandEU 15

USA

Luxem-burg

Norway with oil

Sweden

MainlandNorway

Denmark

There seems, in fact, to be no clear correlation between national R&D investments and economic growth.

So now what do we do? We are convinced

R&D in a narrow sense and learning in a broad sense explains growth.

But where is the convincing story that makes sense to our policy audience?

The Research Council of Norway and Innovation Norway have brainstormed a lot!

We are slowly developing a new story that makes sense to us, working with researchers and ministries

But the new story brings up new questions with no answers

This is not a unique phenomenon. It seems the consequences of a systemic approach to learning and innovation is finally catching up with us.

(Vinnova of Sweden is also taking part)

The main explanation

OECD and EIS indicator sets are misused both by the OECD and the EU and national governments.

The need for benchmarking leads to the idea of “best practice” and model countries (Finland, US, Ireland and Israel)

The heterogeneity of national innovation systems is ignored

Model based on old fashioned health care. One treatment fits all.

Countries may vary a lot as regards industrial structure

Sweden and Finland has some large high tech companies that explain the R&D investment levels

Nokia, Ericsson, Volvo, ABB)

Norway and Iceland are dominated by traditional industries that do not normally invest much in R&D

Fisheries, aquaculture, oil and gas

Denmark is somewhere in between

Note also that the Norwegian GDP is extremely large

A nagging suspicion

Companies may not fully have grasped the Frascati manual.

The understanding of basic concepts like research, (experimental) development and innovation may vary from

Industry to industry (e.g. manufacturing vs. services)

Company to company (different reporting practices)

Country to country (different incentives, e.g. tax)

The main problem: Developmental work“That’s just what we do!”

Norway EU Innovation ScoreboardEIS 2005 Innovation performance (relative to EU average) - NORWAY

76148150

193124

11987

8296

87

111137

65128

9890

12121

3260

69

9877

10827

49

0 20 40 60 80 100 120 140 160 180 200

INNOVATION DRIVERSS&E graduates

Tertiary educationBroadband penetration

Lifelong learningYouth education

KNOWLEDGE CREATIONPublic R&D exp

Business R&D expMed/hi-tech manuf R&D

Public funding innovationUniv R&D financed by bus

ENTREPRENEURSHIPSMEs innovating in-house

% all SMEs collab. on innovationInnovation expenditures

Early stage venture capitalICT expendituresNon-tech change

APPLICATIONEmploym hi-tech services

Hi-tech exportsNew-to-mark product salesNew-to-firm product sales

Employm med/hi-tech manufINTELLECTUAL PROPERTY

EPO patentsUSPTO patents

Triad patentsCommunity Trademarks

Community Designs

Low Medium-low Average Medium-high High

We need to shift from a technology push perspective to a vision of learning

The 3 percent objective is based on a linear model

1. Increase national investments in R&D

2. Produce more knowledge

3. Apply that knowledge to industrial production

4. Result: innovation

5. and wealth creation

There is nothing wrong in having increased investments in R&D as a policy objective, but here it is used as a proxy for innovation

Focus on other aspects of learning

R&D embedded in technology and human capital

Design

Branding

Organisational change

Management practices and types of ownership

A new focus on research as a learning tool

The effects of R&D products on the innovation system

Company profits as result of sales of new or improved products, processes or services

Spillover effect 1: the new products leads to increasing productivity among customers

Spillover effect 2: the new products leads to innovation among customers and suppliers

The effects of R&D on learning in the innovation system

Research builds competences that can be used to absorb knowledge and technologies developed elsewhere

Research may lead to network building

University research teaches students how to use the tools of science

Do we have indicators for learning?

The numbers of students

The size of faculty

The number of Ph.D’s

The size of R&D budgets

These are all input-indicators that say nothing about outcome.

What about research output indicators?

Publications

Citations

• These are important as they are measures of research productivity and quality, • But they are not measuring the effects this research has on society

• Different publication practices in different disciplines• Publish or perish hysteria•Bottom-up perspective on research

Indicators for the interaction with society

Patents

Licenses

Co-publication

Popularization and communication

Income from commissioned research

Collaboration agreements

Participation in the EU Framework Programme

These are measures of interaction, but not of the effect on society.

We have to connect research input to social, economical and environmental output Now research is seen as one

of many “deserving causes”, and it lose out if set up against health, poverty or the environment.

Have to communicate that research is part of the solution of major social challenges.

One example: Connect company R&D investments with accounting data

The Nordic advantage: The autonomous workers make the bumblebees fly

The Nordic advantage: staff that solves problems independently and finds competences outside the organization.

Connected to educational level and research as a learning tool.

(Source: Åge Mariussen, NIFU STEP)

-1,00000 0,00000 1,00000

Market learning

10,00

20,00

30,00

40,00

Denmark

Germany Finland

Sw edenUK

Norw ay

Slovenia

Spain

Hungary

France

Austria

Be lgium

Czech R

Sw itzerlan

Turkey

GreeceIta ly

Netherland

Poland

Portugal

Slovak R

Ire land

Europe: market learningGDP/ capita

Source:Fourth European Working Conditions SurveyOECD

Concepts like “high tech” and “low tech” makes little sense

Understanding innovation in resource based industries Resource-based industries

may be knowledge-intensive and profitable, but not R&D intensive

Farming, aquaculture fisheries, petroleum and mining

The word “high tech” is misleading, as it refers to R&D as a percentage of company turn-over, and not to the company’s use of advanced technology and knowledge

Low tech, but knowledge intensive

A high tech company is per definition a company that invests much in R&D.

Norwegian petroleum companies and Swedish and Finish mining companies do not invest much in R&D, but they do make use of advanced technologies) and they do employ highly competent engineers

A lot of process innovation

Companies make use of other forms of innovation: branding, marketing

Little R&D per company, but branches of industry as a whole may invest much (food)

Oil- and gas is defined as high tech, but is in fact veeeery advanced

Indirect use of research

MS Kristian With Refrigeration/containership built by Vaagland Båtbyggeri AS on the North-West Coast of Norway

Argon AS has installed the electronics

Radar, satellite phone and TV-antenna delivered by ProNav

Sonar, logg, radio and electronic map systems delivered by Furuno

Gyro compass and autopilot delivered by Simrad

The advanced technology has been “black-boxed”. You do not need to know how to build a TV to watch the Simpsons.

We need to get a better understanding of the role of services

The largest part of the economy

A very heterogeneous sector

A residual factor (what’s left when we leave out manufacturing and food)

We need a new categorization

Until now: Industrial policies have been focused on manufacturing, even if 70 percent are employed in services

Important service sectors

B2C services: non-R&D companies, including retail, people care and tourism

Retail: Innovation in transport, storage, delivery and customer care

Tourism: Innovation in product range, presentation, transportation and marketing

Services as partners and knowledge providers for industry

Advanced knowledge providers, e.g. R&D intensive B2B ICT companies (Knowledge intensive business services)

These companies may compensate for the lack of R&D in others.

Low R&D measurements in some industries may be an effect of outsourcing

Understanding the effects of public sector innovation

Innovation in the private sector is understood as an investment, in the public sector as an expense

Innovation in the public sector and the effect on industry

Public/private learning arenas

Public procurement

The effect of social welfare on risk taking and company behavior

We have no output indicators!

The Copenhagen Manual

New Nordic project aimed at developing a statistics manual for public sector innovation

Danish Agency for Science, Technology and Innovation, Denmark

Danish Centre for Studies in Research and Research Policy, CFA,

NIFU STEP, Norway

RANNIS, Iceland

Innovation Norway, Norway

The Research Council of Norway

DAMVAD, Denmark

Statistics Finland

Statistics Norway

Statistics Denmark

Statistics Sweden

In cooperation with: OECD NESTI NESTA UK

Company

LearningNetworksinnovation

Customers and users

Suppliers

Policy-institutions

FinancialInstitutions

R&DInstitutions

Consultants

Public policy Cultural framework

International framework Industrial structure

Understanding competence flows in the innovation system

The role of competence flows

The role of education

User-driven innovation

Customer/supplier relationships

“Open innovation” and industry collaboration

The role of KIBS

The role of public sector institutions

National innovation systems must have porous boundaries.The EU is not competing with the US or Japan. This is not a zero-sum game!

Understanding the heterogeneity of innovation systems

Norway, Sweden, Iceland, Finland, the Netherlands, Bravaria, Catalonia, Northern Italy and the UK have all successful innovation systems that produces wealth.

But they are all totally different from each other.

Unique historical trajectories.

Unique socio-cultural framework conditions

There is no best practice.

Finland and Ireland cannot be used as models for other countries!

Understanding socio-cultural framework conditions Stable macro-economic framework conditions

Disciplined fiscal policy

Competition policy encouraging innovation

Low taxes

An open economy

Socio-cultural framework conditions

Egalitarian culture with high social mobility

High wages for blue collar work gives impetus towards innovation (robots, internet banking)

High educational levels brings flexibility and labour mobility

An efficient public sector helps industry

A trustworthy welfare system reduces risk

Political and social stability engender trust

Where are we now?

The 2008 report on Norwegian innovation tells the new story in full: There is no paradox!

Increasing consensus: The Nordic success is competence based.

DG Enterprise and DG Research are working on more new nuanced interpretations of indicators

White papers in all the Nordic countries try to take the heterogeneity and complexity of innovation into consideration

Online R&D statistics and research and innovation policy information

Cordis http://cordis.europa.eu/indicators/

Pro Inno Europe http://www.proinno-europe.eu

Eurostat http://epp.eurostat.ec.europa.eu

OECD http://tinyurl.com/6dt2qn

Erawatch http://cordis.europa.eu/erawatch/

Worldbank: http://www.worldbank.org/kam

A Norwegian portal for R&D policy info: www.forinn.no