Learning more from evaluationsThe use of thematic approach and impact modelling in

evaluating public support measures

Innovation Policy

Tekes Technology Programmes

Thematic programme evaluation

Conclusions

Jari RomanainenJari RomanainenExecutive Director, StrategyExecutive Director, Strategy

National Technology Agency, FinlandNational Technology Agency, Finland

OutlineOutline

Research and the Knowledge Based Society - Measuring the LinkGalway, 24 May, 2004

Innovation policy governance The role of innovation and innovation policy in other sectors

Increasing number of stakeholders

Regional development, employment, social divide, etc.

Non-technological innovation new business models and value chains

service innovation, social innovation

Renewal of traditional sectors industries

public sector

universities

Maintaining the balance science - industry

economic - social

existing - new

Framework conditions, creating the demand regulatory framework, e.g. IPR, taxation of private VC investments

public procurement, lead markets, etc.

From 2nd to 3rd Generation Innovation Policy

Information and communications technology

Material technologyBiotechnology

Key Areas of Industrial Renewal andWelfare-promotion

Sustainable development

future energy solutions environmental technologies

ecological effectivenessand low-emission processes

life-cycle solutions

Knowledge-intensiveservices

knowledge-intensive business services

product-integrated services new technology-related services

Intelligent products,processes and systems adaptive and intelligent

products, materials and systems navigation and identification

virtual models

Welfare information and communications

technology for health care functional foods

targeted pharmaceuticalsand diagnostics

healthy and safe living environment

DM 1728404-2002 Copyright © Tekes

Business competencein the networked economy

networks with new value cluster cooperation

fast commercialisationof ideas

digital economy

Implementation

Analysingpolicyneeds

Settingpolicyagenda

Impactevaluation

Impactevaluation

Performanceevaluation

Performanceevaluation

PrivateinterestsPrivateinterests

PublicinterestsPublic

interests ResearchResearch

NationalstrategyNationalstrategy

Strategicintelligence

Strategicintelligence

Policyevaluation

Policyevaluation

SectorpoliciesSector

policies

Implementationstrategies

Implementationstrategies

Instrumentset-up

Instrumentset-up

Innovation Policy Cycle

Competitiveness,profitability and

growth

New businesses,start-ups

Impact of Tekes activities

Research institutesand universities

Enterprises

Social and environmental

impacts

Projects and programmesInternational cooperation

Tekesprovides expert services andR&D fundingcoordinates programmes

DM 3218911-2003 Copyright © Tekes

Boosting exports, broadening Finland’s industrial base,generating new jobs and improving wellbeing.

Boosting exports, broadening Finland’s industrial base,generating new jobs and improving wellbeing.

What are Technology Programmes?

extensive programmes initiated by Tekesand consisting of numerous projects

focused on a key technology sectorimplemented in cooperation by companies

and research unitscompanies can participate with their own

projects or by joining commonresearch projects

projects and resultsare partially public

DM 3025603-2003 Copyright © Tekes

Technology programme model

Steeringenterprises

TekesGrants

Company R&D projects

Public research projects

SynergyNetworking

Part financing

Implementation of results of public research is based onexecution and networking in parallel with company projects.

preparationcoordinationdecision

making

DM 3025601-2004 Copyright © Tekes

GrantsLoans

Capital loans

Key figures of technology programmes

23 ongoing programmes from the beginning of 2004 with a total cost of 1.2 billion euros

Each programme typically lasts 3-5 years2,000 company participations annually800 research unit participations annuallyTekes usually finances

60-80% of university projects 25-50% of company projects

DM 3025601-2004 Copyright © Tekes

MARKETS quality

competitiveness price

competitiveness environmental

expertise

F O R E S T I N D U S T R Y

Pulp and paper technology

Wood products industry

Machines, machinery and processes

A11/ 99-08

Securing competitiveness inthe Forest Industry

FORESTRY

CONSTRUCTION AND WOOD TECHNOLOGY modification of wood construction

technology andarchitecture

logistics, assembly

MANUFACTURING TECHNOLOGY forestry machines process and production machinery material technology

BIOTECHNOLOGY enzymes rot prevention gene technology

ENERGY AND ENVIRON-MENTAL TECHNOLOGY biofuels, combustion technology ecobalances closed cycles energy-saving and

emissions

CHEMICAL TECHNOLOGY paper and bleaching chemicals surface treatment

substances pigments, adhesives

INFORMATIONTECHNOLOGY sensors, measuring

and control computational

intelligence,simulation andmachine vision

multimedia and telecommunication tomography

Programme evaluation at Tekes

First generation– Performance evaluation

– Feedback for participants

– Peer reviews

Second generation– Separation of performance and impact evaluation

– Internal performance evaluation supporting monitoring

– External impact evaluation supporting programme design

Third generation– Impact modelling and thematic evaluations

– Supporting renewal of technology programme concept

– Better support for policy design

Knowledge transferawareness, co-operation, mediation

Market conditionscustomers

competitorsvalue-chains

Knowledge baseeducationscience

R&D

Framework conditionsculture, social capital, financial services, innovation services, regulation

skilled labour

knowledgeideas

sophisticated demand

competitionclustering

Learninggovernance, strategic foresight, evaluation, monitoring, assessment

regulation services money

Innovationprocess

innovation capabilities

complementaryknowledge

complementaryresources

Systemic approach to innovation policy

Technologies and innovations that enhance well-fare and sustainability

Impact model of technology programmes

The activities of the programme and the projects creates competitiveness, job, well-fare and environmental impacts

Technologies and innovations that improveproductivity and create new added value

Technology development

Strategicrenewal

Operationalrenewal

New knowledge

Behaviouralchange in

organisations and clusters

Conceptual technology programme evaluations in 2002-2003 (1/2)

Programmes for new business operating models - to develop new business concepts - based on value chain analysis

- 3 programmes from construction and manufacturing- evaluation report 10/2002

Programmes for innovations in changing business regulation

- deregulation (electricity) and increasing regulation (environmental)

- evaluation report 12/02 Industrial cluster programmes

- activating companies to R&D, networking, industrial R&D strategy, strong regional and SME approach- programmes for furniture, stone and computing clusters

- evaluation report 18/2003

- “evolution of sector programmes” – construction cluster report 6/2003

… conceptual evaluations (1/2)

Programmes for innovations and commercialisation- programmes with new product and new business goals- user- and exploitation oriented- programmes for pharmaceuticals, diagnostics, computing etc.

- evaluation in 2003Targeted programmes

- problem-oriented focussed and well planned- “hands-on management” of projects

- programmes for plastics, paper and machinery clusters

- evaluation report 13/2003Programmes looking for technology synergy

- technology and knowledge transfer oriented- cross industrial programmes

- evaluation report 11/2003

Commercialisation

Technology

1980 1990 2000

TechnologyTechnology

Internationalcompetition

Internationalcompetition

Regu-lation

Regu-lation

Commercialisation

Evaluation of programmesduring the transition fromTechnology orientation to

Commercialisation

Thematic evaluation: Programmes targeting behavioural change and clustering

Targeting behavioural change and clusteringrequires a more comprehensive approach

Firm R&D

Firm strategicbehaviour

TechnologyServices

Target

Content

Business models

Now

Previously

Source: Gaia Group

Technology programmes is a concept delivering a tailored mix of value added services – a toolbox

Source: Gaia Group

Internationalisation in technology programmes

Programme proposal preparation:

-Focus,

-Structure,

-Timing

Feasibility check & decision

Expressed or latent need for public action.

Programme launch & Call for projects

Phase I:

Programme promotion, project selection, network building

Phase II:

Joint activities, first results, intensified networking

Phase III

Result collection, dissemination, future orientation

Midterm evaluation

Programme evaluation

Final reporting

DefinitionDefinition ImplementationImplementation Follow-upFollow-up

Programme output, Programme output, impact & triggered impact & triggered changeschanges

Follow-up programmes

Strategic decisionsStrategic decisions

Analysis of Analysis of international international context(?):context(?):- Needs- Needs- Competition- Competition- Partners- Partners- Competence - Competence

Joint launch, Joint launch, international international promotion, open promotion, open call? call?

Finding of foreign Finding of foreign partners / partners / agencies / agencies / programmes? programmes? Foreign Foreign

projects? projects?

International co-International co-operation, market operation, market checking, checking, application application partners? partners?

International International promotion of promotion of results? results?

International business International business promotion, promotion, internationalisation of internationalisation of projects? projects?

Project co-Project co-operation? operation?

Research oriented international activitiesResearch oriented international activitiesCommercially oriented international activitiesCommercially oriented international activities

Source: Advansis

Rationales for internationalisation

Firms

Did not participate/play an important role

Reaching

Intrn leading-edge (upgrading)

Increasing lead in relation to intrn leading-edge

(upgrading)

Exploiting

Finnish competence

Research institutes

Exploiting

Finnish competence

(exchanging knowledge)

International research programmes

Intrn research & activation of business in the sector

Home-base extension(?)

Home-base extension + export orientation

Increasing lead in

relation to intrn leading-

edge (upgrading)

Domestic research (+networking)

Knowledge-flows: Research -> business sector

Domestic R&D – ”No need for others”

Commercialisation of research competence

Reaching

Intrn leading-edge

(upgrading)

Technology transfer(?)

”Lakki kädessä” Knowledge flows: Business sector -> research

Export orientation and activation of supporting research

Did not participate/pl

ay an important

role

No programme Domestic activation of sectors

Domestic R&D, (+networking)

Strong export orientation

Antares?

FFusion2

GPB

Kesto (turb)

KennoPresto

Spin

Kesto (soda)

TeslaNavi

iWell

Source: Advansis

Findings & statements 1(2)

Rationales for internationalisation:

• Key rationales for internationalisation relate to seeking better technological knowledge from lead developers to up-grade Finnish competence and experience on applying & delivering technological knowledge in target markets.

• For most companies, internationalisation aims are naturally linked to business objectives – how to get access to information on the application side of technologies. ‘Internationalisation should not be done just because the programme is building international links. The reasons must be found from each business.’

• Large companies are international, almost by definition. When it comes to promoting international co-operation for commercial aims there is little reason for large companies to join.

Source: Advansis

Findings & statements 2(2)

Impact / added value of technology programmes:

• Programmes can have an important impact on the integration/synergy of the industry. This is important when trying to increase the awareness of Finnish competence abroad – one good reference speaks for the whole sector.

• Being part of a technology programme gives an image abroad that you are among the selected, qualified and backed-up projects. It does improve your credibility.

• International partners often appreciate formal qualifications, along with practical cultural & language skills. It should not be forgotten, that technology programmes are closely linked to graduate schools, which provide young, educated resources to their fields.

Source: Advansis

Benefits of Thematic Evaluations

Provide a much better insight into technology programmes as a concept

Allows stronger conclusions and recommendations

Provides understanding of sector specificity and generality of different impact mechanisms

Create a learning platform for exchanging good practices

Improves the design of new programmes

Conclusions

Innovation policy consists of a mix of schemes targeted at several systemic failures

Evaluation should recognise the role single schemes play in the policy mix targeting these failures

Impact modelling and thematic approach can be useful in better understanding the role different schemes play in policy implementation, impact of mix of schemes and identifying good practices across schemes

Impact modelling and thematic approach require combination of evaluation methodologies and demand methodological development

Evaluation should be recognised as a continuous learning process

Measuring the link

Requires in-depth understanding of the various interconnected mechanisms through which the impact of research and innovation policy are eventually realised

Real understanding of the various impact mechanisms– Focus on systems, processes and policy mix instead of structures,

actors, bilateral interactions and single instruments

– Combinations of methodologies

Focus on systemic failures

Human capital– Focus on SET is not enough!

– Creativeness, innovativeness, multidisciplinary and networking skills, adaptive skill, learning skills

– Life-long learning, links between education and research, social innovation