GenIEGeorgia National Innovation Ecosystem

Istanbul, May 14, 2015

GENIE Project Summary

Intervention Summary

• Objective:– Enhance the Georgian national innovation

ecosystem to fuel economic growth and jobs

• Components to be evaluated: – Regional Innovation Hubs: training and technology

to entrepreneurs and SMEs, e.g. 3-D printers– Enterprise innovation matching grants

Regional Innovation Hubs Concept

Evaluation QuestionsRegional Innovation Hubs• How do we get people in the door?• What is the right menu of services/pricing

encouragement?• What is the impact of the treatment?

Matching grants• What is effect of MGs on innovation, profits, jobs,

firm survival?• Does proposal scoring index work?

Evaluation DesignRegional Innovation Hubs• 1-2 pilot centers (randomizing clients, not centers)• Encouragement design (RCT)

– Coupons, directed marketing (to SMEs, college students)– Services, e.g. use of fabrication machines, training on use of

equipment, business/marketing training

Matching grants• Randomization conditional on score in competition

– Certain score correspond to certain probability of being chosen.– More firms with good scores, less with low scores.– Gives indication of both whether index works and whether

impact overall.

Evaluation Design: RCT encouragement schematic

Encouragement Non-encouragement

Business trainingTechnical trainingUse of fab. machines

DM + CouponsDirect Marketing

Sample and dataRegional Innovation Hubs• Sample: List of all local firms from business

registry; STEM university students• Data: panel surveys, including baseline and annual

follow-ups

Matching grants • Sample: SME applicants that meet minimum

selection criteria• Data: firm reporting

TimelineRegional Innovation Hubs• Sampling frame: October 2015• Survey questions designed and tested: Nov 2015• Baseline survey: January 2016• Pilot Hubs open: March 2016• Send encouragements in April 2016

Matching grants• Awareness raising campaign: January 2016• Call for proposals: February 2016• TA provided to firms to help prepare proposals: Feb to March

2016• Award proposals: April 2016

Thank You!

Team and staffing

• WBG – IE Coordinator; Research Assistants (local and foreign)

• WBG TTL, project team• Private Firm – refine sampling frame;

conducting survey; initial data analysis; • GITA – Mngt of the Hubs, Encouragement,

Portal for Data Collection from Users • Local Hosts – Provision services, Data Collection

from Users

BudgetName of the Service Cost

Baseline Survey Cost 120 K

Researchers: baseline survey & Analysis

20-45 K

Follow up Surveys 200 K

Researchers: follow up survey & analysis

120 K

IE Coordinator 20 K

Marketing

Administrative

Applied R&D ProjectsNational Centre for Research&Development

Adam KaźmierczakPOLAND

Istanbul, May 14, 2015

Intervention SummaryApplied R&D Projects

• Objective: Increase commercialization of R&D (global novelty) by firms

• Components to be evaluated: matching grant scheme offered to consortia of firms and R&D units

1. INPUTSFunds that creates cooperation between science and business

2. OUTPUTSResearch activity&collaboration (global novelty)

3. OUTCOMESCommercialization of the product based on research results

Eligible Applicant

1 RESEAR

CH ORGANIS

ATION

CONSORTI

UMEnterprise

both SMEs and large enterprises

registered in Poland

LEADER OF THE CONSORTIUM

– to be decided by the consortium partners– officialy submits the proposal to NCBR– responsible for official correspondence with NCBR

Evaluation Questions (1)

• What is the impact of matching grant on enterprise innovation and performance?

– What is the impact on the type/quality of innovation by firms?

– What is the impact on the research institute partners: on their research productivity, quality and tendency to collaborate with firms?

Evaluation Questions (2) – additional issues

• How to design the program to maximize chances for commercialization?– Increasing the minimum financial

involvement of firms in the consortia– Changing the scoring system to highlight the

business dimension of the project

Evaluation Design

• RDD is possible because applications are scored by a panel of experts, and only applications above clear threshold are recommended to be financed.– Compare the change in the outcome for

units that scored near the cut-off.• Our previous experience with a similar

program indicates that there may be enough consortia with scores near the cut-off to make RDD feasible.

Sample and data

• Sample will be applicants (around the treshold)• Data sources:

– Information collected during application stage (baseline)

– National census of firms (we will match our applicant to this database)

– Project reports– Dedicated survey for midpoint & endline

Timeline• Program timeline

– Call opening (end of 2015)– Selection process (till March 2016)– Agreements and start of reimbursement (from March 2016)

• Lengh of projects: up to 5 years• IE time

– Design by end of 2015– Baseline data: during the application proces (ending March

2016)– Midline: around 2018/2019– Endline: around 2021

THANK YOU!

Team and staffing

• Siddharth Sharma (WB Facilitator); • Adam Kaźmierczak (NCBR)• Monika Woźniak (NCBR)• Sylwia Lepieszka (MIR)• Chiaki Yamamoto (IFC)

Incubator Coaching IE

Poland Istanbul, May 14, 2015

Intervention Summary

Intervention: External consulting services for regional incubators to improve start-up’s performance

Objective: to improve capacity of incubators to provide high value services to startup companies

Setting the scene

60 heterogeneous incubators in 16 regions (avg. 50 companies per incubator)

25

60 incubators

26

Intervention logic

27

Incubator coachingQuality of

incubator servicesEffect on firms

(sales etc.)

Intervention Summary

“Possible” modules of external consulting to eliminate start-up constraints

– Hands on management coaching (lack of skills)

– Management of Innovation (lack of skills)– Knowledge Sharing (access to knowledge)– Network Strategies (coordination failure)– Access to finance (asymmetric information)

Evaluation Questions

General Question

What is the impact of external consulting services on incubators and entrepreneurs?

The aim is to measure different outcomes related to different aspects of the external consulting services.

Evaluation Design

• Randomized control trial• Phasing randomization of consulting services • The randomization done at start-up level

Services are provided with a one year interval

Up to three rounds of courses (TBD)

Outcome Measurement

• The aim is to measure the following outcomesAverage revenue growthEmployment creation External equity financingBank financingGrant applicationCooperation with other firms/joint projectSuccessful exit

Sample and data

• About 3000 start-ups• To be decided when the list of start-ups is

finalized• It might be possible to survey all start-ups

Timeline

• Prior to interventionList of entrepreneurs for randomization • First YearEntrepreneurs from all incubators are surveyed

but only half start the treatment• Second YearThe control group is interviewedThe short-term impact is measured

Timeline

The short-term impact to be measured is linked to the general performance of start-ups as well as related to each specific modules of the external consultancy

Challenges

• Small number of incubators• Spillover effects on non-participants• Differentiation of services provided by

incubators• Budget constraints (all incubators need to be

treated ???)

35

IE of Myanmar Investment Law

Myanmar Istanbul, May 14, 2015

Intervention SummaryObjectives:

Improve the legal and regulatory framework to (i) generate new, diversified investment (domestic & foreign)(ii) reduce private sector compliance costs

Components:

1. [Retrospective] Easing of foreign investment restrictions on economic activities (since 2013)

2. Rationalizing (fiscal, non-fiscal) incentives (TBD) for domestic and foreign investors

3. Improving Investor protection and dispute settlement mechanisms

4. Streamlining investment approval process

Evaluation Questions

1. [Retrospective] Can removing foreign investment restrictions stimulate overall investment and growth? • Does FDI crowd in, displace, or crowd out domestic investment?

2. Can strategic (fiscal, non-fiscal) incentives increase investment?• Can targeted marketing and outreach activities increase awareness of

the new law and encourage take-up of investment incentives?

3. Can investor protection guarantees and mechanisms attract new foreign investment?• Are guarantees/DRMs more effective for investors with lower risk

tolerance?

4. Can lower investor compliance cost generate more investment?

Evaluation Design

1. [Retrospective] Diff-in-diff with propensity score matching– In 2013, many previously restricted Economic Activities (EAs)

reclassified as open to foreign investors– Compare local and foreign investment in EAs that changed status

(closed -> open) with those that did not

2, 3 & 4. RCT with encouragement design – New investment law (and accompanying regulation) to offer new legal

provisions, investment incentives, and investor guarantees. Assumption: awareness/take-up will be low

– Subsequently, DICA will target randomly selected (domestic and foreign) investors with awareness/messaging campaign

– Compare take-up and subsequent investment activity by investors targeted by DICA vs not

Sample and Data

Domestic: 634 firms surveyed in 2014 Enterprise SurveyMinistry of National Planning and Economic Development’s data?Domestic firms with foreign partnerships post 2013

Foreign:DICA investment proposalsMIGA insurance applicants?Participant list from previous investment fairs?Others?

Timeline

• November 2015: new investment law passes• Also in November: Myanmar votes

?

Improving G2B responsiveness

Rwanda Istanbul, May 14, 2015

E-PPD: Intervention Summary• Investment in Rwanda is 4 times less than what is expected given the business environment

reforms undertaken (WB, 2014). MSMEs claim a lack of predictability, transparency and accountability as well as feeling “alone” after the registration process among the reasons not to invest as initially planned. There seems to be issues around the openness and reach of traditional PPD.

• Objective: The overall investment climate program seeks to improve the business environment so as to increase investments, especially outside of the capital so as to contribute to create jobs throughout the country by:

1) Improving G2B services: using IT solutions to increase reach, transparency and predictability and Institution building.

2) Attracting investments into the horticulture and tourism

• Components to be evaluated

The e-PPD mechanism is a subcomponent of (1). The e-PPD is an electronic platform that allows firms to provide feedback on quality of government service delivery to businesses. Aggregated, anonymous B2G feedback and responses from government can drive meaningful reforms and increase predictability, transparency and accountability.

Evaluation Questions• What is the best variation of quantitative and/or qualitative data that would provide the most

meaningful data for motivating the government to reform? (experiment in designing the e-PPD)

• What is the impact of participating in the e-PPD in firms’ perception about government services and trust in the business environment? Is the e-PPD as effective as a traditional PPD in changing firms’ perceptions?

• Does varying the intensity of information provided back to businesses have different impacts on perceptions?

• What is the impact of the different interventions on firm-level investment (through increased confidence in the business environment)?

Evaluation Design• Randomized controlled trial among local

formal MSMEsSample of local formal

MSMEs using government services

PPD meeting(s)

e-PPD

e-PPD + regular

information to firm on

govt reforms

Control Group

(1) (2) (3) (4)

Sample and data• Population: formal MSMEs (78,000)

• Sampling and randomization when firms use government services

• Randomization stratified by location (Kigali vs outside), size (micro vs SMEs), and gender

• Initial (rough) sample size calculations indicate minimum of 2,500 firms

TimelineActivity Timeline

Pilot of interventions to design e-PPD +6 months

Qualitative data collection on current process +6m

Pilots on measurement of outcomes +6m

Concept note design +6m

Intervention +12m

Follow-ups and analysis +12m; +18m; +24m

Presentation ends here

Team and staffing

• Team: Francisco &TBD• Program Info: P600783, Ashani (TTL), Estem

(M&E), Matina (IT)• GoR: Paula Ingabire (Head of RDB IT), Innocent

Bajiji (Head of aftercare), Daniel (Head of PPD at RDB)

Budget

• Program P600783: covers development of the tool (300K).

• IE budget: field coordinator (STC), 2 researchers (30 days/year), field missions (2/year at 10,000 per year), surveys of 2500 (3*175K= 525K) ---- approx. 865K