Outline for a socio-economic conceptual framework to evaluate the impact of

innovation platforms

Jo Cadilhon

ILRI Internal meeting on Innovation Platforms, Nairobi, 6-7 December 2012

Jo Cadilhon, ILRI 2

Outline of presentation

• Definitions• Why is it important to work on innovation platforms?• Objectives of innovation platforms• Literature review to construct the conceptual

framework• Presentation of the conceptual framework• Monitoring and evaluation setup• First ideas for questionnaires and analysis tools• Discussion and call for help on improvements

Jo Cadilhon, ILRI 3

Definitions

Innovation systems are complex, open and dynamic human activity systems in which actors (individuals, groups, organisations) apply their minds, energies and resources to innovation in a particular domain of human activity (Daane, 2010)

An innovation system can be defined as a network of organizations, enterprises, and individuals focused on bringing new products, new processes, and new forms of organization into economic use, together with the institutions and policies that affect their behavior and performance (World Bank, 2006)

Jo Cadilhon, ILRI 4

More definitions

An innovation platform is a needs-based network bringing together stakeholders from different interest groups, disciplines, sectors and organizations to exchange knowledge, generate innovation and develop joint action. Platforms are more than just places to talk; they create opportunities for stakeholders to test solutions to common problems (Cullen & Ergano, 2011)

Interprofessional organizations are a conventional arrangement, the goal of which is to create value thanks to cooperative behaviour between agricultural producers and industrial firms, and to share the quasi-rent thus created (Valceschini, 2002)

Jo Cadilhon, ILRI 5

A final definition

Commodity associations are organizations that bring together a wide spectrum of interest groups related to a particular commodity or sector (such as horticulture) in a particular country, whether the commodity is for export, for the domestic market or for both. Such associations can draw membership from individual farmers or their associations, from crop buyers, processors, distributors and exporters, as well as from suppliers of support services. Sometimes government agencies are also members (Shepherd et al, 2009)

Jo Cadilhon, ILRI 6

Why work through innovation platforms for agricultural development?

• Because governments and donors have finally recognized the role of the private sector and civil society in agricultural development so as to achieve food security

• Agro-industries and value chains can pull agricultural production in developing countries

• National agro-industrial development policies now encourage strengthening of value chain networks

• Past history of agriculture development projects being confronted with a lack of markets once productivity has increased is encouraging integration of marketing activities in development projects

• Multi-stakeholder interaction recognized as best practice for agricultural knowledge and information systems

(World Bank, 2008; FAO & UNIDO, 2010; Cadilhon & Even, 2012; EU SCAR, 2012)

Jo Cadilhon, ILRI 7

Objectives of innovation platforms• Advocacy of industry interests in policy making circles• Collective promotion of products• Concerted setup of quality standards• Research and development, technology uptake• Capacity building• Market regulation activities

– Market information and statistics– Arbitration of chain conflicts– Limit transaction costs (collective negociation, price

setting)– Setting production targets and allocating production share

among members for specific quality products(Cadilhon & Dedieu, 2011)

Jo Cadilhon, ILRI 8

Literature review to construct the conceptual framework

Based on three socio-economic theories:• Structure – Conduct – Performance• New Institutional Economics • Supply Chain Management and Marketing

Jo Cadilhon, ILRI 9

An elegant economic framework:Structure – Conduct – Performance

• Developed by Bain in an industrial setting (1959)• Posits link between market structure

– Number of players– Market share of players

• The conduct of the players in the market– Competition– Collusion– Price fixing– Raising barriers to entry

• And the performance of the market– Price correlations between different physical markets– Prive variations– Equity of margin distribution among market players

Jo Cadilhon, ILRI 10

The flaws of the SCP framework

• Utilized in developing country settings to make propositions for new market infrastructure, with mixed results

• The model benchmark for the best market conditions is the pure and perfectly competitive market, virtually inexistent in real markets with real people

• SCP completely negates larger environmental influences on the marketing system

• Relies on price data as primary indicator of market performance, always difficult to collect reliably

Þ Remember the elegant logic, don’t mention the outdated framework

Jo Cadilhon, ILRI 11

New institutional economics

• This framework fully recognizes the uncertainty that is endemic in the food industry:– Technical and economic characteristics of the product– Seasonality of production– Weather instability– Unstable food markets

• NIE posits that market stakeholders will create a specific institutional background (laws, norms of behaviour) and organizational setting (associations, co-operatives, contracts, firms) to deal with this uncertain market environment

• It ponders the optimal market institution to reduce transaction costs incurred to secure a deal and the sharing of the quasi-rent (value addition) between buyer and seller

(Williamson, 1991; Hobbs, 1996; Valceschini, 2002)

Jo Cadilhon, ILRI 12

Theoretical inputs from supply chain management and marketing research

• Captures better the continuum of possible marketing relationships that NIE only refers to as ‘hybrid forms’ between spot transactions and firm integration

• More approachable vocabulary for non-economists interested in using marketing concepts

• Re-socializes research on market relations• Provides a range of tested performance measurement metrics• A marketing orientation pushes firms to be focused

throughout their activity:– Implementing market analysis to discover customer needs– Cooperating with chain partners to react to these results– Embedding the marketing concept in all departments of the firm(Webster, 1992; Fearne, 2000; Duteurtre, 2003; Noble et al, 2002)

Jo Cadilhon, ILRI 13

Elements of SCM with positive impacts on chain performance

• Cooperation, coordination and joint planning• Information sharing and communication• Joint conflict resolution mechanisms • Trust, based on:– Credibility of actions and promises– Process, individual characteristics and institutions– Social capital

• Interdependence(Anderson & Narus, 1984, 1990; Kumar, 1996; Batt, 2003; Durlauf, 2002; Hingley & Lindgreen, 2002)

Jo Cadilhon, ILRI 14

The conceptual framework for monitoring and evaluation of innovation platforms‘Structure’ ‘Conduct’ ‘Performance’

Internal structure • Membership

composition• Decision making process• Committees• Source of funding• Staff availabilityLegal and regulatory frameworkCultural normsGenderType of chain stakeholder

CooperationCoordinationJoint planningInformation sharing CommunicationTrustInterdependence

AdvocacyCollective promotionJoint quality standardsResearch & developmentCapacity buildingMarket informationArbitration of chain conflictLimit transaction costsSetting concerted marketing objectives

Jo Cadilhon, ILRI 15

Monitoring and evaluation setup

• Based on questionnaires of facilitators or managers of innovation platforms and their members

• Same questionnaires administered at start of activity, regularly during the activity, and at the end of an activity capture evolutions in the platform’s ‘structure’, ‘conduct’ and ‘performance’

• Statistical tools enable to demonstrate potentially significant relationships between S, C and P over time

Þ Possible to attribute the relative share of the structure and ways of functioning of an innovation platform on its development outcomes

Jo Cadilhon, ILRI 16

First ideas for questionnaires

• ‘Structure’ questionnaire• ‘Conduct’ questionnaire • ‘Performance’ questionnaire • Two latter based on 4–5(?) Likert scales so as to

capture variability of stakeholders’ opinions• ‘Performance’ questionnaire uses selected pages

according to objectives agreed upon by the platform• Statements in questionnaire based on performance

indicators suggested by Swaans et al (2012)

Jo Cadilhon, ILRI 17

‘Structure’ questionnaire

• Structured questionnaire to identify the modus operandi of the innovation platform– Membership composition– Decision making process– Dedicated committees, units or sections– Source of funding– Staff availability, function and numbers– Legal and regulatory framework

• Administered to facilitators or managers of the platform

Jo Cadilhon, ILRI 18

‘Conduct’ questionnaire

• Structured questionnaire to gather the opinion of platform stakeholders on the way the platform facilitates interactions between chain members

• Administered to members and facilitators of the platform, possibly to chain stakeholders who are not involved

• Short section to identify individual characteristics– Age– Sex– Type of stakeholder in chain– Indicator of wealth (type of house)– Level of education

Jo Cadilhon, ILRI 19

Example for information sharing section of ‘conduct’ questionnaire

• On a scale of 1 to 4 where 1 is ‘do not agree at all’, and 4 is ‘completely agree’, please rank your level of agreement with the following statements:– I share information on my business activities with other types of chain stakeholders

in the platform– I share information on my business activities with my peers within the chain– I share information on my business activities with my gender peers– I share information on my business activities with members of the opposite sex– I mainly use platform meetings to share information with other members of the

platform– I mainly use other communication tools to share information with other members

of the platform• Which other communication tools do you use to share information with

other members of the platform?• Are there factors other than the innovation platform that encourage you to

share information on your business activities with chain partners?

Jo Cadilhon, ILRI 20

Example for capacity building section of ‘performance’ questionnaire

• On a scale of 1 to 4 where 1 is ‘do not agree at all’, and 4 is ‘completely agree’, please rank your level of agreement with the following statements:– I actively seek out new opportunities to learn new skills– I participate in all of the capacity building activities of the innovation platform– I am satisfied by the quality of the capacity building activities of the innovation

platform– I am satisfied by the value for money of the capacity building activities of the

innovation platform– I have changed my practices following a capacity building activity I participated in

• Which other means do you use to build your capacity on agricultural production, marketing and business management?

• Are there factors other than the innovation platform that are important in helping you build your capacity on agricultural production, marketing and business management?

Jo Cadilhon, ILRI 21

First ideas for analysis tools

• Factor analysis to identify the dominant constructs of what makes innovation platforms successful

• Cluster analysis to identify different groups of stakeholders according to their perception of the platform

• Regression analyses to determine the relationships between the structure of the platform, the conduct of its stakeholders and whether they are achieving the objectives they set themselves

• Network analysis to represent the institutional elements of livestock markets and how they influence market participation and innovation uptake

Jo Cadilhon, ILRI 22

Links to work by Swaans et al (2012)

Jo Cadilhon, ILRI 23

Links to work by Swaans et al (2012)

• Provides a robust and systematic quantitative method to assess impact pathways along with more qualitative methods from Swaans et al (2012)

• Conceptual framework simplifies the impact pathway diagram while keeping most of its elements (contextual factors, household level, ecological objective missing)

• Swaans et al (2012) wish to evaluate the impacts of IP-based projects: IP as an intervention

• This framework tries to evaluate the impacts of the IP setting on the behaviour of its participants and partners, and ultimately, on the delivery against objectives agreed upon collectively: IP as an intervention tool

Jo Cadilhon, ILRI 24

Discussion• Method relies mainly on quantitative tools so runs the risk of missing

important explanatory factors that have not yet been identifiedÞ Allow questionnaire respondents to propose other impact pathways

than those tested by the framework• Method relies on succession of boring Likert scale statements and there

is a risk of interview bias because of the type of questions being askedÞ Possibly privilege administering the questionnaire by an interviewer

with paper questionnaire or interview tablet to avoid this bias• Method is not commodity specific so can be used on other products

than livestock products and in many country settingsÞ Strong potential for scaleability• To my knowledge, first attempt to quantify the impact of innovation

platforms on development outcomesÞ Strong potential for publishable research

Jo Cadilhon, ILRI 25

Call for help

• Please share case studies from your projects involving innovation platforms so as to illustrate this conceptual framework with concrete examples from the field

• Alternatively, identify situations in your current projects that could be documented further to provide such illustration

• Allow the conceptual framework to be tested on your project setting so as to improve it further while gaining useful evaluation insights

• Inform me of any new project ideas that might be interested in using this framework from its inception so as to undertake the complete monitoring and evaluation process

Thank you for your attentionFeedback welcome

Email: j.cadilhon@cgiar.org