EXPERT GROUP MEETING ON CLUSTER AND NETWORK DEVELOPMENT WITH

EXPERT GROUP MEETING ON CLUSTER AND NETWORK DEVELOPMENT

WITH SPECIAL EMPHASIS ON MONITORING AND EVALUATION ISSUES

Report

By:

Jonathan Dawson and Pierre Paris,UNIDO consultants

and

UNIDO Project Teams in India, Nicaragua, Senegal and Zimbabwe

Edited by:

Giovanna CeglieSmall and Medium Enterprise Branch

UNITED NATIONS INDUSTRIAL DEVELOPMENT ORGANIZATIONVienna, October 2003

Copyright © 2003 by the United Nations Industrial Development Organization (UNIDO)First published 2003

The designations employed and the presentation of material in this publication do not imply theexpression of any opinion whatsoever on the part of the Secretariat of the United NationsIndustrial Development Organization concerning the legal status of any country, territory, cityor area, or of its authorities, or concerning the delimitation of its frontiers or boundaries. Theopinions, figures and estimates set forth are the responsibility of the authors and should notnecessarily be considered as reflecting the views or carrying the endorsement of UNIDO. Thedesignations ‘developed’ and ‘developing’ economies are intended for statistical convenienceand do not necessarily express a judgement about the stage reached by a particular country orarea in the development process. Mention of firm names or commercial products does notimply endorsement by UNIDO.

This document has not been formally edited.ii

SECTION I 1

Introduction 3The UNIDO Cluster and Network Development Programme 3Importance of monitoring and evaluation 3Programme of the Expert Group Meeting 4

SECTION II 5

Project case studiesINDIA Cluster Development Programmes 7

Project basic facts 7Strategy 7Results 8Inputs and success factors 9Lessons learned 9Work Ahead 10

NICARAGUA: Development of local production systems, based on SMEs 11Project basic facts 11Strategy 11Results 12What made the difference? 13Lessons learned and future challenges 13

SENEGAL: Support to the small firms of Senegal 15Project Basic Facts 15Strategy 15Results 16Success factors 16Lesson learned 17Next step 17

ZIMBABWE: Development of the Small-scale Industry Sector throughClustering and Networking 19

Project basic facts 19Strategy 19

Table of contents

iii

Results 20Project Inputs 20Lessons learned 21Work Ahead 21

SECTION III 23

Learning from projects 25Considerations on overall experience of the UNIDO CND projects 25Methodological issues 25Similarities among projects 27Differences among projects 27Successes and challenges of the approach 28Future of the approach 28

SECTION IV 29

Monitoring and Evaluation issues 31Introduction 31What is M&E? 31Why do M&E? : different needs of different stakeholders 31What needs to be measured? 32The log-frame as a tool for M&E 32A. Measuring project performance 33B. Measuring project impact 33C. Measuring cost-effectiveness 38D. Measuring sustainability 39Conclusions: key principles of a CND M&E system 40

iv

SECTION I

INTRODUCTION

UNIDO has been implementing technicalcooperation projects based on a cluster andnetwork development (CND) approach sincethe mid 1990s.

The CND approach is built on three assump-tions:

1. that clustering and networking amongenterprises promotes enterprise competi-tiveness,

2. that public policy can help to facilitateclustering and networking; and

3. that support programmes targeting groupsof enterprises are more cost-efficient andcost-effective than those targeting indi-vidual enterprises.

UNIDO has adopted this approach as one ofits strategies for contributing to the develop-ment of small and medium enterprises indeveloping countries, as a means of promot-ing sustainable and equitable growth.

Encouraging results of, and increaseddemand for, CND projects from withindeveloping countries have led to the estab-lishment of a comprehensive internationalCND programme. This UNIDO programmeincludes1:

• project-level activities;

• the development of methodologies toguide project implementation;

• training programmes for CND develop-ment agents; and

• action-oriented research.

UNIDO is currently designing/implement-ing CND projects in Colombia Ecuador,Egypt, India, Morocco, Nicaragua, Nigeria,Pakistan, Senegal, Thailand, Tunisia,Zimbabwe.

1 See UNIDO publications for more information onthis subject in http://www.unido.org/en/doc/4297.

SESSION I

INTRODUCTION

3

Introduction

The UNIDO Cluster and Network Development Programme

Importance of monitoring and evaluation

Projects implemented to date have adoptedmonitoring and evaluation (M&E) systemsand several have undertaken evaluations,both internal (by the project team and projectmanagers) and external (with external con-sultants recruited by the donors or otherstakeholders). In the light of the differingtypes of information generated by the variousM&E methodologies used, it has been decid-ed that there is a need for the adoption of amore formal, rigorous and standardizedmethodology for the assessment of the per-formance and impact of CND projects. Thedesign and implementation of evaluationinstruments able to provide objective and rel-evant results is an important priority of mostdonors and development agencies. In order

to facilitate this, the Committee of DonorAgencies for Small Enterprise Developmenthas set up a Working Group with the task ofestablishing common M&E guidelines.

The UNIDO CND programme shares this con-cern and, consequently, has included in itsworkplan the development of guidelines to:

• measure the efficiency and effectivenessof the various inputs;

• ascertain the causal links between activi-ties and their outputs;

• evaluate overall project performance; and

• assess project impact, that is, success inthe achievement of the stated develop-ment objective.

Programme of the Expert Group Meeting

The Expert Group Meeting (EGM), held inDelhi from 9 - 14 December 2002, was thefirst step towards the development of theseUNIDO guidelines. The objectives of themeeting were:

1. to exchange information about projectimplementation and M&E methodologiesacross projects;

2. to compare these methodologies and togenerate a common M&E frameworkwith a view to increasing rigor and com-patibility; and

3. to compare experiences, share lessonslearned and promote mutual learningamong project managers.

The meeting brought together five project

teams (from India, Nicaragua, Nigeria,Senegal and Zimbabwe). The workshopprogramme included:

- project presentations

- group work;

- visits to clusters; and

- plenary discussions.

This report describes the outcome of theEGM. The project case studies (based on thepresentation made by each team) are pre-sented in section II. Lessons learned throughthe sharing of experiences are described insection III. The outline of the commonM&E framework that emerged is presentedin section IV.

SESSION I

INTRODUCTION

4

SECTION II

PROJECT CASE STUDIES

SESSION IIPROJECT CASE STUDIES

INDIA

7


INDIA

7

Starting date: January 1997

Donor

Italian Government (approximately US $1million for first project phase and US $1.2million for the second phase).

Swiss Agency for Development andCooperation (approximately US $1.3 mil-lion for the second project phase).

Objectives

• to strengthen the competitiveness ofselected SME clusters by enhancing col-lective efficiency and cooperation (net-working);

• to develop and disseminate a methodolo-gy for cluster development suited toIndian conditions;

• to promote a cluster development move-ment in India; and

• to enhance the contribution of cluster

development to the development objec-tive of poverty-alleviation.

Organizational set-up

National Counterpart: Development Com-missioner, Ministry of Small Scale In-dustries, Government of India.

One Cluster Development Focal Point officein New Delhi (four national officers andadministrative support staff) with fivenational experts recruited as ClusterDevelopment Agents (CDAs) in the fiveclusters currently receiving direct assistance(in seven clusters, projects have been com-pleted). Two more national experts are cur-rently recruited as Technical Advisors (TAs)for the seven Indian organizations currentlyassisted.

A team of international experts.

A Steering Committee with members frompublic institutions currently involved incluster development.

INDIACluster Development Programmes

Project basic facts

Strategy

The Programme seeks to develop sustain-able capacity at both local and national lev-els to promote SME networking and clusterdevelopment through the following strategy:

1. assessment of the competitiveness andinternal organization of SME clusters(including internal strengths and weak-nesses, opportunities and threats present-ed by the global economy, social capitalendowments and support institutions);

2. assistance to actors within the clusters(e.g. suppliers of raw materials &

machinery, buyers, testing laboratoriesand research agencies, industrial associa-tions, training institutions, local govern-ment and financial institutions) in devel-oping a common vision of what they canachieve in national and international mar-kets;

3. building up (through training, workshopsand study tours), the capacity of clusteractors to implement such a vision; and

4. providing advisory services at the policylevel to ensure implementation of cluster

SESSION IIPROJECT CASE STUDIESINDIA

8

development policies by partner institu-tions.2

In the implementation of this strategy, the

Programme actively cooperates with sev-eral public and private SME support insti-tutions in addition to its direct partners.3

Results

The following outcomes have been achievedsince January 1997:

• seven clusters have been assisted on apilot basis under the framework of projectfirst phase, namely Jaipur (textile hand-block printing), Pune (food processing),Tirupur (cotton hosiery) and Ludhiana(knitwear), Ahmedabad (drugs & pharma-ceuticals), Ambur (leather tannery &shoes), and Bangalore (machine tools). Asa result of this project, seven institutions(including export consortia, commonservice centres and SME support institu-tions and associations) have been estab-lished or revitalized. Approximately 1,200firms have benefited from the programmeactivities;

• since July 2002 under the framework ofproject US/IND/01/193, three more clus-ters are being assisted namely Bellary(jeans), Kota (knitwear), and Jallundhar(sports goods);

• a comprehensive data-bank on 350 SMEclusters in India has been made availableto policy-makers through the web-portalof the Ministry of Small-Scale Industries.A list of 1,657 artisanal clusters has alsobeen drawn up;

• more than 600 policy makers, develop-ment agents and academics have beensensitised to the cluster developmentapproach through three national work-shops (Delhi in 1996, Mysore in 1998 andSurajkund in 1999), four state-level work-shops, training modules and various otherseminars;

• seven state governments4 have adoptedthe cluster development model within theframework of their industrial policies andhave started pilot cluster interventionswith UNIDO support;

• an international Joint Learning Workshopfor CND practitioners was organized in

New Delhi in December 2002. Severalstudy tours have been organized to assist-ed SME clusters for practitioners fromPakistan, Thailand, and Bangladesh; and

• methodologies, tools and training mod-ules have been prepared to assist the gov-ernment, public institutions and associa-tions to implement cluster developmentinitiatives. A 12-week programme to pre-pare cluster development agents (CDA)was developed in collaboration with theEntrepreneurship Development Instituteof India (EDII), Ahmedabad and run inthree separate editions. A manual fortrainees and another for trainers were pre-pared and distributed in the three CDAtraining programmes run under the frame-work of the Programme. Over 80 CDAshave been trained to date, the great major-ity of whom are presently operating ascluster development agents in one or moreIndian clusters.

2 such as: Development Commissioner (Ministry ofSmall Scale Industries), State Bank of India (SBI),Textile Committee (Ministry of Textiles), Governmentof Madhya Pradesh, Government of Andhra Pradesh,Government of Gujarat.

3 These include the Small Industries DevelopmentBank of India (SIDBI), National Small IndustriesCorporation Ltd., National Bank for Agricultural andRural Development, Khadi & Village IndustriesCommission, Federation of Indian Chambers ofCommerce and Industry, Confederation of IndianIndustry.

4 Andhra Pradesh, Madhya Pradesh, Karnataka,Rajasthan, Tamil Nadu, Gujarat, and Chattisgarh.


INDIA

9

To achieve the above results, the ClusterFocal Point received the following inputs:

• exposure to methodology on networkdevelopment, building upon the key find-ings from other UNIDO projects in thefield of SME networking, especially fromLatin America;

• exposure to best practices in the field ofcluster development: research studies onthe emergence of cooperative behaviourin Italian SME clusters, and on servicecentres and consortia; study tours toItalian SME clusters; guidelines on clusterdiagnostic methodology; and

• exchange of experience with other practi-tioners in the field of SME cluster/net-work development through a dedicatedUNIDO website and “Joint LearningWorkshops”.

While it is just about impossible to single outall the elements that lie at the root of the suc-cess of the programme, the following pointsstand out as particularly relevant (though notnecessarily in the order presented):

• a highly motivated and internally verycohesive implementation team;

• India is a large country where SME clus-ters are historically well-known andwhere SMEs can count on significant(though not always well-coordinated) sup-port resources and political will;

• a supportive official counterpart, whichgradually developed its own vision ofSME cluster development;

• a strong emphasis on awareness-raisingfor cluster stakeholders;

• an implementation strategy that carefullybalances long-term objectives (trust-building, institutional networking, localgovernance) and short-term objectives(visible impact at the enterprise level,conflict management); and

• continuous investment in identificationand dissemination of information on bestpractice through a variety of media (inter-net, papers, presentations in internationalconferences, video, etc.).

Inputs and success factors

Lessons learned

1. The process of cluster developmentrequires an adequate amount of time forinternal dynamics within the cluster tochange (3 to 5 years).

2. At the level of enterprise development, akey task for the CDA is to identify theappropriate vehicles for cooperation:loosely-coordinated, ad-hoc, self-helpgroups; single-initiative networks; jointventures; consortia; and associations.

3. The dissemination of the cluster develop-ment model to official counterparts (e.g.state governments, national SME supportagencies) has emerged as the only credi-ble strategy for widespread adoption ofcluster development in a country the sizeof India.

4. This dissemination strategy requires athree-pronged approach aimed at sensitis-

ing top policy- makers, transferring com-petence to the apex implementation level,and ensuring empowerment and flexibili-ty at the grass-roots. Synchronization ofall three levels is of paramount impor-tance.

5. The core competences of cluster devel-opment practitioners are in the fields ofcluster diagnostics, support in the for-mulation of cluster action plans, trust-building, institutional networking andimpact assessment. It has also provedhelpful to have fully sensitised focalpoints within the programme manage-ment office to act both as providers offirst-hand information (required to con-vince local stake-holders) and as refer-ees (to be able to mobilise existingfunding schemes and technical assis-tance).

SESSION IIPROJECT CASE STUDIESINDIA

10

6. In relatively large programmes whereseveral objectives are tackled simultane-ously and where experts are dispersedover large distances, it is of paramountimportance that the team of experts: i)ensures proper dissemination of informa-tion; ii) continuously elaborates and

updates information on best practice; andiii) identifies and disseminates shared pri-orities and guidelines. Frequent face-to-face meetings, supplemented by internet-based circulars and visits by seniorexperts, have proved the only sustainable(albeit expensive) solution here.

Work Ahead

• There is a continuous need:

i) to integrate into the methodology andtraining modules lessons learned inproject implementation; and

ii) to ensure that these lessons are employedat the field level in future projects.

• Evidence on the value of SME clusters inIndia should be further disseminated toensure the build-up of a positive momen-tum behind the cluster developmentmodel. This is likely to require moreinvestment in the development of theexisting cluster database.

• The contribution of the cluster develop-ment model to poverty-alleviation must begauged in practice through pilot-levelcluster development initiatives.

• A greater number of official counterpartsshould be drawn into cluster developmentto promote an enhanced level of long-termcommitment and motivation at the toppolicy level.

• The training capacity of senior cluster

experts should be strengthened to meet themounting demand for cluster developmentpractitioners, both in India and in theSouth Asia region. This will require train-ing modules that are less specifically“Indian” in nature.

• In view of the broad liberalisationprocess currently underway in India, itmay be of strategic value to identify offi-cial partners for implementation at thefield level (including NGOs, charitiesand large-scale commercial entities).Where such parnerships prove successful,an appropriate implementation strategyshould also be agreed upon.

• The capacity of the programme to pro-vide an adequate assessment of value-chain dynamics in some of the more suc-cessful Indian clusters remains limited.Affordable, appropriately-skilled consult-ants in this very innovative field appearhard to find. The whole area of clusterbenchmarking requires further explo-ration.


NICARAGUA

11

Starting Date: 1995, now in its third phase ofimplementation: May 2002-November 2004

Donor: Austrian Government

Objectives

The project’s aim is to contribute to thecountry’s equitable development by increas-ing the competitiveness of SMEs, throughthe following:

• foster a shared vision of enterprise devel-opment in the regions of Masaya-Mesetade Los Pueblos and Boaco-Chontales andthe coordination of the activities of localactors with a view to establishing com-petitive, integrated, local production sys-tem;

• promote horizontal and vertical enter-prise-to-enterprise links in the above-mentioned localities and improve access

to business development services(BDS);

• help strengthen inter-institutional coordi-nation at the national level in the imple-mentation of SME development policiesand programmes.

Organisational set-up

National counterpart: Nicaraguan Institutefor the Promotion of Small and Medium-sized Enterprises (INPYME).

Project team: seven Nicaraguan profession-als of diverse backgrounds, located in thetwo selected regions with a central NationalCoordination office in the capital, Managua,supported by 2 Peace Corps volunteers, plusa team of part-time international advisors.

A Steering Committee with members fromprivate and public sector.

Project basic facts

Strategy

Nicaragua is the second poorest country inLatin America and it ranks 5th in the world interms of international aid received.5 Some93% of its enterprises are classified asmicro, small or medium-scale.6

At the outset of the project, the local busi-ness environment was characterized by veryweak institutions, low levels of trust and col-laboration between institutions and enter-prises and a generally passive attitude on thepart of enterprises. This latter attitude wasexacerbated by the common practice amongaid agencies generally to provide grantsrather than promote market-led service pro-vision. In this environment, project strategyfocused, during its initial phase (1995-1997), on: i) a concentrated effort to stimu-

late self-help activities among the enterpris-es in order to promote entrepreneurship andleadership; and ii) joint initiatives amongSMEs (also called horizontal networks) tofacilitate the development of economies ofscale, better products and increased efficien-cy in the organization of production.

During the second phase (1998-2002), inorder to increase impact and promote sus-tainability, the project started training local

5 The Economist, Pocket World in Figures, 2002Edition.

6 MIFIC (Ministerio de Fomento, Industria y Co-mercio).

NICARAGUADevelopment of local production systems, based on SMEs

SESSION IIPROJECT CASE STUDIESNICARAGUA

12

professionals (mainly belonging to localinstitutions) as network brokers and promot-ing the development of vertical networks.7

The local dimension also assumed increas-ing importance during this phase because ofthe need for greater coordination with otherlocal business actors (suppliers of inputs andservices, business associations) and institu-tions (local authorities, service providers,schools, etc.).

This natural evolution towards increasinglycomplex integration at the local level isbeing fostered and consolidated during thethird phase (2002-2004). The aim is to sup-port a truly cluster-based production system,with sustainable relations developing amongthe SMEs themselves and between theSMEs and local institutions. It is anticipat-

ed that such linkages will develop at the fol-lowing levels:

• within integrated networks;

• between BDS providers and brokers;

• between large, medium and small-scaleenterprises;

• between the clusters and local govern-ments; and

• between the local systems and the nation-al support institutions .

7 Vertical networks or supply networks refers tothe promotion of closer commercial relationshipsbetween larger enterprises (industrial buyers) andSME (suppliers).

Results

In terms of promoting self-help efforts, col-laboration and social capital development atthe regional and national levels:

• the project has succeeded in being a cata-lyst for strategic endeavors at all levels(firm, business association, municipal,national);

• the project has succeeded in promotingproject initiatives within all the differentgovernment institutions concerned withsmall enterprise development, in terms ofboth technical interventions and policyformulation, at both the local and nationallevels; and

• increasing the participation of all stake-holders in the design of developmentactivities through efforts to build trust andpromote cooperation.

At the level of capacity-building and pro-moting sustainability:

• new private sector alliances and leader-ship have been stimulated at the locallevel, resulting in the development ofstrategies based on the real needs of localbusinesses: 30 new business networks(incorporating a total of 480 enterprises)have been created and other membershipinstitutions fostered, including, for exam-ple, an association in the dairy sector that

draws its members from various parts ofthe Central American region;

• the achievement of widespread awarenessof networking models and the develop-ment of three specific methodologies andcomprehensive training programmes.These cover methodologies for the pro-motion of both horizontal and vertical net-works and of local-level development.Nine institutions in the country are cur-rently promoting networking/clusteringstrategies, with more than 30 horizontalnetwork brokers. A project has been start-ed to transfer the capacity to provide train-ing in these methodologies to a local uni-versity;

• further training of network brokers hasbeen carried out in Guatemala, ElSalvador and Senegal in response torequests from these countries; and

• cluster development is now under consid-eration as a potential component of thenational development strategy and theproject is actively working with govern-ment authorities in the design of this strat-egy and its implementation.

In the area of service-provision based onmarket development:

• the project has systematically promoted


NICARAGUA

13

the payment of fees for services by enter-prises and institutions: US $30,000 hasbeen generated in sales of training andconsulting services by institutions imple-menting networking strategies and enter-prises in vertical networks.

In terms of enhancing the capacity of partic-ipating enterprises:

• joint initiatives have been launched toenable participating enterprises toenhance their access to new markets, toreduce their production and marketing

costs, to modernize their production tech-niques, to introduce environmentally-friendly production processes and to havegreater influence in national and regionalpolicy formulation. Other achievementsinclude:➢ credit of more than US $ 100,000obtained by enterprises to implement jointactions ; and➢ more than US $300,000 of newinvestment performed by participatingenterprises.

What made the difference?

1. A project team with diverse professionalbackgrounds sharing a common vision ofSME development and capable ofcatalysing resources and actions by localactors.

2. Strong investment in training and moti-vating the project team.

3. Technical assistance principles, tools andmethodologies which makes implementa-tion smoother and more efficient and haspermitted the transfer of know-how to

other institutions to promote sustainabili-ty.

4. The promotion of a culture of self-helpamong client entrepreneurs.

5. The establishment of trust between thevarious stakeholders as the base for jointinitiatives and collaboration betweenactors as a way to achieve concrete eco-nomic gains and not just as a lobbyingmechanism.

Lessons learned and future challenges

• The promotion of clusters and networks isbased on facilitating self-help initiativesin which the principal players are privateenterprises and the organizations that sup-port them. Project teams should not try toreplace or substitute for these actors. Theculture of cooperation must develop at itsown pace within a participative process inwhich the actors themselves recognize theneed for change.

• The degree of decentralization, empower-ment and capacity within local govern-ment is a crucial element in cluster devel-opment. A highly centralized governmentstructure is likely to inhibit the develop-ment of locally-based alliances and ismore likely to require the establishment of

alliances with the centralized institutionsif local strategies are to be successfullyimplemented.

• Implementation of the cluster develop-ment process needs to take into accountthe particular characteristics of each geo-graphic area as the basis for the activitiesto be undertaken. Even in clusters withsimilar characteristics, there is no uniquepath to development. The main factorunderlying the selection of strategy is thedetermination and vision of local actors.

• For a market-led culture in service-provi-sion to emerge, it is essential that coordi-nation mechanisms with other donors bedeveloped to ensure consistency ofapproach.


SENEGAL

15

Starting Date: October 2001


Budget: US $706,250

Objectives:• Organization of business networks and

implementation of joint business initiatives;

• capacity-building of professional associa-tions and development of their services;and

• promotion of the interests of SMEs withinthe framework of the current policyreforms.

Organizational set-up:

Counterpart:

Ministry of Industry and Handicraft andSODIDA (Société de gestion du domaineindustriel de DAKAR, a semi-public compa-ny, operating as the national platform forSME support projects).

Local team:

- 1 project manager,

- 1 business advisor and support staff(located at SODIDA),

- 3 network brokers located in Dakar, Thièsand Saint Louis.

A part-time international advisor.

Steering committee, including representa-tives of public and private sectors.

SENEGALSupport to the small firms of Senegal

Project Basic Facts

Strategy

The project targets small enterprises (about3-30 employees) in 5 sectors: metal-mechanics, agro-industry, wood products,leather products and garments.

The project strategy aims at strengtheningcooperation between stakeholders and socialcapital formation within small enterprises,following a bottom-up methodology andoperating at three levels (as describedbelow).

Results envisaged are as follows:

1. At the micro-economic level,

➢ 20 new business networks, incorpo-rating more than 120 businesses.

➢ Implementation of the strategic proj-ects by these networks, resulting in thelaunch of more than 25 cooperative initia-tives between enterprises.

➢ The training of specialist network

brokers to promote the dissemination ofthe model.

2. At the meso-economic level,

➢ BDS provision, co-financed by theproject, to develop eight professionalassociations (including the training ofassociation leaders) and the design of sixnew services offered to the members ofthese organizations.

3. At the policy level,

➢ Study into the specific concerns ofsmall enterprises and the organization ofevents (conferences, seminars, thematicworkshops), supported by expertise andan information campaign. The objectiveis to increase the capacity of small firmsto dialogue with the state and to promotereforms of the business environment tomake it more conducive to SME develop-ment.

After one year of project activity, 16 busi-ness networks encompassing the three geo-graphic locations and the five designatedsectors, have been established and havereceived training. Demand for assistanceremains strong and 15 further groups areready to enter into the same process with theproject.

Seven of the networks to have receivedassistance to date are now entirely self-man-aging; five are undertaking their pilot projectactivity (three in common purchasing, oneundertaking joint promotional activities, oneupgrading production); and four have under-taken feasibility studies of their strategicprogrammes with the assistance of externalconsultants co-financed by the project.Moreover, the project has attracted the atten-tion of several organizations interested in thepossibility of providing financial support tothe initiatives.

The methodological tools for assisting thebusiness networks have been systematizedand are continually improved on the basis of

best practices learned. This serves not onlyto improve the effectiveness of the project,but also to promote replication and transferof the model to other local SME supportorganizations

Three associations (in the garments, me-chanics and fruit and vegetable sectors) arereceiving support from the project in develop-ing their strategic plans, with support fromexternal consultants. These plans willunderlie the activities, partnerships andorganizational relationships that willemerge under the project. These associa-tions have a national coverage, but theirstrategies are defined on a regional basis, ina manner very congruent with the clusterapproach.

Activities relating to the creation of a dia-logue between small enterprises have not yetbeen implemented. However, the project hasalready negotiated partnerships with a rangeof national-level bodies prepared to providefinancial or non-financial assistance to smallenterprises.

SESSION IIPROJECT CASE STUDIESSENEGAL

16

Results

Success factors

Good local knowledge on the part of theproject team permitted the rapid identifica-tion of potential candidate networks throughdirect contacts with: the enterprises, localleaders, associations and already-existinggroups.

The project team benefited from the outsetfrom a transfer of knowledge from the proj-ect in Nicaragua on: facilitating businessnetworks, reinforced by training of brokersin managing meetings and resolving con-flict; strategic planning; individual and col-lective diagnostics; the design of businessplans, leadership, etc.

This transfer permitted a process of learn-ing by doing on the ground, which wasespecially useful for project officers who

already had solid experience in facilitationof group dynamics.

This participatory approach ensures stronginvolvement on the part of trained enter-prises. This enables them to let go of thedependency mentality created by some pre-vious aid projects and to become more self-sufficient and commercial in their outlook,thus facilitating cooperation among them.

A real willingness exists among the clientsmall enterprises to organize themselves andcooperate: for example, there already existsin four out of the five designated sectors aregional or national producer association.However, none of these organizations had awell-elaborated strategic plan, thus theirinterest in support from the project.


SENEGAL

17

Project experience suggests that an impor-tant part of the group-building process forbusiness networks is the implementation of apilot project, using just their own resources,without project support.

The project came to understand that itseffectiveness is in large part dependent onthe rigour with which it selects groups andnetworks to support.

This selection needs to be made using pre-cise criteria.

Monitoring tools (to measure, for example,increasing maturity of networks) are seen as

indispensable, especially for structuringactivities whose results are relatively quali-tative and intangible in nature.

In numerous cases, active brokering seemsto be an excellent way of breathing life backinto networks or associations that havebecome discouraged by previous set-backsand failures.

Within the overall logic of the project (build-ing from the bottom up the social capital ofsmall enterprise business networks), itmakes sense to support business networksexisting within associations.

Lessons learned

Next steps

• Training of network brokers in other smallenterprise support organizations, using thetools and methods developed by the project.

• Encouraging public authorities to integratethe cluster development approach intotheir small enterprise support programmes.

• Action needs to be taken to implementcomponent 3 of the project: the creation ofa dialogue between the small enterprisesthrough which they can voice their con-cerns and needs in a concrete fashion tonational policy-makers.


ZIMBABWE

19

Starting Date: September 2001 (Phase II)


Total: U$594,000; Phase I: US $ 91,248(Preparatory Phase); Phase II – US $502,752

Objectives:

• enhance the operating environment ofSMEs, promoting changes in policies andregulations;

• increase competitiveness of SMEs by cre-ating and strengthening viable SMEs net-works; and

• strengthen the capacity of SMEs and net-works through accessible, appropriate andcomplementary training, advisory andinformation services.

Organizational set-up

Counterpart: Ministry of Industry andInternational Trade of Zimbabwe

A National Project Coordinator and a techni-cal team organized by the Organisation forSocio-Economic Research and ConsultancyServices (OSERCS) together with theInstitute of Development Studies (IDS) of theUniversity of Zimbabwe. The technical teamincludes 1 Team Leader, 1 Senior TechnicalAdviser and 6 network brokers (one with abusiness background, the other with an engi-neering background) operating in the 3 proj-ect locations: Harare, Bulawayo and Mutare.

A part-time international advisor.

A project Steering Committee with membersfrom private and public sectors.

ZIMBABWEDevelopment of the Small-scale Industry Sector

Through Clustering and Networking

Project basic facts

Strategy

SME networks are being organized by theproject for the network members to exploitopportunities and address common problemsfor mutual benefit. The project operates inthe urban centres of Harare, Bulawayo andMutare where there is a broad base of SMEs.It focuses on three sub-sectors: metal fabri-cation, woodworking and garments. Theproject provides access to training and advi-sory services to strengthen the internal oper-ations of the networks. These services areprovided by the BDS project partners.

The preparatory phase demonstrated thatwhile there already existed support institu-tions with varying capacities to provide serv-ices for SMEs, their needs were not beingsufficiently and effectively addressed. The

project stimulates demand for BDS servicesamong the SMEs through the networks; andenhances the capacity of BDS providers toaddress identified needs in an effective andsustainable manner. This contributes to thedevelopment of a BDS market.

As SME networks are created and/orstrengthened, SME sectoral associations willfind it easier to expand linkages, enhancetheir collective efficiency and become moreeffective in their advocacy activities.

In order to promote a more conducive oper-ating environment for SMEs, a study isbeing initiated to identify direct regulatoryconstraints as well as laws that could bechanged or implemented in a more fa-

SESSION IIPROJECT CASE STUDIESZIMBABWE

20

vourable way for SMEs. The capacity oflocal authorities to make the required

changes and to monitor their implementationwill be strengthened.

Results

1. As of 30 December 2002, 16 networkshave been formed and all have adoptedarticles of association. A further 12 net-works are in the process of being organ-ized. All 16 of the existing networks havereceived training in team-building andawareness of strategic issues relating totheir businesses. The brokers are current-ly helping the networks to identify com-mon projects. One network has receivedfunding from a donor agency to purchaseshared equipment.

2. The project is currently working with 18BDS providers which have receivedcapacity-building from the project in pro-viding training in the creation andstrengthening of networks, quality man-agement and the provision of industrialextension services.

3. Four BDS providers are using the trainingmaterials on group-strengthening for theirown client groups. Two of these are alsousing the training materials in team-build-ing training programmes for the workersand staff of medium-scale enterprises.

4. Six BDS providers are considering offer-ing quality management training to thenetworks. Following on from the trainingprogrammes conducted by the project, theBDS providers are now developing tailor-made business and technical training pro-grammes for the SMEs (a change fromtheir normal practice of offering pre-packaged training programmes).

5. The networks and entrepreneurs havestarted paying for the services of BDSproviders. The Murahwa Green MarketNetwork of metal-fabricating entrepre-neurs, for example, paid ZWD 5,000 to aBDS provider (Women In Business) forthe preparation of a project proposal tosolicit funds from the Ministry of Youthfor a raw material procurement project.Women in Business is using the guide-lines provided by the project to developthis proposal. In Bulawayo, members oftwo associations have paid for an S/IYBtraining course for two BDS providers.Fees paid ranged from ZWD 1,500 toZIWD 6,900 per trainee.

Project Inputs

The network brokers and the BDS partnerswere given the following training:

• Training of Trainers – Presentation andTraining Skills – 1 week - Topics: train-ing defined, training cycle, training needsanalysis, setting training objectives;designing a training course, adult learn-ing principles, adult training techniquesand practice teaching.

• Training on organizing and strengtheningnetworks – 1 week - Topics: definition ofnetworks and clusters; phases of networkdevelopment; group strengthening –vision/mission formulation, effective com-munication, conflict-resolution, problem-solving techniques, effective group leader-ship, and how to handle group mee-

tings; and group project development;

• Quality management and industrial exten-sion – 3 weeks - Topics: industrial exten-sion processes, total quality management,quality management process, problem-solving cycle, error proofing, flow chart,process flow chart, routing diagram,machine utilization, technology audit, 5sand plant design, production planning andmaterials, work-force organization, leanproduction, maintenance, competitive mar-keting, Pareto analysis, estimating workingcapital requirements, integrated cashbook,product costing and financial analysis. Thetraining programme was complementedwith in-plant study for the participants topractice the tools learned during the course.


ZIMBABWE

21

An international expert conducted the firsttwo training programmes. Two internation-al experts and a national consultant conduct-ed the last training programme.

BDS partners selected to attend the abovetraining programmes included non-govern-ment organizations, private consultants andtechnical training institutions.

Lessons learned

• For a project of short duration, it is betterto work with already existing networks orassociations of entrepreneurs rather thanstarting with a fresh group. Organizationalinputs will take less time and the pace ofdevelopment will be faster.

• For greater effectiveness of project staff, itis better to organize/strengthen networksthat are physically contiguous to each other.

Thus, an area where there is a concentrationof entrepreneurs should be targeted first.

• Entrepreneurs are willing to pay for aservice that they believe they need.

• If the project is seeking to develop BDSmarkets, there should be a clear demarca-tion between where the work of the net-work broker stops and where the BDSproviders should come in.

Work Ahead

• There is a need to transfer capacity from theproject to BDS providers to enable themto deliver a package of services and train-ing to the networks and individual SMEs.

• As a way of helping to consolidate thenetworks, they should be helped to plantheir group projects and to follow-throughon vertical network promotion activities.

SECTION III

LEARNING FROM PROJECTS

SESSION III


25

1. The main scope of CND projects is theestablishment of a new culture of cooper-ation both among enterprises and betweenenterprises and other actors in their insti-tutional environment. The starting pointis to help enterprises realize the value ofcooperation by facilitating joint-actionprojects, whose success can be used tofurther promote a philosophy of coopera-tion. The challenge is to ensure that thisapproach contributes to the developmentgoal of increasing SME competitivenessand alleviating poverty.

2. CND is a long-term process. Due to itscharacteristics of promoting participation,empowerment, trust and collective action,the cluster development process needstime to take off and reap all potential ben-efits. Those benefits, on the other hand,can be far-reaching and long-term. Inother words, CND programmes must be

planned and implemented with a long-term vision and not just for quick returns.Short-term activities to generate quickresults can be a good way to stimulateinterest and participation but should notbecome the main scope of CND pro-grammes.

3. The success of UNIDO’s efforts in thisfield should be measured in terms of

• the number of entrepreneurs enteringinto cooperative efforts with their fellowentrepreneurs;

• the impact of this enhanced cooperationon their businesses and the contributionthis makes to the local economy;

• the positive changes in the institutionaland business environment; and

• in terms of the changes that it helpsbring about (an impact that is likely to beespecially long-term in nature).

Learning from projects

Considerations on overall experience of the UNIDO CND projects

Methodological issues

1. As vehicles for cooperation, there is ageneral inclination for cluster develop-ment projects to create “hard” networks(that is, formalized networks). Experienceshows that this type of network offers aneffective learning opportunity in the sensethat, even if they do not survive in thelong term, participating entrepreneursacquire a more favourable attitudetowards cooperation and will be able torecreate networks in other circumstances.

In some cases, however, enterprises mayneed to participate only in “soft” net-works – where alliances are of limitedduration and focused on achieving limit-

ed, time-bound objectives. In fact, loose-ly-coordinated, ad-hoc, self-help groups,single-initiative networks, joint ventures,consortia and associations, are all differ-ent alternatives vehicles for cooperation(each with different needs and capabili-ties), which should be chosen dependingon: i) the task at hand; ii) the investmentcapacity of the potential participants; iii)the extent of competition among thepotential participants; and iv) the expect-ed rate of return from the joint ventures.

2. CND approaches should be seen not asbeing distinct, but as complementary andoccasionally sequential. In some projects

the cluster aspect prevails and the promo-tion of the networks is pursued as part ofthe cluster development strategy. In otherprojects, activities start with the develop-ment of horizontal or vertical networksand then, as networking intensifies, theterritorial or cluster dimension starts toemerge with the involvement of BDS,providers, association, governmentalinstitutions.

3. The role of the cluster development agent(CDA, also referred to as cluster broker)has to be clearly defined in relation tothat of the BDS provider. This is partic-ularly true where the supply of BDS isweak or non-existent and has to be stim-ulated or created. The CDA always needsto try to find an appropriate balancebetween “hand-holding”, in order toensure that projects advance at a reason-able pace, and depending on local serviceproviders, with the possible consequenceof slower progress but greater local own-ership and, ultimately, sustainability ofthe process.

4. The cluster development approach hasrepeatedly proved an effective comple-ment to the efforts of other SME supportinitiatives, (including BDS development,credit support, private-public partner-ships for infrastructure, vendor develop-ment, etc.), enabling them to increasetheir impact and sustainability. Theapproach can also contribute, at least inprinciple, to a whole range of socialdevelopment measures (includingempowerment of women and scheduledcastes, environmental protection andimproving the responsiveness of localgovernment).

5. The length of intervention of cluster ornetwork development programmes mustbe sufficient for internal dynamics withinthe cluster to change. As a general rule,this will not be possible in less than threeyears. While exit strategies need to bedeveloped, the right time to pull out mustbe carefully anticipated. This is particu-larly true in artisanal clusters or in thoselocated in remote areas. In such cases, thetime required both for the necessarychanges in ways of thinking and operat-

ing to take place and for the emergence ofa sustainable governance framework forthe cluster tends to be longer.

6. The specific objectives and operatingmechanisms of business associationsworking in the context of networks andclusters need to be defined. A methodol-ogy for developing business associationsas cluster leaders needs to be established.

7. There is a need to promote vertical net-working within clusters, using specifictools that foster interactions between dif-ferent actors in the value chain. Suchtools could include: promoting specializa-tion within a cluster; facilitating thedevelopment of subcontracting relation-ships; encouraging enterprises within thecluster to fill existing gaps in the valuechain; creating new supply linkages; andproviding information on best practice inthe areas of cluster-based purchasing,subcontracting and marketing.

8. Monitoring and evaluation tools must bedeveloped that are both sufficiently rigor-ous and comprehensive to be useful interms of the information they provide;and sufficiently simple to be easily imple-mented on the ground. There also needsto be a clear understanding of the need togather data not just on activities and out-puts, but also on project impact.

A commonly accepted framework formonitoring the performance and impactof networks and cluster development ini-tiatives is needed. This will permit a bet-ter assessment of the outcomes of supportprojects; easier comparisons betweenthem; and a more transparent presentationto donors, counterparts and beneficiariesfrom the outset.

This performance and impact measure-ment framework should allow us to: (i)assess changes in the level of institutionalnetworking and cooperation; (ii) measureeconomic impact (income and employ-ment gains, etc.); iii) measure impact onpoverty alleviation; iv) assess progress inproject implementation; and (v) monitorchanges over time in terms of the emer-gence of governance structures of thecluster.

SESSION III


26

SESSION III


27

There is a need to monitor the dissemina-tion of information, new skills and work-ing methods to project partners and tomeasure how much of this is retained over

time. Such information needs to berecorded at three levels: among officialpolicy-makers, intermediate organiza-tions and enterprises within the cluster.

Similarities among projects

1. The degree of similarity between the proj-ects described above is remarkably high,in spite of the very different social andcultural environments in which they arebeing implemented: all of them are pro-moting increased cooperation amongenterprises as a way of improving theirindividual businesses through cluster- andnetwork-oriented approaches.

2. The scope for mutual learning amongprojects appears to be largely untap-ped, largely due to a lack of commonchannels of communication. It mightbe useful to experiment with an inter-net-based communication-channel,though the EGM has made it clear thatface-to-face interactions are also ofgreat value.

Differences among projects

1. Differences in approaches are mainly theresult of the differing conditions prevail-ing in each country; the varying levels ofdevelopment of the different clusters; theextent to which there exist other projectswhich complement UNIDO’s efforts; andthe difference in project inputs in eachcountry.

2. The size of enterprises targeted by the dif-ferent projects varies, depending onwhether they are urban- or rural-basedand on the impact of national develop-ment strategies on the original projectdesign.

3. The scope of interventions varies betweenprojects according to the size of theirbudgets. While all acknowledge theimportance of a healthy BDS market to acluster’s performance, for example, notall are active on this level. Similarly,some but not all are active in creatingand/or strengthening business associa-tions, supporting local economic develop-ment commissions; and helping to estab-lish mechanisms through which the smallenterprise sector can influence govern-ment policy.

4. The level of cooperation with and supportto BDS providers also varies between

projects. There is, similarly, significantvariation in terms of how much smallclient enterprises pay for BDS services.This seems to depend on the availabilityof BDS providers, their capacity and will-ingness to work with the networks/clus-ters; the extent to which targeted enter-prises use their services; and also differ-ences in donors’ practices.

5. One other interesting variation betweenprojects is that while some have foundbusiness associations to be a useful start-ing point for the subsequent creation ofsmaller networks, others have first fos-tered the creation of networks that havelater joined together to form businessassociations.

6. Despite the conceptual distinction madebetween a “network” approach and a“cluster” approach, it became clear duringthe course of the meeting that these, infact, appear to be two different stages inthe sequence of project support: all of theprojects began by promoting joint actionbetween firms (network development)before then turning to facilite the devel-opment of local economic developmentstrategies by groups comprised of bothprivate and public sector actors (clusterdevelopment).

SESSION III


28

1. The projects have demonstrated thatenterprises and other actors in theirvalue chain can cooperate, and thatwhen they do, they can improve the effi-ciency of their enterprises and of theirnetworks/clusters. In this self-helpprocess, when the networks or clustersprove successful, it is the entrepreneursthemselves who take the credit, with therole of the project being limited to thatof facilitator.

2. An important challenge facing theapproach is to identify and groom keyleaders within the networks and clusterswho will keep the facilitation processgoing, since this will be needed when theCDA or the network broker leaves.

3. Training courses for brokers need to besupported by user-friendly manuals andregular hand-holding.

4. Ensuring the sustainability of the ap-proach also presents a significant chal-lenge, given the limited willingness on the

part of the cluster actors to pay for CNDservices. Capacity-building work withlocal institutions and business associationsneeds to be geared towards addressing thisissue. Similarly, there is likely to be aneed to encourage governments to becomeinvolved in helping to fund this “partlypublic” function of cluster brokering.That is, it is clear that part of the rationalefor a cluster intervention is that the bene-fits introduced by the project will rippleout beyond direct project clients – as clus-ter dynamics start to function; as network-ing models are replicated, as the capacitydeveloped by participating enterprises andassociations is used to undertake other ini-tiatives beyond the project’s objectives;and as advances in certain strategicallyimportant sectors, light engineering forexample, ripple through in the form ofbenefits to enterprises in other sectors.The ‘public good’ nature of these benefitsmerits careful government attention inspending decisions.

Successes and challenges of the approach

Future of the approach

The EGM arrived at the following recom-mendations for future work:

1. While it is important to encourage large-and medium-scale entrepreneurs to partic-ipate in cluster development initiatives,we need always to bear in mind that thetarget of our assistance is SMEs includingmicro enterprises.

2. Where possible, it is preferable to workinitially with existing networks, in order toshorten the time needed for team-buildingand the creation of trust.

3. Projects should refrain from introducingformal systems into networks that want tokeep it “soft”.

4. It is good to introduce complementarytypes of BDS interventions that will sup-port the CND approach.

5. Adequate time must be devoted to aware-ness-building and promoting the basicconcepts of CND, remembering that theprocess is generally slower and costlierthan anticipated. Rushing to get short-term results ultimately tends to lead tofailure.

6. Enough time needs to be allocated to theproject (generally no less than threeyears) for the culture of cooperation totake hold.

7. Monitoring and evaluation systems shouldbe kept as simple and as well-focused onspecific indicators as possible. It mayoften be preferable to devise systems thatare easy for field staff to use than to aimfor perfectly designed comprehensive datasystems.

SECTION IV

MONITORING AND EVALUATION ISSUES

SESSION IVMONOTORING AND

EVALUATION ISSUES

31

This section is addressed primarily to proj-ect staff, especially those involved in thedesign and implementation of monitoringand evaluation systems. The aim is to pro-

vide an introduction to the key conceptsand methodologies that are likely to be rel-evant to measuring progress in CND initia-tives.

Introduction

What is M&E?

Both monitoring and evaluation (M&E) areinvolved with the gathering of data to mea-sure the work undertaken by a project and tocompare this with the targets set out in theproject document. Monitoring is geared to-wards recording project activities on an on-going basis during the life of the project.Evaluation takes place both during the life ofthe project and retrospectively, at the end of

a project or project phase. It has to do withappraising both the efficiency of the project(its success in achieving the targets set with-in the agreed budget); but also its effective-ness (that is, the degree to which projectactivities have led to the anticipatedimprovements in key impact indicators –income, employment, empowerment, etc.We will return to this below).

Why do M&E? : different needs of different stakeholders

There is no one reason for undertakingM&E activities. Different stakeholders in-volved in promoting CND developmenthave different interests, which in turn deter-mine the type and scope of M&E informa-tion that are relevant to them. The M&Eneeds of the key stakeholders are describedbelow:

For CND project managers, M&E is amanagement tool whose primary functionsare: i) to keep track of whether the variousproject activities being implemented are onschedule and in line with the budget; ii) toanalyse the degree to which these activitiesare translating into the anticipated outputs(are vertical and/or horizontal networksdeveloping in the way that had been hoped?are skills and market access improving in theways that had been anticipated? etc.); andiii) to measure the effect these outputs have

on the project’s key impact indicators – lev-els of income, employment, empowerment,etc. M&E systems need to deliver informa-tion at each of these three levels to enableproject managers to maximise impact, cost-effectiveness and sustainability for their ini-tiatives.

Private sector BDS providers are likely toundertake M&E to monitor customer satis-faction, respond to changes in demand,develop new and better products, managecosts, and establish staff incentives.

Donors need M&E information to ensureaccountability in the use of their funds andto decide between different types ofapproach and project in their funding deci-sions. Donors often focus on broader socialand economic objectives of employment,enterprise competitiveness, and poverty alle-viation.

Monitoring and Evaluation Issues

Within the context of the logical framework(log-frame) planning tool, these four ques-tions relate to:

1. development objective and immediateobjective;

2. outputs and activities;

3. indicators; and

4. sources of verification.

So, using the vocabulary of the log-frame,M&E can be described very simply as theprocess of measuring project achievementsagainst the various targets set for each indi-cator at the four levels of the log-frame(activities, outputs, immediate objective anddevelopment objective).8

The three core problems faced by most SMEprojects are that:

1. they tend to gather much data on activitiesand outputs, but very little on the project’simmediate and development objectives;

2. their collection of financial data on bothcosts and benefits is rarely sufficient toenable them to undertake rigorous andauthoritative benefit-cost analyses; and

3. their indicators for sustainability areinsufficiently clear to serve as a usefulmanagement tool.

Thus, the case studies described above, forexample, are rich in terms of details of man-uals produced, training courses provided,exchange visits undertaken, etc. (activities);and of clusters strengthened, policy-makerssensitised and producer associationsempowered (outputs). This can be describedas project performance. However, theyhave little to say about increases in employ-ment or income, etc. (immediate objective);or about poverty alleviation or other higherorder development objectives. This is proj-ect impact.

As noted at the beginning of this paper, theCND approach is based on the belief (amongothers) that clustering and networkingamong enterprises promotes enterprise com-petitiveness. But can this belief be assumed

SESSION IVMONOTORING ANDEVALUATION ISSUES

32

Governments value M&E because it is canprovide them with useful information on therelative value of different approaches andmodels. This, in turn, can feed into theprocess of policy formulation and the coor-dination of programmes on the ground.

For project client enterprises, participationin M&E exercises can provide an importantopportunity for cooperation and trust-build-

ing and for having a meaningful input intothe design and implementation of initiativesthat directly affect their performance.

In the remainder of this section, priority willbe given to the M&E needs of CND projectmanagers, recognising that their needs will,to a greater or lesser extend, tend to coincidewith or complement the needs of each of theother stakeholders.

What needs to be measured?

The log-frame as a tool for M&E

The first and most important step in thedesign of an M&E system comes at the stageof project design. It is here that the keystakeholders need to reach a common visionabout:

1. what it is they want to achieve;

2. what things the project needs to do forthis to happen;

3. how they would know if they had suc-ceeded in their goals; andm

4. how they intend to measure progress onthese fronts

8 It should be noted here that the log-frame createdat the beginning of the project is not cast in stone:stakeholders can return and make changes to it as nec-essary in response to unanticipated factors or projectresults. Nonetheless, in most cases, the initial creationof the log-frame is most important step in the processof creating a shared vision, indicators and targetsamong the various stakeholders.


EVALUATION ISSUES

33

to be true? We know, for example, that someclusters are highly dynamic while others aremore or less stagnant. The relationshipbetween the outputs associated with CNDprojects (increased cooperation and network-

ing within networks and clusters) and theirimpacts (increased wealth and poverty alle-viation, etc.) is complex and relatively littleunderstood. There are clearly significant dif-ferences between clusters in the degree to

A. Measuring project performance

which increased cooperation and capacitytranslates into solid improvements in thequality of life of the people and performanceof the organizations inhabiting them. It isthe role of a properly functioning M&E sys-tem to throw light on these questions.

Similarly, the case studies contain littleinformation on the relative costs and bene-fits associated with their various initiatives(cost-effectiveness). In addition, while theydo provide some information on the transferof services to BDS providers and otheractors, this is rarely presented in the contextof a clear and time-bound strategy for post-project sustainability.

These various omissions are, in part, due tothe very real methodological problems asso-ciated with data- collection and analysis atthese levels as well as with the “evolving”nature of CND projects. Nonetheless, ifM&E systems are to be an effective man-agement tool, these problems must be satis-factorily addressed. The remainder of thissection describes some of the main method-ological problems and how they might betackled. We will look in turn at the measure-ment of project performance, project impact,cost-effectiveness and sustainability.Finally, some of the principles underlying acommon methodology for CND projects areproposed.

Provided that clear, specific and time-boundindicators and targets have been set in thelog-frame and realistic sources of verifica-tion have been established, the gathering ofdata on project performance should pose fewproblems. Indeed, this has been the experi-ence of the case studies described above,each of which provides substantial informa-tion on the number of clusters strengthened,associations established, business networkscreated, awareness-raising campaigns under-taken, trade fair visits sponsored and so on.

The only methodological problem in measur-ing the performance of CND projects lies inhow to define the ‘strengthening’ of clustersand networks. This is the relationshipbetween project activities and outputs: it can-not be taken for granted that, for instance, theestablishment of a network produces eco-nomic gains for the enterprises that compriseit or providing training to the staff of a pro-ducer associations, will necessarily result in areal strengthening of the association capabil-ity to be useful for its members. As notedabove, the factors underlying the emergenceof dynamic clusters and networks are com-plex: in some cases, for example, the eco-nomic climate can be so unfavorable in thesector concerned that no amount of suchactivities can, in fact, lead to effective jointaction among clustered enterprises. Manyother such factors are also likely to be at play.

What is required is the identification of indi-cators that characterize strong and effectivenetworks and clusters. These may relate tothe types of decisions taken, the nature ofjoint projects undertaken, the quality of therelationships that develop with other clusteractors – the relative importance of these islikely to vary between cultures and contexts.Of key importance is that appropriate indi-cators be identified in a dynamic and con-text-specific process, rather than drawnmechanically from a list.

Here, developing a culture of rigorous andefficient M&E will bring its own rewards.For it is just such a culture that will facilitatethe identification of the key types of behav-iors and factors that characterize trulystrengthened clusters and networks. Oncethese have been identified and demonstratedin a good number of cases, project staff maybe able with greater authority to draw a con-vincing connection between the undertakingof certain activities, and cluster and networkstrengthening.

The impact of a CND project can be definedas those changes, both intended and unin-tended, that occur (especially but not exclu-sively) among its target groups – MSEs, pro-ducer associations, BDS providers, etc. –that can reasonably be attributed to the proj-ect. In this sense, project activities and out-puts (all of the various things that projectstaff do) can be seen primarily as the meanstowards the end of effecting tangiblechanges in the conditions of their targetgroups – which is project impact.

In terms of the measurement of impact, fourpoints need to be made at the outset:

1. Even if there were to be no limits on theresources devoted to M&E (which isnever the case), it would be just aboutimpossible to arrive at an exact and objec-tive calculation of the impact of any proj-ect. Especially in the world of MSE clus-ters and networks, conditions are muchtoo complex to enable the M&E team toeither: i) capture all of the various effectsof project activities that ripple out fromdirect project clients to other enterpriseswithin and beyond the cluster; or ii) pre-cisely attribute benefits to the activities ofthe project, as opposed to all of the otherforces and initiatives at play.

2. Neither donors nor project managersexpect the M&E team to deliver scientifi-cally objective findings on project impact.The aim, rather, is to make an assessmenton the basis of reasonable assumptions(that is, assumptions that will stand upintelligent scrutiny and common sense) ofwhat benefits can reasonably be attributedto the project.

3. Effective M&E systems are those thatfind an appropriate balance betweendelivering useful (that is, specific and rea-sonably accurate) findings without usingup an unreasonably large amount of thehuman and financial resources at the dis-posal of the project.

4. Effective M&E systems tend to be thosethat focus on a small number of indicators(generally including trends in income andemployment) and investigate them thor-

oughly and well. Conversely, projectsthat rely on long and poorly-focusedquestionnaires for their M&E systemstend to alienate both staff and clientswhile delivering information that is oflimited value.

What needs to be measured to deter-mine project impact?Measurement of trends in the followingareas are likely to lie at the heart of all CNDprojects. (Only specific areas may need to bemeasured in particular projects, but the fol-lowing can be considered as to cover thebroad range of impact assessment issues forCND.)

• Scale: how many people, enterprisesand/or institutions were affected?

• Outreach: to what extent did the effects(hopefully benefits!) of the project spreadto specific target groups (the poor,women, specific castes or ethnic groups,particularly isolated or marginal targetgroups)?

• Economic gains or losses among cliententerprises, (e.g. changes in output, pro-ductivity, product range and quality,income, employment, etc.)

• Total economic gains or losses, i.e.including those beyond client enterprises.

• Capacities and strengths of enterprisenetworks, including horizontal and verti-cal linkages achieved during the life of theproject.

• Total entrepreneurial and networkingcapabilities, i.e. including those beyondclient enterprises.

• The development of BDS and financialmarkets: in what way has demand for andsupply of BDS and financial services beenaffected by the project?

• Strengthening of support institutions:in what ways have the various supportinstitutions, including producer associa-tions and government agencies, beenstrengthened by the project?

• Changes in the overall business envi-


34

B. Measuring project impact


EVALUATION ISSUES

35

ronment that have an effect on enter-prises

• Corporate responsibility: that is, capa-bility of firms to be “responsible” forsocial and environmental issues

• Social Capital: including issues such ascollective action and cooperation, socialinclusion and empowerment.

The methodological difficulties and chal-lenges associated with the first four of theseareas (considered as core elements of animpact assessment system) are relativelywell understood and will be discussedbelow. There are significantly greater diffi-culties associated with measurement ofmany of the distinctive elements of the CNDapproach, to do with increasing the capacityof business networks, and support organiza-tions; enhancing the business environmentand the local social capital; and developingBDS and financial services markets. Theseissues are the subject of a further studyundertaken by UNIDO whose results will bepublished at a later stage.

Measuring ScaleHow many institutions, enterprises, house-holds and individuals have derived benefitfrom the project? Of course, in most cases,it is impossible to know exactly: good ideasare self-seeding and such impacts are gener-ally difficult to track. The aim is to make asound estimate on the basis of reasonableassumptions.

A first step is to distinguish between directand indirect beneficiaries. Direct benefici-aries should be easy to count – these are theclients with which the project has direct con-tact. Greater methodological challenges liein the calculation of indirect beneficiaries.This is especially so within enterprise clus-ters, where part of the rationale for interven-tions is that innovations introduced by theproject will spill over beyond direct projectclients, thus increasing the cost-effective-ness of the intervention.

In seeking to quantify indirect beneficiaries,it is necessary to establish what are the mainanticipated benefits (or in the case of post-project evaluation, what have been the prin-cipal benefits) of the project: new techniques

or technologies introduced? new productsdeveloped? joint raw materials purchase?new markets opened up? others? The aimthen is to attempt to gauge the degree towhich other actors that have had no directcontact with the project have also adoptedthe new techniques, technologies, workingmethods, forms of organisation, or whateverthe specific benefits might be.

How one would investigate this and whereone would look for evidence will depend onthe nature of the anticipated benefits andidentity of the likely beneficiaries.Remember that beneficiaries will not neces-sarily be limited to other small enterprises:they may also include other actors bothupstream (those supplying benefiting enter-prises with raw materials, equipment, com-ponents, etc.) and downstream (those usingthe products of benefiting small enterprisesin their various activities). It is importanthere to think in terms of ‘value-chains’ – toattempt to track impact throughout the chainof relationships of which client small enter-prises form part.

In most cases, this is best done relativelyinformally – that is, by visits to other areas orenterprises where it is anticipated that theinnovations may have taken root and the useof key informant and semi-structured inter-views – rather than by highly rigorous andscientific analysis.9 This latter strategy islikely to prove too time-consuming andexpensive. Remember, the principal aim ofM&E for project staff is as a source of infor-mation to improve the quality of manage-ment, not as a propaganda tool. In conse-quence, those undertaking such studiesshould be motivated primarily by curiosityabout the degree to which project strategy isworking and benefits are spreading through-

9 ‘Key informants’ are people identified by theM&E team as particularly important sources of infor-mation by virtue of the position they occupy in theSME world or in the value-chain of which they formpart. Semi-structured interviews can involve the use ofboth questionairesquestionnaires and more informaldiscussions. They provide greater flexibility and per-mit the gathering of more qualitative information thanconventional, questionairequestionnaire-based inter-views.

out the cluster and beyond. If this is happen-ing to a significant degree, what has the proj-ect done right and what lessons can be learnedto guide future actions? If not, what morecould the project be doing to facilitate dis-semination? It serves no one for project staffto actively seek out those cases that justify itsapproach, over-looking cases of failure.

Measuring outreachTo what degree has the project succeeded indelivering benefits to particular targetgroups? Begin by noting which (if any) spe-cific groups the project seeks to reach –women? the poor? specific ethnic groups orcastes? etc? Particular attention is requiredin projects with a strong focus on poverty-alleviation in defining what constitutes ‘thepoor’. Is poverty to be measured in purelyfinancial terms or is there a place for consid-erations such as access (to health, education,land, etc.) or vulnerability?

Having clarified precisely which specialgroups are to be targeted, these need to berepresented to an appropriate degree in theM&E’s baseline sample and control group(see below). If non-financial measures ofimprovement in the condition of the poorhave been adopted, a more qualitativeapproach to impact assessment will berequired. This is likely to entail the adoptionof a highly participatory approach to ensureboth that appropriate indicators are identi-fied and that high-quality information onproject impact is gathered. There is likely tobe a need to complement (or, in some cases,to replace) the questionnaire-based methodof information-gathering, so suited to quan-titative data collection, with key informantand semi-structured interviews and focusgroup formats. (This point is equally truewhen setting and measuring all qualitativeindicators, not just those relating to poverty.)

Measuring economic gains amongclient enterprises. Remember that a core rationale for mostenterprise development projects is to pro-mote an increase in the material well-beingof households and individuals, and the mostaccurate indicators we have for measuring

this is jobs and earnings. A crucial factor tobe considered here, however, is time. InCND projects, in fact, the impact on enter-prise profitability “matures” only over timebecause these projects focus on institutionbuilding and inter-enterprise relationshipsrather that on direct support to individualenterprises

Keeping this factor in mind, it is still impor-tant that economic gains of local enterprisesare adequately accounted for and the firsttask here is to draw up a representativesample of client enterprises to provide thedata base-line. What are the key variableswithin the target group you are working withmost likely to have an impact on enterprise-level trends in employment and income? –sector? enterprise size? level of technologi-cal sophistication? gender of the owner orworkers; caste or ethnicity? (The relativeimportance of these is likely to vary signifi-cantly between projects.) Identify which arethe most important and ensure that the base-line sample offers an approximate reflectionof how these variables are distributed amongthe total universe of enterprises that the proj-ect is targeting. The sample needs to belarge enough to compensate for any particu-larities or exceptional cases at enterprise-level: generally ten per cent or so of the totalnumber of direct beneficiaries is recom-mended.

In general, getting information on trends inemployment at enterprise-level is relativelystraightforward. However, it is important toremember that in many situations, a signifi-cant amount of employment is neither full-time nor permanent. M&E systems need tohave sufficient sensitivity to track trends inseasonal and part-time work. This requireseither relatively frequent monitoring (quar-terly information-gathering should be suffi-cient) or training of sample entrepreneurs torecord this information themselves on sim-ple questionnaires. M&E should attempt torecord not just the number of workers butalso: i) category of worker (skilled employ-ee, apprentice, part-time, seasonal); and ii)how many hours per week they areemployed.

Gathering data on trends in income amongclient enterprises can be significantly more


36


EVALUATION ISSUES

37

difficult. There are numerous reasons whyan entrepreneur might provide inaccurateinformation to a project M&E worker: poormemory recall in a context of little or norecord-keeping, fear of the information leak-ing to the tax authorities; believing thatunder-reporting or over-reporting gainsmight result in additional project assistance;or a simple desire for privacy and/or resent-ment of perceived intrusion. In spite of allthis, and especially where strong relations oftrust have developed between project andclients, direct enterprise-level questionnaireson income trends can deliver valuableresults.

In those cases where it is not safe to trustinformation on income gained from directinterviews, one alternative (or complemen-tary) approach is to identify proxy indica-tors – that is, indicators which are closelyrelated to the trends to be measured andwhich can be expected to throw significantlight upon them. In the case of income, thebest proxy indicator is production. Here, thetask is to identify the principal productsmade by target enterprises and to trackchanges in their levels of production.

This can be done in one of three ways. First,and easiest, in those cases where enterprisesare involved in joint marketing, the recordsof the marketing company can provide allthe necessary information. Second, entre-preneurs can be trained to record productiondata on simple questionnaires. Finally, theinformation can be gathered through regularvisits by field staff. Remember, the aim isnot to record every item produced, but onlythe major ones.

The next challenge for the M&E system isthat of attribution – that is, to what extentcan any gains that are recorded among cliententerprises be attributed to the activities ofthe project, as opposed to other forces atwork within the cluster or network? The bestway of addressing this problem is to estab-lish a control group. A control group is agroup of enterprises that, as far as possible,resembles the base-line sample in everyrespect other than that it derives neitherdirect nor indirect benefit from the project.Thus, in theory, by using a control group, thespecific impact of the project can be isolated.

The use of control groups is rarely withoutits complications. Enterprises enjoying noproject support have little interest in cooper-ating with M&E staff – in many cases wherecontrol groups are used, in fact, they are paida small fee to encourage them to do so. Inaddition, it is rarely easy to find a truly sim-ilar control group, not least because clustersare often selected for participation in CNDprojects because they already enjoy somespecial distinctive characteristics that setthem apart from others.

Within the cluster, it can also be difficult toidentify enterprises that are in no way affect-ed by the project – for one of two reasons.First, where cluster-based projects are suc-cessful, their effects are likely to ripplewidely throughout the cluster, with the inno-vations introduced by the project imitatedand replicated by many others. Second isthe problem of displacement; that is, do thegains recorded among the sample group gen-uinely represent new economic activity, ordo they merely indicate that enterprises ben-efiting from project assistance have dis-placed to others that have not? If this is thecase, the contrast in fortunes between thetwo will be exaggerated (and the projectmay believe it is being very successful),even if little or no new economic activity isbeing generated.

There are no easy solutions to these chal-lenges. The most that project staff can do isto be aware of the dangers in the creation oftheir control group and to aim for a groupthat as nearly as possible resembles the base-line sample in all respects other than partici-pation in the project.

Measuring total economic gainsWe return to the question of how to trackimpact beyond the direct project clients.Within the cluster, as noted above, successfulprojects are likely to generate significantcluster-wide ripples, with new products,techniques, technologies, working practices,forms of enterprise cooperation, etc. beingwidely imitated and replicated. In addition,the capacity of producer associations andother organisations is likely to grow,enabling them to better promote the interestand fortunes of their members. Further,


38

within the cluster as a whole, capacity fordesign may well be enhanced, with addition-al positive consequences in terms ofincreased growth.

External to the cluster, there may well also besignificant benefits to a range of actors alongthe value-chain. An increase in the capacityof small-scale capital goods manufactures,for example, is likely to have a wide anddeep impact through the dissemination ofsmall-scale manufacturing and food-process-ing equipment, creating new opportunitiesfor rural enterprises, with employment andincome gains among both rural entrepreneursand farmers. Increased vitality within MSEclusters, irrespective of the specific sector,will generate additional economic activity,both up-stream among suppliers; and down-stream among clients (except in the case ofpurely consumer goods).

It is important for CND projects to attemptto capture these various indirect benefits, fortwo principal reasons. Firstly, as a manage-ment tool. One cluster development projectin Zimbabwe began by gathering data onlyamong the small-scale engineers that it was

working with. At this level, it concludedthat impact was relatively low – significant-ly lower than project costs. Only later did itrecognise that most project benefits accruednot to the small-scale engineers (their directclients) but to the rural entrepreneurs whobought their equipment and the farmers fromwhom they, in turn, demanded an increasedsupply of inputs. This insight permitted ashift in project strategy that saw a muchgreater focus on the marketing of the equip-ment made by their client enterprises in therural areas of the country. This shift result-ed in the project having a significantlyincreased impact.

Second, to ensure efficient allocation ofdevelopment funding, it is important to beable to compare the total relative costs andbenefits of different projects and of differentapproaches and models. As CND projectsare often characterised by relatively highlevels of ripple benefit (beyond direct proj-ect clients), it is especially important forthem to be able to track these wider impacts.We will return to this in the next section oncost-effectiveness.

C. Measuring cost-effectiveness

There are two dimensions of cost-effective-ness that CND projects need to measure.The first is that noted above, namely relativeproject costs and benefits. There are well-established conventions governing the cal-culation of benefit:cost ratios, including theprojecting of anticipated monetary benefitsfor 10 – 15 years beyond the life of the proj-ect. It is essential that such calculations,whether undertaken during the project orafter its completion, be undertaken in astransparent and professional a manner aspossible.

A negative benefit:cost ratio does not neces-sarily mean that a project has failed; manyare able to argue that certain of the benefitsgenerated have some ‘public goods’ charac-teristics (enhanced skills and other capacityspreading far beyond the direct target group)for which full cost-recovery is neither possi-ble nor reasonable. However, a transparentand professional benefit: cost analysis willhelp to make this rationale explicit and to

make the case for on-going government ordonor subsidy.

It is also useful, where possible, to attempt toseparate out the costs and benefits associatedwith different services provided by a project.This enables project managers to get a feel forwhich of the services (or which combinationof services) they provide are having greatestimpact. It is true that where services are bun-dled together, such a disaggregation of costsand benefits may be difficult. However, cal-culations of the relative costs and benefits ofpackages of bundled services may also beboth possible and useful. The greater thelevel of disaggregation, the more useful it islikely to be to project managers.

The second dimension of cost-effectivenessneeding to be tracked can be described as‘value-for-money’ – that is, are the servicesbeing provided in the cheapest and mostefficient way possible? This is a particular-ly important consideration when considering


EVALUATION ISSUES

39

Is there a need for the services provided bythe project to continue beyond the life of theproject? If so, how are they to be provided?The case studies in section two suggest fivepossible sustainability strategies (that are, infact, complementary – most of the case stud-ies include at least several of the followingelements).

Transfer services to private sector BDSproviders.

Strengthen the capacity of business associa-tions to provide services beyond the life ofthe project.

Client enterprises within the cluster takeover from the project payment for the serv-ices of the cluster or network broker.

Look to donors or government for long-term subsidy. This will be possible onlywhere projects succeed in persuadingdonors or governments of strong ‘publicgood’-type benefits accruing from projectsthat will incline them to provide on-goingsupport. However there is an obvious riskin this type of strategy especially in coun-tries where public budgets are scanty anddevelopment priorities may change. Onlyrarely will this prove to be a viable sustain-ability strategy.

Support ‘soft networks’, for short-term, spe-cific goals, that will not need to continuebeyond the life of the project.

The first task is to be clear about which ofthese elements, and in what combination, areto make up the project’s sustainability strat-egy. Then, targets and timetables need to beset for each. For example:

Which BDS are to be transferred to privatesector providers? What should be the

timetable for this transfer? How is it to beachieved?

According to what timetable should businessassociations take over project activities?Which ones? How?

What is the strategy for engaging donorsand/or government into playing the role oflong-term funder? What are to be the indi-cators and targets for this?

Each of the key stakeholders concernedshould be involved in negotiating and settingthe targets and timetables for the sustainabil-ity strategy. This will create consensusaround the strategy that evolves, thus con-tributing to its chances of success.

A few words are needed specifically abouttracking the development of a BDS market,since this is likely to be a particularly impor-tant element of most CND sustainabilitystrategies. In many (some would say most,or even all) cases, private sector organisa-tions are likely to be able to deliver BDSmore efficiently, cheaply and sustainablythan donor-funded projects. In this context,the appropriate role of projects should be tostimulate private sector BDS provision ratherthan attempting to play this role (in the long-term) themselves. Thus, it is legitimate forprojects to act as BDS providers only as ameans of stimulating demand for and/or pri-vate sector supply of the services in question.

However, it will be difficult for project man-agers to gauge when and at what speed towithdraw from service provision withoutgood information on the levels of existingdemand and supply potential within the mar-ketplace. On the demand side, the M&Esystem needs to be able to track both whatservices are required by small enterpriseswithin the cluster and their willingness to

D. Measuring sustainability

services for which there is the potential forcompetition between the project and otherBDS providers.

Every effort needs to be made to ensure thatdonor funding is not providing hidden sub-sidies in service areas where private serviceproviders could emerge. If project man-

agers are to make informed decisions onthe allocation of resources; and if they areto encourage rather than inhibit the devel-opment of private sector BDS markets,M&E systems need to be designed to per-mit the tracking of service-specific costsand benefits.


40

pay for these. On the supply side, indicatorsneed to be developed and tracked thatdescribe the capacity of private sectorproviders to deliver services of an accept-able quality.

It is important that private sector BDSproviders have the capacity to undertake

market research of this kind into the future, ifthey are to be able to adapt their services tochanging patterns of demand. Consequently,it should be an important part of the capaci-ty-building work of CND projects to under-take this M&E work in close cooperationwith these private sector service providers.

Conclusions: key principles of a CND M&E system

The following are some of the key principlesof an appropriate M&E system for CNDprojects arising out of the foregoing:

M&E for managers of CND projects shouldbe seen primarily as a management tool,whose function is to feed information intothe process of maximising the impact, cost-effectiveness and sustainability of this andother similar projects. Project M&E sys-tems that are geared towards proving impactto donors and governments are too oftenselective in their search for positive evi-dence and thus, miss out on the many posi-tive lessons to be learned from failed exper-iments.

M&E should be seen as a learning experi-ence, an opportunity to engage all stakehold-ers in the process of setting indicators andtargets and measuring performance andimpact against them. This is likely both tobuild the capacity of the various stakehold-ers and to lead to an improved flow of infor-mation at the disposal of the project team.

It is important to deliver high-quality infor-mation not just on project activities and out-puts; but also on immediate and develop-ment objectives.

Indicators need to be adapted to sector- andculture-specific contexts rather than drawnin a mechanical way from a pre-preparedlist.

Effective M&E systems tend to work with arelatively small number of highly-focusedindicators. The process of designing andimplementing an appropriate M&E system

should arise out of the questions: ‘what arewe trying to achieve?’ and ‘how would weknow if we were succeeding in this?’ Ifdone this way, the process of M&E data col-lection and analysis should feel meaningfuland exciting as all involved track progressagainst commonly-agreed indicators and tar-gets.

Neither project managers nor donors expectscientifically rigorous findings from anM&E system. What is required are resultsbased on reasonable assumptions, thatdemonstrate awareness of the factors that aremost likely to distort the true picture.

Devote resources at the outset to the estab-lishment of a base-line data set and of a con-trol group. This is likely to save many M&Eproblems in the longer-term.

It is desirable to provide some form of ben-efit: cost analysis. If this is to be relevant toCND projects, which have the potential tocreate substantial ripple benefits, ways mustbe identified to track and quantify impactbeyond direct project clients, throughout thevalue-chain.

For the M&E system to be a useful manage-ment tool in tracking progress towards sus-tainability, a sustainability strategy must beclearly articulated and appropriate indicatorsand targets set.

Measurement of trends in the supply anddemand of BDS should be done in closecooperation with private sector serviceproviders as a way of transferring capacityto them.

EXPERT GROUP MEETING ON CLUSTER AND NETWORK DEVELOPMENT WITH

Documents

Transcript of EXPERT GROUP MEETING ON CLUSTER AND NETWORK DEVELOPMENT WITH