Indonesia Kecamatan Development Program › content › usergenerated › asi › cl… · assess...

Indonesia Kecamatan Development Program:Building a Monitoring and Evaluation System For a Large-Scale Community-Driven Development Program

Susan Wong

May 2003

Reprinted May 2004

Environment and Social Development UnitEast Asia and Pacific RegionDiscussion Paper

This publication was developed and produced by the Environment and Social Development Unit (EASES),East Asia and Pacific Region of the World Bank. The Environment, Rural Development, and SocialDevelopment are part of the Environmentally and Socially Sustainable Development (ESSD) Network.

Papers in the EASES Discussion Paper Series are not formal publications of the World Bank. They are pub-lished and circulated to encourage discussion and comment within the development community. The findings,interpretations, judgments, and conclusions expressed in this paper are those of the author(s) and should not beattributed to the World Bank, to its affiliated organizations, or to members of the Board of Executive Directorsor the governments they represent.

Copies of this paper are available from:Susan Wong, EASES, Room [email protected]

ii

iii

ABSTRACT

The World Bank has invested heavily in community-driven development (CDD) programs throughout the world since the mid-1990s. As with other development programs, these CDD investments require strong monitoring and evaluation systems toassess progress and evaluate program impact. The Kecamatan Development Program(KDP) in Indonesia is a large-scale CDD program that developed an extensive monitoring and evaluation system to track its progress and evaluate its impact. The system consisted of quantitative and qualitative tools, including internal monitoringfrom program stakeholders and external independent monitoring by Indonesian non-governmental organizations (NGOs), journalists, and audit teams. The Programcommissioned several impact and technical studies to measure its impact on poverty alleviation and local governance. Because of the strong emphasis the WorldBank and the government of Indonesia placed on oversight, monitoring and evaluationbecame integral components of overall program management. Findings were discussedregularly with managers and used to improve the design and operations of KDP.

This paper describes the monitoring and evaluation system developed for KDP. Itincludes a detailed description of its components, how they were constructed and operationalized, and the challenges encountered while incorporating these multi-faceted activities into a large CDD program. It is hoped that this case study will aidpractitioners in the development community who seek innovative and effective tools to monitor and evaluate their programs.

ACKNOWLEDGEMENTS

This paper, and indeed the overall development of the KDP Monitoring and Evaluation System, benefitedgreatly from the vision, support, and guidance of Scott Guggenheim and Victor Bottini of the World Bank.David Timberman also provided fresh insight and comments to improve this paper. The KDP monitoring andevaluation team members deserve acknowledgement and special appreciation for implementing and refiningthe system’s components over the years. These team members are: Sandra Usmany, Lily Basri, Yuli Hastadewi,Yesi Maryam, Ari Siregar, Pinkan Utami W, Budi Hadi Totok, M. Amin, Azlim Fitra, Wirawan Nugrahadi,Kukun Kurnia, Heru Gunawan, Joko Suhartanto, Hotman P S, Yonef Rizal, Kumoro Pristi, Ende Nurdartilah,and Yayuk Wuriati.

The many accomplishments of KDP’s Monitoring and Evaluation component would not have been possiblewithout the commitment and support of colleagues associated with the Ministry of Home Affairs, KDPNational Secretariat, National Management Consultants, field consultants, community members, and thenumerous NGOs and journalists who joined collectively in this massive monitoring and evaluation effort. Theyare too numerous to mention here by name, but their input and support are gratefully appreciated.

iv

TABLE OF CONTENTS

List of Acronyms ......................................................................................................................................................iv

1. Introduction .........................................................................................................................................................1

2. Brief Program Description..................................................................................................................................1

3. Development of KDP’s Monitoring and Evaluation System...........................................................................3

4. Development of the M&E Plan.........................................................................................................................5

5. Components of the Monitoring System............................................................................................................9

5.1 Field Monitoring and Reporting by Government Officials and Consultants...........................................9

5.2 Community Participatory Monitoring ......................................................................................................11

5.3 Case Studies and Documentation of Lessons Learned............................................................................12

5.4 NMC Financial Supervision and Training Unit.......................................................................................14

5.5 Handling Complaints and Grievance Procedures.....................................................................................15

5.6 NGO Independent Province-Based Monitoring......................................................................................18

5.7 Independent Monitoring by Journalists.....................................................................................................20

6. Components of the Evaluation System ..........................................................................................................21

6.1 Impact Evaluation Study ............................................................................................................................21

6.2 Infrastructure and Economic Loan Evaluations .......................................................................................23

6.3 Audits and Financial Reviews ....................................................................................................................24

6.4 World Bank Supervision Missions and Financial Reviews......................................................................25

7. Costs of the Monitoring and Evaluation System............................................................................................25

8. Difficulties Encountered...................................................................................................................................25

9. Conclusions and Future Improvements under KDP Phase Two...................................................................26

Bibliography.............................................................................................................................................................29

Annexes:

A. Brief Description of KDP.................................................................................................................................31

B. KDP Project Design Summary........................................................................................................................40

C. KDP Monitoring and Evaluation Framework................................................................................................42

D. Terms of Reference for NGO Independent Monitoring of KDP.................................................................46

E. Sample Workshop Schedule for NGOs Involved in Independent Monitoring............................................53

F. Complaints/Grievances Database Fields .........................................................................................................55

G. Terms of Reference for Financial Supervision and Training Unit .................................................................56

H. Terms of Reference for KDP Financial Audit ................................................................................................59

v

vi

LIST OF ACRONYMS

AJI (Aliansi Jurnalis Independen)Association of Independent Journalists

Bappeda (Badan Perencanaan PembangunanDaerah), Regional DevelopmentPlanning Agency at the province or dis-trict level

Bappenas (Badan Perencanaan PembangunanNasional), National DevelopmentPlanning Agency

BPD (Badan Pemwakilan Desa)Democratically elected village represen-tative council. This body was intro-duced under Regional Governance Law22 of 1999.

BPKP (Badan Pemeriksa Keuangan danPembangunan) Finance andDevelopment Supervision Agency

BRI (Bank Rakyat Indonesia) IndonesianPeople’s Bank

CDD Community Driven Development

DPRD (Dewan Perwakilan Rakyat Daerah)Local Parliament. DPRDI is the parlia-ment at the province level. DPRDII isthe parliament at the district level.

FD (Fasilitator Desa), Village Facilitator

FK (Fasilitator Kecamatan), KecamatanFacilitator

KDP Kecamatan Development Program

Kecamatan Subdistrict containing on average 20 to 25 villages with as many as 60,000persons

KMKab (Konsultan Manajemen Kabupaten)District Engineer Consultant

KMProp (Konsultan Manajemen Propinsi),Provincial Consultant

KPKN (Kantor Perbendaharaan dan KasNegara) Treasury State Office

LD (Lembaga Demografi) DemographicInstitute, University of Indonesia

LKMD (Lembaga Ketahanan Masyarakat Desa)Village Community Resilience Board

LP3ES (Lembaga Penelitian, Pendidikan danPenerangan Ekonomi dan Sosial)Institute for Social and EconomicResearch, Education, and Information

M&E Monitoring and Evaluation

MIS Management Information System

Musbangdes (Musyawarah Pembangunan Desa) ,Village Development Meeting

Musbangdes I: meeting to provide KDPorientation at the village level and toselect village facilitators.

Musbangdes II: meeting to decidewhich ideas the villages will put forward as proposals.

Musbangdes III: meeting to report uponresults of UDKPII meeting, to organizethe community Implementation Teamto oversee project implementation, andto decide upon the village technicalassistance.

Musbangdes Pertanggungjawaban:meeting to report upon projectprogress and finances halfway throughimplementation.

NGO Non-Governmental Organization

NMC National Management Consultants,located at central level

O&M Operations and Maintenance

PjAK (Penanggungjawab AdministrasiKegiatan) Project Administrator for KDPat the kecamatan level

PjOK (Penanggungjawab OperasionalKegiatan) Project Manager for KDP atthe kecamatan level. The PjOK is thesection head of PMD at the kecamatanlevel

PMD (Pembangunan Masyarakat Desa)Community Development Agencyunder the Ministry of Home Affairs.PMD is the government implementingagency for KDP.

Podes (Potensi Desa) Annual village survey

Susenas (Survei Sosial Ekonomi Nasional)National Expenditure Survey ofHouseholds

TK – Kab (Tim Koordinasi Kabupaten)Coordination Team at district level con-sisting of various government min-istries and chaired by Bappeda

TK-Prop (Tim Koordinasi Propinsi)Coordination Team at provincial levelconsisting of various government min-istries and chaired by Bappeda

UDKP (Unit Daerah Kerja Pembangunan)Kecamatan Development Forum, com-posed of village representatives andkey local officials

UDKP I: kecamatan – level meeting todiscuss what KDP is and which villageswill participate in the program for thatyear.

UDKPII: kecamatan – level meeting todecide which village proposals will beaccepted and elect members of theUPK

UPK (Unit Pengelola Keuangan) FinancialUnit elected by the UDKPII forum usual-ly composed of two or three personswho handle the kecamatan and villagefinances for KDP

Indonesia Kecamatan Development Program 1

1. INTRODUCTION

The World Bank has invested heavily in “communi-ty-driven development” (CDD) programs andapproaches since the mid-1990s. CDD projects arebroadly defined as those that give control of deci-sions and resources to community groups.1 As CDDinvestments grow rapidly, it is important that moni-toring and evaluation (M&E) systems be establishedto assess the program’s progress as well as evaluate itsimpacts. All too often, development programs do notincorporate adequate or rigorous monitoring andevaluation mechanisms to measure progress andevaluate the impact of interventions. These systemsare crucial to provide timely, reliable information andallow program managers to adjust activities as need-ed and to assess development effectiveness.

The Kecamatan Development Program (KDP) inIndonesia is one of the largest World Bank-financedCDD projects, with a budget of some $700 millionfor its first two phases (1998 to 2006). During itsfirst phase (1998-2002), KDP developed an exten-sive M&E system to track the program’s progressand evaluate its impact. The system consisted of bothquantitative and qualitative tools, including internalmonitoring from program stakeholders and inde-pendent external monitoring by Indonesian non-governmental organizations (NGOs), journalists, andaudit teams. The program commissioned severalimpact and technical studies to measure cost effec-tiveness and the impact of the program on povertyalleviation and local governance. Because of thestrong emphasis the World Bank and the govern-ment placed on M&E from the outset, it became anintegral component of overall program management.Findings were discussed regularly with managers andused to improve KDP design and operations.

This paper describes the M&E system developed forthe first phase of KDP. It aims to provide develop-ment practitioners with a case study of how monitor-ing and evaluation approaches and tools can beincorporated into large-scale CDD programs.

Detailed information regarding the components isincluded in the text, while annexes serve as a refer-ence for those developing their own M&E systems.

2. BRIEF KDP PROGRAM DESCRIPTION

The Kecamatan Development Program (KDP) is agovernment of Indonesia program aimed at alleviat-ing poverty in rural communities and improvinglocal governance. The first phase of KDP began in1998 and ended in 2002. The second phase will runfrom 2002 to 2006. KDP provides block grants ofRp350 million to 1 billion ($40,000 to $114,000)directly to kecamatan2 and villages for small-scaleinfrastructure, social, and economic activities.

KDP began in 1998 at a time of tremendous politi-cal upheaval and economic crisis. The economic cri-sis reversed years of progress in poverty reductionand plunged millions of rural poor below the povertyline. Economic growth declined to negative 13 per-cent in 1998, in marked contract to the 8 percentannual growth rates of earlier years. Public demon-strations and riots shook the country and led to thedownfall of President Suharto after 32 years inpower. Indonesia entered into a period of politicaltransition leading in June 1999 to the country’s firstfree general elections in 44 years.

The change in leadership, however, did not solve thecountry’s deeper problems. Despite high rates ofpoverty reduction, vulnerability remained high andmany poor did not benefit adequately from thecountry’s economic growth. Three decades of author-itarian rule had also undermined local capacity andplaced heavy restrictions on local community organ-izing. Local level government was generally unre-sponsive and unaccountable to the needs of itscitizens. Corruption was rampant, and the misman-agement of government projects and funds led to ageneral disillusionment with public programs.

1 This is the broad definition of CDD as defined in the WorldBank’s Poverty Reduction Strategy Sourcebook, May 2001.

2 Kecamatan are Indonesian sub-districts containing on average20 to 25 villages with a population of some 60,000 persons. For afuller description of kecamatan and Indonesian administrativelevels, see Annex A.

Government-sponsored community developmentprograms in Indonesia, as in many other countries,were vulnerable to political manipulation and captureby local elites. While new decentralization laws werepassed in 1999 to give districts greater responsibilitiesand authority, this did not mean that power wouldautomatically shift to communities or indeed bedemocratic.

KDP was implemented in the midst of Indonesia’sbumpy political transition period and decentraliza-tion process. Despite a volatile political and econom-ic backdrop, KDP eventually became a core part ofthe government’s strategy of decentralization andimproving local governance. National and local gov-ernments have “bought in” to the KDP approach asevidenced by increasing budgetary contributions tothe program and the replication of KDP mecha-nisms for other public programs. Current govern-ment policy promotes the KDP design as afoundation of its national poverty reduction strategy.

KDP’s goals are to:• Alleviate poverty by raising rural incomes and

providing increased economic opportunities at thevillage level in the poorest kecamatan.

• Strengthen local government and communityinstitutions by empowering them to manageincreased funding and becoming accountable for it.

• Build public infrastructure through labor-intensivemethods.3

By pushing decision making down to the lowest lev-els, KDP aims to allow villagers to participate and directly manage development resources. Theprogram, in essence, seeks to empower the rural poorand encourage more democratic and participatoryforms of local governance. All KDP activities aim atallowing villagers to make their own choices aboutthe kinds of projects that they need and want. Theprogram gives power to communities by placingfunds and the planning and decision makingprocesses directly in the hands of villagers.

KDP’s key principles are:• Community participation and empowerment of poor

rural communities –– Communities should havecontrol over decisions and resources that affectthem. Communities take ownership of all aspectsof projects, from planning and decision making toimplementation. Participation should be broadbased and include poor people and other margin-alized groups.

• Transparency –– KDP emphasizes transparencyand information sharing throughout the projectcycle. Information is power. Decision making andfinancial management should be open and sharedwith the entire community. Everyone should haveaccess to key program information.

• Sustainability –– Activities should be sustainable,building on community self reliance and villagemanagement of all activities.

• Simplicity –– KDP strives to keep projects simple.There should be no complex rules or procedures;only simple strategies and methods should beused.

• Competition for funds –– There should be open,healthy competition between villages for KDPfunds. KDP encourages villagers to select projectsbased on their individual merits.

Geographic coverage

Based upon a national poverty ranking of kecamatan,the government selected the poorest kecamatan toparticipate in KDP. The program began in 501 kecamatan in 20 provinces throughout the country.By its third project year (2001-02), it had scaled upalmost twofold to cover 984 kecamatan with a totalpopulation of approximately 35 million. KDP isoperational in almost one out of every four villagesin Indonesia. (Table 1).

Management

The overall management of KDP is the responsibili-ty of the Ministry of Home Affairs CommunityDevelopment Agency (PMD). PMD deals with theprogram’s day-to-day operations at the national level.Government coordination teams, representing vari-

2 Indonesia Kecamatan Development Program

3 See First Kecamatan Development Project Project AppraisalDocument, April 21, 1998.

ous ministries, exist from the national down to thedistrict levels to oversee KDP.

PMD hired consultant teams to provide technicalassistance to the program. The office of the NationalManagement Consultants (NMC) based in Jakartaled the teams of consultants and provided overallguidance and supervision to field consultants. Some1,660 consultants from 16 private companies at thenational, provincial, district, and kecamatan levelsgive technical assistance and guidance to KDP.

For a more detailed description of KDP, see Annex A.

3. DEVELOPMENT OF KDP’SMONITORING AND EVALUATIONSYSTEM

The Context

Monitoring is the regular checking of projectprogress through the routine and systematic collec-tion of information. This information, when verifiedand analyzed, highlights progress or problems thatallow project managers to adjust plans and imple-mentation as needed. Evaluation is the assessment ofthe merit of the project by measuring it against theachievement of its objectives. Evaluation answers thequestions: has the project met its objectives and whatfactors led to these outcomes. What were the pro-ject’s impacts?

The KDP M&E System was designed to collect sys-tematically information about the program’s progress

and evaluate its effectiveness and impact over time.The uniqueness and magnitude of KDP in theIndonesian context, especially in stressing the impor-tance of community participation, transparency, andinstitution strengthening, demanded that a solidM&E system be in place to document the KDPexperience and to distill lessons. In many ways, KDPis a pioneering effort for government programs inIndonesia and offers a potential model for futuredemocratization and development efforts. Therefore,findings from KDP had to be fully monitored, docu-mented, and evaluated.

The system also needed to provide effective channelsof communication from the field to the nationallevel—and back—to inform management decisionsand ensure that corrective action was taken if neces-sary. Reliable and timely information had to be inthe hands of those who could act upon it and resolveissues and problems expeditiously.

Challenges in developing the M&E system

There were several challenges in developing theM&E System.• Ambitious objectives and wide variety of activities ––

Among the most difficult challenges in develop-ing an M&E system were the program’s ambitiousobjectives and wide scope of activities. WhileKDP’s overall goal was poverty alleviation, formany stakeholders issues of process and democra-tization were of equal, or greater, importance. Theprogram’s emphasis on citizen involvement, inclu-sion, and open and transparent discourse on devel-


Table 1: KDP Geographic Coverage, 1998-2002

Year One of KDP Year Two Year Three Total in Indonesia(1998-1999) (1999-2000) (2001-2002)

• 20 provinces • 20 provinces • 22 provinces • 32 provinces

• 105 districts • 116 districts • 130 districts • 341 districts

• 501 kecamatan • 727 kecamatan • 984 kecamatan • 4,048 kecamatan

• 3,524 villages • 1,325 villages • 15,481 villages • 69,168 villages

opment priorities provided the fundamental build-ing blocks for democratic practices in a societybranded with three decades of authoritarian rule,top-down planning, and corrupt practices.Capturing the breadth of that process required theuse of complementary qualitative and quantitativeM&E instruments, as well as concerted efforts tosystematically document the process.

The open menu and multi-sectoral nature ofKDP also meant that a variety of outputs had tobe monitored and evaluated for quality and costeffectiveness. This meant designing and imple-menting control systems for roads, water, and avariety of infrastructure projects, as well as eco-nomic activities such as women’s savings and loanschemes.

• Geographical scope and logistics –– The sheer magni-tude and geographical scope of KDP also mademonitoring and reporting a challenge. Indonesia isan archipelago of over 13,000 islands and logisticsand communications posed challenges. Reachingprovincial capitals was easy enough, but visitingremote villages and islands was sometimes diffi-cult, especially in the eastern islands and theMalukus. Also, with over 1,660 consultantsnationwide, obtaining standard, timely, and usefulreports was not always easy. Simply managingdata and reports from over 15,000 villages nation-wide presented enormous data entry and quality-control issues for the management informationsystem.

• Combating corruption –– Fighting corruption wasalso high on the agenda. Indonesia has traditional-ly been ranked as one of the most corrupt coun-tries in the world and tough measures to detect,prevent and deal with corrupt practices had to bebuilt into the M&E system. Corrupt practiceswere endemic, particularly in government devel-opment programs, and it would take a concertedanti-corruption strategy and effort to counterthese tendencies. KDP implemented a three-pronged strategy that guided elements of projectdesign as well as the M&E system: (i) eliminatecomplexity in project design features and fundflows; (ii) ensure transparency of all project infor-mation including financial transactions; and (iii)

respond quickly to complaints.4

• Political change and conflict –– Lastly, KDPPhase One operated in the midst of a tumultuouspolitical and economic backdrop throughout itsproject period: a prolonged drought; severe finan-cial crisis; a political coup followed by the first freegeneral elections held in 44 years; four presidentsand numerous cabinet reshuffles in four years;armed insurgencies; ethnic riots and repeated ter-rorist attacks around the archipelago, and theOctober 2002 Bali bombing. Over one-third ofIndonesia’s provinces had recent experiences withconflict or are currently in the midst of civilunrest. While KDP continued relatively unscathedfrom these political events, project operations weresuspended in certain troubled areas for varyingstretches of time. Insecurity in provinces such asAceh, Maluku, Central Kalimantan, and CentralSulawesi affected the program’s ability to imple-ment and monitor activities.

Opportunities

Despite these challenges, Indonesia had numerousadvantages that made it conducive to implementingthe M&E system.• Nascent democracy and more open political climate ––

After the downfall of President Suharto in 1998,KDP benefited from the more open political envi-ronment in several ways. First, while intimidationand repression were, and still are, all too prevalentin parts of the country, there was a growing trendfor citizens and community groups to speak outand voice their concerns. This greater opennessfacilitated the documentation and reporting ofstakeholder feedback on the program’s progress.

• Growth of civil society organizations –– The bur-geoning democratization process also paved theway for a rapid expansion of media and civil socie-ty organizations. After President Suharto’s down-fall, media organizations flourished and thenumber of provincial NGOs grew exponentially.


4 For further details on anti-corruption measures, see SecondKecamatan Development Program Project Appraisal Document,May 23, 2001, Annex 11.

While questions remain over the quality and regu-lation of some of these groups, there is little doubtthat they have grown in prominence and influencesince 1998. This growth helped tremendously informing partnerships with civil society groups toundertake independent monitoring of KDP.

• Skilled pool of consultants –– Indonesia enjoys ahigh national adult literacy rate of 87 percentand KDP could recruit educated consultants andfacilitators, usually with university degrees. Therewere problems in some provinces, such as Papuaand the outer islands, with recruiting educatedand experienced consultants, but in the majorityof provinces hiring suitable staff did not pose sig-nificant problems. That being said, KDP still hadto invest in staff development, especially forcommunity facilitation and mediation skills,technical supervision, and reporting. Obtaining agood gender balance among consultants, espe-cially for technical staff, also posed an enormouschallenge.

• Relative ease of communications –– Another advan-tage was Indonesia’s generally efficient communi-cations network. Telephones, faxes, post ande-mail could be used down to the district level ifnot lower. By Year 3, over 80 percent of districtfacilitators, and all provincial coordinators, coulduse e-mail to send their reports and communicatewith the national level. The widespread knowl-edge and use of a single national language acrossmost of the archipelago simplified reporting andcommunications between the central and fieldoffices.

• Firm government and World Bank support and com-mitment to developing a strong M&E system ––The establishment of the KDP M&E systemwould not have been possible if the World Bankand the Indonesian government had not commit-ted financial and technical resources toward thiseffort. M&E activities such as journalist monitor-ing, special studies, complaint-handling proce-dures, and impact monitoring were incorporatedinto the Project Appraisal Document and LoanAgreement. The program established a 22-personM&E Unit and Complaints Handling Office todevelop and implement this system. The Loan

Agreement contained a $2 million budget for specific M&E activities and special studies.

4. DEVELOPMENT OF THE M&E PLAN

Developing the overall monitoring and evaluationplan for such an extensive and multi-faceted programwas not an easy task—nor was it a one-off exercise.KDP evolved, and to the program managers’ credit,results from monitoring reports and evaluationsinfluenced project implementation, the annualredesign of the program’s procedures, training plans,and the next KDP2 (2002-2006) phase. For exam-ple, if field monitoring or audit reports revealedproblems with elite interference or incorrect imple-mentation of procedures, managers contacted localstaff to investigate and initiate remedial actions. Inseveral provinces, extreme cases led to the temporarysuspension of activities.

The starting point for development of the M&Eplan was understanding a project’s background anddesign, including its context, objectives, outputs,inputs, and activities. What were the key objectivesand outputs of KDP? How did the steps in the proj-ect cycle work, and what critical information had tobe collected and at what stages? What research ques-tions or hypotheses must the M&E system address?To answer these questions, the KDP M&E teamundertook a thorough review of KDP’s backgrounddocuments and Loan Agreement. This documentreview was complemented with discussions with key KDP actors in the government and World Bank, and provincial stakeholders, to ascertain what they felt were the “heart and spirit” of KDP,and what they hoped to see as a result of the three-year program.

What eventually developed as the basic M&E planor framework is listed in Annex C. Based on discus-sions with stakeholders, key research questions andinformation needs emerged, such as the following:

1. Poverty and Socio-Economic Impacta. What is the socio-economic impact of KDP on

household welfare?


b. What are the social and economic benefits ofthe project’s activities?

2.Technical Quality of Infrastructure and Social Subprojectsa. Is the technical quality of the infrastructure and

social subprojects sound? b. What are the returns on public infrastructure

(KDP vs. non-KDP)?c. What is the cost-effectiveness?d. What are the factors that affected technical

quality?e. Is the infrastructure sustainable? What

actions have been taken to ensure the project’soperations and maintenance?

3. Economic Loan Activitiesa. The economic loans are used for what purpos-

es? Are these economic activities profitable?b. Who benefits from the loan activities (gender,

socio-economic status, etc.)c. What are the repayment rates and the reasons

for high or low repayment?d. Are funds revolving?

4. Institution Strengtheninga. Are the village and kecamatan-level councils

functioning and are they being strengthened?b. What other social organizations exist in the vil-

lage? Who are the members? What roles didthey play in KDP?

5. Community EmpowermentGeneral:a. How do villagers feel about KDP? Is there a

sense of ownership of projects and processes?b. Are the projects that are chosen those that the

villagers want?c. Are there any major problems with the project,

and if so, how are they addressed or solved?d. How do villagers feel about the technical assis-

tance provided?e. Are any of the mechanisms, principles, or prac-

tices used under KDP being replicated in otherdevelopment projects?

f. What impact, if any, does KDP have on localgovernance?

Participation:a. Are villagers actively participating in all stages

of the project?b. Who participates? Who does not? Why? Are

disenfranchised groups, women, poor, partici-pating in all stages?

c. How are decisions being made? Are decision oragreements carried out? If not, why?

d. Do villagers feel as if they benefit from theproject? Who benefits most from the project?In what way?

e. Do villagers feel ownership of the project, com-pared to other projects?

Information Flow and Transparency:a. How is information disseminated and how

often? What issues are confusing?b. What are the best practices for widespread and

transparent information dissemination and edu-cation in specific areas?

c. What is the community’s knowledge andawareness of the general project and specificissues, i.e., budget, payments made, roles andresponsibilities?

6. Project Management Information a. Are the main project outputs being achieved?b. Is information on inputs and outputs being col-

lected and used for management purposes?c. Is the project being monitored and evaluated

effectively and are the results being fed effec-tively into management decisions?

Indicators

Once objectives, outputs, and key research areasare clearly defined, indicators must be developedto measure progress. An indicator serves as a tar-get or an explicit measure of expected results. It isimportant that indicators are clearly defined,gender-disaggregated where appropriate, andrealistic. The development and finalization of thedata points and indicators is usually one of themost difficult phases of establishing the M&Esystem. Managers often have a tendency to col-lect a wide range of information without a clear


idea of why that data is needed, what the purposeis, and how it will be used. Sometimes indicatorsare difficult to quantify (e.g., increased trust ofthe villagers toward government agencies) andproxy indicators must be put in place.Pinpointing the final indicators is usually themost lengthy phase of developing the framework.

The performance indicators listed in the loanagreement (Table 2) were the minimum number ofindicators for reporting. In practical terms, addi-tional indicators for management and reportingpurposes are required and those indicators shouldbe included in the overall M&E plan.

Means of verification and data gathering tools

Once indicators are determined, the project mustdetermine how those pieces of information will becaptured. If indicators are impossible or not feasibleto measure, they are eliminated. For KDP, many ofthe input and output indicators relied on the internalmonitoring system of the project and monthlyreporting by KDP’s 1,600 field consultants.However, there were other indicators that requiredmore technical specialized skills or independentassessment, such as impact surveys, cost benefitanalyses, and financial audits.

The KDP monitoring and evaluation componentswere designed to cross-validate and triangulate infor-mation from various sources. The project benefitedfrom a range of perspectives from different sources.For example, KDP field consultants might reportthat there was active community participation inproject activities and meetings, while NGOs judgethe activities differently based upon their ownresearch and field interviews. External monitoring byoutside parties such as NGOs and journalists wasused to complement perspectives from project imple-menters as well as provide more thorough analysis ofcases and events.

Also, the integration of quantitative and qualitativemethods allowed the program to reach some moregeneralizable conclusions as well as understand in

greater depth the reasons or dynamics behind certainoutcomes. The quantitative household surveys pro-vided useful statistically representative information,while case studies helped interpret some of thesefindings and provide greater insight into the dynam-ics and reasons behind the statistics. The qualitativework provided a richer and more nuanced under-standing of the project landscape. In combining thesetwo methods, the program was able to meet more ofits information needs and gain a broader perspectiveof the progress and impact KDP was making.

As the program progressed, the M&E plan was periodically refined and altered, as, over time, someissues or outputs were stressed more than others orreporting requirements changed. Also, variousresearch questions or hypotheses arose over thecourse of implementation as field experience anddata unearthed interesting areas worth researching.For example, communities’ abilities to manage conflict and undertake dispute resolution became arich area of analysis. The role of women’s groups andfemale leaders in the KDP process was another areaof analysis and the subject of several case studiesleading ultimately to a separate project under theKDP umbrella.

The following sections describe the components ofKDP’s monitoring and evaluation system. In total,11 activities were developed over the four-year pro-gram. Table 3 illustrates the principles and functionsof the M&E system and how the mechanisms wereused in a complementary fashion to provide a com-prehensive picture of KDP.

M&E components

Internal Monitoring:• Reporting by government officials and field

consultants• Community participatory monitoring• Case studies and documentation of lessons • Financial supervision and training• Handling complaints and grievance procedures.

External Monitoring:• NGO independent monitoring



Table 2: KDP Loan Agreement Performance IndicatorsTarget set for Loan Agreement Actual in Yr 3 for Yr 3 Difference for Yr 3/ Comments

I. InputsNumber of project kecamatan 725 986 +261 kecamatan

+304, if East Timor is included, KDP has worked in 1,029 kecamatan since 1998

Number of villages with subprojects 6,000 15,000 +9,000

Grants and subloans disbursed Rps. 450 Rps 1,600 + Rps 1,150 billion(Rp billion) billion billion

II. OutputsNumber of subproject agreements >6,000 15,000 +9,000

Percentage of agreed work completed >80% 95% +15

Percentage of sites visited by >10% 100% +90% , All villages are visited by supervision staff government and project staff

Recommendations from studies received Yes Yes Studies and evaluations completed in 2001-2002. Recommendations incorporated into program design.

III. Impacts

1. Sub-project impacts

Benefits/returns >20% 60% +40%, Weighted average taken from sample of KDP infrastructure

Number of beneficiaries 3 million Est. + 32 million35 million

Sub-loans; % repaid by schedule >80% 45% -35%

2. Poverty assessments using X Yes Power and sample size feasibility Susenas household data study completed in 2001. Economic

impact assessment to be completedmid-2003 after last round of Susenas data becomes available

3. Governance

% of infrastructure subprojects with O&M >50% 86% +36%committees formed

In kecamatan with sub-loans

% of UPKs operative >50% 100% +50%

% of revolving funds with increased capital >80% Survey underway at beginning of 2003 to ascertain final amount for KDP1.

Audits (kecamatan sample) 2.5% 30% +27.5%

Independent monitoring

Number of locations providing 20 240 +220 (distributed through NGOs)

information on kecamatan list

Number of trips by journalists to monitor villages 20 >250 >+230

Agreed study recommendations Yes Yes Studies completed

Source: Schedule 6 of KDP Loan Agreement and KDP Phase One: Final ReportNote: Figures for actuals and differences were reported in KDP Phase One: Final Report

• Independent journalists’ monitoring

Evaluation:• Impact evaluation study• Technical infrastructure and economic activities

evaluations• Audits and financial reviews• World Bank supervision missions

5. COMPONENTS OF THE MONITORING SYSTEM

5.1 Field monitoring and reporting bygovernment officials and consultants

KDP is a government of Indonesia program. KDPfunds are public funds, and government authoritieshave a responsibility to ensure that the program pro-ceeds according to its principles and procedures andthat funds are used appropriately and effectively.Government officials at all administrative levels—national, provincial, district, kecamatan, and village—are responsible for monitoring KDP. One weaknessof the reporting system however, has been that


Box 1. Steps in developing the monitoring and evaluation framework and plan

1. Review all the project documentation. Understand fully the project background and design,especially its objectives, outputs, and project cycle.

2. Meet with key stakeholders such as the government, implementing agencies, fieldworkers,and World Bank, to find out what important information they need from the program andwhat research questions are interesting to them. This can be done through key informantinterviews, focus groups, or workshops.

3. Decide on the key research topics and hypotheses the project wants answered by the end ofthe project. Determine the key questions and issues that the M&E system should answer.

4. Determine the indicators for the project based on the inputs, outputs, objectives, andresearch questions mentioned above. Check that all the indicators are measurable, relevantand gender-disaggregated when possible.

5. Determine means of verification or tools to collect the information for the indicators. Ensurethat these tools are realistic given the country context, budget, and human resource capaci-ties.

6. Decide who is responsible for collecting that information.7. Determine the approximate time frame for collecting the information.8. Develop rough budget approximations for each activity.9. Ensure that there are ways for monitoring and evaluation results to feed into management

channels to improve the program. Link the M&E information to decision-making forums anddevelop regular reporting flows.

10. Review periodically the M&E plan, perhaps every six months or year, to assess what is rele-vant or if additional research areas need to be added or taken out. One of the most commonmistakes made is when managers ask to collect a large volume of information but it’s notclear why that information is needed or what purpose it would serve. If the information is notused, it may be wiser to stop collecting the information.

reporting to the national level occurred only sporadi-cally, usually if there was a problem in the field orspecific cases required follow up action. This lack-adaisical reporting was a program weakness thatneeds to be corrected in the next phase. For KDPPhase 2 there will be a greater institutionalized mon-itoring role for district parliaments, governmentcoordination teams, and local village councils. Thesegroups will receive specialized training in monitoringand reporting.

Most of the reporting for the program was donethrough the monthly reporting of KDP consultantsat the national, provincial, district, and kecamatanlevels. The consultants were responsible for writingmonthly reports. Using the M&E plan and key indi-cators as a reference, the M&E Unit of the NMCdeveloped standard reporting forms and field testedthem in several provinces before finalization. Theforms incorporated critical performance and man-agement indicators (Annex C, KDP M&EFramework, Means of Verification). The programoriginally started with some 90 forms but by Year 3the forms were revised and reduced to 64, includingadministrative, financial, and narrative formats. The

intention was to reduce forms and reporting to theminimum amount needed so as not to overburdenthe field staff.

The monthly reporting formats varied dependingupon the field position, but they generally coveredthe following topics:• Activities completed during the past month and

plans for the following month• Status of activities in the project cycle• Description of infrastructure, economic and social

activities• Financial information• Level of community participation in activities• Information dissemination and transparency• Training and activities with village councils, keca-

matan financial units (UPKs), and other localinstitutions

• Coordination work with government coordinationteams, parliaments, and other government agen-cies

• Coordination with NGOs, journalists, or othercivil society groups

• Problems encountered and proposed solutions• Any other issues


Table 3. Principles, Functions and Mechanisms of the KDP M&E System

M&E System Principles and Functions Mechanisms

To check regularly project progress through Examples:the routine and systematic collection of • Consultant & government field reports/MISinformation. • Financial supervision

• Monthly independent reporting from NGOs & journalists• Complaint handling mechanism• Case studies

To assess the impact of the program • 4,600 household impact survey• Case studies• Audits• Bi-annual World Bank supervision missions

To gain a variety of perspectives from different • Monitoring by implementers, such as consultants, sources communities.

• Monitoring by independent actors such as NGOs, journalists• Participation of other donors and NGOs in World Bank

Supervision Missions

To use quantitative and qualitative methods • Consultant reports, MIS and household surveys provided to complement research findings quantitative data

• The quantitative reports were complemented by qualitative work in the form of case studies and descriptive reports from NGOs, journalists, complaints & grievances

• Attachments: reporting forms, photos, complaintsreceived, etc.

Field staff sent their monthly reports directly to theircompanies for forwarding to the Ministry of HomeAffairs (PMD), with copies to their local field super-visors and the NMC in Jakarta, which was responsi-ble for technical supervision of field consultants. Theregional coordinators at the NMC reviewed thereports and quantitative data was entered into anACCESS Management Information System inJakarta. The NMC and PMD used the monthlyreports to assess progress in the project cycle, moni-tor financial disbursements, and keep track of prob-lems arising in the field. The NMC also encouragedfield staff to review their progress reports at themonthly provincial and district level coordinationmeetings.

Any reporting system dealing with over 1,600 con-sultant reports each month is daunting under thebest of circumstances. While the reporting systemfunctioned and the reports provided very usefulquantitative and qualitative information on progress,there were also several difficulties:• Consultant reporting was sometimes sporadic,

incomplete or late, especially from remoteprovinces. Also, writing skills were often lacking,and some reports were difficult to understand.The central office MIS unit spent considerabletime at the end of each month or project year col-lecting outstanding reports from field consultants.By mid-project, KDP had instituted policies tyingsalaries to reporting, which resulted in a 30 to 40percent increase in reporting overall (with wideprovincial variations). By the end of the program,approximately 90 percent of field consultants werereporting regularly, but this required substantialreinforcement and follow-up.

• Although the overall political climate was moreopen and freedom of speech had generallyincreased since 1998, it was sometimes difficult toencourage whistle blowing and urge consultants orcommunity members to raise complaints or prob-lems openly. Indonesia had a long history ofintimidation and retribution and someIndonesians felt that it was futile reporting prob-

lems so they remained silent. Facilitators some-times experienced physical intimidation in thefield from village elites or other actors. Moreover,some consultants feared that implementationproblems would be seen as a reflection of theirown poor work performance, and tried to hideproblems. Better communications between thenational level and the field helped to improve thisnegative environment somewhat by Year 3.However, project managers need to ensure moreopen reporting and communication channels dur-ing the next phase.

• Entering project data centrally into the MIS sys-tem in Jakarta has meant tighter control over dataquality but placed a heavy burden on MIS staff atthe central level to clean the data and follow upmissing reports and incomplete information.Centralized data management has also meant thatprovincial and district consultants lack directresponsibility for managing the data (they receivedcopies of the reports) and have not made optimaluse of it for their own project management. UnderKDP2 the MIS function will be decentralized tothe regions.

• KDP had great difficulty controlling the economicloan portion of the portfolio. Throughout the pro-gram, repayment rates were low, averaging about40 percent. Accounting systems for the loans wererudimentary and depended for the most part onthe experience local groups already had in manag-ing microfinances. Financial units at the keca-matan level (UPKs) found it difficult at times tocomplete forms for loan reporting. The microfi-nance portion, including the reporting forms, hasbeen completely redesigned for KDP2.

It is anticipated that with the decentralization of theMIS for KDP’s next phase, more effective training,and stronger supervision and enforcement measures,reporting should improve.

5.2 Community ParticipatoryMonitoring

The best type of monitoring is monitoring conduct-ed by communities themselves. Communities takeownership of the process when they are responsible


for monitoring project activities. This promotes par-ticipant learning about the program and its perform-ance, and enhances understanding of otherstakeholders’ points of view. It also increases the like-lihood that evaluation information will be used toimprove project performance.

Monitoring and evaluation by outside parties oftenfocus on issues important to donors and imple-menters; but communities may have other issues thatthey value and wish to monitor. Community partici-patory monitoring allows the communities tobecome the question-makers, the collectors of infor-mation, and, ultimately, the end users.

Over the years, KDP encouraged different kinds ofcommunity participatory monitoring:• Monitoring by village councils (BPDs): Laws for

the election of village councils were passed in1999. Prior to this, village chiefs generallyappointed the village assemblies (LMDs) and vil-lage community resilience boards (LKMDs) fromvillage elites. By the middle of KDP however, vil-lagers were democratically electing their represen-tative councils in many Indonesian villages. BPDshave responsibility for monitoring KDP activitiesat all stages: socialization, planning, implementa-tion, and maintenance. The BPDs select membersto monitor each phase. Results of this monitoringare then discussed at council meetings or fed intolarger village meetings.

• Monitoring by special community groups orteams: KDP encouraged community monitoringgroups in each village during Years 2 and 3.Communities were encouraged to form specialteams or groups at the community level duringthe village meetings to monitor KDP. These com-munity monitoring groups were independent ofthe village implementation teams. The communityteam members shared responsibilities for checkingfinancial accounts, monitoring bank transactionsand material purchases or rentals, visiting suppliersto confirm the costs of goods, and monitoringsubproject activities, including infrastructure con-struction.

• Community participatory monitoring facilitatedby NGOs: In several provinces, including Aceh

and East Java, NGOs involved in province-basedmonitoring helped a number of villages, specialgroups, and teams conduct community participa-tory monitoring. They helped villagers decidewhat questions were important to them aboutKDP, how to collect data to answer those ques-tions, and helped villagers analyze communityfindings. NGO facilitation has been a successstory in several locations, and the activity will beexpanded to every province under KDP2.

5.3 Case Studies and Documentation of Lessons

Because of KDP’s large scale and ambitious goals forimproving local governance, it was important thatthe program research and carefully document fieldresults. The NMC had two full-time staff membersresponsible for conducting research and writing casestudies on KDP, assisted by other NMC staff forlarger or more technical case studies. The case studieswere important monitoring and communicationtools because they provided rich descriptive informa-tion and analyses on multiple themes related to KDPimplementation. The case study work provided qual-itative in-depth information to help explain some ofthe findings from the quantitative surveys andmonthly reporting data.

Case studies began with NMC M&E staff meetingsto discuss possible topics or themes, which usuallyarose from field reports, supervision missions, or gen-eral feedback from KDP staff. The case study staffcompiled a list of topics and developed rough actionplans and schedules, then contacted regional man-agers and provincial coordinators to brainstorm pos-sible locations to research the topics. In general, thecase studies covered at least three different provinces.Once locations were decided, staff developed generalresearch plans, including question guides pertainingand list of persons to interview in the field. In thefield, they used a combination of methodologies tocollect information, including: direct observation;focus group discussions; key informant interviews;mini-surveys, and rapid appraisals. The staff adjustedresearch plans and question guides as needed. Casestudies averaged approximately two to three months


of fieldwork followed by a month of analyzing dataand reporting findings.

Except for a more lengthy study on KDP’s impacton local governance, the case studies were purposelybrief and limited to eight pages so that they were nottoo “heavy” for the KDP audience. The idea was tocapture lessons from different project areas and sharethat information widely with project stakeholders.

Language was kept simple so that village communi-ties could understand the stories.

Under KDP1, the program published and distrib-uted eight case studies on:• Lessons from KDP Year 1• Participation of Women and the Poor• Conflict Resolution• Information Dissemination and the Role of the

Village Facilitator


Box 2. Steps in Conducting Participatory Community Monitoring

NMC training for NGOs assisting with independent monitoring included a module on community participatory monitoring and the steps to begin the process in KDP communities:1. Call a general meeting to ask who would be interested in participating in the monitoring exer-

cise. It is important to check if villagers actually have the time to participate in this activity.2. Once the community decides who wishes to join the participatory monitoring, the facilitator

meets with the group at a time convenient to them.3. The facilitator builds consensus on the purpose of the exercise, clarifies his/her role, expecta-

tions, and reviews the schedule.4. Community members then identify the monitoring and evaluation questions they want

answered related to the KDP activity. Questions come solely from the community; there areno predetermined questions or forms from the facilitators. The group then brainstorms onways to collect the data and who is responsible for data collection.

5. Once the data is gathered, the group meets again to analyze it and reach consensus on find-ings, conclusions, and recommendations.

6. The group then reports to the larger community and KDP field consultants and together theyprepare a plan of action to improve performance if needed.

In Aceh Province the PUGAR NGO provided independent KDP monitoring and facilitated com-munity participatory monitoring in four villages. In Lok Nga Village, for example, PUGAR askedthe KDP Kecamatan facilitator and village leaders to call a general meeting where PUGARexplained its purpose and the benefits of monitoring. At the meeting, the villagers appointedseven members for the community participatory monitoring team representing women, youth,the poor, and community leaders. The team decided on each member’s responsibilities and thenchose their leader, a woman named Ibu Mardiana. PUGAR met with the community monitoringteam each month to review progress. The monitoring group reported back to the village regular-ly on project progress.

“I encouraged women to take part in the community participatory monitoring and now they real-ize how useful it is,” Ibu Mardiana says. “To get a good quality road, I myself sat on top of theroller machine and watched the workers work.”

Another female member of the group adds: “We have a very good quality KDP road and it isvery cost effective as well. We controlled the quality of the materials by checking it when it wasdelivered by the suppliers and we did this every time.”

• Community Participatory Monitoring• Women Leaders in KDP• Infrastructure Best Practices under KDP• KDP’s Impact on Local Governance in Nine

Villages

Each case study was sent to 3,200 consultants, gov-ernment officials, local parliaments, journalists, andNGOs at the national level and in the field. By read-ing these case studies, it was hoped that programstakeholders, especially consultants, would learn fromthe experiences in other KDP areas.

Results of a June 2001 survey questionnaire request-ing feedback on the case studies indicated that read-ers found the studies to be very informative anduseful reference guides, especially for providing infor-mation on problems in other provinces and how theywere solved. The case studies also gave them a richerunderstanding of how rules and procedures were putinto practice in the field. Respondents also found thesimple language of the case studies easy to under-stand. KDP will continue these case studies in thenext phase.

5.4 NMC Financial Supervision andTraining Unit

Financial monitoring was an important part of theprogram’s overall monitoring activities. GivenIndonesia’s endemic corruption, it was important tosupervise financial transactions and procurementclosely, especially at the kecamatan and village levels.Community participatory monitoring played a cru-cial role in overseeing village finances and procure-ment but there was also a need for the program toprovide its own financial supervision and developcommunity capacity for financial management.

The NMC established a five-person FinancialSupervision and Training Unit in May 2001 toimprove the financial skills of the kecamatan finan-cial units (UPKs), village boards (LKMDs), and eco-nomic loan groups, and check their financial andadministrative records at the kecamatan and villagelevels. The unit served a dual function of providingmuch needed on-the-job training and mentoring to

community financial units as well as supervising andauditing KDP finances at the local level. (For Termsof Reference for this Unit, see Annex G)

The unit was not established as only an internalaudit unit because project management felt that itwould be more beneficial for UPKs, LKMDs, andeconomic loan groups to be mentored in financialskills. Auditors would have focused chiefly on point-ing out problems rather than concentrating on waysto improve skills. Thus, the unit was conceived as agroup of “roving accountants” who would reviewfinancial records, but also provide much-neededfinancial training and mentoring.

Despite initial delays in recruiting staff and finalizingthe unit’s scope of work, it accomplished a great dealin a little over a year and strengthened KDP’s finan-cial oversight function. The unit audited and trainedfinancial and program staff in 13 of the 22 KDPprovinces and 140 kecamatan. It visited an average of36 percent of the KDP kecamatan in each province,sampling two or three villages in each kecamatan.The team also played an important supportive role informal World Bank supervision missions.

The unit’s findings varied from province to province.In provinces such as Central and East Java, financialmanagement was, on the whole, fairly sound, withregular consultant supervision and solid bookkeepingby the UPKs and LKMDs. In other provinces, suchas North Sumatra and South Sulawesi, there weresystematic problems related to misuse of funds, lackof financial oversight from district and kecamatanconsultants, and lack of documentation for revolvingfunds. After each trip, the unit presented its findingsto local consultants and government authorities, andthe national level. Problems uncovered were sent tothe complaints handling unit’s consultants and gov-ernment for follow up as part of overall KDP pro-gram management. For example, in North Sumatraand South Sulawesi, the unit’s report led to the tem-porary suspension of project financing until the iden-tified problems were corrected. The unit alsoprovided follow-up financial training in problemprovinces.


The NMC also carried out two confidential surveysusing field consultants to ascertain if consultantswere being paid according to their contracts so thatthe program manager could take action on non-performing or corrupt consultant companies. Thesurveys found that several companies were holdingback on transportation and other allowances to consultants. PMD notified and warned the compa-nies but eventually four of the consulting firms were terminated.

5.5 Handling Complaints andGrievance Procedures

KDP established a complaints resolution process toensure that communities had a forum or outlet to aircomplaints, and resolve problems. Complaints andquestions about KDP could be directed to any KDPfield consultant or government official. Complaintscould also be sent to a special post office box inJakarta and the KDP national project secretariatoffice. Project brochures and posters included infor-mation on where to send complaints. Five staff at theNMC, the Handling Complaints Unit, were respon-sible for ensuring that complaints and questions wereanswered promptly and investigated further in thefield. The unit coordinated with KDP field managersand local government officials to ensure prompt fol-low-up action in response to complaints. KDP com-plaints procedures are illustrated in Figure 1.

There were four general categories of complaints orproblems from the field: those regarding the KDPprocess and procedures; misallocation of KDP funds;

inappropriate intervention from government officialsor KDP consultants, and unforeseeable events suchas natural disasters (force majeure) cases (Table 4).KDP received the largest number from KDP fieldconsultants or NMC staff who visited the field. Theregular monthly reports from field consultants andfield reports from NMC staff visits provided usefulinformation on problems in the field and what need-ed to be followed up by the proper parties. Letterswere also sent in by community members, NGOs,and anonymously. The unit also tracked the newspa-per reports from AJI reporters (see section onAJI/LP3ES journalist monitoring) and ensured thatproblems described in those reports were communi-cated to field staff and government officials for fol-low-up. Regular supervision missions by the WorldBank, NMC, and PMD were also useful sources ofinformation on field problems. The HandlingComplaints Unit, with assistance from MIS staff,entered complaint and grievance information into acentral ACCESS database in Jakarta. (See Annex Ffor Complaints and Grievances database fields).

As of July 2002, 1,900 problems or complaints hadbeen reported—an average of 45 per month—ofwhich 49 percent had been resolved and 51 percentwere still in process (Table 4). The most commonproblems or complaints related to violations of KDPprinciples or procedures (52 percent) followed bymishandling of funds (38 percent) and force majeure(5 percent). Force majeure cases were tracked in casefollow up actions were necessary or funds must berecorded. Some 77 percent of the problems or caseswere reported by consultants followed by NGOs (7


Table 4. Summary of Complaints Received under KDP1 (January 1999-July 2002)Types of Problem or Complaints Number % Resolved Still in Process

Violation of Principles or Procedures 987 52% 499 488

Misuse of Funds 724 38% 304 420

Intervention 75 4% 51 24

Unforeseeable events (Force Majeure) 102 5% 58 44

Other 21 1% 17 4

TOTAL 1,909 100% 929 980


Figure 1. KDP Grievance and Redress Procedures

This diagram was adapted and revised from an original complaints procedures diagram drafted byEnurlaela Hasanah and Sadwanto, NMC in 1998.

percent), community letters or reports (5 percent)and AJI or other journalists (2 percent). WhileNGOs and media accounted for only 180 cases or 9percent of the total number of complaints received,these civil society groups usually played an importantrole in following up and monitoring cases. For exam-ple, once grievances such as misuse of funds werereported to KDP, AJI journalists followed the storiesand reported upon them as the cases unfolded.

As Table 4 shows, it was much more difficult toresolve cases related to the misuse of funds and cor-ruption than those involving procedural violations orintervention by outside parties. Among the difficul-ties faced in resolving problems in general were:• Consultants were sometimes not creative enough

in seeking alternative solutions or persistent insolving problems. Better training and clearlydefined procedures are needed;

• Consultants had differing perceptions of problemsor how to resolve them.

• Consultants sometimes did not want to reportproblems due to physical intimidation or fearedthey would be blamed for the problems.

• In some cases, there was insufficient support fromgovernment officials to solve problems; oftentimesgovernment officials were reluctant to imposesanctions.

• Corruption cases took an especially long time toresolve. Many times, those that stole moneycould—or would—not return funds. Given theweakness of law enforcement and judicial process-es in Indonesia, the prosecution of criminal casesis lengthy. Also, the police and judicial system donot give ample attention to the poor and their

cases. Only five KDP corruption cases went tocourt. These cases were in Lampung, Central Java,North Sumatra, and two in Nusa Tenggara Timor(NTT). As of June 2002, three cases in CentralJava, Lampung, and NTT were finished and theculprits were prosecuted successfully. Two cases, inNTT and North Sumatra, were still in trial.While these court cases were few and far between,the “victories” were widely publicized and sent astrong signal to the public that corruption wouldbe punished in KDP.

Under KDP 2, there will be more training for con-sultants on dispute resolution. Also, the HandlingComplaints function will be decentralized so thatthere will be a complaints officer in each region,which should improve communications with provin-cial and district government coordination teams andresolve grievances more expeditiously.

In response to several corruption and legal disputesarising in KDP areas, the program initiated in 2001 apilot Legal Assistance Project for KDP communitiesin North Sumatra and Central Java. The project pro-vided legal assistance to KDP communities throughNGOs and university legal aid organizations. Theproject has since expanded into a “Justice for thePoor” pilot activity on sub-national legal reforms.

On the positive side, KDP has a wide range of actorsto help push the agenda forward. The most impor-tant actors were the communities themselves whowere usually consulted immediately regarding report-ed problems. Consultants and government, as well as NGOs and the media also assisted in problemresolution.


Table 5. Actors/Sources of Information for Complaints Source of Information Number of Complaints % of Total Complaints

Consultants 1,468 77 %

Monitoring NGOs 140 7 %

Community members 99 5 %

Mass Media (journalists) 41 2 %

Other 161 8 %

TOTAL 1,909 100%

Source: KDP NMC Complaints Handling Unit database

External monitoring by independent organizationsThe program defined external monitoring as the col-lection of project information and data by outsideparties or non-implementers. KDP included externalmonitoring activities so the program could benefitfrom a wide variety of perspectives from independentparties who may have more objective viewpointsthan project implementers. KDP could then crosscheck the information from external monitoringwith its own internal monitoring reports.

The activities of civil society organizations such asNGOs and media were suppressed until the down-fall of Suharto in 1998, when the new governmentlifted media censorship. Freedom of organizationand the press has flourished ever since to the point

now where there are hundreds of registered NGOsand some 700 newspapers and journals, and themedia and NGOs have formed key pillars in build-ing a more open democracy in Indonesia.

Indonesian civil society groups such as NGOs andjournalists provide independent monitoring for KDP.Feedback from these groups reveals that they valuetheir roles as civil society watchdogs to hold govern-ment and policymakers publicly accountable.

5.6 NGO Independent Province-BasedMonitoring

Many Indonesian NGOs have solid skills in com-munity development and extensive knowledge of


Box 3. Handling complaints successfully in South Sumatra

In September 1999, the KDP Kecamatan Facilitator (FK) of Tulung Selapan Kecamatan in SouthSumatra discovered that a village head had apparently stolen KDP funds in one of the villages.The FK reported the case directly to the KDP district coordinator and the government districtcoordination team, who reported the details to the NMC. The Handling Complaints Unit of theNMC informed PMD at the national level about the case and confirmed the details with theprovincial and district authorities in South Sumatra. In the meantime, the district coordinatorand the district coordination team discussed the case and decided upon a course of action. Ameeting was held with the village head to clarify the details of the case, and it was confirmedthat the village head had taken approximately Rps 8 million ($1,000) of KDP funds for personaluse. At the meeting, the village head was asked to return all the funds immediately, and helater returned about half the funds, Rps. 4.5 million, promising to repay the balance as soon ashe could.

The outcome of the meeting was reported to the NMC Complaints Unit, which tracked the casefor the next several weeks and wrote a letter in October, suggesting that another village “auditmeeting” be held to review the case. The meeting was attended by the village head, the villageresilience board (LKMD) and village elders and the village head was interrogated again. At theend of this meeting, the community leaders concluded that the village head should be fired. Hewas relieved of his duties, but the local leaders, KDP consultants and NMC continued to pressfor him to return the stolen funds. Every two weeks, the NMC Complaints Unit phoned the dis-trict coordinator to check on progress and urge local officials to resolve the case. Finally inDecember 1999, the village board together with some of the village elders met formally withthe ex-village head and demanded that he return the debt by the following month or the casewould be sent to the police authorities for further action. He paid back the remaining amount inJanuary 2000.

local conditions. In many cases, NGOs are able toamplify the voices of communities so that their con-cerns are heard. To take advantage of these skills andincrease the involvement of civil society organiza-tions in the program, KDP contracted NGOs ineach province to monitor and report monthly onKDP progress. The NGO Independent Province-Based Monitoring activity began in July 2000. Toensure transparency and wide information dissemi-nation, KDP sent out terms of reference and requestsfor applications to hundreds of provincial NGO foraand NGOs recommended by other donors. Some 27provincial NGOs, including three provincial NGOconsortia, were selected competitively out of 210NGO applicants to monitor KDP in each province.In 2001, the NGO monitoring activity expandedand to date 40 NGOs monitor the program in all 22KDP provinces.

The NGOs were responsible for monitoring (specificterms of reference for NGO independent monitor-ing are in Annex D):• The participation of villagers, especially women

and the poor, in all phases of the program• Information sharing and transparency• Adherence with the general principles and proce-

dures of the program• Financial reviews• Quality of technical assistance• Quality of program outputs • Recommendations from program stakeholders on

how to improve KDP.

The NGOs provided oral and written reports at themonthly coordination meetings for consultants andprovincial and district government coordinationteams. These reports included descriptions of prob-lems encountered and recommendations for correc-tive action. The monthly meetings were important inmaking the NGO findings part of the program’soverall management and problem-solving at the locallevel. While there were some provinces with weakprovincial coordination teams and poor NGO coor-dination, the majority of provinces incorporated theNGO reports and recommendations into theirmonthly meetings and action plans.

The KDP M&E Unit conducted four initial trainingsessions for NGOs to provide an orientation towardKDP, review NGO terms of reference for independ-ent monitoring, discuss data collection methodolo-gies and monitoring techniques, and develop workplans. (See sample NGO orientation workshopschedule in Annex E) These initial trainings werecritical to ensure that NGOs were properly briefedabout KDP’s principles and procedures and theirexpected roles and responsibilities under the con-tracts. Over the last two years, KDP also sponsoredworkshops in Surabaya and Manado that broughtNGOs together to reflect on their experiences in thefield, best practices and lessons learned, technical andfinancial monitoring, and recommendations for pro-gram improvement.

The NGO monitoring activity has been valuable.First, the NGO monitoring provided an external,independent analysis of KDP progress in eachprovince. NGO findings were cross-checked and val-idated against those of KDP’s internal monitoringsystem. The NGOs uncovered some 140 cases ofanomalies. Also, some villages reported that they feltmore comfortable confiding in NGOs than imple-menters. Lastly, NGO reports indicated that NGOstaff improved their monitoring skills and the KDPactivity enhanced their ability to monitor public useof funds and government development programs.Evaluations have, however, indicated several areasthat need improvement. NGOs should have a deeperunderstanding of how KDP works and its proce-dures, their reports were sometimes late or unclear inthe presentation of results, and coordination betweenNGOs, consultants, and provincial coordinationteams could be improved.

One important management consideration in under-taking such an exercise was that contracting NGOsindividually created enormous contract managementburdens on the government. The KDP NationalSecretariat’s contract management unit managed 61service contracts for the entire program, of which 28were individual NGO contracts. These figures donot include contract amendments. One alternativewould have been to contract only one NGO networkor membership association but that option was


rejected because several provincial NGOs did notwish to use intermediary forums or associations.Also, there was some resentment that the largerJakarta-based NGO forums were monopolizingdonor funding. Many of the provincial NGOs pre-ferred contracting directly with the government.

KDP 2 will continue and expand this NGO monitoring activity. Other NGOs will have anopportunity to participate, as NGO monitoring contracts will be re-tendered for the next phase.More emphasis will be placed on community participatory monitoring.

5.7 Independent Monitoring byJournalists

In January 1999 the Ministry of Home Affairssigned a contract with the Association ofIndependent Journalists (AJI) and the Institute forSocial and Economic Research, Education andInformation (LP3ES) to provide systematic externalmonitoring and reporting on KDP. This activity wasdeemed important enough to be included as a provi-sion in the original KDP Loan Agreement betweenthe government and World Bank.

The purpose was for provincial journalists to provide

independent monitoring of KDP and publish orbroadcast their findings. An important principle forthis monitoring activity to succeed was that therewas no prior review or censorship of the stories.Secondly, the journalists did not receive any honorar-ia, only reimbursement for travel expenses to KDPsites. Thirty-one journalists (26 print media and fivefrom radio) from 20 KDP provinces visited projectareas and broadcast radio stories or wrote articlesmonthly in leading newspapers. AJI selected thejournalists based on membership in its network orbecause they represented the leading newspapers orradio stations in a province. In general, the AJIreports fell into three categories of articles:• General KDP information and orientation pieces• Feature stories• Investigative reporting (for example, reports on

corruption or the misuse of funds)

By the end of KDP1, the provincial journalists hadwritten or broadcast some 850 stories and visitedover 250 villages. The KDP Complaints Unitreceived the monthly journal articles or radio tapesand followed up on any reported problems.AJI/LP3ES also published ten editions of a newslet-ter entitled “Transparan,” which printed selectedKDP-related stories.


Box 4. Local NGO reporting mechanisms

The Indonesian NGO SPEKTRA undertakes independent monitoring for KDP in East Java.According to Abdul Azis, SPEKTRA’s executive director and NGO coordinator: “During themonthly coordination meetings at the district and provincial levels, we report field findings tothe consultants and government coordination team for feedback. The KDP provincial and dis-trict consultants are generally responsive and follow up on issues raised by us. They will cross-check, clarify, and report back at the next coordination meeting. Then if it is necessary for thegovernment side to take action, they will.”

He recalls a recent case dealing with corruption. “Someone from the local treasury office(KPKN) asked money from villages and this was discovered during our regular NGO monitor-ing. Together with the KDP district consultant, we reported this case during a coordinationmeeting at the district level. The head of planning was so angry that he immediately made aphone call, then the next day, the money was returned to the villages.”

Annual evaluations and a 2002 end-of-contract eval-uation revealed some of the strengths and weakness-es of the journalist monitoring activity:

Strengths:• Journalists were able to adhere to the principles of

independent, unbiased journalism in coveringKDP activities and were not subject to outsideintervention or censorship that would affect theirreporting.

• AJI reported that the KDP activity improved theirmembers’ reporting skills and practical knowledgeof transparency and good governance. Theyunderstood more fully the free media’s role as apublic watchdog. AJI believed that other journal-ists in Indonesia could learn from its experienceand took the initiative to publish a book entitled,“KDP as a Monitoring Model for a DevelopmentProgram”.

• In several provinces, such as Yogyakarta andLampung, the newspaper articles had a directimpact on policy makers by pressuring them totake action. Once cases of corruption or misuse offunds were reported, the journalists usually wrotefollow-up stories and kept the cases in the publicspotlight. In a corruption case in Lampung forexample, the local AJI journalist reported closelyon the scandal and trial of a kecamatan govern-ment official accused of stealing KDP funds.Despite repeated threats and intimidation, thejournalist reported consistently on the criminalcase and kept the public spotlight on the casethrough to the end of the trial.

Weaknesses:• The abilities of journalists were limited despite the

fact that the KDP contract paid for orientationand refresher training for the journalists. Therewere problems with clarity in writing, accuracy,balanced reporting, cross checking sources, andunfamiliarity with KDP procedures.

• Some KDP villages are located in remote areas, soit was difficult for the journalists to cover them.

• Journalists did not get out to the field as often asthey should have.

• Investigative reporting by the journalists was lim-

ited. Some journalists did not have the skills to doinvestigative journalism; others could not devotesufficient time to investigate cases because theyhad other reporting responsibilities with theirnewspapers. Other journalists, such as those inAceh, reported problems of intimidation wheninvestigating cases.

• Newspaper articles in general did not reach thevillages, so, KDP villagers often did not see thearticles unless consultants posted them on villageboards.

The AJI activity will continue into KDP 2.Additional training and orientation will be providedto the journalists to give them more extensive KDPprogram orientation and improve their journalisticskills. The radio journalism component will also beexpanded. AJI will also try to link up more withnational media organizations in covering KDP.

6. COMPONENTS OF THE EVALUATION SYSTEM

The KDP evaluation system was designed to meas-ure the overall impact of program interventions. Didthe project achieve its objectives? Were resourcesused cost effectively? How could the program be bet-ter designed to achieve the desired outcomes? Thesewere some of the questions to be answered throughevaluations.

It is important that periodic evaluations are incorpo-rated into the design of projects so that managers canuse the results to adjust or redesign program compo-nents. KDP’s original program design includedimpact studies, including the collection of baselinedata, and technical evaluations to assess programimpact.

6.1 Impact Evaluation Study

In 1999 KDP contracted the Demographic Instituteof the University of Indonesia (LD) to conduct animpact study looking at two major issues: KDP’simpact on household welfare, and its impact on com-munity organization and perceptions.


For the household welfare component, LD conduct-ed an initial feasibility study to determine if it wasactually possible to evaluate quantitatively the impactof KDP on poverty, incomes, and livings standardsgiven the available data. The sample had to be suffi-ciently large to distinguish relatively small differencesin per capita income growth rates between thoseareas benefiting from the program and a set of con-trol areas. Furthermore, the control areas had to besufficiently comparable to ensure that the measureddifferences were really due to the program itself. Thestudy recommended the appropriate sample size andthe use of official socio-economic statistics (Susenas)datasets for 1998-99 and 2001-02 to compare databefore and after KDP 1 interventions, and in pro-gram and control areas.7 The World Bank staff inJakarta will complete the study of the Susenas paneldatasets by mid-2003.

For the community organization component, LDconducted a baseline 4,600-household survey in pro-gram and control areas and followed by anotherpanel survey looking at impact after one project cyclewas completed. The survey followed a quasi-experi-mental design using a double difference methodolo-gy comparing the groups before and after theintervention. LD selected 74 pairs of KDP and non-KDP enumeration areas and held interviews with2,600 household members and 2,050 local leaders inthe seven provinces of North Sumatra, SouthSumatra, West Java, Central Java, East NusaTenggara, Central Kalimantan, and CentralSulawesi.

The identification strategy of selecting program andcontrol areas took advantage of KDP’s staggereddesign in village selection within kecamatan.According to KDP1 procedures, for kecamatan con-taining more than ten villages, only half of the vil-lages were allowed to participate in KDP during thefirst year. The other half of the villages participatedduring the second year, and all villages were eligible

to participate in the third and final year. LD used theyear 1 villages, with the villages in the same keca-matan that did not participate that year as controls.

The survey asked questions related to:• Household socio-economic characteristics• Community participation in the various phases of

KDP (planning, implementation, maintenance,cost-sharing) as well as other development proj-ects

• The role of different groups in the project process(village council, women, poor)

• Information sharing and transparency• Impact of the subproject interventions• Problems encountered and handling of complaints• General level of satisfaction with the project

In some enumeration areas, field researchers followedup the quantitative survey questions with more prob-ing qualitative techniques and questions. Detailedfield notes from the researchers were very helpful inexplaining some of the household survey results anddescribing the details of some problem issues.

Results of the household survey were mixed. KDPinfrastructure investments were very popular andbeneficiaries expressed satisfaction over the results.The perceived benefits of the infrastructure develop-ment were fairly equitably distributed to peopleacross income groups. Compared to control areas,community leaders in the KDP areas had signifi-cantly more positive perceptions regarding theimpact of KDP infrastructure projects on householdincome, access to markets, schools and health facili-ties. Women and the poor in KDP areas were alsomore involved in project activities compared to con-trol areas. Lastly, KDP beneficiaries had greaterknowledge of the role of village councils, kecamatanforums (UDKP), women’s groups and the poor inKDP projects than those in control areas. Villageleaders had an even greater knowledge and positiveperception of the program than ordinary citizens. Onthe negative side, about 10 percent of respondentsraised problems such as the lack of financial trans-parency or the negative impact that some of theinfrastructure projects had on specific individuals,e.g., inadequate compensation for their land.


7"Evaluation of Kecamatan Development Program (KDP)Impacts on Community Organization and Household Welfare:Power and Sample Size Calculation for Feasibility Study,"Demographic Institute, Faculty of Economics, University ofIndonesia, May 2001.

The surveys revealed that economic loans were gen-erally not made to poorer families in the villages.Compared to economic loan programs in non-KDPareas, loans were strongly biased toward the richersegments of villages. Beneficiaries were chosen whowere more likely to pay back their loans. There werealso widespread complaints about the complexity andthe lack of transparency of the economic loanarrangements. Respondents stated that loan termswere not clear, calculations of the interest rate to bepaid were complex, and sometimes the terms of pay-ment and repayment were filled with hidden costs.8

The LD survey provided some useful inputs intofuture program design. First, the economic loanportfolio was redesigned to take into account themany shortfalls identified in the LD survey as well asrecommendations from a 2001 economic evaluationstudy (see following section). The LD findingsregarding economic loans were more or less consis-tent with some of the findings from internal moni-toring reports and other evaluations. Secondly, thesurvey showed the need to emphasize transparencyin all project activities. Improved community out-reach materials will be developed to reach targetaudiences more effectively.

Lastly, KDP is examining closely survey methodolo-gies in order to improve the measurement of pro-gram impact during the next phase. Theidentification and matching strategy present a specialchallenge due to KDP’s selection criteria for projectbeneficiary areas. The problem with the identifica-tion strategy used for the KDP1 LD evaluation wasthat most of the “control villages” were recruited intoKDP the following year. Thus, it was very difficult tomeasure subtle changes in attitudes and behaviorover just a one or two year time frame. KDP is cur-rently examining the feasibility of alternative identifi-cation strategies. In addition, the surveyquestionnaire will be improved to include morequestions on social capital and local governance.

6.2 Infrastructure and Economic Loan Evaluations

There were several program components thatrequired more specialized studies to determineimpact. Many questions regarding cost effectiveness,program benefits, and impact were not answeredthrough the routine consultant reporting system,audits, or periodic supervision missions. KDP there-fore initiated several, sectoral studies in 2001 to eval-uate rural infrastructure and economic loan activities.KDP commissioned a team of outside evaluators toundertake a rural infrastructure study that reviewed:• The technical quality of KDP infrastructure proj-

ects• Maintenance plans• Usage of facilities • Employment generation • Economic rates of return • Community perceptions of, and satisfaction with,

the infrastructure • Cost efficiency• Increases in economic activities, savings, and

access directly related to the KDP-built infrastruc-ture.

The evaluators examined KDP infrastructure in 167villages across 18 KDP provinces. They found thatthe general quality of KDP infrastructure was goodto very good. The study also showed that over 83percent of community respondents were satisfiedwith their KDP infrastructure. Ninety-six percent ofthe respondents stated that the quality of the infra-structure was the same or better than that builtthrough other government programs. KDP infra-structure was also more cost effective than equivalentworks built through other government programs.KDP infrastructure was determined to be as much as23 percent less expensive than similar government-funded infrastructure works. In terms of sustainabili-ty and maintenance, field reports and the technicalevaluation revealed that 86 percent of KDP Year 1infrastructure was being maintained, with 94 percentof maintenance done by village O&M teams orcommunity members.

Economic internal rates of return (EIRR) were alsofound to be relatively high for the main infrastruc-


8See "Evaluation of Kecamatan Development Program (KDP)Impacts on Community Organization and Household Welfare",First and Second Round reports, Demographic Institute, Facultyof Economics, University of Indonesia, May 2001 and July 2002.

ture types (Table 6). Generally, the nature of small-scale rural infrastructure projects is such that themarginal productivity of capital can be very high andonly relatively small injections of capital are neededto release large incremental benefits. KDP infrastruc-ture projects illustrate this point. The EIRR for asmall sample of the main infrastructure types rangedfrom 14.8 to 83.3 percent.9

Economic Loan EvaluationKDP also commissioned a separate evaluation of theeconomic loan sector in September 2001. The studyexamined:• The overall design of the economic activities

portfolio• Repayment rates, including timeliness of repay-

ment and reasons for high or low repayments• Relending of funds • Control and management of loans and their uses• Economic return on the loans• Recipients and beneficiaries of the loans • Impact of KDP loan funds on other sources of

local credit• Borrower perceptions of the loans and economic

activities.

The study’s conclusion was that there were seriousdesign and system flaws and that major improve-ments needed to be made in design, credit manage-ment, training, and administration. It found that

KDP’s credit functions were distributed to many per-sons who did not bear credit risks. The evaluationrecommended that the credit management functionsbe viewed separately from the overall project processand that substantial changes be made to the eco-nomic component design and management. As aresult of this evaluation and similar findings throughKDP’s internal monitoring and reporting, KDPassigned a special task force of micro-credit andfinancial management experts to prepare the newKDP Phase 2 micro-credit component.

The importance of these sectoral evaluations wasthat they provided critical and constructive input thatallowed program management to improve the designand operation of KDP. Both studies have helped toinform the design of KDP’s second phase.

6.3 Audits and Financial Reviews

Given the widespread misuse of funds in manyIndonesian government programs, KDP put in placemechanisms to ensure that there was strong financialoversight of program finances. Along with the NMCFinancial Supervision and Training Unit, the govern-ment audit agency, the Finance and DevelopmentSupervision Agency (BPKP), and the World Bankconducted regular financial inspections and audits ofKDP. BPKP and the NMC Financial Supervisionand Training Unit audited almost 30 percent ofKDP kecamatan in 2000-2001.

BPKPBPKP audited KDP annually. It usually audited 14 to30 percent of all KDP kecamatan. By Year 3, auditsamples were taken from 17 provinces, 61 districts,137 kecamatan and five to eight villages in each keca-matan. BPKP reviewed four aspects of the program:• Project financial statements and their accuracy• Adequacy of project financial management sys-

tems including internal controls• Compliance with provisions of financing agree-

ments, especially those relating to accounting andfinancial matters

• Achievements of planned project results as meas-ured by the performance indicators stipulated inthe financing agreement.


9 “Ex-Post Evaluation of Kecamatan Development ProgramInfrastructure Projects: Final Report”, prepared by GeoffreyDent, Project Appraisals Pty Limited, November 2001, and"Technical Study on Infrastructure", Infrastructure TechnicalStudy Team, September 2001.

Table 6. Economic Internal Rate ofReturn for KDP InfrastructureProjects EIRR

Water Supply 83.3%

Bridges 58.7%

Roads 32.8%

Irrigation/Drainage 14.8%

Weighted Average 60.1%

For the complete terms of reference for the BPKPKDP audit, see Annex H. In general, over the lastthree years, BPKP found that PMD had followedthe terms of the loan agreement in program execu-tion. The selection of sites was in accordance withpredetermined guidelines. Preparation of proposalshad been well planned and followed program poli-cies. BPKP found levels of participation and trans-parency adequate, although there were severalprovinces with weaknesses. Benefits from the eco-nomic and infrastructure activities were adequate,although problems were mentioned regarding eco-nomic loans not being targeted towards the poor.Maintenance of infrastructure was also found to beadequate.

Special problem areas included weak and incompletedocument administration in some locations, and loanrepayment in numerous provinces. In general, theBPKP audits were helpful in uncovering problems inthe field. The Handling Complaints Unit incorpo-rated the audit report findings into the complaintsdatabase for follow up action. The overall audit find-ings were consistent with KDP’s general findingsfrom its own internal monitoring and reporting.

6.4 World Bank Supervision Missionsand Financial Reviews

The World Bank, together with the government andNMC, undertook semi-annual supervision missionsthat proved helpful in identifying management issuesand evaluating the progress of the program at thecentral level and in the field. To gather a wide spec-trum of perspectives, these missions usually includedrepresentatives from other donor agencies andIndonesian NGOs. The Bank also undertook occa-sional financial spot checks of consultant companiesand contractors. World Bank financial specialistsexamined invoices and payments made to consult-ants for salaries, transport allowances, travel and theuse of operational expenses in the field. Staff alsooccasionally intervened on corruption cases and pres-sured local stakeholders to solve problems expedi-tiously. KDP program management and the Bankidentified a number of consultant firms who wereperforming below standard and gave written warn-

ings to rectify problems. Financial supervision andauditing from BPKP, the NMC (FinancialSupervision and Training Unit) and the World Bankwere all valuable in ensuring that program fundswere not diverted. Field consultants reported thateven the threat of audits or financial reviews helpeddeter misuse of funds. These periodic audits werereinforced through more regular field visits by con-sultants, government officials, and NGOs. UnderKDP 2 these audit activities will increase along withmore financial management training for consultantsand kecamatan- and village-level actors. KDP2 willalso sponsor cross-village visits to “audit” and learnfrom the successes and weaknesses of other villages.

7. COSTS OF THE MONITORINGAND EVALUATION SYSTEM

The total program cost for monitoring and evalua-tion activities was approximately $2.4 million. Thisfigure does not include costs for general field con-sultant staff or World Bank supervision missions.The M&E budget was less than 1 percent of theoverall program budget of $273 million.

There were 22 full-time monitoring and evaluationstaff (including the complaints/grievances unit) basedin Jakarta. They spent on average 50 to 75 percent oftheir time in the field. (See Table 7 for cost break-down of activities)

8. DIFFICULTIES ENCOUNTERED

Implementing the M&E procedures in the field cre-ated its fair share of challenges. Among some ofthese difficulties were:• Difficulty in managing contracts –– On average,

it took seven months to a year to contract out eachof the monitoring and evaluation activities. Inlarge part, this was due to the difficulties the gov-ernment experienced in initiating and managingsome 61 contracts to various consultant compa-nies, suppliers and service agencies for the entireprogram. Contracts related to M&E (e.g., NGOs,journalists, studies, evaluations) comprised one


half of the KDP contracts. Delays in finalizingcontracts created significant implementation prob-lems. For example, the NGO monitoring con-tracts took over a year to finalize. Contractextensions also took unduly long to complete.

• Incomplete reporting by consultants –– KDPfield consultants on occasion did not report prob-lems in the field for fear of being blamed for theproblems or drawing attention to their situation.Despite repeated training and instructions, therewere consultants who would rather keep quietabout problems rather than report them to man-agers. The project also experienced problems withincomplete reporting from certain remoteprovinces, and eventually sanctions had to beestablished to deal with consultants who did notreport regularly. More training should also begiven in the future in the identification of prob-lems, technical and financial reviews, and reportwriting.

• Limited capabilities of M&E implementingagencies –– Contracting out to IndonesianNGOs and journalists also meant a certainamount of capacity building had to accompanythe activities. For NGOs, KDP provided not onlyprogram orientation, but also training on simpledata-collection methodologies, interview tech-niques, reporting and technical and financialreviews. For journalists, KDP supported trainingfor investigative journalism as well as interviewingtechniques and media resources. The quality ofthe monitoring suffered somewhat from the lackof deep data analysis and high quality writtenreports. Also for the impact survey, KDP had sig-nificant problems with the implementing organi-zation and the quality of questionnairedevelopment and data analysis was not as high ashad been originally anticipated.

• Improvements needed for impact survey design–– While KDP succeeded in putting in place aquasi-experimental design for its impact survey,improvements could be made. The questionnaireshould be more focused and include more ques-tions related to social capital and local governance.Data analysis and reporting were also had prob-lems and must be improved. KDP’s design and

selection of kecamatan based on a national povertyranking presented unique problems in developingan identification strategy to determine controlareas. That problem is still being wrestled with forKDP2.

9. CONCLUSIONS AND FUTUREIMPROVEMENTS UNDER KDP2

KDP employed a broad mix of approaches to moni-toring and evaluation. As KDP is a large multi-sec-toral program with a wide range of targets andoutput variables, capturing these program aspectsrigorously necessitated a variety of approaches andtechniques. The program combined quantitativeinstruments (household surveys, regular monthlyfield reporting) with qualitative research (case stud-ies). It supported participatory methods such as com-munity participatory monitoring along with morehighly specialized monitoring and evaluation activi-ties such as financial audits and sector studies. Lastly,to provide a wide array of perspectives as well ascross-triangulation of data, KDP incorporated bothinternal and external monitoring from a host of civilsociety actors and other donors.

But did the system work? Did the monitoring com-ponents allow program managers to keep track ofprogram progress and take corrective action as neces-sary? Did the evaluation components allow KDP toassess the impact of its activities?

Program managers in general were able to assess reg-ularly whether program activities were being imple-mented as planned. Through internal monitoring byprogram stakeholders, NGOs, and journalists, pro-gram managers obtained feedback and informationon the status of program implementation. Whilethere were problems with the consistency and qualityof reporting, the information was generally usefuland timely. The findings provided information forprogram managers to take corrective action andadjust program procedures. The direct link betweenM&E findings and actions taken was evidenced inseveral tangible ways: the handling of grievances andcomplaints; suspension of activities in problem areas;


termination of four company contracts; reformulationof procedures each project year, and the incorpora-tion of lessons and best practices for the design ofKDP Phase Two. In the cases where communitiesundertook participatory monitoring, feedback wasalso very positive because villagers were able to mon-itor and assess project aspects that mattered to them.

The evaluation components also provided vital infor-mation to assess impact. The final impact evaluationsto measure changes in household welfare are stillunderway. However, the sectoral evaluations gaveconstructive recommendations for the redesign of theeconomic loan portfolio and the cost effectivenessand use of infrastructure. Financial audits and theperiodic supervision missions again led to furtherfinancial training in certain areas, program suspen-sion in problematic areas and reformulation of designissues.

The next phase of KDP (2002-2006) will allow forfurther refinements and improvements in the moni-toring and evaluation system. Specifically, the follow-ing changes will be incorporated:• KDP2 will decentralize many of its management

and monitoring functions by establishing fieldoffices at the regional level. Several M&E func-tions such as the management information systemand handling complaints unit will be decentralized

to the regions and districts. This could presentmore difficulties in controlling and managing thequality of data and processing of complaints.Conversely, with additional personnel in the field,it should be easier to gather and process informa-tion as well as respond to grievances in a moreefficient manner.

• The impact survey work is being redesignedbuilding on the baseline work captured underKDP1. The survey instruments will be refinedand further qualitative and participatory work willcomplement the quantitative survey work. Casestudies and more ethnographic research areplanned.

• Under KDP2, the government, especially districtparliaments, will take a more active role in super-vising the program. KDP2 places greater emphasison involving the local parliaments and govern-ment officials to monitor activities more regularly.

• KDP2 emphasizes to a greater extent capacitybuilding especially to improve the skills of com-munities and consultants in technical supervision,financial oversight, and contracting. During thenext phase, KDP’s M&E system will need toincorporate additional mechanisms to track andmeasure the impact of capacity building and train-ing interventions.



Table 7. Summary of KDP1 Monitoring and Evaluation Activities and CostsActivity Objective Implementing Outputs Approximate

Agencies/Actors Cost ($)

INTERNAL MONITORING ACTIVITIES

Monitoring by consultants/MIS

To monitor and regularly reportupon KDP progress

Consultants at the nation-al, provincial, district andkecamatan levels

Monthlyreports, data-base, prob-lems solved.

$540,000 MIS Unit, M&EAdvisor, and full-time M&E staff.**

0Reports andproblemssolved

Govt. officials at thenational, provincial and district levels

Monitoring by Govt.Officials

To monitor and regularly report upon KDP progress, trouble-shooting

0Activities moni-tored by the community

Community membersCommunityMonitoring

To monitor projects within their communities

$230,000Staff salaries & travel costs plus @$100,000 for pro-duction & distribu-tion of case studies

6-8 case studies per year

2 full-time KDP staff plusoccasional consultants

Case Studies To document lessons learned andbest practices of KDP

$145,0006 staff persons’salaries, travel

Resolved casesDatabase ofcomplaints

6 full-time KDP staff inHandling Complaints Unit

ComplaintsResolution Process

To document and resolve fieldproblems

$168,0005 staff persons’salaries, travel

Training toUPKs and eco-nomic groups.Field financeschecked andaudited.

5 full-time staff withaccounting backgroundsin the FinancialSupervision and TrainingUnit

Financial Supervisionand Training

To improve the financial skills ofUPKs and economic loan groupsand to check financial/adminrecords at the kecamatan and vil-lage levels.

$616,000($10,000 – 14,000/NGO/province/year)

Monthly reports40 provincial NGOsNGO Province-basedMonitoring

To monitor and provide an inde-pendent source of qualitative infor-mation on KDP

$ 390,000$130,000/yr

Newspaper arti-cles, radioreports

18 provincial journalists,LP3ES/AJI

Journalist monitoringand reporting

To provide independent monitoringand reporting on KDP by local jour-nalists

Aide-memoire(usually twice ayear)

World Bank , PMD, BAP-PENAS, NMC, NGOs

World BankSupervision Missions

To supervise KDP progress andachievements

Audit findingsand reports

BPKPFinancial Audits To audit KDP finances

$300,000Baseline surveyof 4500 HHsand communityleaders.Economic feasi-bility assess-ment

Demographic Institute,University of Indonesia

Impact Evaluation (a) To evaluate KDP’s impact oncommunity organization and perceptions; (b) to assess the feasibility of measuring KDP impacton poverty, incomes, and livingstandards.

$30,000Study evaluat-ing the qualityof KDP infra-structure projects

External consultant team(5 persons) including oneexpatriate

TechnicalInfrastructure Study

To evaluate KDP infrastructure proj-ects and recommend improve-ments

EXTERNAL MONITORING ACTIVITIES

EVALUATION ACTIVITIES

$30,000Study evaluat-ing the qualityof KDP eco-nomic loanactivities

External consultant team(4 persons) including oneexpatriate

Economic Loan Study To evaluate KDP economic loanactivities and recommend improve-ments

TOTAL M&E BUDGET $2.4 million* Costs are over four-year program period. World Bank supervision costs are not included. **Costs for consultant monitoring include only the costs of full-time M&E staff.Costs for field consultants involved in technical assistance, management, and supervision are not included here.

BIBLIOGRAPHY

CESDA – LP3ES. 2001. Transparency inDevelopment: The Experience of Mass Media inMonitoring KDP. Jakarta, Indonesia. (Indonesian)

Demographic Institute, Faculty of Economics,University of Indonesia. 2001. Evaluation of theKecamatan Development Program (KDP) Impacts onCommunity Organization and Household Welfare: Book– 1. Jakarta, Indonesia. (Indonesian and English)

———-. 2001a. Evaluation of KecamatanDevelopment Program (KDP) Impacts on CommunityOrganization and Household Welfare: Power andSample Size Calculation for Feasibility Study. Jakarta,Indonesia.

———. 2002. Evaluation of Kecamatan DevelopmentProgram (KDP) Impacts on Community Organizationand Household Welfare: Second Round Survey. Jakarta,Indonesia.

Dent, Geoffrey, Project Appraisals Pty Limited.2001. “Ex-Post Evaluation of KecamatanDevelopment Program (KDP) InfrastructureProjects”. Jakarta, Indonesia

Economic Study Team. 2001. “Report of theEconomic Study Team.” Jakarta, Indonesia.

Evers, Pieter. 2001. “Legal Assistance Program forKDP Communities: A First Assessment of theLegal Assistance Pilot Project in Central Java andNorth Sumatra, August – October 2001.” Jakarta,Indonesia.

Holloh, Detlev. 2001. “Review of the KDPMicrocredit Approach: Consultancy Report.”Jakarta, Indonesia.

Infrastructure Technical Study Team. 2001.“Technical Study on Infrastructure.” Jakarta,Indonesia.

Johns Hopkins University/Center forCommunication Programs. 2001.“Behavior ChangeCommunication Strategy, KDP: Final Report.”Jakarta, Indonesia.

———-. 2002. “Focus Group Discussion Results forthe Development of Communication Materials”,(presentation materials). Jakarta, Indonesia.

Ministry of Home Affairs, CommunityDevelopment Agency, KDP National Secretariat andNational Management Consultants. 1999.Kecamatan Development Program, First Annual Report(1998-1999). Jakarta, Indonesia.

————. 2000. Kecamatan Development Program,Second Annual Report (1999-2000). Jakarta,Indonesia. (English and Indonesian versions)

————. 2002. Kecamatan Development ProgramPhase One: Final Report. Jakarta, Indonesia. (Englishand Indonesian versions)

Ministry of Home Affairs, CommunityDevelopment Agency, KDP Central CoordinationTeam. 2000. Kecamatan Development Program:Technical Operations Manual for 1999/2000. Jakarta,Indonesia. (Indonesian)

————-. 2000. Kecamatan DevelopmentProgram: Forms. Jakarta, Indonesia. (Indonesian)

National Management Consultants, InternalMonitoring Case Studies: (in Indonesian)

Edition 1: Year 2/2001 “Lessons Learned FromKDP Year One”

Edition 2: Year 2/2001 “Participation of Women andthe Poor in KDP”

Edition 3: Year 2/2001 “Conflict Resolution inKDP”

Edition 4: Year 2/2001 “Information Disseminationand the Role of the FD”

Edition 5: Year 3/2001 “Community ParticipatoryMonitoring”

Edition 6: Year 3/2001 “The Role of WomenLeaders in KDP”


Edition 7: Year 3/2002 “The Impact of KDP1 onLocal Governance”

Edition 8: Year 3/2002 “Infrastructure Best Practicesunder KDP1”

National Management Consultants. 2001. “BriefOverview of KDP.” Jakarta, Indonesia. (English andIndonesian)

National Management Consultants. 2002. “KDPActivities in Conflict Areas.” Jakarta, Indonesia.(English and Indonesian)

Wong, Susan. 2002. “Do Women Make AnyDifference? Gender Data Analysis Interim Report.”Jakarta, Indonesia.

World Bank Supervision Mission Reports (semi-annual)

World Bank. 1998. “Project Appraisal Document ona Proposed Loan in the Amount of US$225 millionto the Republic of Indonesia for a KecamatanDevelopment Project.” Washington, D.C.

————-. 2001. “Project Appraisal Document on aProposed Loan in the Amount of SDR 87.5 million(US$111.3 million equivalent) to the Republic ofIndonesia for a Second Kecamatan DevelopmentProject.” Washington, D.C.


This Annex describes briefly how KDP Phase One(1998-2002) operated, starting from the initial selec-tion of project locations to implementation in thefield. Additional information can be found in themain text of this report.

1. Selection of KDP Locations

The selection of KDP kecamatan and provinces wasbased upon poverty incidence. The Program targetedthe poorest kecamatan in the country. Studies sug-gest that populations within kecamatan are relativelyhomogenous in terms of poverty incidence, andthereby form a sound basis for poverty targeting andfor reaching a large number of the rural poor. Thenational planning agency, Bappenas, analyzed pover-ty incidence statistics based upon government officialSusenas data (expenditure) and Podes (mostly infra-structure surveys). Bappenas then prepared a long listof some 1,500 kecamatan with a high incidence ofpoverty.

The long list of kecamatan was sent to the KDPgovernment Coordination Teams (TK-Prop and TK-Kab) at the provincial and district levels for reviewand to confirm poverty incidence and ranking. Ashort list was formed of the poorest kecamatan.Once the list was finalized, the Government ofIndonesia sent it to the World Bank for finalapproval.

For village selection, if a kecamatan contains ten vil-lages or less, all villages were allowed to participate.For kecamatan with more than ten villages, only halfof the villages in a kecamatan were allowed to partic-ipate in the first year of KDP (up to a maximum often). This was primarily done to limit the amount offacilitation required. In Year Two, the other half ofthe villages were eligible to participate. All villageswere eligible to participate the third year. Sincehealthy competition among village proposals wasencouraged, not all villages necessarily received fundsduring any given year.

2. How KDP Works

KDP provides funds at the kecamatan level.Villagers then decide to use these funds for infra-structure, social or economic activities. These fundsare available to each kecamatan each year for up tothree years.

The distribution of funds within the kecamatan isthrough the UDKP Forum (Unit Daerah KerjaPembangunan) to the village board, LKMD or villagecouncil BPD. The UDKP Forum consists of UDKPmembers, appointed by government, plus additional,broadly respected persons (such as religious and tra-ditional leaders, teachers) and three additional repre-sentatives (one man and two women) selected fromeach participating village. The UDKP also creates aunit called the Financial Management Unit (UnitPengelolaan Keuangan (UPK)) to manage KDP fundsand to oversee any large procurements.

The grant size to each kecamatan varies by popula-tion size (Table A.1).

3. KDP Project Activity Cycle

The KDP Project Cycle goes through various stages:information dissemination; planning; proposal prepa-ration and verification; funding decisions; implemen-tation; and follow-up. The Project Cycle generallytakes 12 to 14 months to complete although there isvariation among provinces. The socialization andplanning stages take approximately four to sixmonths. All stages aim to achieve a high degree ofcommunity participation and transparency through-out the process.

Information DisseminationInformation dissemination about KDP occurs in sev-eral ways. Workshops are held at the provincial, dis-trict and kecamatan levels to disseminateinformation and popularize the program. The work-shops involve community leaders, local government


ANNEX A: BRIEF DESCRIPTION OF KDP


Kecamatan in IndonesiaKecamatan are the sub-district level of administration in Indonesia. There are some 4,000 keca-matan in the country. On average a kecamatan contains approximately 20 villages and a popula-tion of over 68,000 people. Though these units are already fairly large, the kecamatan office is stillseen as being “approachable” by the community (whereas the district (kabupaten) is usually seenas too distant on the administrative scale). The administrative head of a kecamatan is called a“Camat”. He is appointed by government.

_____________________________The administrative levels in Indonesia are as follows:

National Level or Pusat

Province or Propinsi (often referred to as Tingkat I or Level I)

District or Kabupaten (Tingkat II or Level II)

Sub-district or Kecamatan

Village or Desa

Box A.1: Profile of Villages and Kecamatan in Indonesia

Villages in IndonesiaVillages (desa) in Indonesia are formal administrative units containing on average approximately2,700 people. There are some 69,000 villages in Indonesia and most of these are predominantlyrural. Villages vary greatly in terms of population size and land area, as well as physical layoutand land use. Culture also differs significantly between villages across the Indonesian archipela-go. The head of a village is called a “Kepala Desa”. The Kepala Desa selects and is assisted by acommittee called a Village Community Resilience Board (LKMD) or Village Assembly (LMD), thatis mandated to guide development within the village through periodic consultation meetings.The Decentralization Laws of 1999 also called for Village Councils (BPDs) to be democraticallyelected. LKMDs or BPDs play an important role in facilitating KDP in the villages.

Source: Adapted from the KDP First Annual Report (1998-1999), National Management Consultants for the KDP NationalSecretariat, Ministry of Home Affairs, Community Development Agency

officials, local press, universities and NGOs.Dissemination of information at the village leveloccurs through large village meetings (Musbangdesmeetings) as well as through group and sub-villagelevel meetings to disseminate information andencourage villagers to propose ideas for KDP sup-port. Elected Village Facilitators play an importantrole in disseminating information to communitygroups. On average, approximately 50 to 100 vil-lagers attend Musbangdes meetings, although insome areas, attendance is reported to be several hun-dred participants.

PlanningPlanning meetings occur at the sub-village (hamlet)and village levels. The Village and KecamatanFacilitators disseminate information about KDP pro-cedures and encourage villagers to submit ideas forKDP funding. Women also hold their own separatemeetings to decide upon women’s proposals. At thesecond village meeting (Musbangdes II), villagers’ideas are discussed and the forum decides whichideas the village should put forward to the UDKPIIForum as proposals. The planning stage of KDPusually takes one to two months for villagers to learnabout KDP procedures and submit ideas for funding.

Proposal Preparation and VerificationEach village can submit up to two proposals to theUDKP. If there are two proposals, one must comefrom village women. Proposals from villagers rangebetween a minimum of Rps 35 million ($4,375) to amaximum of Rps 150 million ($18,750).

KDP can finance productive infrastructure, socialand economic activities. Proposals can include a mix-ture of social, economic and infrastructure activities ifthe villagers so choose. Project menus are open to all


Table A.1: Grant Size Eligibility Thresholds for Kecamatan(Exchange rate: $1=Rps 8,000)

Kecamatan in Java: Kecamatan outside Java: Maximum Grant to Kecamatan

> 50,000 persons > 25,000 persons Rps 1 billion/year ($125,000/year)

25,000 – 50,000 persons 15,000 – 25,000 persons Rps 750 million/year ($93,750/year)

< 15,000 persons (NTT) Rps 350 million/year($43,750/year)

productive investments except those on a negativeshort list.10 Villagers can prepare joint proposals, forexample, for multi-village irrigation, road or watersupply systems. KDP does not support district-levelinfrastructure since districts have their own budgetsand areas of responsibility. The Open Menu policy isan important element of KDP. It allows villagersthemselves to decide what they want.

For economic activities, the borrowing groups arerequired to be in existence for at least one year.Interest rates should follow prevailing market ratesand loans should be repaid within a maximum of 18months. The repayment schedule should fit the typeof economic activity undertaken and benefit flows,and should be agreed upon by all members of thegroup. The repayment schedule should be attachedwith the proposal.

For infrastructure projects, village technical assistanceshould be identified and used in proposal prepara-tion. Public infrastructure projects must meet techni-cal design quality standards, benefit the poor, andinclude a plan and budget for maintenance aftercompletion, including arrangements to collect andmanage user fees as appropriate. Villagers are expect-ed to contribute labor and/or materials so that con-tributions are maximized. There is no preset targetfor community contributions, but the greater thecontribution, the more favorably the proposal isreviewed by the UDKP Forum.

10 KDP funds cannot be used for: military or paramilitary pur-poses; civil works for government administration or religiouspurposes; manufacture or use of environmentally hazardousgoods, arms, or drugs; or financing of government salaries. Landacquisition is also restricted.

Verification of the proposals’ technical elementsoccurs during the proposal review stage prior to theUDKP Forum selection. A kecamatan verificationteam is formed and usually includes communityleaders, the Kecamatan Facilitator (FK), and appro-priate technical staff recommended by the DistrictEngineer (KMKab). The District Engineer also doesa final check before the results of the verification arepresented and considered in the UDKP forum. Theverification team looks at the following questions:• are proposals technically and economically

feasible? • do they benefit large numbers of people, especially

the poor? • are there maintenance plans (or repayment plans

in the case of economic loans) in place? • do people genuinely participate in the formulation

of proposal ideas, and what is the community contribution?

This verification phase usually takes three to fourweeks and is seen as crucial to screen for projects thatare only of sound design and quality.

Project SelectionVillage proposals are discussed at the kecamatan levelduring the second kecamatan level meeting,UDKPII. The UDKP Forum reviews the findings ofthe verification team and discusses the merits andbudgets of each proposal. On average, about 60 per-sons attend these meetings. Final decisions from theUDKPII meeting are posted on KDP InformationBoards and shared with the villagers through thethird village level meeting, Musbangdes III, andsmaller sub-village and group meetings.

Financial Management and AccountsThe Head of the UPK, the Kecamatan Facilitatorand all the LKMD Heads open a KDP account atthe local Bank of the Republic of Indonesia (BRI).When the subproject agreements are finalized, theyare endorsed by the PMD Kecamatan ProjectManager, PjOK, and a copy is sent to the govern-ment treasury office (KPKN). KPKN then orders aninitial transfer to the KDP-BRI account. The install-ments are made in 40%- 40% - 20% installments.

The UPKs, Village Councils, and implementationteams must report to the community on the use ofthe funds after each installment has been utilized.The last 20 percent payment to villages cannot bereleased until the District Engineer approves andsigns the completion certificate.

Although money is deposited in the kecamatanaccount in three installments, there is no limit on thenumber of village withdrawals and there are no mini-mum periods between withdrawals by villages.Disbursements are made against certified requests ofexpenditure by the village’s LKMD Head, the UPKHead and the FK. The UPK with the assistance ofthe FK must account fully for the kecamatan grant.

Basic signature requirements are as follows:Establishment of kecamatan bank account: FK, UPK Head, All

LKMD Heads

Withdrawal of funds: FK, UPK Head, LKMD Head (3 persons)

Project agreement: PjOK, FK, LKMD Head, Kecamatan Head

Project completion report: KMKab, LKMD Head, UPK Head

Project ImplementationOnce it has been decided which projects should befunded, villagers mobilize technical assistance.Contracts are signed with the LKMD and acknowl-edged by the District Engineer. Technical designs arefinalized and the mobilization of workers takes place.During the Musbangdes III meetings, villagers electa community Implementation Team of five personsin each village to oversee the project implementation.Funds are withdrawn from village level accounts forthe construction of infrastructure and economicloans. The community contributions are also collect-ed. The Kecamatan Facilitators and PjOK (PMDkecamatan level project manager) support groups andLKMDs on administrative and financial matters.Communities, government officials and KDP con-sultants supervise the implementation of activities.Village meetings are encouraged to report upon thestatus of project implementation. At least one villagemeeting occurs when activities are halfway throughimplementation. There is also a final village meetingwhen the project is completed and final accounts are


reported. Infrastructure implementation usually tookthree to four months for completion.

MaintenanceOnce the infrastructure is built, villagers formOperation and Maintenance Committees who areresponsible for overseeing the proper maintenance offacilities and fee collection. The user fee system isalso reconfirmed. For economic loans, loan repay-ments are collected by the UPK for a maximum loanperiod of 18 months.

4. Management of KDP

The management of KDP is the responsibility of theMinistry of Home Affairs, CommunityDevelopment Agency (PMD). At the national level,PMD deals with the day-to-day operations of theProgram. Government coordination teams, repre-senting various ministries also assist with KDP. Atthe national level, the Deputy for RegionalDevelopment of the National DevelopmentPlanning Agency (Bappenas) heads the nationalCoordination Team. At the province and district lev-els, the government Coordination Teams are headedby the Regional Development Planning Agency(Bappeda). At the kecamatan level, the role of thePjOK is very important because he serves as KDP’slocal project manager. The PjOK is assisted byadministrative staff (PjAK) also from PMD.

KDP is assisted technically by consultant teams.Some 1,662 consultants from 16 private companiesat the national, provincial, district and kecamatanlevels provide technical assistance and guidance tothe project.

In terms of roles and responsibilities for managingKDP, the PMD Project Secretariat at the nationallevel issues formal guidelines and determines projectpolicy based upon the Loan Agreement with theWorld Bank. The Secretariat also performs all con-tracting and invoice verification. In addition, theSecretariat undertakes all formal contact with theWorld Bank and local government agencies.

The consultants are responsible for all daily technicalmatters, clarifying and operationalizing policies,supervising and evaluating all consultants, draftingTerms of Reference and guidelines, and advisingupon all monitoring and evaluation work for theProgram.



Figure A.1 KDP Activity Cycle––Year 3


Table A.2: Phases of ImplementationKecamatan Development Program Year 3


Figure A.2: KDP Funds Flow System in Year 3


Figure A.3: The KDP management System


ANNEX B: KDP PROJECT DESIGN SUMMARY

CAS Objectives:

• Reducing poverty

• Efficient provision ofbasic services

• Participation in development

• Decentralization

% of absolute poor

Increased transfers tokecamatan and villagelevels

CAS and CEM reports (CAS Objective to BankMission)

No gaps between macroand project objectives.

Project DevelopmentObjectives

• Strengthening kecamatan and villgelevel government byempowering them tomanage increased funding and becomingaccountable for it

Rural development byproviding increased economic opportunities atthe village level in thepoorest kecamatan

# participating kecamatan:Year 1: 350Year 2: 725Year 3: 725

Transfers to kecamatan:Baseline:Rps 6m per villageKDP: Rp 500m/year forkecamatan with25,000/50,000 people onJava, 15,000/25,000 people off JavaRp 750 million for kecamatan exceedingthese thresholds;Rp 250m on Timor Timur

#participating villages:Year 1: 2000Year 2: 4000Year 3: 6000Susenas householdexpenditure surveys

Bappenas’ recordsKPKN recordsProvincial recordsKecamatan UPK recordsJournalist reports

Susenas analysisAd-hoc surveys and kecamatan field visitsProvincial records

(Development Objectivesto CAS Objective)The project’s developmentobjectives are fully in linewith CAS objectives.Risks:(a) Scale up too fast;(b) Central government

does not completetransfers;

(c) Districts appropriatefunds forkecamatan/villages

(d) No improvement inlocal mobilization ofmaintenance resources

(a) Current economic crisis may increasepoverty; thus changesin the % poor bySusenas may notreflect on KDP results;

(b) Inadequate targetingmay miss the poorest

Narrative Summary Key Performance Indicators 1 Monitoring and Supervision Critical Assumptions andRisks

1 See additional Performance Indicators as listed in KDP1 Project Appriasal Document, Annex 1

Source: KDP Project Apraisal Document, Annex 1


Project Outputs

• Improved planningprocess and plans

• Public infrastructures

• Economic investments

• Studies, in particular toimprove kabupaten pro-curement guidelines toencourage small con-tractors

# of villages with sub-village/village meetingsheld

implementation agreements

UPKs established/ opera-tive

Transparency rulesapplied

#proposals implemented

#completed works by typeand total amounts

# O&M committeesformed and funded/operational from Yr 2onwards

#proposals implementedby type and amounts

Compliance with repayment

Benefit returns

Issued new regulation andif necessary MoHA clarifi-cation, repeal of SKs (localregulations) Project Year 2

UPK/KPKN & BRI records

Overall:

Technical audits

TA qualitative evaluationsof participation

Random field checks

Summary reports fromfacilitators.

Bank-GOI site supervisionof 3-5% of recipient kecamatan

Annual review of projectperformanceBappenas recordsDatabase on contractswith communities

(a) Risks: Elite and higher-level domination ofplanningprocess/exclusion ofpoor;

(b) Directed investment

(a) Poor technical quality, inadequateengineering support

(b) inadequate maintenance;

(a) Inadequate facilitators,poor choice of activi-ties could limit sustain-ability;

(b) High leakage rates;(c) Low repayment

Risk: removing one set ofbarriers to local contract-ing may prompt issuanceof other, new restrictiveguidelines.Critical assumption: localcontractors and artisanscan provide quality goodsand services at competi-tive prices if procurementis opened to them.

Narrative Summary Key Performance Indicators 1 Monitoring and Supervision Critical Assumptions andRisks

ANNEX B: KDP PROJECT DESIGN SUMMARY


KEY RESEARCH QUESTIONS

INDICATORS *

Monitoring Tools Evaluation Tools

ANNEX C: KDP MONITORING AND EVALUATION FRAMEWORK

1. Poverty & Socio-Economic Impact

TOOLS/ MEANS OF VERIFICATION

a. What is the socio-eco-nomic impact of KDP?What are the social andeconomic benefits ofthe project’s activities?

1. Household socio-economic poverty indicators, % ofabsolute poor

2. #/type of beneficiaries3. #/type of workdays

created4. #/type of persons who

gained employmentfrom the project.

5. Community perceptionsof project benefits (e.g.usage/increase in eco-nomic activities, cost &time savings, accessdirectly related to infra-structure)

1. Case studies2, 3,& 4 MIS – Forms 2.1-

2.2, 81. – 8.75. Field reports from

consultants,govt.,NGOs, journalists

Impact survey usingSusenas panel dataImpact survey on community perceptionsCase studies

2. Quality and Effectiveness of Infrastructure Projects

a. Is the technical qualityof infrastructure proj-ects sound? How doesthe quality of KDP infrastructure comparewith non-KDP?

1. # /$/type/dimensions/quality of infrastructurebuilt

2. Satisfaction levels ofcommunity with infra-structure projects

1. MIS - Forms 3.1, 3.2,8.1, 9.9 -9.14, 10.1- 10.2,11.1 – 11.6, NGOreports

2. Consultant regularmonitoring & reportsHandling Complaints, NGO reports, community participatory monitoringCase studies

Infrastructure evaluationstudyImpact survey on community perceptionsSupervision missions

b. Cost/benefit, cost-effectiveness relative to non-KDP projects

1. $/type of infrastructureproject, KDP vs. non-KDP

2. Economic internal rateof return

3. Feedback from community regardingproject costs

1. District engineer reportsForms 4.1, 5.1, 8

3. Field visit reports Community participatory monitoring

Infrastructure evaluationstudySupervision missions

c. Factors which mayaffect technical quality(e.g, simplicity ofdesign, use oflocal/external expertise,quality of district engineers and villagetechnical assistance)

1. Use of villagers vs. outside contractors

2. Quality of technicaldesigns

3. # TA visits4. Performance of District

engineers and villagetechnical assistance

5. Satisfaction levels ofcommunities with TA

1. MIS - Form 3.1 2. Form 2.3, 3.1 3. Prov & district

consultant reports4. MIS-Form 8.1

Prov & district consultant reports

5. Handling ComplaintsreportsField visit reports

Infrastructure evaluationstudySupervision missions

d. Are the projects beingmaintained?

1. No. of O&M committees

2. No. of membersinvolved in O&M

3. Schedule of O&M activities

4. Quality of O&M activities

MIS- Form 2.1, 11.1-11.6 Infrastructure evaluationstudy

– All indicators are gender-disaggregated where appropriate. Indicators listed in the framework include, but are not limited to, the key performanceindicators required for World Bank reporting per the KDP Loan Agreement Schedule 6. – MIS Forms refer to the specific reporting forms to be completed by field consultants.

a Benefits of EconomicLoan Portfolio – Forwhat purposes were theeconomic loans used?Were these economicactivities profitable?Who benefited from theloan activities?

1. #/size of loans2. #/type loan recipients3. use of loans4. economic return on

loans5. borrower perceptions

1&2 MIS – Form 8.2, 8.4, 8.6

3 Form 2.24&5 Field monitoring

reports, community monitoring

Economic loan studyImpact survey on community perceptionsSupervision missions

3. Quality and Use of Economic Activities

b. Repayment &Revolving Funds – Whatwere the repaymentrates and the reasonsfor high or low repay-ment? Did fundsrevolve?

1. Repayment rates (bylocation, gender)

2. Timeliness of repay-ment

3. Reasons cited forhigh/low repayment

4. $revolving funds (bylocation, gender)

1,2,4 MIS – Form 8.2, 8.4, 8.6

3. Field reports, NGOreports

Economic loan studySupervision missions

4. Other Social Activities(Followed same kinds of indicators as above for technical and economic activities)

5. Institution-Strengthening

a. Are the village councilsand inter-village forumsfunctioning and arethey being strength-ened?

1. #/type of personsattending committeemeetings

2. quality of participationat meetings

3. #/type of decisionsmade,when, how often

4. community aware-ness/satisfaction withcommittee decisions

1&2 MIS-Form 9.5 3 Form 4.1, 9.5 4 NGO & journalists’

monitoring, field visits, case studies

Impact survey on community perceptions

b. What other socialorganizations exist in thevillage? Who are themembers?What role do/can suchorganizations play inKDP?

1. #/type social orgs2. composition of mem-bers3. perceptions by thecommunity

1&2 LLI study

Field reports

Impact survey on community perceptions

6. Community Empowerment

General

a. In general, how do villagers feel aboutKDP? Sense of owner-ship of projects andprocess? Were the projects chosen thosethat they wanted?

1. Community perceptions 2. #/type of complaints 3. Level of community

contribution (financial,time, in-kind)

4. Willingness to partici-pate in future activities

Field reports, handlingcomplaints reports, NGOs,journalists, case studies, community participatorymonitoring

Impact survey on community perceptionsSupervision missions



INDICATORS *



b. Recourse - were thereany major problemswith the project, and ifso, how were theyaddressed and/orsolved?

1. community perceptions2. #/type of complaints,

resolutions

Field reports, handlingcomplaints reports, NGOs,journalists, case studies,community participatorymonitoring



c. Participation - Are vil-lagers actively partici-pating in all stages ofthe project?

1. #/type of villagers/%HHparticipating actively invarious stages

2. # consultation meetings 3. Amount/type of com-

munity contributions

1. MIS-Form 9.5NGOs & journalists’reports, case studies

2. Consultant reports3. Form 2.1,2.2, 2.4, 10.1


d. Inclusion - Who is participating? Who do not? Why? Are disenfranchised groups,women, poor participating in allstages?

1. #/% women, poor, marginalized villagersparticipating actively invarious stages

2. # women proposalsreceived and approved.

3. Perceptions of differentgroups

1&3 MIS-Form 9.5NGOs & journalists’ reports, case studiesConsultant reports2. Form 4.1


e. Decision-making - howare project decisionsmade? Are decisions oragreements carried out?If not, why?

1. Quality of decision-making

2. Perceptions by thecommunity on howdecisions were made

MIS-Form 9.5NGOs & journalists’reports, case studies Consultant reports

Impact survey on commu-nity perceptionsSupervision missions

f. Benefits - Do villagersfeel like they benefitfrom the project? Whobenefits most from theproject? In what way?

1. Community perceptions NGOs & journalists’reports, case studies Consultant reports


Information-Sharing & Transparency

g. How is information disseminated and howoften? What issues areconfusing? What arethe best practices forwidespread and trans-parent information dissemination and education in specificareas?

1. # of info boards2. Regularity of info

boards being updated 3. Type of info posted 4. # community info meet-

ings 5. Knowledge/awareness

of the community aboutvarious aspects of KDP

6. Frequency/type of infodisseminated

7. Community response

NGO reportsCase StudiesField supervision reportsInformation, education,communication consult-ants’ reports


h. What is the community's knowl-edge and awareness ofthe general project andspecific issues, i.e.,budget, paymentsbeing made, roles andresponsibilities?

1. Knowledge/awarenessof the communityregarding variousaspects of KDP

NGO reportsCase StudiesField supervision reportsCommunication consultants’ reports




INDICATORS *




a. Are the main projectoutputs beingachieved?

1. $ disbursed by activitytype, beneficiary group,area

2. #/name of provinces,districts, kecamatan and villages where KDP operates

3. # subproject agreements

4. #/type of total beneficiaries

5. #/type of training provided & beneficiaries

1. MoF, PMD,MIS reports2. MIS3. Form 4.14. Form 4.1, 9.5Field reportsTraining reports

Supervision missionsPKP audits

7. GENERAL PROJECT MANAGEMENT INFORMATION

b. Is the project beingmonitored and evaluated effectively?

1. # supervisory visits byfield consultants/ % ofsites visited

2. # audits 3. # visits/reports/stories

of independent monitoring by journalists, NGOs andother 3rd parties

4. #WB supervision missions

5. #independent evaluations, surveys,studies conducted

6. # communities conducting participatorymonitoring

7. Complaints information(see database info)

Field reports, handlingcomplaints reports, NGOs,journalists, case studies,community participatorymonitoring

Supervision missions

c. Are the project’sfinances being managed properly?

1. # villages/kecamatanaudited and audit findings

2. #/type of complaintsreceived regarding mishandling of funds

Financial Supervision and training unit audits,field reports, handlingcomplaints reports, NGOs,journalists, case studies,community participatorymonitoring.

BPKP annual auditsWorld Bank supervisionmissions and financialreviews



INDICATORS *




A. Background

The Kecamatan Development Program (KDP) is aGovernment of Indonesia program aimed at alleviat-ing poverty in rural communities and improvinglocal governance. KDP provides block grants of 350million to one billion rupiah ($40,000 to $114,000)directly to kecamatan and villages for small-scaleinfrastructure, social and economic activities. KDPemphasizes the principles of community participa-tion, especially for women and poor villagers, trans-parency, competition for funds, and sustainability. AllKDP activities aim at allowing villagers to maketheir own choices about the kinds of projects thatthey need and want. (See attached brief description ofKDP)

Phase One of KDP began in August 1998 through a$280 million World Bank loan. By 2002, theProgram will cover 984 kecamatan across 22provinces.

The uniqueness and magnitude of the programrequire a solid monitoring and evaluation systemto be in place to document the KDP experience

and lessons learned from this program. KDP’s monitoring and evaluation system consists of several components:• Monthly field reporting from consultants at the

kecamatan, kabupaten, provincial and national lev-els;

• Complaints/problems monitoring and resolution• Community participatory monitoring• Monitoring by government officials• Case studies documenting best practices and les-

sons learned;• Special studies on infrastructure, economic activi-

ties, and other themes• Financial supervision and auditing• Joint World Bank/GOI supervision missions

• Impact studies• Independent monitoring and reporting by the

Association of Independent Journalists (AJI)

In order to strengthen its monitoring system, KDPwill invite Indonesian NGOs to monitor independ-ently program progress in each province. This activityis intended to provide valuable inputs for improvingthe program.

B. Description of NGO IndependentMonitoring

The purpose of the NGO independent monitoringis to provide an independent source of qualitativeinformation on KDP implementation from a selectnumber of project locations to complement theinformation collected through the project’s internalmonitoring system.

Currently, KDP monitoring centers on regular inter-nal reporting from project implementors at all projectlocations. Although the reporting system is effectivefor monitoring progress on program implementation,inputs, and direct outputs, it is less effective at regis-tering the program’s ability to affect less tangiblechanges (such as improvements in the quality ofwomen’s participation and institutional strengthen-ing), and a wide range of specific problems and les-sons learned from overcoming them. Also, becauseall information stems from program staff, data arenot collected by objective parties.

In order to ensure that emerging problems are notedand addressed, additional monitoring is required toprovide data on: (i) common problems that areoccurring in many or all locations; (ii) specific prob-lems limited to a particular location; and (iii) short-and medium-term changes brought about by theprogram. The NGO independent monitoringdescribed herein mainly concerns the collection ofqualitative data through participatory methods,although a small number of quantitative indicatorswill also be incorporated in the NGO monitoring.

ANNEX D: TERMS OF REFERENCE FOR NGO INDEPENDENT MONITORING OF KDP1

1 These Terms of Reference were adapted and revised from anearlier draft version prepared by Anna Wetterberg, World Bank,in 1999.

The main source of information will be villagers andstakeholders in KDP kecamatan, both those thathave participated directly in the program and otherswho have not. Some data may be collected at thekecamatan level, but only as they relate to programimplementation at the village level.

C. Specific Tasks and Responsibilities

NGO independent monitoring is currently plannedfor all 22 KDP provinces. NGOs will monitor regu-larly the program’s developments and provide anindependent source of information to improve KDP.The primary users of the collected information willbe province and district- based project staff. TheNGO monitoring is not intended to cover all projectareas; the emphasis is for the NGO to provide rich,descriptive information in a sample of select loca-tions.

In each chosen province, one or two NGOs (orNGO consortia) will be identified to carry out themonitoring. The Community Development Officein the Ministry of Home Affairs (PMD), theNational Management Consultants (NMC) andWorld Bank will select the NGOs on a competitivebasis.

1. Monitoring ScopeNGOs will focus on these specific areas for monitor-ing:• Participation of villagers, especially women and

the poor, in all phases of the program;• Information-sharing and transparency;• Adherence with the general principles and proce-

dures of the program;• Quality of technical assistance• Quality of program outputs• Financial review• Recommendations from program stakeholders to

improve KDP.

A list of sample questions for each of the topics listedabove is attached in Annex D.2. The NGO is free toadd other questions as it deems necessary.

2. Geographic CoverageIn most provinces, only one NGO or NGO consor-tia will be chosen for each province. Due to the geo-graphical distances involved and the number of KDPkecamatan, two NGOs will be chosen for theprovinces of Irian Jaya, Nusa Tenggara Timur, NorthMaluku, and North Sumatra. More NGOs may beadded at a later date if necessary.

Each month, the NGO must visit a total of eightvillages in a minimum of three kecamatan. Two vil-lages should be monitored continuously each month(through community participatory monitoring) inorder to follow the progress of the complete projectcycle in two set locations. The NGO can select anyother six villages to visit. The NGO should rotate itsmonitoring of kecamatan and districts so that it cov-ers as wide an area as possible (taking into accountbudgetary considerations). The NGO should try tomonitor a wide sample of KDP locations in varioussites around the province.

Summary of Monitoring each Month:• District : sample as many districts as possible• Kecamatan: Minimum of three kecamatan• Village: Minimum of eight villages, monitor two

villages continuously, monitor six other villages

3. Methodology to be UsedThe NGO shall decide what monitoring tools itwishes to use. Some monitoring and data collectiontools include:• Direct Observation• Informal interviews• Focus Group Discussions• Mini-surveys• Participatory rural appraisal• Community participatory monitoring• Financial review of book-keeping

Community participatory monitoring will berequired in the two villages that are continuouslymonitored each month. Using this methodology, theNGO will facilitate the community members in datacollection, compiling questions and analysing theirdata. The NMC will provide training on thesemethodologies at the start of the contract.


Monitoring should include direct observation of thevarious stages of the planning process (such asobserving village and kecamatan-level meetings), aswell as more general monitoring of developments atthe village level (such as informal discussions withvillagers about their involvement in the KDPprocess). NGOs should make a special point ofinterviewing the poorer segments of the communityto solicit their views on KDP and the process. Directquotations from the field are highly encouraged andshould be included in the NGO reports.

Under this TOR, NGOs will also be responsible forfinancial monitoring and checking simple book-keeping at the kecamatan and village levels. While afull-scale audit is not required, the NGO will beexpected to spotcheck financial documentation andbe on the lookout for cases of corruption or pricemark-ups.

The NGO should strive to be as thorough as possi-ble in its monitoring. It should cross-check its find-ings and determine the validity of the informationwith a range of sources, including villagers, contrac-tors, government officials, and consultants.

It is important to note that the implementing orga-nization’s role is to observe, collect, analyze, andreport data. The NGO must not compromise itsindependent observer status by intervening to correctperceived problems or otherwise become involved inthe process. The role of the NGO is to monitor theprocess and then report its findings to KDP imple-menters.

4. Analysis of collected informationNGOs will clean and analyze the collected data,drawing attention to problems encountered, villagers’recommendations for changes, and general findingsof program impacts. NGOs are expected not only topresent field findings but also to analyze systemati-cally the results and provide recommendations.

5. ReportingDuring the first week of each month, the NGO willmake an oral presentation of findings to provincialKDP consultants and the government coordinationteams. In addition, the NGO will submit a brief

written report on the previous month’s monitoringresults to them with a copy to the NMC using theconcise standard reporting format in Annex D.1.The main text of the report should be no more thanten pages in length, followed by attachments.

E. Management and ContractProvisions

Upon selection, NGOs will be asked to attend anational orientation about KDP, led by NMC staff,to be introduced to project objectives, design, andprocedures. A range of monitoring methodologieswill also be reviewed. NGOs will then be asked tofinalize a time schedule and workplan for the moni-toring activities. The NGOs’ main counterparts forthe provincial monitoring will be the KDP field con-sultants, although KDP central-based staff will bedirectly involved at early stages to provide feedbackon reports as well as provide technical assistance andguidance throughout the monitoring process.

The contract will be signed between PMD and thecontracting NGO for an initial one-year period.That contract is subject to renewal after one yeardepending upon contract performance and availabili-ty of funds.

The NGO will negotiate a lump-sum fee contractwith PMD. Payments to NGOs will be made inthree installments. Thirty percent of the total fee willbe paid out at the initiation of the contract. An addi-tional 60 percent of the total fee will be paid basedupon receipt and acceptance of the first three month-ly reports. The remaining ten percent of the total feewill be paid after all twelve reports have been sub-mitted as agreed.


Given that different organizations will be undertak-ing the NGO monitoring in numerous parts of thecountry, there is a need for a standard reporting for-mat to ensure that similar issues are covered in dif-ferent locations and make it easier for readers to findinformation on specific issues. The reporting formatis intended to provide concise information on basicissues and should be no more than ten pages for eachmonthly report. However, additional detail (includ-ing brief descriptions of villages visited, photos, fieldnotes, and other logistical information) should beattached as annexes to ensure that the richness ofcollected information is not lost. Brief case studiescan be compiled if instructive and interesting. Directquotations from villagers and stakeholders are highlyencouraged and should be included in the report.

The reports should be as specific as possible. In all thecomments, please be specific about details and loca-tions (name of village, kecamatan, to whom theNGO spoke, etc.)

Detailed questions for each topic are attached on thefollowing pages.

Reports must follow the following format:1. Cover Page2. Table of Contents3. Summary Table of Problems and

Recommendations (see below)4. Table of Geographical Areas Covered5. Methodology Used6. Participation (especially of poor people, women,

and the “disenfranchised”) and barriers to activeparticipation and decision-making

7. Information-sharing and transparency (issues ofawareness, information dissemination and on-going flows)

8. Role of technical assistance in the broad sense(FK, FD, KMT-Kab, local officials, NGOs,technical team, others) and the following of pro-cedures

9. Quality of Program Outputs, Technical qualityof infrastructure, economic and social activities.

10. Financial Review11. Strong and weak points of the program, recom-

mendations for improvement


ANNEX D.1: NGO INDEPENDENT MONITORING REPORT FORMAT

No Location (Name of village and kecamatan) Problem Recommendation Follow-up needed &encountered by whom

1

2....

Name of Name of Name of persons Position/Title of persons interviewed (i.e. villager, Village Council Kecamatan visited village visited interviewed (optional) member, Village Head.)

Summary Table of Problems and Recommendations

Table of Geographical Areas Covered

These questions are the most basic required for inde-pendent monitoring and reporting on findings. TheNGO is encouraged to use a variety of techniquesand interview several respondents to answer thesequestions and explore other questions as needed. Itshould also be noted that the purpose is to get generalanswers to the questions, rather than ask all questions ofall respondents in all locations.

Participation –– The NGO should monitor partici-pation during the different phases of the KDP cycle.

During the planning and socialization phases, are vil-lagers aware of KDP and do they understand itsprinciples, regulations, and requirements? Who isaware of the project and who isn’t? Why? Do facili-tators, consultants and local officials understand theprinciples and process?

Are sub-village and village meetings well attended?Who attends and who doesn’t? Why? Who makesproposals? Is the process open and transparent andare the decisions made participatory? What degree ofparticipation do “regular” villagers have, especiallywomen, poor people, and the “disenfranchised”? Dodecisions represent the needs and wishes of themajority of villagers? Is there evidence of interven-tion from outside or domination by the village heador village elite?

Are separate women’s meetings held? Are they wellattended? Is there active participation from allwomen participants or do just a few women domi-nate the proceedings? Are poor women activelyinvolved?

Who is involved in the village mapping and wealthranking exercise? Is it well attended and are villagersactively participating? Do most villagers seem tounderstand the process? Are the real needs of thecommunity identified?

Are villagers involved in the proposal writingprocess? What role do they play? Are proposalsdeveloped based on the actual needs of the originat-

ing group or are design details left up to the proposalpreparation team (Tim Penulis)?

What role does the Technical Assistance Team playin developing proposals? How were they selected?Do villagers find their input useful? What role doesthe Verification Team play before the UDKP meet-ing? How were they selected? Are villagers aware oftheir recommendations? If so, what action was takenbased on Verification Team’s recommendations?

Are the UDKP meetings well attended? Whoattends and who doesn’t? Why? Are decisions negotiated in the group or decided by a small number of people? Is there competition based onquality and poverty impact of proposals? Are thereelected representatives from all participating villages,including at least two women from each village? Doother villagers attend, especially members of groupswith proposals? What degree of participation do village representatives have in decisions? What wasthe role of the Verification Team in the UDKPIImeeting? Are UDKPII meeting decisions reportedback to villagers who did not attend? How and bywhom? Do groups who did not receive fundingunderstand the reasons why they were not funded?

Who is involved in the implementation process?Who isn’t? Why? What roles do poor people,women, and marginalized groups play?

Who benefits from funded activities? Who doesn’t?Why? Are beneficiaries those that were intended inthe original proposal? Do projects especially benefitpoor people and women? Are there projects thatbenefit only a small group of people? Are there peo-ple who are excluded from project benefits? If so,who? Are beneficiaries satisfied? Why or why not?Are those who do not benefit upset? How is thisdealt with?

Are villagers aware of the projects that are fundedunder KDP, who is supposed to benefit, and howfunds are supposed to be used? Do they knowwhether projects are being implemented and funds


ANNEX D.2: SAMPLE OF MONITORING QUESTIONS

being used as intended? If yes, how? If not, why not?Has there been any leakage of funds or other corrup-tion in projects (diverting benefits, changing projectgoals, etc.)? How and by whom are problemsencountered dealt with? To whom are they reported?What was the response? If problems have notoccurred, do villagers know who to report to orwhere to report problems?

Information and Transparency –– The NGO shouldmonitor if program information is transparent andshared widely within the communities.

Are villagers aware of KDP and do they understandits principles, regulations and requirements? Areinformation boards well-placed, clear, up to datewith all relevant information posted in kecamatan,villages, and hamlets? Do villagers know about them?Do they use them? Why or why not? What othermechanisms are used to disseminate basic informa-tion on project principles and overall process? Whatmechanisms are used to keep villagers informed ofprogress, including decisions at village and keca-matan meetings, disbursements, and payments? Arethere breakdowns in the information flow at specificstages or in getting information to certain villagers(women, poor people, distant hamlets, illiterate/non-Indonesian language speakers, etc.)? Are villagersinformed of decisions after they are made and dothey have opportunities to give input? During imple-mentation, is financial information on individualprojects and on bank withdrawals posted?

Quality of Technical Assistance and Institutions ––NGOs should monitor the role of TechnicalAssistance Teams, Verification Teams, UPK, consult-ants and other government officials in implementingKDP.

How are local officials supporting the project? Howare the facilitators and local consultants assisting theproject? Do they work together well? Are involvedofficials and consultants often in the villages? How isthe quality of the technical assistance provided byconsultants and contractors? Do they understand theprinciples and procedures of KDP and are they fol-lowing them? What are villagers’ views on their

involvement? How can support from technical con-sultants and officials be improved?

Quality of Outputs,Technical Soundness –– TheNGO should monitor the quality of the economic,infrastructure and social activities.

Are infrastructure projects technically sound? Whatimpact have these projects had on the village? Arethey maintained? By whom and how? If not, whynot? Are there sanctions for incomplete or technical-ly flawed projects? If so, are they enforced?

Are economic activities successful enough for loansto be repaid? Are the loans being put to productiveuses? What impact have the economic activities hadon the household welfare of the recipients? Arerepayments on time? If not, why not? Are theresanctions for late or missed payments? If so, are theyenforced? Do group members know the conditionsand amount of the loan and repayment schedule?Are there regular meetings of the group to reportupon the loan status and repayments?

How are repaid funds being used? By whom andhow are decisions made for use of revolving funds?

Are the social activities successfully completed? Arethey technically sound? What impact have theseactivities had on the village?

What do villagers think about the projects, especiallypoor people, women, and the “disenfranchised”? Arethey satisfied with the projects? What impact havethe projects had on their lives (economic, social,etc.)? Have there been problems at any stage? Howhave they been dealt with? How could the process beimproved? Do they see any differences from howplans and decisions are usually made and their role inthe process? Are there changes in villagers’ under-standing and views of the LKMD and UDKP?

Financial Review of Bookkeeping –– The NGOshould do a spotcheck of the UPK, LKMD, andeconomic group financial records.

Are the UPK and the LKMD/TPK handling project funds appropriately? Are standard prices for


goods made public? Are financial transactions transparent and properly documented, and are theymade known to the community? Are there any signsof mark-ups on goods? In the case of economicloans, are the loan repayments being recorded accu-rately? Are the FKs and the KMKabs checking thebooks regularly?

Strong or Weak Points of the Program –– The NGOshould mention any strong or weak points of KDPbased upon their observations and interviews withvillagers and KDP stakeholders. Quotations fromstakeholders should be provided as much as possible.

What are the strong points of the program as seen inthe field? What do villagers cite as the benefits ofKDP? What do government officials such as thePjOK, Kecamatan Head, Village Head or LKMDview as benefits or positive impacts from KDP? Canyou describe any success stories or positive examplesfrom which others can learn?

According to your NGO team members, what arethe weak points of the program? What do villagersand other program stakeholders cite as the weakpoints? What are recommendations for improvingthe program?



ANNEX E: SAMPLE WORKSHOP SCHEDULE FOR NGOS INVOLVED ININDEPENDENT MONITORING

Sample Schedule for One-Week Orientation Workshop1

Time Activity Person-in-Charge

Day One

08:30 – 9:00 Registration09:00 – 09:45 Opening: PMD, NMC Government

• Welcome remarks Provincial Coordination Team• Introductions• Review Agenda• Procedures for Workshop

09:45 – 10:30 KDP Orientation: Principles and Project Cycle KDP Provincial Coordinator

10:30 – 10:45 Coffee Break

10:45 – 12:00 KDP Orientation Continued KDP Provincial Coordinator, Question & Answer session NMC

12:00 – 13:00 LUNCH

13:00 – 14:00 KDP M&E System Overview M&E Unit

14:00 – 14:30 Presentation by Association of Independent Journalists (AJI) AJI


14:50 – 15:30 Question & Answer Session with AJI AJI

15:30 – 16:00 Small Working Groups to discuss the day’s presentations and answer questions

16:00 – 17:00 Plenary to answer any general questions Evaluation of Day 1 M&E Unit NGO Coordinator

17:00 CLOSE DAY 1

Day Two

09:00 – 09:15 Review day’s agenda and evaluations from Day 1 M&E Unit NGO Coordinator

09:15 – 10:30 Terms of Reference for NGO Independent Monitoring, Q&A M&E Unit NGO Coordinator


10:50 – 12:00 Presentation by NGO currently undertaking monitoring: NGOexperiences and lessons learned

12:00 – 13:00 LUNCH

13:00 – 13:15 Team game KDP Trainer13:15 – 15:00 Presentation of various types of monitoring tools, quantitative M&E Unit

and qualitativeAdvantages & disadvantagesReview four different M&E methodologies and techniques:• Direct observation• Informal Interviewing • Focus Group Discussions• Community Participatory Monitoring


15:30 – 16:30 Small working groups – role play using different M&E techniques

16:30 – 17:15 Plenary session – discussion of advantages and disadvantages of methodologies.

Evaluation of Day 2 M&E Unit

1 One week was considered the minimum time needed for the initial NGO orientation. After the initial orientation, KDP provided additional trainingand meeting opportunities for the NGOs twice a year after they had gained field experience doing the independent monitoring. Additional sessionson monitoring tools and practicum should be included if time and budget permit.

Time Activity Person-in-Charge

Day 3

09:00 – 09:15 Review of day’s agenda and evaluations from Day 2 NGO Coordinator

09:15 – 10:30 Financial Reviews: Financial tips for non-finance people. Head of NMC Financial UnitHow do we check simple book-keeping?


10:50 – 12:00 Group exercises on financial review and checklists NMC Financial Unit

12:00 – 13:00 LUNCH

13:00 – 13:15 Team Game KDP Trainer

13:15 – 14:30 Technical Reviews: Technical tips for non-engineers. What should we check in simple infrastructure projects? NMC Engineer

14:30 – 15:00 BREAK

15:00 – 16:30 Group exercises on technical review and checklists NMC Engineer

16:30 - 16:45 Evaluation of Day 3

Close Day 3 NGO Coordinator

Day 4

09:00 – 17:00 All day field trip to KDP field sites; practice different KDP Provincial Teammonitoring techniques

Day 5

09:00 – 10:30 Discussion of field observations, sharing of general NGO Coordinatorimpressions, plenary and group sessions


10:50 – 12:00 Review of reporting format and requirements for M&E Unit, PMD ContractsNGO monitoring, Q&A Officer

12:00 – 13:00 Lunch

13:00 – 13:30 Games KDP Trainer

13:30 – 14:45 Development of workplans by NGO NGO Coordinator


15:00 – 16:30 Sharing of individual workplans in plenary NGO Coordinator

16:30 - 17:00 Any other business, establish tentative date & venue for NMCnext meeting

17:00 – 17:30 Official Closing Ceremony PMD


ANNEX E: SAMPLE WORKSHOP SCHEDULE FOR NGOS INVOLVED ININDEPENDENT MONITORING

It is important that the Program establish a simple,password-secure database (in Access or another data-base program) to record complaints or problemsbeing reported. Keeping a detailed, accurate accountof each complaint is critical to facilitate follow-upaction as well as program reporting. The HandlingComplaints Unit should generate a monthly reportlisting complaints received and other key informationas listed below. This report should be circulated toProject Senior Management, World Bank, and sen-ior field staff. The following information fields aresuggested:1. Date complaint received2. Location where problem occurred (province, dis-

trict, sub-district…)

3. How the complaint was received (i.e., letter, tele-phone, field report, NGO,etc.)

4. Name of plaintiff (optional)5. Contact information for plaintiff (optional)6. Type of complaint. This can be divided into sev-

eral categories: a) violation of principles or pro-cedures; b) misuse of funds; c) improperintervention; d) force majeure; e) other

7. Brief description of complaint (text field)8. Who should handle the complaint9. Actions taken to respond to the complaint and

dates10. Status of complaint (in process, solved)11. Date complaint or grievance was resolved


ANNEX F: COMPLAINTS/GRIEVANCES DATABASE FIELDS

I. BACKGROUND

The Kecamatan Development Program (KDP) is aGovernment of Indonesia effort to alleviate povertyin rural communities. KDP is a three-year programbegun in 1998 through a World Bank loan (IBRDloan number 4330-IND) to finance village activities.KDP aims to strengthen kecamatan and villagecapacities and improve community participation byproviding block grants directly to kecamatan and vil-lages for small-scale infrastructure, economic andsocial activities. KDP works directly with villagecouncils (LKMD or BPD) and kecamatan develop-ment forums (UDKP). KDP is supported by teamsof facilitators and consultants from village to nationallevels who provide technical support and training.

Most importantly, KDP emphasizes the principles ofcommunity participation, especially for women andpoor villagers, transparency, and sustainability. AllKDP activities aim at allowing villagers to maketheir own choices about the kinds of projects thatthey need and want.

During its first year of implementation (1998-1999),KDP provided assistance to 501 kecamatan spreadacross 105 districts in 20 provinces throughout thecountry. In its second year, 2000/2001, an additional269 kecamatan were added, covering over 11,325 vil-lages.

As of mid-February 2001, KDP has disbursed a totalof Rps. 671 billion (US$75 million) to kecamatan.While the government audit branch, the Financeand Development Supervision Agency (or BadanPengawasan Keuangan dan Pembangunan (BPKP)) isresponsible for regularly auditing KDP, there stillremains much to be done to strengthen local finan-cial management, especially with the kecamatanfinancial units, UPKs (Unit Pengelola Keuangan) andeconomic loan groups, and to prevent the misuse ofprogram funds. In its 1999 audit findings, BPKPfound in general that administration and documen-tation were not sufficient in places they visited.

Furthermore, the National Management Consultants(NMC) completed a brief ten-day needs assessmentin July and August 2000, and many UPKs requestedadditional training to improve their financial man-agement skills and their understanding of KDP prin-ciples and financial reporting requirements. Thus forYear Three, the NMC will form a five-person rovingteam of accountants in a Financial Supervision andTraining Unit within the NMC to coach and worktogether with UPKs and economic loan groups.

II. PURPOSE AND SCOPE OFTHE FINANCIAL SUPERVISIONAND TRAINING UNIT

The purpose of the Financial Supervision andTraining Unit is: (1) to improve the financial skills ofUPKs and economic loan groups; and (2) to checktheir financial and administrative records at the keca-matan and village levels.

Specifically, the Unit will:

Training:• Review UPK financial records and provide guid-

ance to UPK units on ways to improve theiraccounting and administrative skills. This includesthe formulation of balance sheets and properincome statements;

• Train UPKs on ways to improve loan repaymentand collection from economic groups;

• Improve the UPKs’ capabilities to report on KDPtransactions, especially to the UDKP fora (e.g,what should be reported, when, and how) and thecommunity at large;

• Review financial records of the economic groupsand provide guidance to the groups in recordingtheir transactions.

• Provide recommendations to PMD and NMC onways to improve the economic loan portfolio. Thisincludes developing an understanding of howother village micro-lending activities are operatingand providing some comparative analysis.


ANNEX G: TERMS OF REFERENCE FOR NMC’S FINANCIAL SUPERVISION AND TRAINING UNIT

Supervision:• Check UPK financial records for proper account-

ing, adequate documentation, contracts, bidding,unit prices of goods, and cost efficiency;

• Check bank statements and signature cards, allfinancial records and cashbooks, receipts, expendi-ture records for operational costs, and economicloan records;

• Ensure that documents have been properly signedand approved by appropriate parties;

• Check economic groups’ records and sample indi-vidual borrowers’ records of loan payment andrepayment;

• Ensure that financial and contractual informationis being shared with the community and has beenposted on the information boards.

• Visit the project site to verify that the activity/sub-project is actually in place.

• Participate in supervision missions to performfinancial reviews.

Other:• Coordinate with members of the NMC, including

the Regional Managers and Handling ComplaintsUnit to monitor financial progress of kecamatanand follow up on supervision recommendations;

• Brief and debrief KDP provincial, district andkecamatan teams before and after missions.

The members of the Financial Training andSupervision Unit will travel to various KDP loca-tions to provide training and check the financialrecords of UPKs and a select number of economicgroups. The Unit will be a five-person roving teamwhose members will spend at least 75 percent of thetime in the field. In general, the team should notoverlap with kecamatan already visited by the gov-ernment audit agency, BPKP.

Members are expected to meet with relevant actorsin the field to explain their tasks and findings beforeand after missions. Along with the UPKs and eco-nomic groups, this includes meeting when possiblewith government coordination teams, PjOKs,KMProps, KMKabs, FKs, TTDs, and village management teams.

III. MANAGEMENT OFTHE WORK

The NMC will form a Financial Training andSupervision Unit consisting of five persons trained inaccounting. The M&E Advisor will supervise theUnit. The Unit will coordinate closely with PMDand the NMC’s Regional Coordinators, FinanceUnit, and the Complaints Resolution Unit for trou-ble-shooting in special situations where corruption orfinancial anomalies are reported.

After recruitment, the Unit will spend two to threeweeks in Jakarta reviewing KDP documentation,UPK training materials and meeting with relevantPMD, NMC and World Bank staff for orientation.The Unit will also meet with BPKP to discussBPKP’s findings so far under KDP and their termsof reference and schedule for the upcoming year.During this initial orientation period, the Unit willdesign and agree upon operational procedures in thefield, standard reporting formats, and their quarterlyworkplan.

IV.TIMING

This activity is expected to begin in March 2001, assoon as five qualified persons are recruited. Theactivity will continue until the end of KDP PhaseOne.

V. REPORTING

The team will brief and debrief KDP provincial, dis-trict and kecamatan teams before and after the fieldmission.

The general report format will be:1. Introduction – Dates of the mission, timetable of

visits, and list of mission participants2. Specific location of mission and the number and

names of districts, kecamatan and villages coveredvisited. Description of methodology and criteriaused for site selection.

3. Main findings listed by kecamatan and village(reported in priority order)


4. Description of training provided by team members (topics covered, number of days, nameof participants)

5. Recommendations for follow-up action.

Annexes including financial tables, matrix of cases,list of trainees, photos

The Unit will share its written mission reports withPMD, World Bank, NMC, relevant consultants inthe field, as well as the provincial government coor-dination teams. The Unit will also organize meetingsto discuss the reports with senior program manage-ment.


TERMS OF REFERENCE for theAudit of Special Purpose ProjectFinancial Statements

ObjectivesThe overall objectives of the audit are: (i) to enablethe auditor to express a professional opinion on theproject financial statements, the operation of theoverall financial management system including inter-nal controls, and compliance with financing agree-ments; (ii) to provide project management withtimely information on financial aspects of the projectto enable follow-up action; and (iii) to assess on theachievements of project objectives as measured byperformance indicators.

The audit should cover the entire project, i.e. cover-ing all sources and application of funds by all imple-menting agencies. The auditor should visit thevarious implementation units and other agencies asconsidered necessary for the audit.

ScopeThe audit will be carried out in accordance withInternational Standards of Auditing and with theAudit Manual for World Bank Financed projects( July 1998). It will include such tests and controls asthe auditor considers necessary under the circum-stances. Specific areas of coverage of the audit willinclude the following:(1) an assessment of whether the project financial

statements have been prepared in accordance withconsistently applied Generally AcceptedAccounting Principles (GAAP) and give a trueand fair view of the operations of the project dur-ing the year and the financial position of the proj-ect at the close of the fiscal year. Any materialdeviations from GAAP, and the impact of suchdepartures on the project financial statements aspresented would be stated

(2) an assessment of the adequacy of the projectfinancial management systems1, including inter-nal controls. This would include aspects such asadequacy and effectiveness of accounting, finan-

cial and operational controls, and any needs forrevision; level of compliance with established poli-cies, plans and procedures; reliability of account-ing systems, data and financial reports; methodsof remedying weak controls or creating themwhere there are none; verification of assets andliabilities; and integrity, controls, security andeffectiveness of the operation of the computerizedsystem (if any), and

(3) an assessment of compliance with provisions offinancing agreements, especially those relating toaccounting and financial matters. This wouldinteralia include verification that:(a) all external funds have been used in accor-

dance with the conditions of the relevantfinancing agreements, with due attention toeconomy and efficiency, and only for the pur-poses for which the financing was provided.

(b) counterpart funds have been provided andused in accordance with the relevant financ-ing agreements, with due attention to econo-my and efficiency, and only for the purposesof which they were provided;

(c) expenditures charged to the project are eligi-ble expenditures and have been correctly clas-sified in accordance with the relevantfinancing agreement;

(d) goods and services financed have been pro-cured in accordance with the relevant financ-ing agreement;

(e) all necessary supporting documents, records,and account have been kept in respect of allproject activities;

(f ) clear linkages exist between the accountingrecords including accounts books and theProject Financial Statements;

(g) where Special Account has been used, it hasbeen maintained in accordance with the pro-visions of the relevant financing agreement;

(h) statement of expenditures (SOE) used as thebasis for the submission of withdrawal appli-cations accurately reflect expenditures andactivities on the project;


ANNEX H: TERMS OF REFERENCE FOR KDP FINANCIAL AUDIT

(i) project expenditures as reported by the proj-ect implementation agencies are reconciledwith the amounts withdrawn from theSpecial Account and the amounts depositedto the special account are reconciled with theamounts disbursed from the IBRD Loan.

(4) an assessment on the achievements of theplanned results of the projects as measured by theperformance indicators as stipulated in the rele-vant financing agreement.

Project Financial StatementsProject Financial Statements prepared by the projectexecuting agency would be based on the projects’sFinancial Monitoring Reports (FMR), and shouldinclude:(a) project accounts, covering (i) Annual Project

Expenditures and Financing; and (ii)Cumulative Project Expenditures and Financing.Sources of funds would show IBRD/IDA, otherdonors, and GOI counterpart funds separately.Project expenditures would be summarized bymain project components, disbursement cate-gories and by project location (province or dis-trict) both consolidated for the current fiscal yearand accumulated to date.

(b) financial statement of special account, covering(i) deposits and replenishments received fromthe Bank; (ii) withdrawals from the specialaccount; and (iii) the remaining balances at theend of the fiscal year

The auditor should provide an opinion as to thedegree of compliance with the Bank’s procedures andthe exactitude of the balance of the Special Accountat year-end. The audit should examine the eligibilityand integrity of financial transactions during theperiod under review and fund balances at the end ofthe period, the operation and use of the specialaccount in accordance with the financing agreement,and the adequacy of internal financial controls.

The auditor must assess a reconciliation reportbetween the project expenditures made from the spe-cial account and the withdrawals from the specialaccount. Reconciliation should also be made with theamounts paid from the pre-financing account anddirect payments.

Statement of Expenditures (SOEs) The auditor is also required to audit all SOEs (paidfrom the special account and/or other accounts) usedas the basis for the submission of withdrawal applica-tions. The auditor should apply such tests and con-trol as the auditor consider necessary under thecircumstances. These expenditures should be careful-ly compared for project eligibility with the relevantfinancing agreements, and with reference to theProject Appraisal Document for guidance when con-sidered necessary. Ineligible expenditures identified ashaving been included in withdrawal applications andreimbursed by the World Bank should be noted sep-arately by the auditor. The total withdrawals underthe SOE procedure should be part of the overall rec-onciliation of Bank disbursements described above.

Audit ReportThe audit report shall contain the auditor’s opinionon the fairness of the project financial statements.The audit opinion should include a separate para-graph commenting on the accuracy and propriety ofexpenditures withdrawn under the SOE proceduresand the extent to which the Bank can rely on SOEsas a basis for loan disbursement. The report shouldrefer to the auditor’s TOR.

The auditor should submit the report to the projectexecuting agency who should then promptly forwardone copy of the audited accounts and report to theBank. It should be received by the Bank no laterthan six months after the end of the project’s fiscalyear.

Management Letter In addition to the audit reports, the auditor will pre-pare a “management letter,” in which the auditor will:(a) give comments and observations on the account-

ing records, systems, and controls that wereexamined during the course of the audit; andidentify specific deficiencies and areas of weak-ness in systems and controls and make recom-mendation for their improvement;

(b) give comments on economy, efficiency, andeffectiveness in the use of resources;

(c) report on the achievement of the planned resultsof the project


(d) report on the degree of compliance of each ofthe financial covenants on the financing agree-ment and give comments, if any, on internal andexternal matters affecting such compliance;

(e) communicate matters that have come to atten-tion during the audit which might have a signifi-cant impact on the implementation of theproject, and

(f ) any other matters that the auditors considerspertinent.

GeneralThe auditor should be given access to all legal docu-ments, correspondence, and any other informationassociated with the project and deemed necessary bythe auditor. Confirmation should also be obtained ofamounts disbursed and outstanding at the Bank.

The auditor should be familiar with the Bank’s AuditManual for World Bank Financed Projects whichprovide guidance to auditors conducting audits ofWorld Bank financed projects.


Indonesia Kecamatan Development Program › content › usergenerated › asi › cl… · assess...

Documents

Transcript of Indonesia Kecamatan Development Program › content › usergenerated › asi › cl… · assess...