1 | P a g e

Review of Peace Dividend Marketplace

Afghanistan

Final Report, August 2010

2 | P a g e

Contents

Introduction 3

Market and Policy Context 6

Market Constraints 9

Program Performance 14

Relationships with Clients 24

Regional Offices 30

Risk Factors 33

Future of Project 36

Summary of Conclusions and Recommendations 40

Annex One: Interview Respondents 44

3 | P a g e

Introduction

The Canadian International Development Agency (CIDA) is providing funding of $9.0m over

four years from 2008-2012 for the Peace Dividend Marketplace Afghanistan (PDMA)

program. The program is delivered by the Peace Dividend Trust (PDT), a Non-Governmental

Organisation (NGO) headquartered in Ottawa, Canada. PDMA seeks to apply the spending of

the international military, donor and diplomatic community in Afghanistan to assist the

growth of the country’s private sector. It does this by providing information and advice to

tackle information and knowledge barriers to trade between them.

The intended beneficiaries of the program are Afghan private sector businesses, particularly

SMEs. The principles behind it are that the private sector is the main driver of growth and

that increased support for entrepreneurship should happen early in the stabilisation

process. Eighty percent of local businesses which PDMA works with employ less than 50

people. The international community is not the intended beneficiary, though the project

should contribute to stabilisation and improve the efficiency of procurement methods.

The PDMA project followed on from a pilot project, also funded by CIDA, which PDT

delivered during 2006-08 covering only Kabul. This pilot facilitated around $47m of business

between international buyers and local vendors, compared with its initial target of $5m, and

was judged to be sufficiently successful to justify a larger, nation-wide project launched in

2008 as the PDMA.

The project is intended as a short term intervention, tackling the information market failure

then either winding down or transferring any activities still required to a local delivery

agent. The end-point planned for the CIDA project is March 2012.

Specifically, the activities delivered by the project are:

Management of a database of potential suppliers, all of whom must be registered

with the project and have their basic details verified through periodic telephone

survey and visits. There are 5,500 businesses (May 2010) on the national register.

Distribution of tenders to registered businesses.

Advice to international procurement officials on sourcing local suppliers, including a

matchmaking service to put buyers and sellers into direct contact. This matchmaking

service accounted for over 70% of the contract value facilitated by the program up to

March 2010.

Training for suppliers in the standards and procedures required by procurement

offices.

Support and mentoring to local suppliers to improve their business capacity.

Organisation of a series of Afghan Business Conferences (ABC) to bring buyers and

sellers into direct contact.

4 | P a g e

Specific initiatives, primarily training, for female entrepreneurs.

The program is managed from PDT’s office in Kabul and also has field offices in Mazar-e-

Sharif, Jalalabad and Kandahar. In parallel, PDT delivers a specific project with similar

objectives in Helmand province, on behalf of the UK’s Department for International

Development (DFID). DFID has committed £720,000 ($1.16m) to the Helmand project for

the period December 2008 to December 2010.

In June 2010, Upper Quartile was commissioned by CIDA and DFID to review the progress of

the PDMA to date. Each donor had slightly different objectives for their part of the review

and for CIDA the requirements were to:

Assess the program’s achievements against its logic model and Performance

Management Framework (PMF), making recommendations for improvements if

necessary.

Discuss any unintended consequences of the project.

Assess the effectiveness of PDMA’s services for increasing local procurement and

improving the capacity of local businesses.

Assess the effectiveness of PDMA’s relations with its clients and client satisfaction

with its services.

Assess whether PDT has undertaken sufficiently deep analysis of the reasons why

local suppliers do not secure a greater share of international business.

Assess whether the program demonstrates value for money and whether it is

possible to determine the “on the ground” impact of the contracts PDMA has

facilitated.

Assess the impact of the newer field offices in terms of their sustainability,

effectiveness and operating costs as compared to the Kabul office.

Assess the sustainability of results and whether there is ownership, financing

mechanisms and institutional capacity to sustain the program after CIDA has

completed its support.

Suggest options for improving sustainability as appropriate.

Assess risks and the effectiveness of mitigation measures.

Assess the effectiveness of PDT’s management of the project and provide

recommendations for any improvement required.

The review was undertaken over June and July 2010 with an integrated work programme

covering the needs of both CIDA and DFID. It included:

A review of background information.

Interviews and observation of work with PDMA.

Interviews with CIDA and DFID staff in Ottawa (CIDA), East Kilbride (DFID), Kabul,

Helmand and Kandahar.

5 | P a g e

Interviews with procurement personnel in embassies, donors and the military in

Kabul, Helmand and Kandahar.

Interviews with Afghan businesses and representatives in Kabul, Helmand and

Kandahar.

Field visits to Camps Bastion and Leatherneck, the Provincial Reconstruction Team

(PRT) in Lashkar Gah, Helmand, and the Kandahar Air Force (KAF) base in Kandahar.

Over 50 meetings took place and a full list of respondents is included as Annex One. One gap

in the intended methodology for the review is that we were unable to visit the field offices

in Jalalabad and Mazar-e-Sharif. The costs and visibility of security were agreed to be too

high and our assessment of them has therefore had to be based on secondary data.

6 | P a g e

Market and Policy Context

Market context

There are no reliable estimates of the total expenditure by the international community in

Afghanistan. One estimate from the International Security Assistance Force (ISAF) put the

figure in a range of $US10-20 billion each year, of which around $US2 billion is spent within

the country. This contrasts with Afghanistan’s estimated GDP of $US12.9bn in 2008,

emphasising the huge impact that international expenditure can have on the Afghan

economy.

There is a wide range of buyers, nationalities, departments, types of purchases and

procurement procedures involved – a very complex demand side. The broad categories of

buyer are:

NATO headquarters, which buys goods and services on behalf of ISAF as a whole.

US Military headquarters in Kabul, which is the largest single purchaser of supplies.

US Military in the field, both “inside the wire” on major bases such as Baghram and

KAF and “outside the wire” in field operations.

The military procurement sections of other members of the coalition, notably the

UK, Canada, the Netherlands (until 1st August, 2010) and Germany but also smaller

members such as Denmark and Estonia. As with the US, all of these buy for

headquarters and for inside and outside the wire.

The international donor community along with their implementing partners.

Other international agencies such as UNAMA.

International Embassies in Kabul, many of which function as central procurement

offices for other parts of their national mission.

The range of purchases made is similarly complex: small household items; security, medical

and other life support services; food; and construction. The final item, construction, is by far

the largest in value terms and covers anything from small projects within Embassy

compounds to larger projects within military bases, through to major infrastructure projects

in the field.

Each buyer, and in most cases each category of purchase, has to operate within set

procurement legislation and procedures which are generally defined from the home

country. Procurement officers are accountable to headquarters for ensuring compliance

with these standards.

7 | P a g e

Therefore the route to winning contracts with the international community varies by

nationality, the function and location of the buyer and the products or services being

purchased, all within a procedural framework set from home base. Added to this,

procurement personnel are generally rotated in and out of the country on the same basis as

other civilian and military staff, meaning that vendors need to continually rebuild

relationships with their individual customers.

This would be a difficult market environment for any supplier but for the still under-

developed Afghan private sector facing barriers of language, limited understanding of

procurement procedures, limited knowledge of international quality standards and

constraints on direct access to military bases it is a daunting challenge.

The proportion of international expenditure realistically available for local vendors to bid for

also varies considerably by market segment. In most Embassies, service requirements such

as security cover and medical support are unlikely to be purchased locally for the

foreseeable future, yet are generally the largest single items of expenditure. Once they are

taken into account the proportion available to purchase locally is much more limited, often

in single figure percentages. At the other extreme, for many military construction projects in

less secure Districts there is little option other than to use local suppliers – building new

infrastructure or community facilities will depend almost entirely on entering into contracts

with local businesses.

Policy context

The policy context for local procurement in Afghanistan is evolving quickly. It influences the

current operation of PDMA and may influence decisions by DFID and CIDA regarding their

future support for the program.

When PDMA was first established in Afghanistan, local procurement was accepted as a

“good thing” which had potential to promote private sector and economic development.

DFID and CIDA were early movers in recognising this potential, reflected in their agreement

to provide funding for the project. But within the rest of the international community, whilst

there was a willingness to support the principle, there was only limited commitment to

putting it into practice. Local procurement was not a mandated responsibility and was not

being institutionalised in procurement practice, which is in reality the only way it can be

independent of the goodwill of individual buying agencies and officials.

Since then, a number of developments have changed the policy context:

The Afghan First initiative was launched by Ambassador Eikenberry in November

2009, creating a greater degree of profile, urgency and advocacy on behalf of local

procurement. Afghan First has rapidly become part of the lexicon of international

procurement officials.

8 | P a g e

In parallel with this, UNAMA set a target for the international community to increase

its level of local procurement by 10% (albeit without a definition of the baseline).

The US Government has embedded a policy of local procurement within its

procurement law and this is now rolling out to the regulations within which officials

must operate. This is the first example of local procurement becoming

institutionalised in buyers’ day to day practices.

Most recently, Task Force 2010 has been launched under the leadership of Rear-

Admiral Dussault. It places local procurement very much at the heart of the

coalition’s counter-insurgency (COIN) strategy, alongside the military and

governance strands. Its high profile is already influencing procurement strategy and

it will be supported by a team of intelligence and forensic accounting experts to

ensure the benefits accrue to the legitimate economy.

The combined significance of these developments is that local procurement has a much

higher profile, is being promoted as an essential part of the coalition’s strategy and has

senior political and military buy-in. Commitment to it will be expected to shift from “in

principle” support to practical implementation with procurement officials being held

accountable for the results. This means that decisions on the role, future direction and

funding of PDMA will have to be made in a more visible and more politically charged

context.

One final contextual point is that until now, PDMA has not been mandated to promote local

procurement of fresh fruit and vegetables (FFV). This sector was initially in DFID’s contract

with PDT for the Helmand project but was removed in response to concerns over security

and quality – the same concerns that have constrained similar efforts in the past. A pilot

project is now however being planned at KAF, with participation from the military, PDMA

and Supreme (the lead catering contractor on site). If successful this would reopen the case

for more extensive local purchasing of FFV by the international community, which in turn

has huge potential to stimulate the Afghan agricultural and agri-processing sectors –

significantly more so than the current dominance of the construction sector in the PDMA

program.

9 | P a g e

Market Constraints

PDMA’s approach rests on the premise that international buyers and local vendors find it

difficult to connect in the Afghan market place. This is primarily caused by information

market failures, with buyers unable to identify or assess potential vendors, and suppliers

insufficiently aware of the processes and standards they need to meet. We were asked to

examine whether PDT has undertaken sufficiently deep analysis to validate these

assumptions.

Constraints on buyers

There is a wide range of buyers in the market with different procurement requirements and

procedures. They cover four main categories of spend:

Civilian purchases, mostly Embassies, donors, NGOs and other agencies supporting their staff in Kabul or in field offices.

Military domestic procurement, supporting the housekeeping and administrative needs of the military.

Military construction expenditure within bases, meeting the logistics needs of the very large communities on Camps Bastion, Leatherneck and KAF.

Military construction expenditure in the field, supporting stabilisation and reconstruction activities.

Civilian purchases: The main items in this category are security, medical support and other

life support services. It also includes household and office supplies and occasional small

construction, repair and maintenance projects.

The scale of purchases realistically available to local vendors from this category is in practice

very limited. Security, medical and catering services account for by far the greatest

proportion of expenditure and, with limited exceptions where local labour can be employed,

there is little likelihood of these being sourced locally for the foreseeable future.

This leaves the relatively small budgets for basic household and office supplies. Most buyers

have the option to order these from home national base or from a regional centre, usually

Dubai. There is however a general willingness to source from local suppliers where possible,

not just because of Afghan First policies but also for practical reasons such as speed of

delivery or lower costs. Most procurement offices have local national staff who are able to

10 | P a g e

directly source and check potential local suppliers and are often able to visit their premises.

As a result, there is not a major information failure other than in the early stages of

establishing the office.

Procurement offices in the field are in a more difficult position than those in Kabul. There is

a smaller local supply base and more restrictions on movement, including for local national

staff. Supplies are also needed to support civilian staff in operating bases in outlying

districts. The information gap is therefore wider, though should be temporary as links to

local suppliers develop.

Military domestic procurement: Military bases in Kabul, provincial centres and in theatre

also require basic household and office supplies – items that are not mission-critical. For

these domestic items, a similar situation applies as with civilian buyers. The easiest option is

simply to order up items from home base. These orders are however of low priority for

freight transport from home and there can be substantial delays in their arrival. Local

sourcing therefore becomes an attractive option, driven by convenience rather than policy.

On Bastion and Leatherneck, there is no significant commercial centre nearby and

procurement officers generally cannot move outside the wire. It is not possible to employ

local nationals on the site. Information on and access to local suppliers is therefore very

difficult. Transport of supplies into the bases is also constrained, with trucks often waiting

up to five days for access. Local procurement is not an easy option, despite the potential

time and cost advantages.

Military construction expenditure within bases: Military bases such as KAF have continuous

construction programs under way. These are generally handled by foreign contractors

located within the perimeter of the base. Afghan contractors are usually not allowed to

operate within the wire, though the foreign contractors do employ and accommodate

Afghan labour on site subject to security restrictions.

The foreign contractors face similar problems in local procurement as buyers for civilian

items. They are not able to deal with Afghan contractors directly because of movement

restrictions into and out of the base. It is therefore difficult both to identify local providers

and assess their reliability. They are encouraged to adhere to the Afghan First policy but

receive little guidance on how to achieve it.

Military expenditure in the field: The military has significant requirements to undertake

construction work in the field: bases for coalition, Afghan National Army (ANA) and Afghan

National Police (ANP) forces; infrastructure improvement in support of military operations;

construction projects in support of stabilisation objectives, such as those managed by the

11 | P a g e

Special Team of Royal Engineers (STRE) at the Helmand PRT and by the US Marines

Expeditionary Force (MEF) using CERP funds.

There is little option other than to use local contractors for these projects both for practical

and policy reasons. Foreign contractors find it difficult to mobilise for logistical and security

reasons (though there are some Turkish and UAE-based lead contractors on-site). And for

stabilisation reasons there are advantages in demonstrably using local contractors and

labour.

Many of these projects are driven by commanders in the field and with a great deal of

urgency. Field engineers are under considerable pressure to identify, validate and

commission local contractors on very short timescales. They are also subject to short

rotations, often of six months, and so it is difficult to accumulate and retain knowledge

about the local contracting market. In these circumstances there is a clear information gap,

particularly as stabilisation and reconstruction work extends into more outlying districts.

Engineers need a quick and reliable means of identifying local contracting capabilities.

Overall therefore, the validity of PDT’s diagnosis varies by category of buyer. It is at its most

relevant in military field construction work, especially in outlying districts. It is least relevant

for civilian domestic purchases in central locations where market information is much easier

to obtain.

Constraints on suppliers

We identified a number of constraints facing local suppliers.

Lack of access to invitations to tender (ITT): This can happen for a number of reasons. Only

10% of Afghan businesses use the internet as their main communications method, though

59% of construction companies use it as their main source of information.1 This makes e-

procurement methods inaccessible for many suppliers. ITTs are normally written in English

so there is a language barrier. For some military construction contracts, ITTs were often

simply posted outside procurement offices within bases meaning that local contractors had

no access to them.

Procurement procedures: Most procurement officers are obliged to work to a standard set

of procedures defined from headquarters in their home country. There can be some

variations to meet local requirements but these are generally minor. The procedures are

1 Afghan Business Attitudes on the Economy, Government and Business Organisations, 2009-2010, Afghan

Business Survey; National Endowment for Democracy, 2010

12 | P a g e

designed to demonstrate fairness and transparency but are largely alien to Afghan

businesses which have little experience of dealing meticulously with every detail of a

complex procurement process.

Quality standards: Procurement officers also have to work to a set of quality standards, on

health and safety for example, which have been developed for good reason in their home

region but again may be largely alien to Afghan businesses. Even where they are capable of

meeting the right standards, they may find it difficult to demonstrate that. Quality problems

extend even to simple commodities where language or literacy problems often cause

misunderstandings over what the buyer requires.

Fear of corruption. Afghan companies cite corruption as one of the main barriers to

developing their business. It is so endemic that they tend to assume it is there even when it

is not. This fear extends to their perceptions of the international community. They are

concerned that contracts are open to only a small network of businesses with the right

connections, or that local Afghan staff will favour their own contacts when awarding or

advising on contracts. This fear can lead to suppliers not even bothering to pursue business

with international customers. Another concern is the selling on of contracts, where a large

contractor wins the work and then initiates a chain of selling the contract to other

businesses, taking a margin on the transaction and leaving contractors at the lower end of

the chain to do the work with an inadequate budget. This in turn affects the quality of work

delivered.

Payment difficulties: Vendors have experienced difficulties in receiving payment from

international customers. This has included a complete failure to pay by some foreign

companies and a failure by procurement offices to meet promised payment times: 30 days

extending to 50-90 days.

Lack of feedback: Vendors rarely receive feedback on the bid process. They are often not

informed about project delays or decisions. The reasons for unsuccessful bids are rarely

explained. They therefore have no opportunity to learn from experience and in many

instances lack of feedback has led to unwillingness to continue making the effort.

Much of PDMA’s work rests on tackling these constraints: encouragement to buyers to

make their ITTs more accessible through translation into Pashtu and Dari; circulating notices

of ITTs to relevant businesses; training in understanding and following procurement

procedures; matchmaking services and business events to build more direct relationships

and trust.

13 | P a g e

We met a number of local businesses, including service providers (recruitment,

communications, business consulting) and construction businesses in Helmand. We also met

with the Afghanistan Chamber of Commerce and Industry (ACCI) to hear the views of the

main business advocacy and representative body and with the Afghanistan Investment

Support Agency (AISA).

Without exception these respondents were able to confirm the above diagnosis. Most had

experienced problems of corruption in the international community market place, not

necessarily from foreign nationals but in areas like the selling on of contracts and

favouritism by locally employed nationals. All had initially baulked at the complexity of

procurement procedures and some hired consultants to prepare proposals on their behalf.

Many had found difficulty in gaining direct access to buyers for transport or security

reasons. And many had experienced difficulties in understanding buyers’ quality and

technical requirements, though were confident of being able to meet them if they could just

work out what they were.

Interestingly, none of the businesses or agencies we met wished to see any relaxation of

standards in favour of local procurement. They accepted that they were a necessary part of

doing business with the international community and regarded them as an opportunity to

develop the experience and competence necessary to tackle export markets.

We therefore believe that the diagnosis on which PDMA is based remains an accurate

reflection of conditions in the procurement market.

14 | P a g e

Program Performance

This section reviews the performance of the PDMA to date from a number of perspectives:

1. Comparison of results against the targets set in the original Logic Model and Performance

Management Framework (PMF).

2. Economic impact of the program.

3. Performance of the Logic Model and PMF

4. Value for money

1. Performance against targets

CIDA’s current Logic Model for PDMA was prepared in November 2008 and sets out the

ultimate, intermediate and immediate outcomes anticipated from the program, along with

the outputs and activities required to achieve them. The PMF was finalised at the same time

and details the baseline, targets, data sources, data collection methods, reporting frequency

and responsibility for each of the outcomes and outputs defined in the Logic Model.

The ultimate and intermediate outcome targets defined in the PMF are by their nature set

at too broad level to isolate the specific impact of PDMA. But there are indicators from

secondary sources that the broad targets are moving in a positive direction. The latest

Economist Intelligence Unit report on Afghanistan2 provides a number of indicators of this:

The Afghan economy grew by 22.5% over 2009-10, making it one of the fastest

growing in the world.

This was largely driven by a boom in agricultural output and also by growing

Government and international donor expenditure. Performance in the construction

and manufacturing sectors was less strong.

Government tax revenues increased by 53%, reflecting improved tax administration.

Also, the Afghanistan Investment Support Agency (AISA) continues to report growth in the

number of businesses registered with it and therefore participating in the formal economy.

Its register now includes 12,000 businesses and it is processing 50 new licences and 50

renewals each day.

To this extent, therefore, the long and medium term outcomes to which PDMA is intended

to contribute are moving in a positive direction though it is impossible to say how much if

2 Country Report, Afghanistan; Economist Intelligence Unit, July 2010

15 | P a g e

any of this is attributable to the program. This also means of course that the business

environment for PDMA-registered companies is increasingly favourable: therefore the

hurdle that PDMA must jump to demonstrate an additional impact is getting higher.

The more specific and measurable targets for PDMA are in the immediate outcomes section

of the PMF. The results to date, as reported by PDT3, are summarised in the table below.

Immediate Outcome PMF target Cumulative results since

April 2008

Increased access to and

awareness of locally available

goods and services by

international buyers.

6,000 new entries on Business

Portal.

2,400 new entries, with 5,348

businesses now listed on Portal.

Increased access by local business

to international tenders.

9,000 tenders received and

distributed.

3,165 tenders obtained and

distributed 45,776 times.

Strengthened business

environment, improving formal

economy and tax revenues.

Matchmaking agreements for 100

businesses nationally.

205 contracts facilitated, worth

$340m.

Increased capability of local

suppliers to win contracts from

international buyers.

840 businesses attending training

seminars.

Over 1,300 attendees,

representing over 1,000

companies.

Increased visibility and

participation of female

owned/managed businesses.

25% of registered female

businesses registered for training.

90 of 207 (45%) registered female

businesses attended training.

Increased capacity of partner

organisations.

Continually expand marketing

strategies in line with region-

specific best practices.

Increase in impact of partners’

expenditure due to local

procurement. Increased

confidence in Afghan businesses.

Reporting of the quantitative targets indicates that at the half-way point in the program,

PDMA is mostly on track to achieve its final targets. The number of new businesses

registered on the Portal has reached 47% of the final target of 6,000. The number of

matchmaking agreements, expressed in the report as contracts facilitated, has reached 205,

over double the final target. The number of businesses attending training has already

exceeded the final target by almost 20%. The percentage of registered female businesses

attending training is almost double the final target. The main target still behind schedule is

for the number of tenders distributed. Just over one third of this target has been achieved

3 PDMA Semi-Annual Progress Report, October 1, 2009 – March 31, 2010

16 | P a g e

half way into the program. At this stage, we suggest this requires attention rather than

major concern. Our field work has shown that the program’s awareness amongst buyers

continues to increase and so the number of tenders notified should accelerate in the

remaining two years.

In addition to the PMF indicators, PDT reports a total number of contracts won by PDMA-

sourced suppliers of over 600, with a total value to date of over US$465m. There are no

reliable estimates of the international community’s spend in Afghanistan. Using the

previously quoted ISAF-sourced figure of $10-20bn annually, this suggests that PDMA is

associated with 1-2% of current annual expenditure. Given that only a limited proportion of

total coalition expenditure is realistically available to local suppliers, the actual level of

penetration of the potential market will be much higher. Again using an ISAF-sourced

estimate of $2bn being available for local businesses, the percentage of total expenditure

with which PDMA is associated could be as high as 20%. These estimates are based on very

rough estimates of the baseline but they suggest that PDMA is now involved with a

significant proportion of total available purchasing.

There are a number of important qualifications on the results reported against the PMF:

The original targets may have been set too low. Initial baseline data are difficult to

establish for any development initiative in Afghanistan and this makes target-setting

very difficult. Also, the systematic encouragement of local procurement is a

relatively new activity in Afghanistan and elsewhere, so there is not much previous

evidence on which to build meaningful targets. However, the Kabul pilot project for

PDMA did run for two years and exceeded its initial targets by a large number so this

should have provided a reasonable basis for calibrating the PMF.

There was no systematic collection of information on local procurement by the

international community until PDMA began. Since then, with exhortation from

UNAMA and PDT, the level of reporting has improved considerably. Part of the

apparent impact of PDMA may therefore be due to fuller reporting rather than to

actual increases. We cannot gauge the size of this effect but it will be having some

impact on the numbers. On the plus side, it does at least mean that there is now

more comprehensive baseline information available which can be used to recalibrate

the PMF targets.

The figures reported by PDT relate only to outcomes associated with PDMA’s

facilitation. That is different from genuinely additional outcomes which would not

have happened without the program’s intervention. Some of the progress reported

would therefore have happened without the program being in place. PDT has

however recently commissioned a job survey for the DFID-funded Helmand PDMA

17 | P a g e

project to attempt an estimate of the program’s economic impact and the

preliminary results of that are discussed later in this section.

The Afghan economy has grown rapidly during the time PDMA has been operating.

Thus an increase in business formation, business performance and tax revenues

would be expected anyway. It would require a wider study using a control group of

businesses with which PDMA has not been involved to isolate the program’s

contribution from the wider economic climate.

The figures are based on self-reporting by PDT. We have no reason to doubt the

validity of the data but it is important to note that they have not been independently

verified through.

Despite these qualifications, we are confident in concluding that PDMA is making very good

progress against the targets initially set for it – it is delivering the anticipated immediate

outcomes, in turn contributing to achievement of the medium and longer terms aspirations

set for the program.

2. Economic impact

Our terms of reference include a request to determine whether it is possible to calculate the

economic impact of contracts which PDMA has facilitated: for example employment; and

benefits to SMEs versus large businesses. In order to do this properly information on a

number of variables is required:

The level of employment and revenue in registered businesses before they received

support from the program.

The level of employment and revenue in supported businesses after intervention.

Separation of the employment and revenue increases due to the specific contracts

facilitated through the PDMA from the business’s overall performance.

For impact by size of company, a breakdown of contracts won by registered

businesses according to their employment size.

Also, to measure the additional economic impact of the program, i.e. the extent of impact

which can be attributed solely to PDMA’s intervention, further information would be

required. This would include matched data from a control group of businesses which have

not been supported by the program; and data on employment and income growth in the

overall economy to adjust for growth that would have been expected anyway.

Most of this required information is not available. PDT estimates that employment in

registered businesses is around 138,000. But this is a snapshot and does not cover the

18 | P a g e

“before and after” of PDMA intervention. Information is not collected on employment

related to specific contracts nor is this broken down by company size. For measuring the

additional impact, information is available on growth in the wider economy though not at

the level of disaggregation ideally required for a robust impact analysis. Also, there are no

data from any control group of businesses and to gather this was outside the scope of this

review.

In summary therefore, there is insufficient information collected by PDT (who were not

mandated or resourced to do so) or available from government or elsewhere to undertake a

proper economic impact analysis.

PDT has however just completed a survey in Helmand which provides some relevant data.4

Thirty one companies were surveyed, handling 61 contracts representing 65% of contract

awards reported by PDT. The highlights from the survey are summarised below:

The contracts had a combined value of $22m.

Baseline employment of the 31 companies before contracts were awarded was 945.

A total of 335 permanent staff (employed for more than six months) were recruited

after the contract award, and 1,546 temporary staff (less than six months

employment).

39 staff were kept on permanently after the contracts were completed, and 32

temporarily.

Of the 61 contracts, respondents reported that 57 (93%) would not have been

awarded without PDMA support.

The most generous interpretation of the survey is that 1,881 jobs in Helmand were

supported by the contracts awarded (335 permanent and 1,546 temporary). The strictest

interpretation – number of staff kept on permanently after completion of the contract –

gives a total of 39.

We have made an alternative calculation of job impact based on figures provided by the

Special Team of Royal Engineers (STRE) based at the Helmand PRT on construction wage

rates and on labour costs as a proportion of total contract value.5 With a total contract value

of $22m, we estimate that the full time equivalent (FTE) number of jobs supported would be

in the range of 140-275, depending on the daily wage rate assumed. The value of contracts

reported by PDT in the most recent semi-annual report is $465m. A simple extrapolation

4 Helmand Job Creation Survey; Peace Dividend Trust, July 2010

5 Assumptions: 1. Labour costs 15% of contract value

2. Daily wage rates $4-8, depending on security of area 3. 300 working days per annum 4. 10 jobs required for each full-time equivalent (FTE)

19 | P a g e

from the Helmand survey would therefore suggest an employment impact of 2,959 -5813

jobs, using the rather strict assumptions in our alternative calculation.

Another approach is to take the current accumulated value of contracts facilitated ($465m),

estimate the labour content of this and divide that by an average annual salary figure. The

labour cost share of construction contracts was estimated above to be 15%. Nationally,

other sectors with higher labour costs are covered by PDMA so we have assumed a labour

content of 20%. This gives a labour cost associated with contracts facilitated of $93.1m over

the two years of PDMA to date, an average of $47m each year. Estimates of average annual

salaries in Afghanistan vary widely, differing by province and by sector. A top end estimate,

and therefore a conservative one for the purpose of this calculation, is $1,500 annually. This

would mean that the contracts identified would have supported around 31,000 jobs.

Applying our assumption of 10 jobs per full time equivalent gives an estimate of 3,100 FTE

jobs. This is within the range of estimates we made from the Helmand survey.

These estimates must be treated very cautiously. The assumptions on labour content, wage

rates and jobs per FTE are based on very limited evidence though we have tried to be

conservative. We present them only as ballpark figures to give some order of magnitude for

the employment impact of the program.

To build a more robust picture of the PDMA’s impact on jobs and income would require

some additional research and analysis. This could be seen as an extension of the Helmand

study to the national level. It would require survey work in two main areas. The first would

be with a sample of Afghan businesses, some which have received support from PDMA and

a matching sample of others who have not. This part of the survey would gather data on

jobs and income growth in each group over the period of PDMA’s operation, comparing the

two and assessing how much if any of the difference could be attributed to PDMA’s

intervention. The second would be with a sample of procurement offices, probing more

deeply on the difference PDMA has made to their purchasing patterns and also tracking

information on the extent of repeat business which suppliers are now winning without the

need for PDMA intervention.

This type of survey work is not within PDT’s current mandate or resources. It could however

be commissioned as a separate exercise, ideally from a local research or consulting business.

This would both follow the principles of Afghan First and also offer a better opportunity to

question the sample of local businesses in more detail than a foreign contractor can

generally achieve. It would also ideally be constructed in a way which allows for regular

repeat surveys, to enable progress to be tracked over time. It could be funded from a

separate budget or, more likely, could make use of any underspend in the current PDMA

budget.

20 | P a g e

3. Performance of the Logic Model and PMF

The project still has around 20 months to run, leaving enough time to amend the Logic

Model and PMF if necessary.

The Logic Model itself mostly remains a clear presentation of the flow from program

activities through to outputs and incomes. It may require some updating to recognise that

the relative value of certain activities has changed with experience. The “pursuit plans” that

were envisaged as a service to institutional buyers do not appear to have materialised to the

extent envisaged and in practice the regular working discussions which PDMA has with key

buyers are the most effective way of developing their “Afghan First” strategies and actions.

Apart from minor adjustments like this, we do not believe that there are any flaws in the

Logic Model, any major omissions in the program’s implementation of it and no

requirement for significant changes to it over the remaining life of the program agreement.

The PMF would however benefit from some recalibration of targets. With 20 months still to

run on the initial agreement it would be advisable to do this to help re-focus activities where

necessary. Our suggestions on this are listed below, focusing on the immediate outcomes:

The target of 6,000 new entries on the Business Portal is almost 50% achieved. We

believe there are still many potential vendors to the international community to be

identified and registered and it would make sense to increase the target to some

extent. However, we do not recommend that this should simply be done pro rata.

The size of the database is not in itself evidence of an effective program. The key is

the quality of vendors listed, the accuracy of information on them and the growth in

their capacity. Focusing only on the numbers, with the same overall resources

available, will detract from these more qualitative but hugely important outcomes.

Also, the larger the database the more difficult it will be to maintain after the

program finishes when some other, local agency may become responsible for it. We

therefore suggest only a modest increase in the target for entries, say 10-15%, with

PDMA in parallel encouraged to spend more time on quality improvement and

assurance.

There are no targets in the PMF for the value of contracts facilitated. We believe this

was a weakness in the original version – the equivalent document for the Helmand

program includes value targets (which have been substantially exceeded). Given that

PDT collects and reports information on contract value, it should not represent an

additional demand on resources to include it in the PMF. There is also now a more

robust baseline, which was not the case at the beginning of the program. We

therefore suggest including a value target in the PMF for the remainder of the

21 | P a g e

agreement. Again, this should not just be a pro rata figure which would encourage

the program to pursue only large, construction contracts. It should be coupled with a

higher target for the number of contracts facilitated and a target for the sectoral

breakdown of contracts facilitated, to encourage a stronger focus on, say,

manufacturing and professional services.

The targets for training have already been achieved. This is one of the services most

highly valued by vendors and is also the main activity which has a specific focus on

female entrepreneurs and managers. In this activity we recommend a higher target

of, say, an 80-100% increase in throughput over the remaining lifespan of the

contract.

The PMF does not contain any targets for managing the transfer of the program’s

services over time to local agents. PDT is mandated to achieve this transfer and the

original program design included a planned approach to it. We return to this issue

later in the report but little progress has been made to date on a transfer strategy

and we recommend that its preparation is now included specifically as a target in the

PMF.

4. Value for Money

CIDA has committed marginally over $9m to the PDMA, a significant investment over four

years. The breakdown of costs by main category of expenditure is summarised in the table

below.

Table 2: Budget breakdown

Item Total program budget, $m

Fees 2.81

Reimbursable expenses

- of which security

5.27

- 2.38

Overhead 0.92

Total 9.00

Security is included explicitly in the budget, in contrast to the DFID project in Helmand

where security and most transport costs are covered off-budget under DFID’s duty of care.

22 | P a g e

In terms of value for money (VFM), it is necessary to define what the program is there to

“buy”. A more audit-driven assessment would look at PDMA’s impact on procurement costs

for the funders and possibly for other international buyers, looking for savings at least as big

as the program expenditure. Against this measure, it is not possible to demonstrate VFM.

The Canadian mission overall makes very little use of the program, and any procurement

savings it has made through buying locally can only partly be attributed to PDMA’s

intervention. The same applies to the DFID part of the program and generally to all other

buyers, though there is more evidence beginning to emerge of procurement savings in some

cases.

However, the Logic Model and PMF do not set any targets for procurement savings. The

ultimate outcome defined in the Logic Model is to reduce poverty through economic

development and private sector development (PSD). The intermediate outcomes are all

similarly defined in economic development and PSD terms, with an emphasis on SME and

female enterprise development. These are therefore the benefits to be assessed when

considering VFM. Any parallel benefits to international procurement offices, whether in cost

or efficiency, are valuable but not central to appraisal of the program.

In economic development terms the main benefits which the program is there to generate

are:

The value of contracts facilitated and therefore private sector revenues

The employment impact of the program

The specific impact on SMEs and female entrepreneurs

Value of contracts: Up to March 2010, PDT report the total value of contracts facilitated as

$465.6m. To the same date, actual expenditure on the project was $5.1m, giving a leverage

of 1:91, i.e. for every $1 spent on the project, $91 were generated through contracts

facilitated. Some of these contracts would have been let to local vendors without PDMA

intervention. Nevertheless, the leverage is high and represents good value for money in

terms of supporting private sector revenue growth.

Employment impact: To estimate employment impact we made estimates using two

different methods, one producing a range of 2,959-5,813 FTE jobs and the other a figure of

3,100 FTEs. The lower end of the first range is close to the 3,100 estimate so, again to be

conservative, we can take a figure of 3,000 for FTE jobs facilitated.

With expenditure to date of $5.14m, this produces a cost per job supported of $1,713.

There are no “norms” for what is an acceptable cost per job figure in development work in

Afghanistan but this figure is certainly much lower than would be the case in infrastructure

investment but probably higher than for, say, education and training programs. On balance,

we believe that it represents reasonable value for money in terms of employment impact.

23 | P a g e

Impact on SMEs and female entrepreneurs: We do not have sufficient data to assess VFM

in achieving these two objectives. The PDMA database predominantly includes SMEs, 80%

of the total employing less than 50 people, but many of the larger contracts will have been

won by relatively large companies. Nevertheless we believe it is reasonable to assume that

the majority of vendors supported have been SMEs. There was also evidence from our field

work that SMEs were winning more and bigger contracts than they would otherwise have

had an opportunity to bid for, creating genuinely additional business for the SME sector and

a shift of the overall market in its favour.

Overall therefore we conclude that PDMA represents good value for money as an

initiative to promote economic growth, PSD and SME development.

24 | P a g e

Relationships with Clients

In this section we report on PDMA’s relationships with its client base. The findings combine

both the CIDA and DFID-related field work in order to give as complete a picture as possible.

Buyers

Relationships and satisfaction with PDMA vary considerably and follow a pattern defined by

geography, procurement purpose, coalition membership and product category.

Geographic location: There was clear variation along geographic lines, illustrated by Figure

1. Use, understanding and appreciation all increase closer to operations in the field. The

Ottawa and London offices of CIDA and DFID respectively were somewhat sceptical about

PDMA’s value. They have no reason to use the service and their contact with PDT’s

operation in Kabul is limited. Their perceptions are primarily shaped by costs and the

periodic reports submitted by PDT. In CIDA’s case, management responsibility for the

project rests with Ottawa and there is a more regular relationship with PDT’s Ottawa office

than with Kabul. Both offices are concerned about the lack of any exit strategy. Awareness

of the context in which PDMA operates and the specific services it provides understandably

diminishes by distance.

Figure 1: Geographic perspective

The perspective from Kabul is slightly different since there is more direct contact with and

understanding of the project - lead responsibility for DFID’s support rests in its Kabul office.

The overall perception of PDT remains largely sceptical. Neither the Canadian nor UK

missions in Kabul make much use of it and therefore question its funding and value for

money.

In the field, in the Helmand PRT, Bastion, Leatherneck and KAF and outlying districts of

Helmand, the perception is much more positive. The service is much more frequently used

and there is less concern with its funding and management structure. Procurement officers

in the field consistently praised the value of the service, primarily citing the time and

manpower savings of using it but also recognising the wider range of supplier options and

the validation of basic vendor information. The ABCs were also well attended and highly

regarded.

Sponsors’ HQ Kabul Field operations

25 | P a g e

Procurement purpose: There is a further spectrum of responses to the project based on the

products and services being purchased, illustrated in Figure 2.

Figure 2: Functional perspective

The value of the project is least for procurement departments purchasing only for civilian

use: primarily office and household supplies. Within those budgets there is relatively little

scope for local procurement given the dominance of security and life support requirements.

The sums spent on supplies that can be purchased locally are small, there are normally local

nationals who can source them and the only real use made of the project is to use the

database to add to the supplier list.

For domestic purchases for the military, the perception of the project improves. This applies

both to military bases in Kabul, primarily Camp Eggers, and the bases in Helmand and

Kandahar. All of these still have the option to source from their home base but are now

adopting the Afghan First policy more explicitly (fully, in the US case). They also see

potential cost and time advantages in local sourcing.

The project is most highly valued by those responsible for military construction projects

both on the large bases and out in the field. Although they have little choice other than to

source locally, constraints on information, time and security mean that PDMA is used

frequently and valued highly.

Coalition members: The next variable is the nationality of the coalition members.

Figure 3: Coalition perspective

The smaller international missions have limited procurement budgets, especially in the

areas where PDMA is most active: construction and domestic supplies. The program is used

mainly as a source of additional suppliers, and is most valued in the field where it is hard to

find vendors outside the wire. Those of the smaller missions which are showing a growing

interest in local procurement are beginning to use the project more, initially to test local

vendors for quality and price. For example, the current Danish military procurement team at

Camp Bastion is the first rotation to try local purchasing. But in general, the service is not

crucial for them.

Civilian Military domestic Military construction

Smaller missions UK Military US Mission ISAF

26 | P a g e

For the larger countries, the value of the program increases. The UK military, and in

particular the STRE in Helmand, rate it as very important to achieving their objectives in the

field, enabling a quicker and more efficient response to operational demands for

stabilisation and reconstruction projects. The STRE now only use contractors who are

registered with PDMA.

The US mission overall has more fully adopted the Afghan First policy and local procurement

is now embedded in its regulations. Its scale of procurement is also by far the largest in the

coalition. Its use of PDMA services is more extensive than other coalition members and its

stated dependence on the project is higher. The Regional Contracting Centre at Camp

Eggers, the largest in the country, now requires that any local vendor is registered with

PDMA. It believes that if the service did not exist, it would have to be re-invented in some

form. The procurement personnel do not have the time or resource to deal with local

procurement and need external help to strike a balance between Afghan First and actually

getting the job on the ground done.

Finally, the most positive advocacy for the project came from ISAF. This is largely due to

Task Force 2010 which positions local procurement as one of the main pillars of the COIN

strategy. PDT is well known to and regarded by ISAF headquarters and the program is seen

as fundamentally important to its strategic objectives.

Unfortunately we were not able to gather information and opinion from the Canadian

military in Kandahar: no one was available to talk with us during our field work. Our

understanding is that they make little if any use of the PDMA service, possibly reflecting

both their relatively small requirements in relation to the US and UK and also the early stage

of PDMA’s presence in Kandahar.

Product category: Perceptions of the project also vary according to the type of product or

service being purchased.

Figure 4: Product sectors

The program does not generally cover the fresh fruit and vegetable sector, although it was

originally included in the Helmand project. Other products and services mainly include the

household and office supply needs of both civilian and military missions. The scale of these

is relatively small but there is a growing commitment to procuring them locally as far as

possible, even where this only involves using local wholesalers and retailers – few of the

necessary supplies are manufactured locally. In addition there are categories of services

such as recruitment, repair and maintenance, communications and business development

FFV Other products/services Construction

27 | P a g e

services, where local the local supply base is increasing. In all of these cases, the project is

recognised as a valuable source of information on suppliers and an effective means of

feeding ITTs into the local market.

Finally, the construction sector is the main user of the project. The scale of expenditure is

much higher than in any other category, the requirement to source locally is driven by both

policy and practicability and the level of urgency demands quick and reliable access to

information on potential contractors. For these reasons, the project is consistently viewed

very positively and considered essential to meeting operational needs.

Suppliers

We interviewed vendors, including local and foreign construction contractors in Helmand

and professional services providers in Kabul. We also met with the Afghanistan Chamber of

Commerce and Industry (ACCI), the Afghanistan Investment Support Agency (AISA) and an

Afghan NGO, Harakat.

Construction contractors: Construction businesses described the “before and after” of their

contact with the project. In the past, they were aware that business was available from the

international community but they had no means of gaining direct access to it due to a lack

of contacts, information and physical access. They did benefit to some extent but only

through being at the low end of a chain of on-selling of contracts.

They now deal directly with buyers, winning contracts in their own right, delivering work to

the right specification and retaining the profit margin. They have invested in new

equipment, increased their permanent employment (from two to 25 in one case),

diversified into new areas of business and expanded their operations geographically.

The most valued support they had received from PDMA was direct introductions to new

customers, including through the ABCs. They had also received help in completing

procurement documentation and more detailed training in bidding procedures. They had

recently won new contracts with the international community of $10-20m which PDMA was

unaware of, demonstrating their progress towards doing business directly with buyers.

The companies did not find the process of registering with PDMA onerous, nor were they

aware of any other source of support available to them: AISA was seen as a necessary

registration process but not a direct means of winning business and chambers of commerce

or other trade associations were not seen as relevant. The beneficial impacts on their

business were attributed fully to the support which they had through the project, matching

the findings of the Helmand Job Creation Survey.

In terms of additionality, the contracts themselves would have had to have been let anyway

– no new business has been created. However, the business is unlikely to have gone directly

28 | P a g e

to local SMEs. They may eventually have benefited from some work towards the lower end

of the contract selling chain but with lower revenues and margins. The benefits of local SME

growth in business and employment would not have been achieved and in this sense the

program has created both additional and accelerated benefits for the SME sector. It has also

helped direct growth to legitimate business.

International contractors: The major military bases have a number of international

contractors based on site, mostly undertaking construction projects within the wire. They

are in the unique position of being both suppliers to the international community and

buyers of local services and materials. In their supplier role, they are expected by customers

to follow the principles of Afghan First and increasingly to transfer business and technical

knowledge to local vendors. For the latter, they are beginning to look for more formal joint

venture agreements with local businesses.

Whilst there is a commitment to Afghan First in principle, no support is provided by

procurement offices in the practice of delivering it. Foreign contractors face the same

problem as the military in not being able to go outside the wire to find and validate

suppliers. They have previously had problems with the quality of local work and in filtering

out the common practice of local contractors claiming to have much greater capacity and

experience than exists in reality. PDMA has provided them with a wider range of options for

potential suppliers and an initial filter on their credentials. Once these are provided, the

contractor can build relationships more directly with local suppliers and the PDMA service is

not vital to them longer term.

Service businesses: The PDMA operation in Helmand deals primarily with the construction

sector. There is a wider vendor base in Kabul, including a growing and increasingly

professional service sector. The initial support received from PDMA was to make

introductions to buyers and provide information on bid solicitations: a first point of entry

into the market. Training in the procurement process was also provided and continuing use

is made of the tender lists.

In the cases of the companies interviewed, the hands-on PDMA support was only required

for an initial period. Their business and communication skills are already well developed and

after relationships with international buyers are established, the businesses become

capable of dealing directly with them.

They saw no options to delivery of the program. AISA has a different function in terms of

business registration and overseas investment promotion. ACCI is viewed with suspicion and

not considered relevant to smaller businesses. For those who can afford it, there is the

option of buying in external expertise, including from international consultants, to assist in

preparing bids. But the PDMA service was seen as the most relevant and professional and

was instrumental in assisting them to make their initial entry into the international market.

29 | P a g e

Business agencies: We met with AISA, ACCI and Harakat. Their perceptions of PDMA are

generally positive, with neither AISA nor Harakat planning to provide it themselves. ACCI’s

position is different. They claim to provide a similar service, sending information on

contracts to their members, advising on bids and providing procurement training. They also

have plans to expand the service on a chargeable basis. In contrast, other respondents,

including an ACCI office bearer, were unaware of this. SMEs interviewed felt that ACCI was

not relevant or trusted, while most respondents did not believe that ACCI had the capacity

to deliver a similar service to PDMA.

All three however agreed that it was not sustainable to have the service provided long term

by a foreign NGO and that there should be a phased transfer to Afghan ownership and

management. The issue of transfer is covered later in this report.

30 | P a g e

Regional Offices

The initial pilot program for PDMA over 2006-08 only covered the Kabul area. The full

program has extended its geographic reach to three provincial offices, in Jalalabad, Mazar-e-

Sharif and Kandahar. The DFID-funded project covers Helmand. The rationale for opening

field offices was based on a number of arguments: many of the major international buyers

had procurement offices outside Kabul, in particular the military; many of these offices in

turn were “inside the wire” and could only be approached by international staff based in the

field; the private sector in the provinces could not realistically be supported from Kabul; the

identification and validation of provincial businesses would only be possible if done by staff

in local offices.

Due to the costs of security and other logistical problems, we were not able to visit the

offices in Jalalabad and Mazar. Nor did we visit the PDMA premises in Kandahar city or

Lashkar Gah, for security reasons: PDT’s international staff do not visit them either in order

to preserve their low profile within the community. Therefore our coverage of PDMA’s

regional operations was much less comprehensive than would have been ideal for the

review. We did however spend time working with PDMA international staff at Camps

Bastion, Leatherneck and Shorabak, at the Helmand PRT and on KAF, where most of the

procurement client base is located. We have based our assessment of the regional

operation primarily on these visits and on a review of the provincial data in PDT’s regular

reports.

Table 3 summarises the results so far achieved by the field offices in terms of the value of

contracts facilitated.

Table 3: Value of contracts facilitated by field office, $m, to March 2010

Activity $m contracts

Kabul 419.2

Mazar-e-Sharif 9.9

Jalalabad 13.8

Kandahar 6.2

Lashkar Gah 16.4

Total 465.5

The Kabul office clearly dominates the values with 90% of the total to date. It is of course

the longest established, including the two year pilot project, and operates from the largest

31 | P a g e

economic centre in the country. The Helmand office in Lashkar Gah is the second biggest

contributor, at 4% of the total, with Jalalabad, Mazar and Kandahar following behind. The

low proportion of the total contributed by Kandahar is surprising given the concentration of

buyers at KAF and the city being the second largest economic centre in the country. There

are some mitigating factors, in particular the difficulty experienced in recruiting national

staff and a turnover of international staff – the latest field manager arrived only in March

after these figures were collated. Within KAF the combination of a new manager and

rotation of procurement personnel means that the program has started from a low base

and a small contact network each time. Further the security situation prevents personal

contact between the international and national staff. With a more stable staffing position,

the returns from Kandahar should increase but this needs to be monitored closely.

Table 4 shows the projected costs of the field offices over years two to four of the program,

year one having been essentially a start-up phase for the offices outside Kabul.

Table 4: Costs of field offices, years 2-4, $000

Item Kabul Mazar Jalalabad Kandahar Total

Personnel 1510 68 92 258 1929

Operations 684 36 38 46 804

Total 2194 104 130 304 2733

It is expected that the Kabul office will account for 80% of costs, suggesting that it is

currently over-performing in relation to the others (having generated 90% of reported

contract value). This is to be expected given that it is the longest established. The Mazar

office will account for 4% of costs, slightly under-performing in relation to results. Jalalabad

will account for 5% of costs, again slightly behind its results profile. Kandahar will account

for 11% of costs against just over 1% of results to date. This indicates the extent of catch-up

required if the Kandahar field office is to achieve its potential.

Overall the field offices, excluding Helmand, contribute 7% of contracts facilitated by value

but require 20% of projected costs. It is reasonable to expect a higher level of cost in the

field offices compared with Kabul where there are more economies of scale and a longer

track record. For the Mazar and Jalalabad offices, we do not think that the current level of

apparent under-performance is of major concern. But Kandahar should be performing

better given the scale of potential and the higher operating costs of the office.

During the field work we were able to work closely with the international staff of the

Helmand and Kandahar offices. Although the former is part of the DFID-funded project, the

observations from our visit are relevant to the national program.

32 | P a g e

The Helmand field manager has been in post longer than the Kandahar manager. This was

clear from the level of contact and rapport with the main procurement offices. This personal

networking is essential to the success of the program because with buyers it is face to face

contact, advocacy and occasional nagging which makes the difference to their purchasing

behaviour. The field manager is also able to meet regularly with DFID and PRT personnel.

The lead local national working in Helmand is also very capable and experienced, building

personal contact with vendors and also able to meet the field manager on a face to face

basis at the Helmand PRT. Finally, the frequency of ABC events and the ability of vendors to

visit buyers at PRT also help in building confidence in the program.

Kandahar in contrast has faced a number of disadvantages to date: turnover of international

staff; difficulty in recruiting local nationals; lack of access for locals to KAF; and, to date, lack

of access for the field manager both to the Kandahar office and the Kandahar PRT. Contact

between the field manager and the lead CIDA officer at the PRT has been minimal. The two

field offices are therefore almost diametrically opposite in these important regards. As the

new field manager becomes established and benefits from greater freedom of movement

with a security contract now in place, the situation should improve.

These extremes illustrate one main driver of success in the field offices, one which Kabul

had already benefitted from – the importance of regular, face to face contact with the main

procurement offices. Although the policy environment is now moving more rapidly in favour

of local procurement, this level of contact remains essential to PDMA’s ability to influence

its key clients.

33 | P a g e

Risk Factors

This section examines the risks associated with the program and the effectiveness of

mitigation measures taken to date. The original Program Design document6 identified four

main risks to achieving the project outputs:

Political instability

Security

Capacity of Afghan SMEs

Economic conditions

The first two of these remain facts of life when operating in Afghanistan and generally have

not improved to a significant extent since the program was launched – deteriorating in

some cases. There is nothing that the program management can do to reduce the risk of

political instability – it can only mitigate its impact on the program by monitoring the

situation and continuously assessing any implications for the PDMA.

In terms of security, this remains a major risk and constraint for the program. The Helmand

project staff operate under DFID duty of care, which assists with freedom of movement and

accommodation for international staff. The other offices make their own security

arrangements within an overall security contract with Edinburgh International. Provincial

offices maintain a low profile, avoiding prominent signage and avoiding visits from

international staff. The field managers in Helmand and Kandahar as well as the Project

Director in Kabul are ex-military and understand the requirements of working within

insecure environments. Security considerations will continue to constrain the geographic

reach of the program and to limit the amount of time that can be spent on direct capacity

building with local suppliers.

The capacity of Afghan SMEs to meet the demands of the international community remains

a risk. Particularly in the early days of working with buyers, and where new products or

services are being sourced, there have been cases of suppliers not being able to meet the

required specifications. Many vendors have overstated their capacity, suggesting that they

can handle all types of work asked of them, misrepresenting their technical and equipment

capacity (often claiming capacity that belongs to other businesses and then seeking to sub-

contract) and providing misleading information on previous work. The basic validation of

vendor details which PDMA undertakes on its registered companies cannot adequately test

for this. Spotting overstatement depends on site visits which are in turn dependent on

6 Afghanistan Marketplace, Detailed Program Design; PDT, December 2007

34 | P a g e

freedom of movement and access. The biggest mitigating factor in these circumstances is

that buyers normally only make the mistake once. If a contract is awarded which the vendor

does not deliver on, they are unlikely to get a second chance and PDMA can remove them

from the register if there was blatant misrepresentation involved. Time and resource will of

course have been wasted in the process.

The value of contracts now being placed, the level of repeat business and the increasing

prevalence of direct contracting without PDMA intervention suggest that the capacity risk is

reducing. But it remains an important issue and any disappointment of buyers in local

vendors introduced by the program still carries a reputational risk. This is another argument

for focusing less on the absolute number of registered businesses and more on ensuring the

quality of those already on the database.

With regard to economic conditions in the country, this has not materialised as a significant

risk. Growth in the overall economy has been high and spend by the international

community has continued to increase year on year.

There are other risks which have become apparent but which were not identified in the

original program design.

Corruption: Fear of corruption within the international community and the larger

contractors has been a disincentive to SMEs in pursuing business opportunities. It is more of

a constraint than, say, lack of confidence in meeting technical standards. PDMA will remove

from the register any contractor found to be selling contracts. But there is little it can do to

mitigate the risk and reduce SME concerns – it is a wider national problem.

Vetting of suppliers: PDMA takes steps to validate basic information about suppliers on its

register. It does not however vet them for technical competence or illegality, so there is a

risk that buyers will be introduced to inadequate or illicit vendors. This in turn would

damage the reputation of the service. Despite PDMA making it clear to buyers that it cannot

vet for these problems, a number of buyers raised this as a weakness in the service. In terms

of mitigation, lack of technical capacity will become evident once work on a contract begins

but clearly time and resource will have been lost. On illegality, military contractors generally

accept this as their responsibility and military intelligence services are increasingly involved

in tracking business legitimacy – Task Force 2010 has this as one of its principal objectives.

Dependency: There is a risk that some buyers and suppliers become dependent on

continued provision of the service, reducing incentives for buyers to institutionalise local

procurement and for vendors to build direct relationships with their customers. As ISAF and

the US mission embed local procurement into their procurement legislation and procedures,

35 | P a g e

this risk diminishes but this is not yet the case for other coalition members. It remains a risk

for suppliers but PDMA’s training programmes and the evidence from both buyers and

vendors that they are now trading directly with one another are reducing it. The fact that

PDMA was unaware of some repeat business which had been won in Helmand is actually a

good indicator that the market is beginning to work without intervention.

Rotation of procurement personnel: Both civilian and military procurement personnel are

subject to regular rotation and PDMA is already working with second and third cohorts of

buyers. In most other aspects of military and development work in Afghanistan, corporate

memory is short-term and the transfer of knowledge and information to successors is not

systematic. This risk has materialised in many cases, with incoming personnel starting from

scratch on the sourcing of suppliers and suppliers losing consistent contact with their

customers. The only long term mitigation of this is for local procurement to be fully

institutionalised, which PDMA can help with but which is primarily dependent on the

buyers’ commitment, policies and procedures.

Inadequate vendor specifications: The ITTs prepared by buyers have often hindered local

vendors. Specifications written in English will only rarely be understood. Overly general

specifications have caused problems of interpretation and disappointment on both sides.

PDMA has sought to mitigate this risk in a number of ways. It has arranged for specifications

to be translated into Pashtu and Dari. It has acted as an intermediary in helping to explain

specifications to vendors. It has encouraged buyers to use photographs and catalogues to

illustrate their requirements. This problem appears to be diminishing, according to both

buyers and suppliers.

36 | P a g e

Future of the Project

It is part of PDT’s mandate to transfer responsibility for the project to a local organisation.

The original agreement with CIDA set out the likely timescale for such a transfer. It said that

the need for the project would decrease over time as market information and business

capacity improved, envisaging a demonstration phase of 18-24 months, a consolidation

phase of 24-48 months and then either a wind down or transfer to a local body.

It was envisaged that there would be a different exit strategy for each of the main activities

in the project. The business database would be reviewed after three years then phased out

or transferred. The verification activity would be transferred in years three and four. The

matchmaking service would be phased out quicker, after year two. The tender distribution

point would be transferred in year four and the training activities in year three. Overall, the

program was intended to be either wound down or transferred by the end of the funding

agreement in 2012.

The need for a continuing service: On the demand side progress is being made on the

institutionalisation of local procurement, especially in the US mission which is the largest

buyer. We believe that there is now sufficient momentum behind the US efforts on local

procurement to begin reducing the need for third party support. It may take another 12-18

months before the process is fully self-sustaining but the combination of regulatory change

and growing practical experience means that it is already well under way.

Other coalition members remain some way behind the US. Afghan First is accepted in

principle but is still not fully embedded to the extent that it is independent of staff rotations.

Realistically it may take another 18-24 months before this is achieved during which time

external support will still be required.

For vendors, there is already encouraging evidence of suppliers winning repeat business

without the project’s intervention and as local procurement becomes institutionalised this

will become more prevalent. This is not yet the case for all PDMA-registered businesses

however and most are still in the early stages of understanding the international market.

Another factor determining the need for the service is the level of penetration it has so far

achieved amongst buyers and vendors. PDMA lists 95 procurement offices which it works

with. The list of participants in the UNAMA Local Procurement Campaign lists only 20

agencies who signed up to the UNAMA campaign. The two lists are not directly comparable

and PDMA’s efforts to involve buyers have been more intensive than UNAMA’s. But the

37 | P a g e

figures do suggest that PDMA has already works with most of the relevant procurement

offices.

On the supply side, there are now around 5,500 businesses on the PDMA register. A recent

survey of Afghan businesses7 reported that 4% of businesses were making sales to foreign

companies based in Afghanistan. The survey did not separately ask about sales to coalition

members but applying that 4% to an AISA estimate of there being over 300,000 businesses

in the country suggests that there are up to 12,000 businesses selling to foreign customers.

It is unlikely that all of these will be relevant as suppliers to the coalition and so it does

suggest that PDMA has already achieved a good penetration of the likely supply base.

Both estimates confirm that there can be less emphasis on adding to the number of clients

or the size of the register and more on ensuring that the quality of interaction and trade is

improved.

We believe that the need for and value of the service will not disappear entirely within the

current funding commitments of both DFID and CIDA. But there is now good evidence that

the market information failure on which PDMA was founded is reducing. This was always

the aim of the project and so is an indicator of its success. The initial plan for the national

project to have a 24 month demonstration phase followed by a 24 month consolidation

phase is behind schedule by perhaps 6-12 months but is still moving in the right direction.

An exit strategy is now required to enable a phased run-down of parts of the program which

are not required long term, for example the direct matchmaking and advocacy elements,

and for other parts to be transferred to local management, such as maintenance of the

database and procurement training. Again this is as originally intended. Despite our view

that the original plan is 6-12 months behind schedule it should still be possible to implement

the exit strategy by March 2012, though with provision for a limited extension of up to six

months to ease the transfer process.

Preparation and implementation of an exit plan requires a systematic and properly

resourced effort. It has drifted to date as PDT itself would probably admit – other priorities

for the use of scarce time and resources have prevailed. The risk however is that the end

point of CIDA’s funding agreement will be reached without a plan in place to transfer the

services still required to a local provider. This would result in the service simply fading away

or the current package being extended only because there is no option to do otherwise –

neither would be good outcomes.

7 Afghan Business Attitudes on the Economy, Government and Business Organisations; 2009-2010 Afghan

Business Survey; Center for International Private Enterprise, 2010.

38 | P a g e

To prepare the plan properly and in a reasonable timescale a dedicated and separately

resourced effort is needed. PDT should be fully involved in the plan but it could be prepared

by an independent advisor, perhaps funded separately by CIDA and DFID or funded out of

any program underspend. As with economic impact work, this would ideally be done by a

local advisor though care would be required to avoid any conflict of interest – the advisor

should not be one of the options for delivering any transferred services. A timetable should

be established for the plan, ideally by the mid-2011 so that there is sufficient time available

to ensure a smooth transfer to the new providers.

The options for transfer: There will be elements of the project which merit continued

provision after March 2012. The register of companies should still be valuable to

procurement offices as a source of additional vendors, provided it is regularly updated and

cleaned. A distributed list of ITTs would also assist vendors in finding opportunities. Training

in procurement procedures will still be valuable to new entrants to the market. What will be

less necessary as local procurement becomes institutionalised is the intensive, hands-on

work done with procurement departments, primarily by PDMA’s international staff. The

longer term services could therefore be provided by local agents.

Little progress has been made within the current program on identifying and developing

local partners to deliver all or part of the service. PDT initially worked closely with ACCI as a

potential future provider, including operating from shared premises. However the

relationship appears to have broken down. ACCI still says it is interested in delivering the

service on a chargeable basis and claims to be providing it already for members. But other

respondents were sceptical about their capacity and their relevance to the SME sector. ACCI

confirmed that is focusing more on larger companies than on SMEs. We have not

undertaken an analysis of ACCI’s activity and capacity but the consistency of feedback

received was a strong indication of this not yet being a credible transfer option.

AISA and PDMA both maintain registers of businesses and exchange information across

them. Registration with AISA is a condition of being listed on the PDMA database. AISA does

play a role in business development, for example through mediation and arbitration of

commercial disputes. Whilst it probably has the competence to deliver the service, it is not

however its core business and the agency did not express a desire to take it over.

Harakat, a local NGO, is also interested in procurement policy and practice and is currently

discussing a procurement initiative with the Afghan government. This would be for

government business only, however, and Harakat has not expressed an interest in providing

a service for international purchasing. As with AISA, it would be a distraction from core

business.

39 | P a g e

It would not have been appropriate to try to shift the opinions of AISA or Harakat during this

review. Any further probing of their position regarding transfer would best come directly

from CIDA on completion of the review. As indicated below, however, we believe that there

are more appropriate options for transfer.

Transfer to a Government department is not considered an option. Vendors expressed

concerns over corruption and competence. This would be the least favoured option at

present.

Transfer to a local business development service (BDS) provider is an option, provided there

is a means of generating revenue. Feedback from respondents was that vendors would be

willing to pay if they believed that it would guarantee business. No provider can guarantee

that of course. Also, once a charge is made there will be concerns over the provider’s

independence and objectivity. However, operation of the service by a commercial provider

is the best option at present. There is no reason in principle why a commercial business

model could not be designed, for example through charging registration fees for entry to

the database or charging for training programmes. It is also not necessary that all the

continuing services should be delivered by a single provider: they may be more easily

absorbed commercially within a number of providers with the best suited business models.

A variation of this model would be to transfer ownership and management of parts of the

current service to Afghan staff, with members of the existing team and/or new recruits. This

would allow for a more seamless transfer but would still require a planned and phased

approach running beyond the current funding period.

The PDMA has made good progress towards reducing the market information failure

which it was set up to address. It has been a successful program. The policy context is also

moving in favour of local procurement. The market failure will not have disappeared

completely within the lifespan of the current agreement with CIDA. Nor is there a clear

exit plan to wind down or transfer the relevant parts of the service to a local provider. But

progress to date means that the original objective to achieve transfer after four years is

still valid, albeit with provision for a limited extension to ease the transfer process if

necessary.

40 | P a g e

Summary of Conclusions and Recommendations

This final section presents our conclusions and recommendations from the review under the

following headings:

Changing context

Market constraints

Project performance

Validity of logic model and PMF

Value for money

Client perceptions

Risk factors

Future of the project

Changing context: Since the project was launched, the coalition’s approach to local

procurement has moved on considerably. The Afghan First policy has been launched with

strong advocacy and branding behind it. The US mission has embedded local procurement

more fully in its laws and regulations. ISAF has launched Task Force 2010, positioning local

procurement at the heart of the COIN strategy. These changes reflect well on CIDA and DFID

for being first movers in supporting the PDMA project. They will however make decisions on

its future more sensitive – there will be more interest and comment from other members of

the international community.

Market constraints: On the demand side, the market is highly segmented by type and

location of buyer and the purpose of procurement. Generally the constraints include: the

small discretionary budgets available for civilian spend; a small local supply base;

restrictions on movement for both buyers and sellers; pressure in the field for quick delivery

of stabilisation projects; and short rotations of procurement staff. We believe that PDT

properly diagnosed the constraints facing buyers. The core problem is an information

market failure, the main premise on which PDMA has been designed.

For suppliers, the principal are: lack of access to invitations to tender; lack of physical access

to customers; the complexity of procurement procedures; language; uncertainty regarding

quality standards; lack of feedback from buyers; and fear of corruption. Again, these

findings confirm PDT’s diagnosis and the appropriateness of their interventions: once again

the core problem is an information market failure.

Project performance: The project has performed strongly against the original PMF targets.

There are some qualifications on this: the original targets were set too low; some of the

performance data is due to improved reporting by buyers rather than actual increases in

41 | P a g e

local procurement; the reported data are on contracts and jobs associated with the project

rather than created by it; and the results have not been subject to independent monitoring

and evaluation.

Nevertheless, the results are impressive and we have no doubt that PDMA has made a

measurable difference to both buyers and sellers. For buyers, it has enabled local

procurement to progress more quickly, more confidently and more efficiently than would

otherwise have been the case. For sellers, it has generated additional business for SMEs and

improved business capacity to deal directly with international customers.

Ideally the economic impact of the project should be measured more robustly than was

possible in this review, through periodic sample surveys of beneficiaries, a matching group

of non-assisted businesses and procurement offices.

Validity of Logic Model and PMF: The original Logic Model is still robust and there are no

major gaps in PDMA’s adherence to it. No significant changes are required in it for the

remainder of the program. We do recommend some re-setting of targets in the PMF for the

remaining 20 months. These include: a 10-15% increase in the number of vendors

registered on the Business Portal, though with an increased emphasis on the quality of

businesses registered rather than the absolute number: a new target to be included for the

value of contracts facilitated; an increase in the targets for training activities; and the

inclusion of a timetable for the managed transfer of appropriate services to local providers.

Specific targets for the field offices should also be set, particularly for Kandahar which is

currently under-performing in relation to its cost and the scale of purchasing potential from

KAF. PDT should work closely with CIDA in re-setting these targets to ensure that they buy in

to the change in expectations of them.

Value for money: The purpose in supporting PDMA is to promote economic and private

sector development. We believe that it provides good value for money in delivering this

purpose. It is levering $91 business revenue for every $1 spent on the program to date. A

very rough estimate of cost per FTE job supported to date is $1,173 which we believe is

good value compared with other options to promote economic development.

Client perceptions: The perceptions held by buyers and their degrees of satisfaction with

the project vary according by market segment. Geographically, perceptions improve the

further the buyer is from administrative headquarters, with military clients in the field

valuing the service most. In terms of the goods being purchased, buyers of civilian, domestic

items had much less requirement for the service compared with a high level of need and

satisfaction from military construction buyers. For coalition members, the value attached to

the service was lowest for those with smaller missions in the country and considerably

42 | P a g e

higher within the US and central ISAF missions. And in terms of product sectors, the value of

the project was by a large degree highest in the construction sector.

In summary, the highest degree of use and satisfaction came from the US and UK military in

the field and the lowest from Embassy-based buyers responsible for household items,

including the Embassies of Canada and the UK.

For vendors, who are after all the intended beneficiaries, the perception of the project was

consistently very positive. They highlighted a range of beneficial impacts: improved access

to ITTs; better understanding of procurement procedures and standards; reduced

susceptibility to the practice of selling on of contracts; increased scope for repeat business

won through direct customer contact. These benefits are now leading to increases in

employment and income.

Risk factors: The risks originally identified in the program design were: political instability,

security, SME capacity and economic conditions. The first two have not reduced and in

some cases are higher but PDT has done as much as could reasonably be expected to

mitigate their impact on the program. The third also remains a risk but PDT has taken active

mitigation measures which have minimised its impact to date – to a large extent it is now a

risk managed as part of the normal business of client/supplier relationships. The fourth has

not materialised, as economic growth has been strong.

We identified additional risks which should be made explicit for the remaining 20 months of

the program: corruption in the business community and government; the vetting of

suppliers; dependency of buyers and sellers on the program; frequent rotation of

procurement personnel; and inadequate vendor specifications.

Future of the project: The project at both national and Helmand level was designed as a

temporary intervention to remove a market failure of information. Nationally, a two year

demonstration phase was to be followed by a two year consolidation phase, after which

there would be a planned transfer of those services for which there was still a need to a

local delivery partner.

The evidence from our review is that the market failure is reducing, with local procurement

becoming institutionalised in the main buyer, the US mission, and direct trading without

intervention growing. This is a sign of success for the project. We do not believe that the

market failure will be fully resolved within the lifespan of the current CIDA and DFID funding

agreements: we think the process is probably 6-12 months behind schedule. But we do

recommend that a detailed exit strategy should now be prepared, including provision for

the transfer of longer term services such as management of the database of suppliers to

43 | P a g e

local commercial providers. The exit plan should aim as originally intended for

implementation by the end of CIDA’s contracted commitment in March 2012, with some

provision for a limited extension to smooth the transfer process. A timetable should be

agreed with PDT for preparation of the plan, ideally by mid-2011 to give sufficient time for a

properly managed handover. The resources needed to prepare the plan should be specified

and sourced separately if necessary, and the possibility of commissioning a local advisor to

do the work should also be explored providing no conflict of interest arises.

For DFID, we have recommended that the Helmand project should be placed on the same

timescale of the national project. This should not be assumed to require continuation of

DFID funding and discussions should begin now on finding alternative sources for the

approximate £430,000 required, initially with the ISAF, US and UK military procurement

offices who benefit most as buyers.

44 | P a g e

Annex One: Interview Respondents

International donors

Jenny Hill CIDA, Ottawa

Michael Creighton CIDA, Ottawa

Mariam Wafa CIDA, Kabul

Abdul Hadi Karwandoy CIDA, Kabul

Zoe Kahn CIDA, Task Force Kandahar, KAF

Janet Lam CIDA, Kandahar PRT

Doreen Broska DFID Afghanistan, Kabul

Tim Stern DFID Afghanistan, Helmand PRT

Pat Hynes DFID, East Kilbride

Pat Thomas DFID, East Kilbride

David Bailey USAID, KAF

Peace Dividend Trust

Wayne Nightingale PDT Ottawa

Mike Capstick PDT, Kabul

Leigh Ryan PDT, Helmand

Zahin PDT, Helmand

Tom Viscelli PDT, Kandahar, KAF

International Embassies

Jackie Elliot British Embassy Kabul

Simon Diggins British Embassy Kabul

Jeff McIlroy Canadian Embassy, Kabul

Ramin Hamkar Canadian Embassy, Kabul

Erika Zielke US Embassy, Kabul

Mark Ward UNAMA, Kabul

Afghan businesses and organisations

Assad Nissar Harakat, Kabul

Yar Taraky Harakat, Kabul

Abdul Karim Safi Afghanistan Investment Support Agency (AISA)

Marshall Ferrin AISA

Atiqullah Nusrat Afghanistan Chamber of Commerce and Industry (ACCI), Kabul

Aziz Ahmad Afghan Barez HR Services Co., Kabul

Musada Sultan Insight Communications, Kabul

Edriss Raha Fazilat Group, Kabul

45 | P a g e

Sayed Ahsan Al Muslim Steel Works/Services and Sayed Tajdar Construction

Abdul Wahab Helmandwal Construction

NATO Military

Dave Babineau ISAF, Kabul

Capt. Headley US Regional Contracting Centre, Camp Eggers, Kabul

Maj. Voth US Regional Contracting Centre, Camp Eggers, Kabul

Maj. D. Brown UK Works, Camp Bastion

WO2 Gary Wills JSF HQ/J8, Camp Bastion

Ms. Veejena JSF HQ/J8, Camp Bastion

Capt. Assoo Estonian military procurement, Camp Bastion

Maj. Allen Danish military procurement, Camp Bastion

Capt. Pieter Danish military procurement, Camp Bastion

Col. Mimms US military C-9, Camp Leatherneck

Col. Kelly US military C-9, Camp Leatherneck

Maj. T. Gates US Regional Contracting Centre, Camp Leatherneck

Maj. M. Hand ANSF procurement, Camp Shorabak

Cap. D. Sureen ANA procurement, Camp Shorabak

Capt. P. Thorneycroft STRE, Helmand PRT

WO2 P. Villa, FCO Helmand PRT

Maj. Lehman Marine Expeditionary Force, Helmand PRT

Maj. Chio Marine Expeditionary Force, Helmand PRT

Lt. Col. M. Heywood RC(S) HQ, CJ4, Kandahar (KAF)

Lt. Col. F. Percival RC(S) HQ, KAF

R. Nowicki USACE, KAF

Sq. Ldr. S. Keenan RC(S) HQ, CJ4, KAF

Contracting businesses

Jarrett Schmitt Green Leaf, Camp Bastion

Frazer Myers Prolog, Camp Bastion

Ian Tinniswood Mott MacDonald, Helmand PRT