Sustainabilty

USA

ID Policies

Local Participationin Project

Implementation

Support toCountry Priorities

Local Accessto Resources

Accountability toLocal Stakeholders

Tracking USAID’s

A Review of Select Procurements in Six Countries

Efforts on the Local Solutions Initiative

Acknowledgments

This study was conducted by Sylvain Browa and Gretchen King. We thankCourtney Cauthen for her technical assistance and data analytics. The studyteam is also grateful to Jenny Russell, Nora O’Connell, Patrick Crump,Muluemebet Chekol, Hajira Shariff, Pat Daly and David Oot at Save theChildren and members of the Modernizing Foreign Assistance Network(MFAN) for their invaluable feedback. In addition, special thanks go to Save theChildren country directors in Bangladesh, Dominican Republic, Haiti, Kenya,Philippines, and Uganda for their guidance and support. For any question,please contact Gretchen King at gking@savechildren.org.

Save the Children USA501 Kings Highway EastFairfield, CT 06825USA1-800-728-3843savethechildren.org

© Save the Children, December 2014.

ISBN 1-888393-29-7

This publication is copyright, but may be reproduced by any method withoutfee or prior permission for teaching purposes, but not for resale. For copyingin any other circumstances, prior written permission must be obtained fromthe publisher, and a fee may be payable.

Save the Children invests in childhood – every day, in times ofcrisis and for our future. In the United States and around theworld, we give children a healthy start, the opportunity tolearn and protection from harm.

I. Background 4

II. Research Scope and Methodology 5

III. Research Findings 7

IV. Recommendations 12

V. Conclusion 14

VI. Annexes 15

Endnotes 24

Contents

In 2010, the United States Agency for InternationalDevelopment (USAID) launched a reform agendacalled USAID Forward, aimed at “embracing newpartnerships, investing in the catalytic role ofinnovation and demanding a relentless focus onresults.”1 USAID Forward spurred various inter-related initiatives within the agency, including somespecifically dealing with aid effectiveness to help theU.S. Government fulfill its commitments to the ParisDeclaration on Aid Effectiveness as well assubsequent agreements in Accra, Busan and Mexico.These include, but are not limited to, the CountryDevelopment Cooperation Strategies (CDCS), LocalSolutions initiative (formerly known asImplementation and Procurement Reform or IPR)and the 2011–2015 USAID Policy Framework.

After four years of implementation, the agency andinternational aid community are beginning to examine theresults of USAID Forward. USAID issued progress reportsin 2012 and 2013 that cover all the reforms under thisagenda. Included in the reports was progress madeon the Local Solutions initiative, which aims tostrengthen partner-country capacity to implementprograms, enhance and promote countryownership, and increase sustainability.2 However,because progress reporting on Local Solutions relieson only one indicator – percentage of USAIDMission funding obligated to local institutions –trends in the other key areas of country ownership,capacity building, and sustainability are very difficult

to assess. A recent study conducted by the GovernmentAccountability Office (GAO) released in April 2014 alsoexamined the progress of USAID’s implementation of theLocal Solutions initiative.3 Using the single indicator, theGAO assessed the extent to which USAID haddemonstrated progress toward achieving its 30 percenttarget of Mission direct funding to local institutions by2015. In addition, it tracked progress on the three afore-mentioned goals. The GAO recommended that theUSAID Administrator “identify additional indicators tobetter capture progress toward the Local Solutionsinitiative’s goals.”

Since 2008, Save the Children has advocated for moreeffective U.S. foreign assistance, and, similarly to theGAO, sees an opportunity for USAID to adoptadditional indicators that will allow its Missions to betterevaluate the breadth and depth of their Local Solutionsinitiative-related interventions. The goal of promotingcountry ownership is central to USAID’s efforts to helpdeveloping countries assume full responsibility for theirown development in the long run.

4

I. Background

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

Country Ownership is “the practice ofpartner countries taking the lead indesigning and implementing clearly defineddevelopment strategies and managing theirown development processes.”

USAID ADS Chapter 220 – Partial Revision, 03/26/2012

5

1. Scope

This research analyzes a sample of 55 (out of 103identified) procurements issued by USAID Missions in2009 (the year before the launch of USAID Forward), 2012,and 2013 in six countries to track how they addresscountry ownership over time, and across differentcountries.4 The six countries are: Bangladesh, DominicanRepublic, Haiti, Kenya, Philippines, and Uganda.

Procurements – also known as mechanisms or instruments –are an essential element of USAID’s Program Cycle.5 Theyare one of the primary means by which the agency designs,implements, and later evaluates its activities. Procurementsare also used to evaluate USAID’s policies and relatedstrategies. If USAID procurements contain the key criteriaof country ownership, the agency’s partners – includingcooperating agencies, contractors, civil societyorganizations, and host governments – will likely addressthese criteria in their proposals, when bidding on theseprocurements, and when implementing resulting projects oractivities.

The six countries in this study were selected from the high-investment countries identified in the GAO’s April 2014report on Local Solutions and based on the need for diversegeographical representation. High-investment countries arecountries where USAID made the most Local Solutions-related investments in 2012 in order to achieve its fiscal year2015 30 percent target for direct funding to localinstitutions. High-investment countries were selectedfor this study because they would have a greaterdiversity of country ownership-relatedprogramming, thus providing more opportunitiesfor learning.

2. Methodology

To select the procurements for this analysis, a wide net wascast to identify all publicly available procurements issued byUSAID Missions in these countries in 2009, 2012, and 2013.The USAID Forward launch year 2010 as well as 2011 wereconsidered too early for noticeable implementation reformsand were therefore left out. In all, 103 procurements wereidentified – a random mix of mainly Request For Assistance(RFA or grant) and Request For Proposal (RFP or contract)issued by country and regional Missions and centralbureaus.6 The number of procurements per year, percountry, and per sector varied considerably – seeProcurement Details table in Annex 1. Fifty-fiveprocurements were selected out of the 103 identified fordata gathering and analysis, with the aim of a balanceddistribution across year, country, sector, and procurementtype, and the goal of reviewing at least50 percent of the gathered procurements.7

To review the procurements selected, four countryownership criteria were adapted from the U.SGovernment Interagency Paper on Country Ownershipand the Modernizing Foreign Assistance Network’s“Way Forward: A Reform Agenda for 2014 and Beyond.”8These criteria are:

1. Support to country priorities (specific host country priorities or plans referenced);

2. Local participation in project implementation (local institutions’ roles as lead project implementers or sub- grantees/sub-contractors);3. Local access to resources (USAID Mission direct funding to local institutions); and

4. Accountability to local stakeholders (alignment of project indicators with those in the host country plan, local participation in project performance monitoring and oversight, or mechanisms to share project results with local stakeholders).

II.Research Scope and Methodology

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

“These principles of effectiveness –host country ownership and strategicpartnering – should be applied appropriatelythroughout the Program Cycle”

“USAID Program Cycle Overview,” December 9, 2011,page 4.

Procurements received a score of either 5, 3, or 1,depending on the existence or lack of specific attributes orcharacteristics under each country ownership criteria – seeScoring Matrix on the 4 Country Ownership Criteria tablein Annex 1. For example, under the support to countrypriorities criteria, a procurement received a score of 5 if itreferenced a specific host country priority or plan itintended to support; a score of 3 if it only recommendedmeasures for the participation of local institutions in projectactivities instead; and a score of 1 if it had neither. Becauseof the importance of USAID Mission direct fundingto local institutions, this characteristic was weightedin the methodology. For example, USAID directfunding to local institutions is a determiningcharacteristic for local access to resources andparticipation in project implementation.

Further, a bonus point was awarded to procurements thatexplicitly referenced USAID Forward or other U.S.Government policies promoting country ownershipprinciples. An additional bonus point was given to those thatrequired the potential contractor or grantee to provide asustainability plan. Ultimately, each procurement couldtherefore receive a high score of 5 points for each criterionfor a total of 20 points and 2 bonus points for a possibletotal score of 22 points. A detailed scoring matrix can befound in Annex 1.

In addition, an anonymous survey of 80 USAID MissionDirectors was conducted to gauge their perspectives onissues related to country ownership implementation.Although the 21 percent response rate did not meetminimum requirements for inclusion in the research data,the responses (and particularly the comments received)provide insights into the challenges and opportunities facingUSAID Missions in their attempt to translate USAID Forwardinto practice. Some of these comments are highlightedthroughout this report and have been edited to ensureanonymity. A summary of the survey responses can be foundin Annex 4.

3. Limitations

Local contribution to project results was an additionalcriterion considered but proved difficult to score in thisexercise. There are three possible ways in which localinstitutions can contribute to USAID’s project results: directimplementers, sub-grantees or sub-contractors, and localinstitutions not receiving any type of funding from USAID.The contribution of local institutions to USAID results couldhelp assess the capacity of these institutions moreaccurately. This criterion should be considered when usingthis methodology at the project implementation level, afterthe procurements have been awarded.

Initially, the research aimed to compare and contrastprocurements released before and after USAID Forward.However, due to limitations on how long procurementsremain publicly available, the majority of the procurementsidentified were in years 2012 and 2013. The procurementsample for 2009 was too unevenly distributed across the sixcountries (eight procurements across three countries) anddid not allow for overall comparison between pre- and post-USAID Forward periods.

This report focuses only on USAID’s internal efforts –through its procurement process – to implement its goal ofpromoting country ownership. The research did notperform any analysis beyond the content of theprocurements reviewed. However, supporting policydocuments from USAID, USAID Missions, and partnercountry governments were reviewed to provide context fornational development plans and initiatives mentioned in theprocurements.

Although proposals responding to USAID procurementstend to adhere to the content and requirements in theseprocurements, it is fair to assume the possibility of adifferent reality on the ground during award negotiationsand project implementation.

Finally, government-to-government agreements and relatedfunding mechanisms that are also part of the Local Solutionsinitiative were not analyzed for this review becauseinformation on these agreements is not publicly available.

6

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

7

1. A systematic approach to trackingand reporting on country ownershipimplementation is feasible and informative.

This research illustrates a revealing and reliableapproach for assessing the consistency of USAID’smultifaceted efforts to implement the LocalSolutions initiative and promote country ownershipthroughout its Program Cycle. It could help USAIDcapture and share the type of comprehensiveimplementation update that the Local Solutionsinitiative is currently missing. The USAID ProgramCycle includes different stages and levels related to policyand strategy development, country developmentcooperation strategy formulation, project design, projectimplementation, performance monitoring, programevaluation, and learning and adapting to improvedevelopment outcomes. This approach could be used fortracking and analyzing progress on country ownershipimplementation at any relevant level of the Program Cycle,from the agency’s intentions (as laid out in policies andfunding mechanisms) to the implementation of fundedactivities and their outcomes. Finally, a systematic approachalso has the advantage of creating a management awarenessof Missions’ operations based on facts or data that canprevent conflicting perspectives on their work. For example,the Mission Directors Survey responses show thatresponding Missions overwhelmingly fund local institutions(#7) and jointly plan and co-finance projects with localinstitutions (#8), but the majority of them do not allowcountry voices in Missions’ funding decisions (#4). Annex 2shows additional analytics possible with a systematicapproach.

2. Countries reviewed show efforts on integratingcountry ownership principles.

Average scores across countries are encouraging. This might bea result of USAID’s significant resource flows in these countriesto implement the Local Solutions initiative. The ProcurementScores Across Countries graph below shows average procurementscores in each country in 2012 and 2013. Whileacknowledging that a two-year period is not longenough to constitute a trend, the changes in scores

between 2012 and 2013 are noticeable and create alevel of awareness that should be significant to USAID.Bangladesh and Uganda appear to have improved most from2012, followed by the Philippines, Kenya, and the DominicanRepublic. Only Haiti shows a slide back from the previous year.

Several factors may explain the difference in procurementscores across countries, including factors outside of USAIDMissions’ control. Each USAID Mission is responsible forformulating its own procurements or endorsing andparticipating in multi-country procurements issued byUSAID bureaus or regional Missions. There are alsodifferences in context-specific approaches from Mission toMission that are reflected in the Missions’ CountryDevelopment Cooperation Strategies (CDCS). Five of thesix countries have approved CDCS in place.9 Most of theseCDCSs are in the early stages of implementation, but theyoutline USAID’s commitment to promoting countryownership, including strengthening its engagement with localstakeholders.

III. Research Findings

Procurement Scores Across Countries

Average total score for each country, 2012-2013. Score out of 22.0points includes 5.0 points per Country Ownership Criteria and bonuspoints for U.S. Policies and Sustainability.

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

Another significant differentiator is the number ofprocurements falling under U.S. Government initiativesthat integrated country ownership principles in theirformulation, such as Feed the Future, the Global HealthInitiative, the Global Climate Change Initiative, and thePartnership for Growth. These procurements integratedcountry ownership objectives more effectively thanprocurements outside of these initiatives. Countries withthe majority of procurements under these initiatives alsotended to score higher. This is because, in procurementsused to implement these initiatives, USAID requestedproposals that often addressed all four criteria used in thisstudy. For example, U.S.-Haiti Feed the FuturePartnership: Northern Corridor procurement (RFP SOL-521-12-000021), issued by USAID Haiti in 2012, sought toincrease agricultural incomes in Haiti’s NorthernCorridor. The procurement required proposed activitiesto: (1) support Haiti’s National Agricultural InvestmentPlan; (2) receive guidance in some instances from theMinistry of Agriculture and/or other capable localorganizations; (3) provide sub-grants and sub-contracts tolocal organizations to implement activities; and (4) useFeed the Future Indicators and involve the Ministry ofAgriculture and other local organizations in monitoringand evaluating activities, including a complete transition tofull local implementation in Year 4.

3. The procurements reviewed address thecountry ownership criteria satisfactorily.

The majority of procurements reviewed did well acrossthree of the four criteria. For example, 92 percent of theprocurements reviewed referenced specific countryplans or priorities they intended to support, and90 percent recommended some sort of mechanisms beput in place during the implementation phase to shareproject results with local stakeholders. Only 20 percentaimed to establish direct partnerships between USAIDMissions and local institutions.10 However, hadgovernment-to-government agreements been publiclyavailable and included in this sample, it is likely that thescores under this criteria would have been slightly higher.

Overall, these scores provide a solid basis for theUSAID Missions’ intentions to promote countryownership and lay the ground for monitoringprogress during the implementation phase, afterthese procurements have been awarded. Forexample, in this study, the Local participation in projectimplementation criteria showed a slight decrease in scorebetween 2012 and 2013, which could be due to adecrease in USAID Mission funding (direct or indirect)provided to local institutions or the unsuccessful launchof projects involving local institutions10 (see ProcurementScores Across Country Ownership Criteria graph in

8

Total Award Amounts and Number of Awards by Country, 2012-2013

Sum of Awarded Amount (in U.S. Dollars) per year broken down by Country. Color shows details about institution Type. Labels specifynumber of awards.

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

Annex 2). To investigate this drop in scores, the studyreviewed data reported by USAID on Missions’ directfunding to local institutions in 2012 and 2013. Thefollowing graph – Total Award Amounts and Numberof Awards by Country, 2012–2013 – summarizes thefindings.11 It shows a slight decrease in the number oflocal partners in Bangladesh (17 to 13), Haiti (33 to 30),Kenya (72 to 71), and Uganda (59 to 57). In Bangladeshand Kenya, however, the total award amount doubledfrom 2012 to 2013, despite the slight decrease in thetotal number of local partners.

4. RFAs and RFPs scored equally on thefour country ownership criteria.

The 55 procurements were also analyzed to find outwhether RFAs (also referred to as cooperativeagreements or grants) or RFPs (also referred to ascontracts) were more conducive to implementingcountry ownership principles. The prevailing wisdom isthat the flexibility and relatively limited oversightassociated with grants and cooperative agreements makeRFAs more conducive to address thematic issues such asthe goal of promoting country ownership than RFPs orcontracts.12 At the procurement formulation stage,the data show no significant difference betweenthese two mechanisms.13 The difference might be atthe project or activity implementation level. Nonetheless,the lack of clear difference at this critical stage raisesquestions about the likelihood of a practical difference atthe implementation level. A detailed scoring under eachcriterion is provided in Annex 2c.

5. Critical data within USAID’s reach is notbeing used to better understand the agency’sefforts to implement country ownership.

It would have been unproductive to review the 55procurements for country ownership compliance usingonly the Local Solutions’ single indicator: percentage ofUSAID Mission funding obligated to local organizations inpartner countries. The overall purpose and structure ofeach of these procurements would have been missed.Each country ownership criterion required theconsideration of at least one or two additionalcharacteristics. For example, to get a true sense of howmuch USAID funding flows to local institutions, sub-grants or sub-contracts to local institutions were alsotracked, in addition to the USAID Mission direct fundingtargets. All data on sub-grants and sub-contractsare readily available through USAID grantees’and contractors’ approved budgets. Suchinformation is one tool USAID could use to collect morequantitative and qualitative data about its interventions.Yet, the aggregated amount of USAID Mission fundingchanneled this way to local institutions annually and overtime is unknown.

Similarly, most of the procurements reviewed, andparticularly the RFAs, encouraged cost share as animportant element of the USAID-recipient relationship.In Kenya, the Mission is also exploring the use of a“Partnership Fund” to raise additional resources from thelocal private sector. The resources USAID leveragesthrough cost share, co-financing, and other meansfrom local institutions – host governmentagencies, civil society organizations, and localprivate sector actors – might not be significantcompared to the agency’s own contributions, butthey should be nonetheless tracked andmonitored across projects and activities, and overtime. They complement USAID Missions’ ownresources and demonstrate local commitment to theagency’s development objectives.

9

“The PMP will include benchmarks forprogram performance over the courseof the implementation period andcontain metrics to assess countryownership and the sustainability ofsuccessful interventions introduced withprogram support.”

Performance Monitoring Plan (PMP) requirement forRFA-OAA-12-000011, page 34, Dominican Republic.

“[There is a] significant cost shareelement with the Host Government, over50 percent [of] many programs minussecurity costs.”

Comment from USAID Mission Directors Surveyresponses (on joint planning and co-financing, #8).

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

In general, all procurements reviewed for this analysisrequired grantees and contractors to address specificissues or to collect specific information that hadrelevance for the Local Solutions’ three main goals.USAID Missions collect a wealth of information throughtheir projects and activities. For example, in RFA-OAA-12-000011 issued for the Dominican Republic, USAIDrequired applicants to provide metrics on how to measurecountry ownership and sustainability of activities.It is safe to assume that all the responses to this RFA,including the winning bid, provided a number of metricson these two issues for USAID’s consideration. Trackingand analyzing this readily available information in astandardized way would greatly inform and expand theagency’s efforts on the Local Solutions initiative.

6. A shared definition of success is still lacking.

Local access to information is necessary for stimulating localengagement. Engagement leads to stronger ownership,which in turn is an essential step toward the sustainability ofeach procurement’s programmatic objectives. Valuing andmeeting the “need to know” of local stakeholders is quitenew and different from USAID’s usual focus on donor-grantee or contractor accountability mechanisms. It istherefore encouraging that the CDCSs of the countriesreviewed contain plans to take initial steps to improve onthis criterion. For example, as part of its CollaboratingLearning and Adapting (CLA) agenda, USAID Uganda plansto conduct at least one portfolio review a year with allpartners and stakeholders.14 Similarly, USAID Kenya plansto use Project Implementation Committees with hostgovernment representation to foster “joint management andimplementation’’ and dialogue throughout the ProgramCycle.15

However, the most meaningful way for USAID toimprove its accountability to local stakeholders is toshare the same vision and definition of success withlocal institutions, based on indicators that are jointly

agreed upon. This might require a joint and open exerciseto harmonize indicators between USAID’s supportedprojects and activities and the host country priorities theyintend to support. In this study, only a limited number ofprocurements, mostly in the health (HIV/AIDS) andeducation sectors, recommended using the same indicatorsas those in the host country plans they intended to support.Promoting country ownership requires both host countriesand donors to improve the way they do business. Hostcountry priorities and plans need strong indicatorsto achieve their objectives, and USAID should helpaddress that need whenever it arises. Doing so wouldstrengthen the monitoring and evaluation systems that theseplans are based on, by ensuring that they employ strongerindicators, and making it easier for donors to align theirinvestments and objectives.16

7. The role of local institutions in USAIDprojects is broader than reported.

In addition to the 20 percent of procurements intended tofund local institutions directly, through limited competition,75 percent of the procurement sample studied allowed theuse of sub-grants and sub-contracts to fund localinstitutions.

Yet, USAID monitors only program budgets and the resultsprovided by direct grantees or contractors who are chargedwith managing, consolidating, and reporting on the work oftheir sub-grantees and sub-contractors. This businessprocess might be missing an important step on the countryownership continuum, which is fostering relationships withlocal institutions that can deliver development projects.With USAID’s reporting requirements in the procurementsstudied focusing only the primary partner, the work ofhundreds of local organizations could go unrecognizedbecause they are unattributed in consolidated programmaticreports by USAID’s primary partners. In addition to theimportant goal of working directly with localinstitutions, USAID has an opportunity through its

10

“Mostly, [the] Mission uses Washington-developed standard indicators, and therewas no indicator harmonization exercisedone with host country.”

Comment from USAID Mission Directors Surveyresponses (on use of host country indicators, #9).

“This year we obligated 39% of ourfunds directly with the local governmentor local companies/NGOs. We expect toreach 52% next year.”

Comment from USAID Mission Directors Surveyresponses (on funding local institutions, #7).

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

sub-grants and sub-contracts to lay the foundationfor future partnerships with local institutions. Thiscould be done by tracking and documenting resultsdelivered by local sub-grantees and sub-contractors in waysthat could inform other aspects of its developmentobjectives. This would generate useful information for the

agency to help (a) understand the local institutionalecosystem, (b) determine the status of local capacity and itstrue (not perceived) ability to carry out any givendevelopment objective, and, most importantly, (c) define thenecessary capacity-building investments and their expectedoutcomes over time.

11

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

Recommendation 1: USAID should conducta comprehensive review of efforts acrossMissions implementing the Local Solutionsinitiative.

Despite the inherent complexity, the Local Solutionsinitiative needs a comprehensive assessment of its efforts toidentify and share its early successes and challenges. Thecontent of the 55 procurements reviewed in this studyshows USAID’s multifaceted efforts to promote countryownership. Unfortunately, similar conclusions cannot bemade of the hundreds of other USAID procurements issuedin 2012 and 2013 across the other 71 USAID Missionsimplementing the Local Solutions initiative in the developingworld because this study focused only on a sample of high-investment countries.17 A comprehensive assessment istherefore needed.

Four years of implementation of the Local Solutionsinitiative is a long enough timeframe to start reviewing atleast how and to what extent the agency’s reformedbusiness processes address the initiative’s goals at key stagesof its Program Cycle – i.e., project design andimplementation. Such a review should be structured andsystematic – based on defined criteria and relatedcharacteristics – to allow for tracking and analyzing effortsover time and across countries, interventions, and sectors.This type of assessment would help USAID identify andaddress emerging challenges, and investigate and learn frompromising practices across Missions, as well as provide atimely progress update to the development community.

Recommendation 2: USAID should adoptadditional standardized indicators tocomplement its current single indicator andbegin to track country ownershipimplementation more broadly.

The Local Solutions initiative’s single indicator fails toprovide an overview of the efforts it takes to promotecountry ownership. For example, to be meaningful, USAID’ssupport to country priorities should be put into context, interms of change over time in the proportion of the agency’sprojects and activities aligned with these priorities vs. thosethat are not. Similarly, if resources are spent to build thecapacity of local institutions to receive funding directly fromUSAID, the annual results of this effort in terms of thenumber of new local partners – i.e., local institutionsgraduating from sub-grantee or sub-contractor status toprime – in the fiscal year is critical information to evaluatethe success of the initiative. Potential indicators based oncharacteristics recurring consistently across theprocurements reviewed include:

• Number and percentage of projects supporting specified country priority or plan;

• Number of new local institutions leading project implementation in the fiscal year;

• Number of local institutions implementing sub-grants or sub-contracts;

• Total value and percentage of project funds awarded to local institutions in the fiscal year (disaggregated by type of funding);

IV. Recommendations

• Value and percentage of project funds contributed by local institutions;

• Number and percentage of projects with results achieved by local institutions;

• Number and percentage of projects with a sustainability plan;

• Number and percentage of project indicators aligned with indicators in host country development plans;

• Number and percentage of USAID-funded projects with oversight or steering structures that include local institutions.

These standardized indicators would allow all USAIDMissions to speak the same language, and to compare“apples to apples” across countries and over time. The dataneeded to report on these indicators are readily availableacross USAID Missions.18 An Indicator Matrix with detailedinformation, including sources of data for reporting, is inAnnex 3.

Recommendation 3: USAID should requiredisaggregated reporting of funding to andresults delivered by local sub-grantees andsub-contractors.

Over 75 percent of the procurements analyzed allow for theuse of sub-grants and sub-contracts to engage localinstitutions in USAID-funded activities. Local recipientsimplement specific activities proportionate to the fundingreceived, and report their results to USAID through theirlead implementing partners. However, in practice, they haveno official relationship with USAID. The agency should findcreative, but practical, ways to pay more attention to thisgroup of local institutions. It could use sub-grants and sub-contracts as a means to initiate and nurture relationshipswith them, in ways that still preserve the “prime/sub”rapport per USAID’s own rules and regulations. Moreover,sub-grants and sub-contracts could also be used tointentionally build a pipeline of future local primary partnersto carry forward the development objectives of the agency.USAID should therefore approach sub-grants and sub-contracts with an explicit learning agenda, broader than, butinclusive of, its financial accountability needs.

12

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

13

The study shows that the use of a structured andsystematic approach to assess country ownershipimplementation is informative. The approachmodeled in this paper could be adapted and scaledfor tracking efforts over time, and across multipleinterventions and countries. This type of analysis ofthe Local Solutions initiative is urgently needed aswe enter its fifth year of implementation.

On average, the procurements across these high-investment countries performed well. Their formulationreflected, to some degree, efforts at the USAID Missionslevel to support country priorities, promote localparticipation in the implementation of USAID-supportedprojects, facilitate local access to USAID funding, and beaccountable to local stakeholders. The research coveredprocurements across only six of the 77 countries wherethe Local Solutions initiative is being implemented.USAID should therefore undertake a more exhaustivereview across its portfolio to better document its effortsto promote country ownership.

There is more to promoting country ownership than theamount or proportion of USAID Mission funding

obligated to local institutions. While this is an importantand accurate indicator for measuring USAID’s fiscal year2015 target of 30 percent, it is simply not sufficient tocomprehensively monitor and assess the agency’s goal ofpromoting country ownership. The Local Solutionsinitiative’s reliance on this single indicator has led theagency to miss capturing other critical aspects ofimplementing the goal, and could thereby undervalue thelocal expertise, commitment, and resources that are socritical to the country ownership process.

The additional indicators suggested in this report couldallow USAID to capture the breadth and depth of itsefforts to implement the Local Solutions initiative, andprovide much-needed comprehensive reporting onprogress and challenges. The procurements reviewedshow that most of the data needed to inform theseindicators is already being collected at the Missions level.

Overall, for the USAID Forward reforms to generate theexpected impact, the agency must monitor meticulouslyand learn from each stage of their implementation,including the critical Local Solutions initiative.

V. Conclusion

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

14

1. Research Methodology Scoring Matrix on the Four Country Ownership Criteria

A procurement received either 5, 3, or 1 points depending on the existence of specific language describingattributes or characteristics of each criteria as follows:

Scoring Examples

• Support to Country Priorities: RFA-617-13- 000004 in Uganda seeking to address climate change through a Feed the Future partnership received 5 points for supporting the National Development Plan and the Ministry of Water and Environment’s strategy on climate change resilience.

• Local participation in Project Implementation: In Bangladesh, RFA-388-13- 000008 seeking to promote wildlife and environmental conservation received 5 points because the lead implementer role was limited to Bangladeshi organizations.

• Local access to Resources: RFP SOL-521-13- 000013 in Haiti supporting post-earthquake reconstruction received 1 point for not recommending direct funding or sub-contracting as a means to fund Haitian institutions.

• Accountability to local stakeholders: RFA- OAA-14-000001 promoting an AIDS-free generation in the Dominican Republic received 5 points for seeking to align project indicators and reporting requirements with the national HIV/AIDS monitoring and evaluation plan.

VI. Annexes

Country OwnershipCriteria

Support to countrypriorities

Local participation inproject implementation

Local access to resources

Accountability tolocal stakeholders

• Specific country priority referenced

• Participation of local institutions in project implementation

Neither

Neither

Neither

Neither

• Participation of local institutions in project activities without USAID funding

• Only sub-grants/ subcontracts recommended

• Only USAID - grantee/contractor accountability required

• Lead implementer or

• Sub-grantee /sub- contractor

• Direct funding

• Alignment of indicators or

• Role in performance assessment & oversight or

• Sharing of results

Characteristics (Score of 5)

Characteristics (Score of 3)

Characteristics(Score of 1)

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

15

* (11) represents total procurements per country.

Region

Country

Africa Latin America /Caribbean

Asia

Philippines(8)

Bangladesh(10)

DominicanRepublic (6)

Haiti (9)Uganda (11)Kenya (11)*

8

1

2

3

8

4

0

1

1

0

2

0

1

2

1

1

6

5

0

3

8

4

0

2

1

0

1

2

1

2

2

1

4

5

0

1

8

3

0

1

4

0

0

0

1

4

1

1

6

0

0

2

4

2

0

2

1

0

1

0

1

1

1

0

8

2

0

1

9

0

0

3

1

0

2

2

0

2

2

2

3

5

0

1

7

0

5

1

1

0

2

4 2 2 2 0 0

2

2

1

1

0

Procurement Types

RFA

RFP

APS

Competition Types

Limited Competition

Open Competition

U.S. Initiatives

Feed the Future

Partnership for Growth

Global Climate Change Initiative

Global Health Initiative

Major Sectors

Agriculture

Education

Environment

Business

Health

Governance

Other

Procurement Details

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

16

2. Data Analysis of the 55 Procurements

a. Procurement Scorecard for 6 countries, 2009, 2012, 2013.

Notes: U.S. Policies scores the existence of explicit language ina procurement referencing theParis Declaration on Aid Effectiveness, U.S. Governmentbilateral initiatives promotingcountry ownership principles,and/or specific initiatives underUSAID Forward;

Sustainability scores explicitrequest in a procurement forpotential contractors orgrantees to provide asustainability plan for theirproposed activities.

Total score per procurement out of 22.0 points: 5.0 points per Country Ownership Criteria, 1.0 point for U.S. Policies, and 1.0 point forSustainability. Procurements ordered by year and country. Green bars signify a Country Ownership score of 5.0. Yellow bars signify aCountry Ownership Criteria score of 3.0. Red bars signify a score of 1.0. Stars represent 1.0 bonus point earned.

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

17

b.

c.

Procurement Scores Across Country Ownership Criteria

Average Procurement Scores Across Sectors

Average score out of 5.0 points per year across the Country Ownership Criteria.

Total Average score out of 22.0 points for selected sectors with three or more procurements in 2009, 2012, and 2013. Score includes5.0 possible points per Country Ownership Criteria, 1.0 possible points for U.S. Policies, and 1.0 possible points for Sustainability. Sectors orderedby total average score. Size of stars proportional to average bonus area.

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

18

d.

e.

RFA and RFP Across Country Ownership Criteria

Bonus Scores

RFA and RFP Procurements compared by Total Average Score and broken down byCountry Ownership Criteria.

Average out of 1.0 points per year in the Bonus areas.

Bonus – U.S. Policies

2009 2012 2013 2009 2012 2013

Bonus – Sustainability

RFA Procurements19.459

AccountabilityCountry PrioritiesImplementationResourcesBonus – SustainabilityBonus – U.S. Policies

RFP Procurements19.611

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

AssessmentCriteria

Relevance Possible Indicators Definition Source of Data Baseline Target Application of Metrics/Data Utilization

Support tocountrypriorities

Project supportsspecific country(sectoral ornational) plansor priorities(referenced inprocurement orCDCS)

USAID CDCS

Project documents

Relevant nationaldocuments

Project reports

List of awardees

Capacity-buildingplans

Sub-grant/sub-contractdocuments

Evaluation reports

Assessment criteria for (a)review of draft solicitationdocuments by the public;(b) applicant proposals; (c) portfolio review foralignment with countrypriorities

Evaluation criteria forincreased projectimplementationresponsibilities to localinstitutions

Proxy for improvedcapacity of localinstitutions over time

USAID Missions orUSAID Forward data

Host governmentproject documents

National/localgovernmentbudgets

Financial accountability

Evaluation of capacity-building investments

Sustainability of projectresults

Assessment of domesticresource mobilizationefforts

Number and percentageof projects supporting aspecific country priorityor plan

Projects that are aligned with orsupport country priorities (asdocumented in national or locallevel strategy documents,national action plans, policydocuments, and other relevantglobal and regional conventionsand bilateral initiatives such asthe MDGs)

Projects that increaseprogrammatic responsibilities oflocal institutions, includingcapacity-building-focused projectswith exit strategies such astransition of programresponsibilities to local institutionsas a result of strengthenedcapacities; Local institutionsinclude national and localgovernments, Non-GovernmentalOrganizations, and private sectorentities; Data should bedisaggregated by type ofinstitutions and by year

Projects that are partially or fullyimplemented by local institutions;Data should be disaggregated bydirect and sub-grant and sub-contracts

Existence of cost share, matchingfunds, or co-financingmechanisms (whether requiredin solicitations or not)

Number of new localinstitutions leadingproject implementationin fiscal year

Number of localinstitutionsimplementing sub-grantsor sub-contracts

Total value andpercentage of projectfunds awarded to localinstitutions in fiscalyear

Value and percentageof project fundscontributed by localinstitutions

Local institutionslead or arequantifiablyinvolved in theimplementationof the project

Local institutionsmanage all orportion of projectfunds

LocalParticipationin ProjectImplementation

LocalAccess toResources

3. Indicator Matrix

19

Number and percentageof project indicatorsaligned with indicatorsin host countrydevelopment plans

Number and percentageof USAID-fundedprojects with oversightor steering structuresthat include localinstitutions

AssessmentCriteria

Relevance Possible Indicators Definition Source of Data Baseline Target Application of Metrics/Data Utilization

LocalContributionto ProjectResults

Project results arepartially or totallyattributable tolocal institutions

Project documents

Publiccommunicationdocuments

Project documents

CDCS

PerformanceMonitoring andEvaluation (M&E)Plans

Project documents

Proxy for improvedcapacity of local institutions

Sustainability of projectresults

Learning and developmentof effective capacity-building models

Strengthening ofnational systems (M&Ecapacity)

Donor coordination

Transparency of donorpractices

Proxy for financial andprogrammaticinformation sharingwith local institutions

Number and percentageof projects with resultsachieved by localinstitutions

Number and percentageof projects withsustainability plan

Contributions of localinstitutions are clearlydocumented in project resultsreports and/or publiccommunications.

Presence of a reasonablyachievable sustainability plan(financial and technical) inprojects awarded to or managedby local institutions.

Use of shared or complementaryindicators to demonstratealignment of project measurementsand reporting requirements withhost country national or sectoraldevelopment plans – making keyindicators relevant to or usable byhost country and vice versa.

Projects with processes orstructures that allow local voicesto inform planning and fundingdecisions – i.e., ProgramAdvisory/Steering Committees toguide program design andimplementation.

Project reportingrequirementsaddress bothdonor’s and localstakeholders’“need to know”

Accountabilityto LocalStakeholders

20

21

4. Mission Directors Survey Responses

“Yes” or “No” responses to Questions 1-16 of the Mission Directors Survey. Answers based on 17 respondants. Survey had a 21.25%response rate.

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

22

5. Survey Questionnaire

1. The Mission has shared relevant aspects of the USAID Forward agenda (such as the Implementation and Procurement Reform and Monitoring and Evaluation Policy) with local institutions (government ministries and agencies, civil society organizations, and private sector actors).

2. The Mission invited local institutions (government ministries and agencies, civil society organizations, and private sector actors) to contribute content to the formulation of the Mission’s Country Development Cooperation Strategy (CDCS).

3. At least once a year, the Mission shares programmatic and funding information with local institutions (government ministries and agencies, civil society organizations, and private sector actors) in a comprehensive and relevant manner.

4. The Mission regularly invites local institutions (government ministries and agencies, civil society organizations, and private sector actors) to participate in USAID’s resource allocation discussions and decisions for prioritized programs.

5. The Mission’s programs are designed to support specific country development plans, including sectoral development plans.

6. The Mission funds multi-stakeholder development priority setting and strategic planning processes at the national and/or local levels.

7. The Mission directly funds or provides sub-grants to local institutions (government ministries and agencies, civil society organizations, and private sector actors) to implement shared priority programs.

8. The Mission actively encourages or seeks joint planning and co-financing mechanisms with government institutions (ministries and agencies).

9. The Mission’s priority programs use the same key indicators as the national or sectoral development plans.

10.The Mission’s annual programmatic results are captured in the national or sectoral development progress reports.

11.The Mission helps government institutions (ministries and agencies) define and establish systems and mechanisms to engage and/or fund non-governmental organizations.

12.The Mission’s priority programs support capacity- building needs expressed by or jointly identified with local institutions (government ministries and agencies, civil society organizations, and private sector actors).

13.The Mission encourages the use of social accountability mechanisms (through its procurement) to foster the participation of hard-to-reach community groups in its programs.

14.The Mission engages the local private sector in development programs.

15.The Mission works with the local private sector to deliver development programs.

16.The Mission invests in building the capacity of government ministries and agencies, and civil society organizations, to work with the private sector – such as funding, supporting, or promoting the development of Public-Private Partnerships between local institutions.

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

23

1 http://www.usaid.gov/usaidforward/

2 For Local Solutions’ overarching goals, see: “USAID Forward Progress Report 2013,” page 29

3 See: “Foreign Aid: USAID Has Increased Funding to Partner-Country Organizations but Could Better Track Progress,” April 2014, http://www.gao.gov/products/GAO-14-355

4 See Annex 2a for a detailed yearly breakdown of procurements per country.

5 “USAID Program Cycle Overview,” December 9, 2011, page 4, http://pdf.usaid.gov/pdf_docs/pdacs774.pdf

6 The selected RFAs and RPAs were gathered from Fedbizopps.gov and grants.gov between June 1, 2014 and July 31, 2014; the sample excludes government-to-government procurements because they are classified information.

7 The data sample studied includes 35 RFAs, 19 RFPs, and 2 Annual Program Statements (APS).

8 “U.S. Government Interagency Paper on Country Ownership: Global Health Initiative, 07-2012,” http://www.ghi.gov/principles/docs/ownershipInteragencyPaper.pdf; and, “Way Forward: A Reform Agenda for 2014 and Beyond,” http://www.modernizeaid.net/documents/MFAN_Policy_Paper_April_2014.pdf

9 Haiti is the exception. It does not have a CDCS and operates under the U.S. Government Haiti Rebuilding and Development Strategy, which also promotes a country-led development approach.

10 The timing and number of procurements may vary from year to year based on Missions’ strategies, needs, and activity cycles, and other external factors such as congressional appropriations.

11 Graph data based on “USAID Forward 2013 Progress Report.” See USAID Forward Results Data Table “Strengthen local capacity”, http://www.usaid.gov/usaidforward/.

12 For more information on the differences between USAID funding mechanisms, see: http://www.ngoconnect.net/documents/592341/749044/NGOTips+-+Introducing+USAID+Solicitation+Instruments

13 The data sample studied includes 35 RFAs, 18 RFPs, and 2 Annual Program Statement (APS).

14 “USAID Uganda Country Development Cooperation Strategy 2011–2015,” page 26.

15 “USAID Kenya Country Development Cooperation Strategy 2014–18,” page 97

16 The justification provided a few times (through comments in the Mission Directors Survey responses) for not using the same indicators as host countries is that their plans have either weak indicators or no indicators at all.

17 Local Solutions initiative data reported in “USAID Forward 2012 Progress Report” cover 77 USAID Missions.

18 With the exception of data related to the two indicators suggested under Accountability to local stakeholders, all other necessary data are readily available at USAID Missions and/or with USAID’s primary partners.

Endnotes

TRACKING USAID’S EFFO

RTS O

N THE LO

CAL SO

LUTIONS IN

ITIATIVE

Efforts on the Local Solutions Initiative

Since 2008, Save the Children has advocated for more effective U.S. foreign assistance that allowsfor greater collaboration and stronger partnerships with aid recipient countries. While manyaspects of foreign assistance contribute to the overall effectiveness of aid, this study has specificallyanalyzed how USAID procurements – one of the primary means by which the agency designs,implements, and evaluates its activities – have been used to translate the agency’s vision of aideffectiveness into activities on the ground.

The research analyzes a sample of 55 USAID procurements in six countries – Bangladesh,Dominican Republic, Haiti, Kenya, Philippines, and Uganda – to assess in each case how USAID haspromoted country ownership, a goal of its Local Solutions initiative. This goal is central to USAID‘sefforts to help developing countries move towards assuming full responsibility for their owndevelopment.

The research also explores the need for additional standardized indicators to better measure andshare USAID’s progress at putting Washington-inspired policies into practice in aid recipientcountries.

For more information, please contact:

Gretchen KingSpecialist, Aid Effectiveness 2000 L Street NW, Suite 500Washington, DC 20036Phone: 1.202.640.6780Email: gking@savechildren.org

savethechildren.org

A Review of Select Procurements in Six Countries

Tracking USAID’s