PIA 2734

Privatization and Contracting Out: The Knowledge and Skills Base

Assessing Performance in Contract Relationships

Contracts, Foreign Aid and International Development

The Policy Process Department of State

U.S. Agency for International Development

Office of Management and Budget- Executive Office of President

Congress

Policy Making and the Interaction of Major Agency Processes

Office ofManagementAnd Budget

(OMB)

Evaluation

Ex-PostFacto

Evaluation

Implementa-tion

Pre-Implementa-

tion

ProjectPaper(PP)

ProjectReviewPaper(PRP)

ProjectIdentificationDocument

(PID)

Field ofConcentration

Strategy(DAPII)

CountryProgramStrategy(DAPI)

PriorEvaluation

OperationalYear Budget

(OYB)Appropriation

CongressionalPresentation

(CP)

BudgetSubmissions

OngoingProjects

HostCountry

Legis-lation

ForeignPolicy

LDCNeeds

Agency PolicyGlobal Sector Strategies

Regional StrategiesResearch Strategy

Management Objectives

Financial MANAGEMENTProgramming INFORMATION Management ReportsImplementation SYSTEM External NeedsProgram Support Data Bank (CPDB, PAIS, DIS, ESDB)Personnel Administration Support Database for Future Decisions, PolicyLessons Learned

Evaluation Criteria

Project ReportingProject PerformanceTracking (PPT);Financial Reporting

Planning

Budgeting

Design Approval

Implementation

Evaluation

Reporting

Foreign Aid?

From Policy to Project

Grants vs. Contracts: Assessing Sub-Grants

RFA’s and Implementation

Project is the Common Denominator for the International Donor

Resourcing Projects

Project Identification, Formulation, Preparation & Design Problems in project identification

In developing countries: Lack effective procedures for project identification

within national planning agencies and operating ministries

Weak conceptual and operational links exist between various national, regional, local, and special interest constituencies

National plans often fail to provide a strategy for development

Allocation of resource issues Issues of priority

Project Identification, Formulation, Preparation & Design

Problems in project identification In developing countries, cont.:

Influence of interest groups Limited international assistance agency staff

time to help government planners Excessive turnover and rotation of field

representatives of assistance agencies Weaknesses in the overall planning system Design Primarily focuses on Contracting Process

The Name of the Game

Blueprint Approach to Development Planning

Pilot ProjectResearchers

Before-AfterSurveys

Planner

Administrators

TargetPopulation

EvaluationResearchersActions

ProjectBlueprints

Actual ChangeVersus

Targeted Change

Tested Models

The Blueprint

Contract Analysis Assessment vs. Evaluation

Impact Assessment

The Need for Quantitative Data

The Reason for Blueprints

The Project Cycle and the Contract Analysis--collection of:

Social Analysis targeted groups: women, minorities, indigenous peoples

Economic Analysis--Cost Benefit Institutional Analysis

Sustainability Organizational Requirements Recurrent Cost Implications Human Skills Needed Social Acceptance

The Project Cycle & Contracts

Design

Identifying nature of problem and possible solutions--specific needs and desired changes

Appraisal

(Mandatory) data needed to prepare project plan and measure completion

COMMUNITY PROJECT DESIGN

The Project Cycle

Analysis--collection of information

Prediction

Selection of preferred alternatives

Measurement of Impact to determine contract fulfillment

Source: Project Management System, Practical Concepts, Inc., Washington, DC 1979.

Project Objectives Achieved

3. Evaluation 2. Execution

1. Design

The Project Cycle

The Project Cycle Analysis

The Logical Framework: (LOGFRAME)

If-then conditions Some donors have moved away from Log-

frame Was replaced by a system based on

identifying Strategic Objectives, Intermediate Results, Measurable Indicators, etc.

That system was recently "de-emphasized." AID mission requests for funds were tied to

promises of specific results Results Framework system is "under review."

The Log Frame

Logical Framework Performance Networks

Practical Concepts, Incorporated

Project Objectives Achieved

3. Evaluation 2. Execution

1. Design

Evaluation System Reporting System

Evaluations assess performance against plans and analyze causal linkages

Progress indicators and formats for communicating project information

Networks display performance plans over time

ACHIEVEMENT

EXCEPTION

Project Management System Provides Management Toolsto Support all Stages of the Project Cycle

Preparation of Documents: Donor – USAID Development Assistance Programs (DAPs)

Country Strategy Paper

Concept Paper

Project Identification Document (PID)

Policy Making and the Interaction of Major Agency Processes

Office ofManagementAnd Budget

(OMB)

Evaluation

Ex-PostFacto

Evaluation

Implementa-tion

Pre-Implementa-

tion

ProjectPaper(PP)

ProjectReviewPaper(PRP)

ProjectIdentificationDocument

(PID)

Field ofConcentration

Strategy(DAPII)

CountryProgramStrategy(DAPI)

PriorEvaluation

OperationalYear Budget

(OYB)Appropriation

CongressionalPresentation

(CP)

BudgetSubmissions

OngoingProjects

HostCountry

Legis-lation

ForeignPolicy

LDCNeeds

Agency PolicyGlobal Sector Strategies

Regional StrategiesResearch Strategy

Management Objectives

Financial MANAGEMENTProgramming INFORMATION Management ReportsImplementation SYSTEM External NeedsProgram Support Data Bank (CPDB, PAIS, DIS, ESDB)Personnel Administration Support Database for Future Decisions, PolicyLessons Learned

Evaluation Criteria

Project ReportingProject PerformanceTracking (PPT);Financial Reporting

Planning

Budgeting

Design Approval

Implementation

Evaluation

Reporting

PP (USAID)(PP = Project Paper)

Program Agreement(Donor)

Technical Proposal(Contractor to Donor)

Country Context(Contractor to Country)

Implementation Documents

Characteristics Affecting Project Implementation

Less Problematic 1. Simple technical features2. Marginal change from status quo3. One-actor target4. One-goal objective5. Clearly stated goals6. Short duration

More Problematic 1. Complex technical features2. Comprehensive change from status

quo3. Multi-actor targets4. Multi-goal objectives5. Ambiguous or unclear goals6. Long duration

The Project Cycle

Implementation (Data- Country Program Data Bank, Economic and Social Data Bank, Project Accounting Information System, Development Information System)

Carrying out actions planned

Personnel- local (and foreign)

Budget and Accounting Information

The Project Cycle Monitoring and Evaluation: Focus on Verification of Completion of

Contract Linked to End of Contract and Verification of

Objective indicatorsUnderstanding what has happened and assessing

changes and quality of change Issue: sustainability regarding follow-on within the

country and replicability from one country to another

Monitoring and Evaluation Nature of Data:

Interview vs. surveySeat of the pants observation

"the old quick and dirty"

The problem of project goals: Goals are to be limited and bounded Specific activities are to be clearly defined and

achieved Short run success leads to successful evaluation Short-term loop is five years

Monitoring and Evaluation

Nature of Data:Judgment: Evaluation vs. Assessment

Two views: a. Learn from experience b. Judge performance

Problem: judgment requires clear goals, in contradiction with learning

Problem: power of the expert Problem: Contract limits judgement

Monitoring and Evaluation

Nature of Data:Evaluation is a donor requirement

External activity Targets blueprint activity (CPA) Critical path analysis (Time based action) PERT chart (Project Evaluation Review

Technique) very technical, programmed Evaluation often the need for more action Contracts Lack Flexibility

Monitoring and Evaluation

Nature of Data:Evaluation as an end product:

Separate from implementation Action pre-determined in design prior to

evaluation

Separates evaluation from the on-going activity

Evaluation Used to Determine Certification of Completion

Monitoring and Evaluation

IssuesProblem with Evaluation concept

Implementation suggests a finished product Bureaucratic action is ongoing Part of larger system with ambiguous boundaries

Assessment Ongoing, part of implementation process Not conducive to Contracts Management

End of Project Status

Central to the Contracting Out Process

End of Project Status (EOPS)

Are of great importance and are primary target of project efforts and discussion

Projects are usually very complex

It is common to find that no single indicator is sufficient to describe the project achievement completely

End of Project Status (EOPS) In determining EOPS we apply following

principle: If all EOPS conditions are satisfied, then there

would be no credible alternative explanation Except the purpose of the project (and the

contract) has been achieved

Good project design will include the conditions that demonstrate successful achievement of the Project Purpose

End of Project Status (EOPS) Example

PROJECT PURPOSE IN THE DEVELOPMENT OF AN OIL FIELD:

Export oil

EOPS 50,000 Barrels of crude/day transferred to tankers at

nearest port; Quality of crude produced is competitive with that

currently sold on world market. To verify, one needs a) the purity of oil, b) the world price, c) price sold, d) amount sold

Group Presentations