WOPs: how to make them even better? An approach to inform about WOPs’ performance
-
Upload
salvador-gilmore -
Category
Documents
-
view
17 -
download
0
description
Transcript of WOPs: how to make them even better? An approach to inform about WOPs’ performance
WOPs: how to make them even better?
An approach to inform about WOPs’ performance
M.Pascual, S.Veenstra, U.Wehn, R.van Tulder, G. Alaerts 30th May 2013
What are WOPs?
• WOPs encompass a great range of collaboration approaches, each able to foster different type of results
Source: IWA & UNHABITAT., 2009
“Any form of simple or structured partnership aimed at capacity building on a not-for-profit basis” (IWA, UN-HABITAT and VEI,2009)
"(...) a structured programme of cooperation among water operators, based on mutual support and on a not-for-profit basis (UN-HABITAT, 2007).
Minimum of
P-WOPsPerformance based non-delegation driven
partnerships
Key elements for WOPs’ success
• WOP evaluation studies converge in the following recommendations for WOPs to be effective mechanisms for change– Long term collaboration
• The transformation of new knowledge into organisational change is a slow process and requires time.
– Strong engagement from local management and sector stakeholders• The change process has to be led by the local organisation
– Funds for investments • KT is not enough to enable change. Limited operational improvements are feasible if no funds
for investments are available to scale up small steps into change.– Flexibility to emerging circumstances
• WOPs as a CD intervention require flexibility to adapt to the unpredictable evolution of the change process in order to ensure relevance and effectiveness
– High relational quality between partners• Particularly in WOPs, where there is no delegation of decision making power to the external
partner, change is socially constructed by partners interaction.– Report on achievements of WOPs during implementation of the project
• In order to gain and keep confidence of partners and stakeholders to keep supporting the process
State of the art on reporting performance of WOPs
• Change in KPIs are rarely achieved by WOPs except for long term structured comprehensive WOPs associated to investment programs
– Yet, in those cases, KPI improvements take years to be achieved• Even in cases of delegated management through PPPs KPI improvements take at least 4-5 years to
be achieved. I.e. Senegal PPP ,10 years or Mali, 5.– KPIs are insensitive to improvements demonstrated in pilot areas and to nascent changes
like strengthened capacity and improved processes, typical outputs of first years of WOPs implementation
• Hence, other measures are commonly used to report WOPs achievements. However, results reported are :
– Highly heterogeneous, which hinder comparative analysis, learning and dissemination– The dominant approach is to report on the basis of
• Inputs:– No and type of trainings– No of exposure visits, etc.,
• and not so much on the basis of outputs:– No of employees that gained specific knowledge– Degree of change in working routines, etc.
• WOPs have been criticised for being an accepted mechanism based on their solidarity nature despite the lack of empirical evidence of their effectiveness (Boag & McDonald, 2010)
There is need for an innovative standardised system to report performance of WOPs
Multi-path approach to inform about WOPs’ performance*
Project inputs consolidation into change
Degree of satisfaction
KPIs changes
Partners’ relational capital
Partners’ interaction Partners’ roles Dominant governing mechanisms Conflict management
Satisfaction with the project
Selected KPIs of the targeted water operator
Most valuable aspects
Capacity changes
Specific KPI- related capacity
Plausible attribution to the partnership
-
+
Time
Other effects of the projectSustainability of the change trend
Project activities-related knowledge consolidation into change
* performance of a WOP different from performance of water operator
WOP performance is measured through the ability of the partnership to effectively support the local operator in achieving ‘progress towards impact’
Cases in which the multi-path approach to WOP performance
• 2 parallel projects• Same type of contract: 4-years service contract 2009-2013 – no delegation
– VEI and Lilongwe Water Board (LWB)– VEI and Blantyre Water Board (BWB)
• 2 main components:– (i) CD and operational support– (ii) support in the supervision of the implementation of an investment program
• VEI tasks– 1. Organisational structure and staffing – 2. Financial and commercial management – 3.Management information system – 4. Reduction of NRW– 5. Operation and maintenance plans– 6. Optimising pumping regimes – 7. Extension of water supply services to LIAs– 8. Access to water facility– 9. Public Relations works– 10. HIV/AIDS programme– 11. Implementation of investments programmes for improvement of production capacity, reduction of
NRW and extension of services to LIA
• KPIs targets– NRW, Working Ratio, No kiosks built, Production capacity (only in BWB)
• Information collected after 2 years of implementation of the project
KPI achievements in each project
• KPIs targets were not met for NRW. • Only No of kiosks built in BWB and Working Ratio in LWB were met, however, partners argued the
limited impact of VEI inputs in the achievement of those 2 KPIs.
BWB LWB
48
4231
49.1 48.1
25303540455055
Baseline Year 1 Year 2
NRW (%)
Target
Achieved
1.14 1.25
1.01 1.09 1.0611.21.4
Baseline Year 1 Year 2
Working ratio
Target Achieved
030
120
0100200300
Baseline Year 1 Year 2
No of new kiosks built
Target
Achieved
39
3430
3938.7 36.9
2530354045
Baseline Year 1 Year 2
NRW (%)
Target
Achieved
1.22 1.271.141.2
1.38
1
1.2
1.4
Baseline Year 1 Year 2
Working ratio
Target Achieved
0
90192
0 0 270100200300
Baseline Year 1 Year 2
No of new kiosks built
Target
Achieved
Change in capacity to reduce NRW and plausible attribution
BWB LWB
Highest capacity change
Organisational structure, availability of resources Strategic management, resources and capability to reduce leakages and overflows in storage tanks
Project strongest contribution
Change of organisational structure, organisational motivation, resources, capability to reduce leakages, and learning capability
Organisational motivation, resources, capability to reduce leakage and learning capability
Strategic managerial level
Responsiveness of departments
Organisational motivation
Capabiilty to reduce customer metering errors and data handling errors
Capability to reduce leakages on service
connections up to pint of customer
48%48%47%
57%50%
43%50%
41%42%
53%46%51%
44%50%
25%33%36%
27%33%
34%36%
28%29%
28%34%
32%30%
25%
28%20%18%17%17%
23%14%
31%29%
20%20%16%
26%25%
Plausible attribution of changes by BWB staff (n=15)
Internally Partnership Externally
Project inputs consolidation into change
• Szulanski (2000) defines KT as– “ an unfolding process comprised of several stages that start with the
opportunity identification for knowledge transfer and ends once the recipient unit is able to maintain satisfactory performance derived from the integration of knowledge into working routines”
Project inputs consolidation into change - Project tasks-related knowledge translation into change-
Project inputs consolidation into change - Other effects from the project -
BWB LWB
CEO A force supporting organisational change via:-Reducing internal resistance to change;- Limiting political interference;- Increasing accountability for results;
A more accurate picture of the current status of LWB
WBs’ staff -Motivation to improve and engagement of staff;-Willingness to learn and improvement of staff attitude-Sense of urgency to improve;-Commitment of the different sections with a role in NRW reduction;-Improvement of management and coordination skills
-Stronger commitment to reduce NRW throughout the organisation; -Higher sense of urgency to improve
Project inputs consolidation into change - Degree of sustainability of perceived change -
– BWB members showed mixed opinions while LWB staff argued that if the project stopped the urge to improve would not be there any more.
– The conditions identified to sustain the change trend are mostly shared by both WBs:
BWB LWB
- Strong support from BWB management to the project and increase efforts to complete implementation of improvement plans
- Capacity to retain qualified staff- Maintain joint work with VEI- Keep the momentum of change and spirit of
improvement of the local staff- Enhance management and operational skills
- Executive management engagement and higher commitment and ownership of the project, particularly in implementation of plans
- Increased ability to retain staff- Foster good relationship and understanding
with VEI to favour joint work- Stronger communication, coordination between
different sections and team spirit and good working relations within LWB
- Additional training
Partners’ relational capital- Interaction, roles, governing mechanism, conflict-
BWB LWBFormal interaction Management: high to medium
Operations: high to highManagement : high to lowOperations: high to low
Informal interaction Management: low to mediumOperations : low to high
Management: low to lowOperations: low to medium
Roles external partner Consultant -> colleague Consultant -> controller
Engagement of local partner Low to high Low to low
Dominance of governing mechanisms
Contract -> contract + trust Contract -> contract
Conflict occurrence and impact in the partnership
Yes (KPIs, role of VEI )Positive impact on the partners relationship
Yes, (KPI, role of VEI, engineering tasks take over)Negative impact in relationship
Satisfaction of partners and stakeholders
• Degree of satisfaction
• Observations from partners and stakeholders– MIWD, EIB, EU
• Appreciate the innovative approach of an external operator working together with the WBs in strengthening their capacity while there needs to be accountability for results.
• Positively valued in terms of sustainability of the achieved improvements• They call for the need to go beyond KPIs to evaluate progress of project
– WBs• BWB, appreciation of the project at all levels.• LWB, appreciation of the project at operational levels, but not at executive management.
– VEI RPM • BWB emphasised the critical role of the CEO’s support and its translation into approx. 10 members of
WB actively supporting the change process.• LWB points at the lack of support from executive management, which translates into low
commitment at operational levels (only 2 members engaged)
BWB LWB
WBs’ executive management High Low
WB’s staff at operational and management level
High High
VEI RPM (resident), VEI PD High Medium
EIB, EU, MIWD High Low- Medium
Conclusions
• Findings– KPIs achievements provided vague information on the achievements of the recipient local operator as well as
effectiveness of the partnership – The proposed complementary paths of evidence proved highly informative on the progress of each project and the
effectiveness of the partnership
• The proposed paths of evidence serve different purposes
• Data sources are highly based on perceptions. Reliability of findings requires:– An objective (external) evaluator– A standard multi-method for data collection– A standard method for data analysis– Representative sample of key informants from both partners and stakeholders– Triangulation of information
Progress towards impact Effectiveness of the WOP Learning and adaptingKPIs XCapacity changes and attribution
X X X
Knowledge consolidation into change and obstacles
X X X
Partners’ relational capital X XPartners’ satisfaction X X
Statements for future improvement of WOPs
1. P-WOPs have the potential to be a slow but firm route to public driven utility reform processes, yet they have to be accountable for results and demonstrate its effectiveness in order to become a widely legitimised approach.
2. A widely used standard system to report P-WOP performance will not only build credibility of WOPs but also enable learning towards improvement of its effectiveness.
3. P-WOP contracts should be designed according to the notion of WOP performance and not merely to performance of the local water operator – Hence, incorporate ( shared ) incentives based on the multiple evidence
paths proposed