Nepal - Assessing the Social Impact of Rural Energy Services Alternative Energy Promotion Center...
-
Upload
joel-marsh -
Category
Documents
-
view
215 -
download
0
Transcript of Nepal - Assessing the Social Impact of Rural Energy Services Alternative Energy Promotion Center...
Nepal - Assessing the Social Impact of Rural Energy Services
Alternative Energy Promotion Center
Monitoring and Evaluation Framework
November 2009
Insert relevant PwC graphic in this space
*connectedthinking
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
2
Table of Contents
Section Page
1 5
2 10
4 15
3 12
5 206 35
Development of M&E framework – the context
Development of M&E framework – our approach
Familiarisation and sensitisation
Review of existing M&E framework
Proposed M&E framework
Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
3
AGENDA of the programme
Day 1: Nov 5th 9:15-9:30 - Welcome and Introduction of participants 9:30:9:45 - Introduction and Objective of the Workshop, Dr. N.P. Chaulagain 9:45-9:50 - Inauguration of the Workshop, Honorable Minister, Thakur
Sharma 9:50-10:20 - Overview of M&E framework for AEPC, RakeshJha, PwC 10:10-10:50 -Remarks: By all program officials10:50-11:00 - Vote of Thanks and Closing of the session, Dr. N.P. Chaulagain 11:00-11:15 - Tea Break Technical Session-I
11:15-13:00 - Introduction to Management Information Reporting & Monitoring System (MIRMS) Mr. Rakesh Jha and Mr. Manoj Mania, PwC 13:00-14:00 - Lunch Break 14:00-15:00 - MIRMS: Manual Data Entry and upload provisions for all
renewable technologies, Mr Manoj Mania, PwC15:00-15:45 - MIRMS: MIS reporting, Mr Manoj Mania, PwC151:450-16:00 - Tea/ Coffee break16:00-17:30 - Project Monitoring – MH programmes of AEPC, Mr Manoj
Mania, PwC
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
4
AGENDA of the programme
Day 2: Nov 6 Technical Session-II 9:00-10:00 - Demonstration of MIRMS: Scorecards, KPI Monitoring &
Dashboards – Mr. Rakesh Jha and Mr. Manoj Mania, PwC
10:00-11:00 - Training and live run, Mr Manoj Mania, PwC
11:00-11:15 - Tea/ Coffee break 11:15-12:45 - Training and live run contd…Mr. Manoj Mania, PwC
12:00 – 13:00 - Concluding remarks and vote of thanks, AEPC
13:00-14:00 - Lunch 14:15 - Departure to AEPC
5
Mo
nit
ori
ng
an
d E
valu
atio
n
Section 1
Development of M&E framework – the context
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
6
Context
Low electricity access in Nepal – 35%
total; Urban- 85%; Rural – 30%
Low per capita electricity consumption
– 76 unit
Cost of grid connection not feasible at
all places – Renewable can play a
major role
Grid extension in majority of villages
not viable - poor financial condition of
NEA
Total MW
HH served by INPS
HH served by Micro and
Small Hydro
HH served by Alternative
Energy
2007 700 35% 8% 2%2017 2035 50% 12% 3%2027 4000 75% 20% 5%
Electrification target under the Water national Plan , 2005
Source: NEA Annual Report, FY 2008 -09
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
7
Context
• Nepal endowed with huge potential in hydropower (>40,000 MW) and other
Renewable energy sources.
• Increasing investment in hydropower – GoN, donors
• AEPC created in 1996 to promote renewable in the country.
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
8
Context
• Diverse programmes – Anecdotal evidence of positive impact but little systematic quantification of results on the ground
• WB funded study on ‘Assessing the Social Impact of Rural Energy Services’ – Oct 2008 -Sep, 2009
• Systematic M&E framework to assess the development impact of MH schemes
• Update the business plan for AEPC
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
9
Why M&E
Improving internal management
– Better planning and budgeting
– Better implementation and reporting
Generating support
– Respond to needs of stakeholders
– Explicit commitment to track contribution to global indicators e.g. MDGs
Ensuring accountability
– Demonstrating results against money spent - Demanded by govt., donors, NGOs
– Ultimately results on the ground matter - not implementation progress
Just in case: How M&E helps ME?
– Motivates by seeing the difference it makes for people
– Learn from experience: Build on success and learn from failures…
– Incentives for better performance…?
10
Mo
nit
ori
ng
an
d E
valu
atio
n
Section 2
Development of M&E framework – our approach
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
11
Development of M&E and MIS system – our approach
Section 1 – development of M&E framework
12
Mo
nit
ori
ng
an
d E
valu
atio
n
Section 3
Familiarisation and sensitisation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
13
Familiarization of the context and the program by the project team
Review of policies, regulations and impact studies
Interaction with REDP, ESAP and AEPC officials
Field Visit
– Dhunne Khola (REDP)
– Tanahun and Banglung (ESAP)
Complete overview of existing M&E
Section 2 – Familiarization and Sensitization
Focus Group Discussion at Dhunne Khola Micro Hydro site
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
14
Project sensitization of the AEPC staff
Organized a sensitization workshop at the outset of the project
DAY 1: November 18, 2008
– Introduction and Context
– Overview of M&E Concepts
– Developing the Monitoring Framework for AEPC
– The Role of Evaluation
DAY 2: November 19, 2008
– M&E framework for Assessing the Social Impacts of Rural Energy Services
– Monitoring: Key Performance Indicators
– Evaluation: Survey Conceptualization and Design
Section 2 – Familiarization and Sensitization
Sensitization workshop at AEPC
15
Mo
nit
ori
ng
an
d E
valu
atio
n
Section 4
Review of existing M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
16
M&E diagnostic review - REDP
Section 3 – Review of existing M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
17
M&E diagnostic review - ESAP
Quarterly reports are sent to AEPC with detail work plan based on the results of the survey.
Each installation is surveyed after a month of installation by the LPO and the promoter.
Technical evaluation – after 3 to 4 months by the LPO and RRESC staff.
Mid term review of the annual work plan to check whether program objectives are met.
Evaluation of LPOs by RRESC
The field technical coordinator from the RRESC office submits progress reports to AEPC by the 12th of every month – based on pre-defined formats
Section 3 – Review of existing M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
18
MIS – Existing & Proposed for REDP
Existing
Focus more on data gathering
Targets not reflected in MIS
Manual calculation of formula fields in few formats
Project monitoring report captured only status of completion of activity
Proposed
MIS aligned for performance monitoring
Captures targets for each measurable indicator and helps evaluation
System calculates formula based on dependent fields
System would capture date of completion to highlight delay in implementation, if any
Section 5 – Development of MIRMS
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
19
MIS – Existing & Proposed for ESAP
Existing
Client server mode visible to select group of people within AEPC office(max 8 to 10 users) – local installations required
Absence of integrated information system - separate databases
Targets not reflected in MIS
KPIs not linked to performance management of business units
Requires design change in system for meeting any future requirements
Static reports with no feature to compare and perform trend analysis
Proposed
Web based system – information available to all users at any location having internet connection
Integrated AEPC database -warehousing periodic information for analysis and evaluation in the system itself
Target driven system for measuring performance every month
Dynamic system to incorporate any future requirements
Multiple ways of viewing data and conducting analysis
Section 5 – Development of MIRMS
20
Mo
nit
ori
ng
an
d E
valu
atio
n
Section 5
Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
21
• Primary reporting interface for meeting all the reporting requirements of the
management(financial, technical, operational, projects, community
participation, capacity building)
• Eliminate the burden of disparate electronic and paper based data collection
requirements
• Provide the top management specific information about the programs and its
business units at a glance.
• Enabling the management to drive strategy by evaluating performance
against the KPIs.
• Protect the confidentiality , availability and integrity of data
Section 5 – Development of MIRMS
MIRMS - what it offers
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
22
•Using existing computer infrastructure of AEPC server for hosting the
system on the internet
•Utilising existing manpower and computer infrastructure at the field level
•Supplements existing information base of AEPC to enhance decision
making abilities
•Integration with REDP and ESAP databases for information
consolidation
•Existing formats have been retained and new formats designed and aligned
to the KPIs in the proposed M&E framework
•Reduction of work load on M&E staff by leveraging the information for
performing evaluation and meeting top managements reporting
requirements
•
MIRMS – leveraging existing capability of AEPC
Section 5 – Development of MIRMS
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
23
•Comprehensive MIS formats designed to facilitate regular review of project
activities for all RETs
•Structured and unified formats for ease in data gathering
•Mapping of data fields to existing transactional system
•Yearly / Monthly Targets to be set in the system to compare progress
towards achieving outputs
• Flexibility in reporting and analyzing the data
•Trend analysis
•Comparative analysis
• Dynamic reporting system with no static reports – user can extract reports
in desired format at any given point of time from any location
MIS - Design and Reporting
Section 5 – Development of MIRMS
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
24
MIRMS will interface with existing systems either through on-line or off-line mode and extract information
User friendly screens provided for mapping and uploading of external data source to MIRMS
Information extracted either on monthly or trimesterly basis depending on availability of information
MIRMS – Mapping with existing database of AEPC
Section 5 – Development of MIRMS
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
25
Build on the existing infrastructure - optimal utilization•Existing AEPC server
•SQL server database
Developed on a 3-tier web technology framework comprising of java technology integrated with MS SQL server
Leveraged industry best practices for designing and developing a scalable system
Creation of a centralized database repository
MIS formats designed for all programs
Development of MIRMS
Section 5 – Development of MIRMS
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
26
Proposed MIRMS
Section 5 – Development of MIRMS
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
27
Institutional arrangement for M&E
The proposed M&E framework is aimed to:
Align and design to meet AEPC’s vision, mission and strategic priorities.
Use M&E as a strategic planning tool.
Optimise the existing set up
Establish a drill down feature for analysis of information
Generate MIS reports at every level
Top Management, Programme Managers
Department head and District Level Monitoring
Field based operation
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
28
Functions:Operational/Technical/
Financial/CapacityBuilding
Advantages of MIRMS
Level of Management
ProjectSites
Middle Management
TopManagement
Functions:Operational/Technical/
Financial/CapacityBuilding
TransactionalActivities/Complaint
Status
Process Performance
Indicators
ProgrammePerformanceIndicators/StrategicObjective
Parameters
Review Timeline Monthly
Monthly/Trimesterly
Monthly/Trimesterly/Yearly
Need based MIRMS
Monthly monitoring of operational data is done by the field staff responsible for the respective functions.
Trimesterly monitoring of performance parameters - by the district heads or monitoring officers of respective programs
The top management reviews the program and strategic parameters on a monthly, quarterly or yearly basis to take decisions and decide the policy level interventions.
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
29
Development of KPIs
To evaluate the performance of the project against defined objective/target
Facilitate review and comparison across project/ location.
Useful for setting performance targets and efficient assessment.
Input indicator
– mostly the programme fund support, contribution in kind and technical assistance by community and awareness generation programmes
Output Indicators
– mostly the number of projects/systems installed, capacity enhancement of the target beneficiaries and the community mobilization process.
– show the benefits accruing to the community through the awareness generation and mobilization process.
Outcome Indicators
– indicators that mostly are a measure of the affordability, operational efficiency and reliability of the MH programs.
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
30
KPIs – Top Management (MH – REDP & MGSP)
No. of MH projects commissioned
Installed capacity of MH
Percentage coverage of target population through awareness generation programme
Progress against projects schedule*
Project implementation delay*
Utilization of Funds
Percentage of HHs with electricity access through MH
No. of enterprises with access to MH
Value of proposed equity by the community (equivalent to labour and cash amount) vis-a-vis actual contribution
% of community facilities with access to other RETs
% of community facilities with access to MH
Percentage of Complaints resolved (consumer level) in stipulated time period
Power availability*
Collection efficiency*
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
31
KPIs – Top Management (Biomass)
Achievement of ICS installations
Achievement of Metallic Stoves installations
Achievement of Gasifier installations
Percentage coverage of target population through awareness generation programs
Total subsidy allocation
Utilization of Funds
% Achievement of Training programmes for program staff
% Systems functioning during verification
% of community facilities with access to MH
System performance in ICS
System performance in Metallic Stoves
System performance in Gasifiers
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
32
KPIs – Top Management (Solar)
Achievement of Solar systems installed
% subsidy disbursement
Avg. delay in subsidy disbursement in all Solar systems
Percentage coverage of target population through awareness generation programs
System performance for ISPS, SHS & PVPS
% of complaints resolved in stipulated time period
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
33
Proposed Institutional arrangement for M&E
Section 5 – Proposed M&E framework
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
34
Snapshot of MIRMS
Section 5 – Proposed M&E framework
35
Mo
nit
ori
ng
an
d E
valu
atio
n
Section 6
Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
36
Periodic Data capture through MIS formats Monthly MIS Reporting MH Project Monitoring KPI Analysis & Scorecards for Business Units Performance Review through Dashboards
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
37
Monitoring & Evaluation Framework
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
38
Data Capture Populate monthly data in MIRMS through two options:
File uploads – mapped to existing AEPC databases
Manual Entry – data entry screens
Periodicity of data capture
One time – DPR, Funding programs, etc
Monthly
Trimesterly
Annually
Data captured at different levels of the organization depending on data availability
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
39
MIS Reporting
Dynamic reporting engine supporting multiple ways of viewing data and conducting analysis
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
40
MIS Reporting
Perform trend analysis on any selected parameter
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
41
MIS Reporting
Consolidated view of past performance data
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
42
Project Monitoring
Tracks project schedules and assists program managers to take corrective measures
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
43
KPI Analysis
Single window for top management to evaluate performance through KPIs
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
44
Program Dashboards
Single view of performance data in graphical form to different levels of user
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
45
Sample - Dashboard
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
46
Snapshot of dashboards
The dashboards would facilitate the Program Mangers and M&E staff to monitor project development
designed to provide specific information about the performance of each component of the programme and present critical measurable parameters of each program at a glance.
The dashboards would provide a graphical display of the overall performance of the programs on key performance parameters like achievement of programme objectives, community reach, financial performance, etc.
The dashboards can be generated starting from the zonal level to individual level dashboards for each project/[programme
Section 6 – Process flow for MIS Reporting and Evaluation
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
47
How to migrate to MIRMS
Form a core group of people for taking ownership of MIRMS and formulating strategies for information
Identify at least one district form each region for testing and roll out
Run this parallel to the existing M&E arrangement
Develop a team of ToT
Handholding of people engaged in M&E by ToTs
Use of operation manual
Periodic review of the system – frequency of monitoring on the higher side in the beginning
– Assess performance of the M&E team and MIRMS
– Take appropriate measures as per requirement – training, modification in MIRMS
Section 6 – Process flow for MIS Reporting and Evaluation
48
Mo
nit
ori
ng
an
d E
valu
atio
n
M&E Framework
Assessing the Social Impact of Rural Energy Services
November 2009
Mo
nit
ori
ng
an
d E
valu
atio
n
49
Regulatory and policy overview
1975 – Formation of Nepal Electricity
Corporation
1984 – Nepal Electricity Act
1985- Formation of NEA
1992-Electricity Act, Water Resources Act
Classification of Hydro projects
Promotion of RE technologiesLarger role for AEPCCreation of REF
De-licesing forgeneration up to 100KWDe-regulation of powersales in isolated areas Encouragement of Private sector 1992 and 2001 - Hydro
Power Development Policy
2006 - Subsidy development Mechanism
1975 – Small Hydro Development Board
Incentives to promote MH up to 1 MW
2003 -Community Electricity Distribution by
law
Encourage Private sector in DDG based RE
Creation of community REF
2006 - Rural Energy Policy
Promotion of RETsSale of power to grid by DDGs
Emphasis on decentralized models
Three Year Interim Plan
Ex-post evaluation - replicable case studies