Government Wide Monitoring and Evaluation...
Transcript of Government Wide Monitoring and Evaluation...
1
Government Wide Monitoring Government Wide Monitoring and Evaluation Systemand Evaluation System
Ronette Engela The PresidencyConrad Barberton National TreasuryAkiiki Kahimbaara StatsSaMohanuoa Mabidilala DPLGHenk Serfontein DPSA
2
Content of presentationContent of presentationOverview of the GWM&E framework Programme Performance Information Evaluations Census and Surveys - National Statistical System Derived and Transversal Information Systems
DPLG Local Government SystemDPSA Systems
3
In 2005 Cabinet approved In 2005 Cabinet approved an an implementation planimplementation plan
to develop to develop a monitoring and evaluation a monitoring and evaluation systemsystem
for use across governmentfor use across government
To encompass To encompass Monitoring implementation, effectiveness,
validationEvaluation impact and process evaluationEarly warning proactively identify blockagesVerification validates integrity of dataData collection using existing capacitiesAnalysis research driven assessments Reporting appropriate & customised
to target groups
The centre of government
needs to provide
clear policies and frameworks
MandatePresidency
4
Composite system that draws its data from contributory systems Emerging system built up over time with consistent and sustained participation by all stakeholders
Policy and Standards
Reporting and Databases
Capacity Building
Lead Agency DPSAPresidency SAMDI
Partner Agencies OPSC
Stats SA
National Treasury
Presidency
Stats SA
DPLGNational Treasury
DPSA
Presidency
Stats SA
DPLGDPSA
Phased implementation through 3x work streams
5
Are we still on track?Are we still on track?Missed a number of deadlinesInformation needed to inform next phase of development not in place
Audit of reporting requirements and M&E systems in gov late P&P shifted to Policy and Standards
Independent development of systemsNearly all provinces
Lack of own capacity - many consultants in sectorsMisconception re ‘System: IT’ vs‘Framework’
Reporting labyrinthReporting labyrinthplethora of dataplethora of data
For example:For example:Provincial dept Provincial dept reports to:reports to:•Own dept and exec authority•National concurrent dept•Premier Office•National Treasury•Presidency
6
Entry pointsEntry points
Executive reportingEvidence based decision making for:
Resource allocationPolicy refinement
Extensive executive interestSupport government implementation focus
RefineRefine
7
GWM&E System: data terrainsGWM&E System: data terrains
Census and Survey Information
Registers and Admin data
Evaluations
Programme Performance Information
8
Census and Survey Information
Registers and Admin data
Evaluations
Programme Performance Information
GWM&E System: policy platform
Policy FrameworkPolicy Framework
9
GWM&E Framework: Programme Performance Information
PPIE
C&SDept Dept Dept Dept
Derived information system DPLGDWAF
Transversal systems PERSAL BAS
10
GWM&E System: GWM&E System: Executive reports
Census and Survey Information
Registers & Admin data
Evaluations
PPI
Dept Dept
Derived information system
Transversal systems
Dept
PoAPoA
NationalIndicators
11
10 categories of 10 categories of Development IndicatorsDevelopment Indicators
Economic growth & transformationEmploymentPoverty & inequalityHousehold & community assetsHealth
EducationSocial CohesionSafety & SecurityInternational relationsGood governance
12
Economic growth & transformation
GDP, GDP per capitaFDICFCGovernment debtBudget deficit
CPI, CPIXBonds point spreadR&DExportsBEEBlack managers
13
Employment Poverty and Inequality
EmploymentUnemploymentExpanded Public Works
Per capita incomeInequality (Gini Theil)Poverty headcountPoverty gap analysisSocial assistance grantsLife expectancyLiving Standards Measures
14
Household & Community Assets
DwellingsWatersSanitationElectricityLand restitutionLand redistribution
15
Employment Poverty and Inequality
EmploymentUnemploymentExpanded Public Works
Per capita incomeInequality (Gini Theil)Poverty headcountPoverty gap analysisSocial assistance grantsLife expectancyLiving Standards Measures
16
Health Education
HEALTHInfant and child mortality rate Severe malnutrition under five yearsImmunisation coverage Maternal mortality ratio HIV prevalenceTuberculosis (TB) Malaria
EDUCATIONEducator - learner ratioEnrolmentMatric pass rateMatriculants with mathematics higher grade passesAdult literacyGraduating science, engineering and technical (SET)
17
Social cohesion
Strength of civil societyVoter participationVoters per provinceWomen who are members of legislative bodiesConfident of a happy future for all races
Country going in the right direction Identity in the form of class Pride in being South African Public opinion on race relations
18
Safety & Security International relations
SAFETY & SECURITYBroad categories of crimes Contact crime Property crime Detection rate Charges referred to court Conviction rateInmatesRoad accidents and fatalities
INTERNATIONAL RELATIONSPeace operations Democratically elected governments in Africa GDP growth in Africa Tourism Mission operations Diplomats trained
19
Good governance
GOOD GOVERNANCETax returns Audits Corruption perceptions Budget transparencyPublic opinion on delivery of basic services Red tape, business Biodiversity
20
Standard format
IndicatorCategory linked to 10 Year ReviewGoalTrend analysisTableGraphDefinitionData source(Data note)
21
Indicators future
Publication - annuallyWill add some indicatorsMinimum requirement for all deptsReplicated in all spheres (as applicable)Develop data for
Community assetsHealthEducationSafety & Security on district level
22
GWM&E System: GWM&E System: Executive reports
Census and Survey Information
Registers & Admin data
Evaluations
PPI
Dept Dept
Derived information system
Transversal systems
Dept
PoAPoA
NationalIndicators
23
Census and Survey Information
Evaluations
PPI
Dept Dept
Derived information system
Transversal systems
Dept
Area of Responsibility
24
EvaluationEvaluationNeed for detailed policy evaluations on a number of different levels:
Departmental led reviews and evaluation of policies and programmesSectoral reviewsBroad, cross cutting reviews led by centre of government
Monitoring and evaluation have different purposes
25
Programme Performance Programme Performance InformationInformation
Kuben Naidoo National Treasury
26
Constitution S.215 S.216Budget & Exp Management
PFMA & MFMA
Ensure information on inputs, outputs and outcomes …
MandateTreasury
…underpins planning, budgeting, implementation management and accountability reporting…
…to promote transparency andexpenditure control.
27
Programme Performance Information
E
C&SDept
Derived information system
Area of Responsibility
Transversal systemsLine manager
Dept exec
Programme manager
Exec Authority
Dept
28
Framework for Managing Performance Information
Key Performance
Concepts
IMPACTS
OUTCOMES
OUTPUTS
INPUTS
ACTIVITIES
The developmental results of achieving specific outcomes
The medium-term results for specific beneficiaries that are the consequence
of achieving specific outputs
The final products, or goods and services produced for delivery
The processes or actions that use a range of inputs to produce the desired
outputs and ultimately outcomes
The resources that contribute to the production and delivery of
outputs
What we use to do the work?
What we do?
What we produce or deliver?
What we wish to achieve?
What we aim to change?
Plan, budget, implement and
monitor
Manage towards achieving these
results
IMPACTS
OUTCOMES
OUTPUTS
INPUTS
ACTIVITIES
The developmental results of achieving specific outcomes
The medium-term results for specific beneficiaries that are the consequence
of achieving specific outputs
The final products, or goods and services produced for delivery
The processes or actions that use a range of inputs to produce the desired
outputs and ultimately outcomes
The resources that contribute to the production and delivery of
outputs
What we use to do the work?
What we do?
What we produce or deliver?
What we wish to achieve?
What we aim to change?
Plan, budget, implement and
monitor
Manage towards achieving these
results
29
Framework for Managing Performance InformationPerformance
Indicators
IMPACTS
OUTCOMES
OUTPUTS
INPUTS
ACTIVITIES
RELATIONSHIP INDICATORS
OPINION-BASED INDICATORS
Types of direct indicators
QuantityQuality
Cost / Price Timeliness
Start and end timesDistributionAdequacy
Accessibility
DIRECT INDICATORS
Economy
Efficiency
Effectiveness
Data gathered through surveys
Equity
Indicators
Indicators
Indicators
Indicators
IndicatorsData gathered mainly by management information
systems
Calculated using a combination of direct indicators and other
dataIMPACTS
OUTCOMES
OUTPUTS
INPUTS
ACTIVITIES
RELATIONSHIP INDICATORS
OPINION-BASED INDICATORS
Types of direct indicators
QuantityQuality
Cost / Price Timeliness
Start and end timesDistributionAdequacy
Accessibility
DIRECT INDICATORS
Economy
Efficiency
Effectiveness
Data gathered through surveys
Equity
Indicators
Indicators
Indicators
Indicators
IndicatorsData gathered mainly by management information
systems
Calculated using a combination of direct indicators and other
data
30
Link to individual performance Link to individual performance agreements of line managers and HOD agreements of line managers and HOD
Ensure Programme Performance Information is appropriately used for planning, budgeting and managementplanning, budgeting and management purposes •set performance standards and targets priorto the start of each year•review performance and take management action•evaluate performance at the end of a service delivery period
PPI manual per sector - extensive consultation to determine needs of
different usersNext steps
Treasury (2)
31
National Statistical SystemNational Statistical System
Akiiki Kahimbaara StatsSA
32
Evaluations
PPI
Dept Dept
Derived information system
Transversal systems
Dept
Area of Responsibility
Census and Survey Information
Registers and Admin data
33
The Statistics Act (No. 6 of 1999) The Statistics Act (No. 6 of 1999) 2002 January Cabinet 2002 January Cabinet LegkotlaLegkotla
State of the Nation Addresses State of the Nation Addresses 2004 and 20052004 and 2005
•Section 14 subsection 6 clauses (a), (b) and (c) where the Statistician-General may advise an organ of state on the application of quality criteria and standards
•Section 14 subsection 7 clauses (a) and (b) grants the Statistician-General power to designate statistics produced by other organs of state as official statistics
•Section 14 subsection 8 clauses (a) and (b) authorisesthe Statistician-General to comment on the qualityof national statistics produced by another organ of state; and to publish such other department’s statistics
MandateStatsSA
34
Demand for information[user groups or
indicator categories]
Stats SAOther producers
[departments, CSOs, etc]
• Insufficient supply of quality information• Uncertain quality?Poor comparabilityIsolated producersNo shared standards• Insufficient statistical skills
Official statistics
Unknown quality
Problem
35
Demand for information[user groups or
Indicator categories]
Stats SAOther producers
[departments, CSOs, etc]
Regulatory environment• Statistics Law• Governance structure• Quality standards• Advocacy programme• Code of conduct
• Adequate information• Reliable information
[quality, sustainable]• Sufficient skills
Coordination
Feedback
Proposed solution
Feed
back
36
Proposed new approachProposed new approach
Use official statistics to ensure quality
• SASQAF [quality framework]•Framework of international best practice•UN Principles of Official Statistics
Maintain the decentralised system of statistical production
Transform all national statistics into official statistics
SASQAFDraft available
37
Transform existing departmental Transform existing departmental data (registers & surveys) data (registers & surveys) into sustainable sources into sustainable sources
of official statisticsof official statistics
•More use of administrative data than of surveys - sustainability and cost
• Agreements and collaboration between Stats SA and individual dept
• Joint Working Party between Stats SAand each of the departments to improve quality
ProgrammeNSS
38
Access registers or datasets and map Access registers or datasets and map them against indicators in the compendiumthem against indicators in the compendium
•Identify gaps – where the compendium of indicators should be coming fromthe department• Assess quality of registers or datasets for usability• Suggest improvements
Working with individual departmentsNext steps
StatsSA
39Build statistical capacity in departmentsBuild statistical capacity in departments
•Encourage departments to establishstatistical capacity – as part of existing M&E units or as components on their own•Allocate sufficient resources: part of MTEF budgeting for sustainability
•Implement statistical training programmeswith support from Stats SA
Audit departments for statistical capacityNext steps
StatsSA (2)
40
ConcernsConcerns
Inadequate appreciation of the practice of managing for results
• Measurement of performance and “objective” info for planning and decision-making not a priority
• Current practice of existing M&E units without measurement
Circumlocutionary behaviour of public officers (multiple formalities)
• Waiting for approval (all the way to the minister!): inadequate delegation of authority
• Lack of institutional memory in government departments [change the head: start afresh]
•Operational silos – protecting one’s turf•Cover-ups: fear of exposure
41
Derived system:Derived system:DPLG local government DPLG local government
Mohanuoa Mabidilala DPLG
42
GWM&E Framework: DPLG Derived Information
PPIE
C&SDept Dept Dept Dept
Derived information system
Transversal systems
43
Constitution Chapter 3 & 7 Constitution Chapter 3 & 7 Municipal Systems ActMunicipal Systems Act
Municipal StructureMunicipal Structure’’s Acts Act
Function of DPLGFunction of DPLG•develop national policies and legislation with regards to Provinces and Local Govt, and to monitor the implementation•to support Provinces and Local Govt in fulfilling their constitutional and legal mandate
MandateDPLG
44
DPLG Monitoring, ReportingDPLG Monitoring, Reporting& Evaluation& Evaluation
ResponsibilityDPLG
•Developing and implementing an integrated monitoring and evaluation system (dplg)
•Providing leadership and support to Local Govt for the successful implementation of the GWM&E framework
•The development of an MR&E capacity in the provinceand local government to monitor and reportimplementation of the 5 Year Local GovernmentStrategic Agenda (2006 – 2011)
45
ResponsibilityDPLG 2
5 Local Govt 5 Local Govt KPAsKPAs•Municipal Transformation and Organisational Development
•Basic Service Delivery•Local Economic Development•Financial Viability and Management•Public Participation and Community Involvement
DPLG will develop and implement a system for assessing local govt
service delivery
46
Collaboration with key Collaboration with key departmentsdepartments
ResponsibilityDPLG
•The Presidency•National Treasury•Dept of Public Service and Administration•Dept of Water and Forestry•Dept of Mineral and Energy•Dept of Housing•Statistics South Africa•South African Institute for ManagementDevelopment (MR&E capacity building)
47
Varying degrees: Varying degrees: ofof understanding understanding and and
ofof M&E capacity and readinessM&E capacity and readiness
•Lack of integrated approach to local governmentM&E across 3 spheres
•Lack of appropriate M&E reporting structures tomonitor local government service delivery
•Capacity and capability to monitor local govtservice delivery
DPLG assessed readiness of provincial depts of local govt and Office of Premiers to do
Local govt M&EProblem
48
Standardise approach to Monitoring, Standardise approach to Monitoring, Reporting and EvaluatingReporting and Evaluating
Develop and implement MR&E policy Capacity building strategyInstitutionalise MR&E systems across the three
spheres of govt by using existing structures withinIGR Act as M&E forums
DPLG did assessment of business processes that inform MR&E
system Proposed Strategy
49
Types of local government Indicators
43%
36%
21%
9%
2%
20%
0%
45%
27%
8%
0%5%
10%15%20%25%30%35%40%45%50%
Input Activities Output Outcome Impact
Indicator category
Perc
enta
ge
Initially 500
Collated allCollated allindicators indicators
on local on local govtgovtPhase 1Phase 1
Now ±150
50
Shared understanding of Local government indicator Shared understanding of Local government indicator development across all three spheres Phase 2development across all three spheres Phase 2
GWM&E System
National
Provincial
District
Local
GPOA
PGDS
IDP
IDPIndicators
51
Achievements to dateAchievements to date Done
Readiness assessment of provincial dept of local governmentDraft Local Government MR&E framework, aligned to GWM&EMR&E business process mapping of Local government indicatorsNational Local Government MR&E forum establishedCollaboration with SAMDI and NT TAU for capacity building
52
Next stepsNext stepsPlans
Finalise Local Government M&E policy frameworkDevelop a compendium of indicators for Local GovernmentDesign and implement Local govt MR&E system that extracts information with multiple sector departments that deal with Local Government –based on impact model Capacity building for provinces and municipalities
53
Derived system:Derived system:DPSADPSA
Henk Serfontein DPSA
54
Public Service ActPublic Service Act•Responsible for public service transformation •Custodian of public management frameworks •Performance and knowledge management •Service delivery improvement
CoCo--Chair of the Governance Chair of the Governance and Administration Clusterand Administration Cluster
CoCo--Chair the GWM&E Task Chair the GWM&E Task Team Team
Aim Increase public
service effectiveness and improve governance
MandateDPSA
55
GWM&E System: Executive reportsGWM&E System: Executive reports
Census and Survey Information
Evaluations
PPI
Dept Dept
Derived information system
Transversal systems
Dept
PoAPoA
NationalIndicators
Area of Responsibility
PM WatchPM WatchCabinet Lekgotla Cabinet Lekgotla ReportReport
56
Lead agency on databases Lead agency on databases and Reporting Work streamand Reporting Work stream
ResponsibilityDPSA
Two major initiatives currently in developmentTwo major initiatives currently in development
Public Management Watch Public Management Watch Extracts HR and Budget expenditure data Identifies vulnerable departments
HR HR utilisationutilisation Report Report Derived from departmental annual reports Focuses on how well HR is used
57
• Will draw from various GWM&E sources •To provide useful, practical updates•Executive level information •High strategic overview.
New development
Creation of Creation of Year end Report Year end Report bibi-- annual Cabinet annual Cabinet maKgotlamaKgotla
58
Linking the performance management and measurement systems needs dedicated attention
•DPSA M&E capacity overstretched (APRM etc.) •Building public participation systems a key issue •Must enhance GWM&E stakeholder relations •Overcoming formalistic compliance by sectors and provinces.
Task: Improving public management outside financial and
expenditure issues Challenges
DPSA
59
•• Priority streamlining reporting burden Priority streamlining reporting burden •• Compilation of reporting requirement Master list Compilation of reporting requirement Master list •• Undertake consultations with systems usersUndertake consultations with systems users•• Focus on improving understanding of their needs Focus on improving understanding of their needs •• Develop database architectureDevelop database architecture
Priority: Work stream 2 objectives
Data bases and reporting
Next stepsDPSA
60
ConcernsConcernsNeed to incorporate Public Opinions/ Imbizo processesPublic Entities and Constitutional Bodies
Conceptual clarity - IGRKey role players involvedConvergence around indicators
Data quality will improve with public attention and utilisation
Steady does it
Myth: Myth: ““at the press of a at the press of a buttonbutton””Systems to alert about Systems to alert about problem, need political problem, need political will to deal with itwill to deal with it
Conclusion
61
KeKe yaya lebogalebogaKeKe a a lebohaleboha
KeKe a a lebogalebogaNgiyabongaNgiyabonga
NdiyabulelaNdiyabulelaNgiyathokozaNgiyathokoza
NgiyabongaNgiyabongaInkomuInkomu
NdiNdi khoukhou livhuhalivhuhaDankieDankie
Thank youThank you