'Strengthening the MIS in Social Protection Programs: A...
22
Strengthening the MIS in Social Protection Programs: A Toolkit Maria Arribas, LCR Cesar Baldeon, ISG May 17, 2007
Transcript of 'Strengthening the MIS in Social Protection Programs: A...
Strengthening the MIS in Social Protection Programs: A Toolkit
Maria Arribas, LCR Cesar Baldeon, ISG
May 17, 2007
2
Purpose, Audience, Methodology
A Mitigation tool for Control and Accountability Risks & A framework for the Implementation of an MIS in CCTs/SP Programs
A practical tool for TTLs an WBG clients working on SP Programs (with a focus on CCTs)
Based on the MIS assessment of Ecuador’s CCT; LCR’s2 day CCT review workshop; Visits to Colombia, Chile, and Argentina; Literature Review; and Conversations with several TTLs
3
Two Dimensions
A look at Control and Accountability Processes in CCTs: An MIS Perspective
A framework for the Implementation of an MIS in SP/CCTs
Beyond having a Unique Identifier and Cross-checksA more systematic and strategic approach to systems managementUseful for both paper-based and automated programs/processes
4
Toolkit Components
Control & Accountability: MIS PerspectiveMIS Framework & Operational Risk Management FrameworkCase StudiesMIS Implementation and Monitoring ChecklistConsiderations/Recommendations
Not an ICT Project Implementation Toolkit!
5
MIS for Safety Net ProgramsIn
form
atio
n f
low
sM
ISCom
ponen
ts
Beneficiary Id.
Targeting
Registration
Graduation
Registry
Database
Validation
Updates
Payment
Eligible P
Payment
Reconciliation
Control
Grievances
Processes
Impact
Conditions
Data Collect
Verification
Penalization
PotentialBeneficiaries
GovernmentInstitutions(Ministries)
ServiceProviders
FinancialInstitutions
CivilSociety
IT Expertise & Organization Structure
Information Management
Information Providers and Consumers
Application Quality Management
Information & Communication Technology Infrastructure
Program Process
6
Info
rmat
ion f
low
sM
ISFu
nctio
ns
Beneficiary Id.
Targeting
Registration
Graduation
Registry
Database
Validation
Updates
Payment
Eligible P
Payment
Reconciliation
Control
Grievances
Processes
Impact
Conditions
Data Collect
Verification
Penalization
PotentialBeneficiaries
GovernmentInstitutions(Ministries)
ServiceProviders
FinancialInstitutions
CivilSociety
IT Expertise & Organization Structure
Information Management
Information Providers and Consumers
Application Management
Information & Communication Technology Infrastructure
Program ProcessProgram Process
Beneficiary Id.
Targeting
Registration
Graduation
Registry
Database
Validation
Updates
Payment
Eligible P
Payment
Reconciliation
Control
Grievances
Processes
Impact
Conditions
Data Collect
Verification
Penalization
IT Expertise & Organization Structure
Information Management
MIS
Com
ponen
ts
Application Quality Management
Information & Communication Technology Infrastructure
Control & Accountability: An MIS Perspective (1)
7
Control & Accountability: An MIS Perspective (2)
Spot checksAuditsError/inconsistencies alerts
Data GovernanceDatabase ManagementData QualityData Architecture
DishonestyInconsistent collection/recordingHuman ErrorsComplex Data Integration
Unwarranted penalizationUnreliable dataMisleading Impact Evaluations
Monitoring of Co-responsibilities
Back ups and redundant systemsDowntime managementAccess Control (logical and physical)Unique identifiersCross-checks
Disaster RecoveryAvailability ManagementCapacity PlanningSecurity ManagementData Warehouse
Environmental eventsSystem breakdownMalicious ActsHuman Errors
Service InterruptionUnauthorized access to infoUnauthorized changesCreation of false info/transactions
Beneficiary Registry
Cross-checksAccess Control Audit TrailsSecurity ClassificationsArchiving Strategy
Data QualityData SecurityRecords ManagementSeparation of Functions
Ineffective eligibility criteriaImperfect targeting toolIncoherent Registration ProcessesHuman ErrorsFraud
Inclusion ErrorsExclusion Errors
Beneficiary Identification
Tools (examples)
MIS FunctionsCausesRisksProcesses
8
Control & Accountability: An MIS Perspective (3)
Reporting ToolHistory TrackingManagement ReportsTransparencyCase Management
Data Quality/IntegrityRecords ManagementData AvailabilityData UsabilityData warehouse
Lack of ReportingLack of Transparency
Uninformed decision makingReputation RisksPolitical Risks
Monitoring & Evaluation
Error/inconsistencies alertsService Level Agreements
Data QualityMaster Data ManagementSystems Availability Mgt
Insufficient FundsHuman ErrorsSystem AvailabilityBribery/DishonestyService Disruption
Irregular PaymentsInaccurate PaymentsInterruption of Payments
Payment of Benefits
Tools (examples)
MIS FunctionsCausesRisksProcesses
9
Control & Accountability: An MIS Perspective (4)
Case Management SystemService level monitoringAlerts for systemic appeals (category, geographic, etc.)
Records Management Process Monitoring
Inexistent feedback systemsFeedback black-boxBroken processesCulture (resignation)
Program Abuse“Ping-Ponging”between officesProgram credibilityMisleading Impact EvaluationsMissed Opportunity for demand driven improvements
Complaint Resolution and Appeals
Service Level AgreementsPublication & TransparencyInformation Management Agreements Publication on Website
Data GovernanceOrg. Structure
Confusing Roles and ResponsibilitiesPolitical VolatilityWeak InstitutionsLack of Political Will
Lack of inter-ministerial coordinationLack of EnforcementPolitical ManipulationEnforcement of decentralization arrangementsLack of funding
Institutional Arrangements
Tools (examples)
MIS FunctionsCausesRisksProcesses
10
Integrated Framework – MIS ComponentsIn
form
atio
n f
low
sM
ISCom
ponen
ts
Beneficiary Id.
Targeting
Registration
Graduation
Registry
Database
Validation
Updates
Payment
Eligible P
Payment
Reconciliation
Control
Grievances
Processes
Impact
Conditions
Data Collect
Verification
Penalization
PotentialBeneficiaries
GovernmentInstitutions(Ministries)
ServiceProviders
FinancialInstitutions
CivilSociety
IT Expertise & Organization Structure
Information Management
Information Providers and Consumers
Application Quality Management
Information & Communication Technology Infrastructure
Program Process
11
Why a Framework?
Logical Structure Systematic Approach to Risk ManagementMIS Framework & Operational Risk Management FrameworkIntegration among building blocks (and how they constrain one another)PrioritizationCost/Benefit Analysis
12
Information Management:• Data Governance• Data Architecture, Analysis & Design• Database Management• Data Quality Management• Data Security• Master Data Management• Data Warehousing Management• Records Management• Meta Data Management
Application Quality Management:• Quality Planning• Quality Control• Configuration Management• Change Management• Version Control• Release Management
IT expertise and Org. Structure:• Organizational Structure• Staff Development• External Expertise• Separation of Duties
Infrastructure:• Capacity Planning• Availability Management• Disaster Recovery• Security Management
Integrated Framework – MIS Components
13
IT Expertise and Organization Structure
Identification of functions that require separation of dutiesDefinition of roles & responsibilitiesEvaluation of roles & responsibilities
Separation of Duties
Outsourcing strategy: identify external expertise requiredProcurement Plan for external expertise Assessment Process for external parties performance
External Expertise
Assessment of in-house skills and gapsOutsourcing strategy: Identify skill to retain internallyStaff Training Plan
Staff Development
Inclusion of the IT team in the design of the programDefinition of roles & responsibilities in the operation manualInclusion of the IT team in the operational assessment of the program
Organizational Structure
OutputsFunctions
14
Information Quality Management
Definition of Metadata RequirementsMetadata Dictionary
Metadata Management
Definition of Record Management processes (procedures and standards)
Records Management
Identification of monitoring & evaluation indicatorsDefinition of reporting requirements (managerial, detailed,…) Training & roll out strategy
Data Warehousing
Information security policiesImplementation of user profiles (logical security)
Data Security
Implementation Plan of data flows and schedules with data custodians
Master Data Management
Data validation processes Definition/Implementation of data validation rulesData quality evaluation
Data Quality Management
Beneficiary databaseMonitoring Plan for database performance
Database Management
Business data model: entities, attributes, relationshipsLogical data modelPhysical database modelData inventory: data inputs and outputs for each processData dictionary
Data Architecture
Identification of authoritative sourcesFormalization of institutional arrangements with data custodiansFormalization of data governance processesData Governance Control
Data Governance
OutputsFunctions
15
Application Management
Definition of user and technical documentation requirementsDocumentation for final release and subsequent changesTraining Plan
Release Management
Version all components and final release (baseline)Version all components and up-to-date release after final versionMonitoring Plan for version control
Version Control
Definition of change management processes (change planning, impact, feasibility, cost analysis, documentation, release)Assessment of Change Management Process
Change Management
Systems Change Management PolicyDocumentation - List of MIS components included in the final releaseList of modified and new components after final release
Configuration management
Definition of functional and acceptance test processesAssessment of quality control processes
Quality Control
Definition of Quality Management processes including:Quality controlChange managementVersion controlConfiguration managementRelease management
Definition of roles & responsibilities for processesAssessment of Quality Management processes
Quality Planning
OutputsFunctions
16
Infrastructure
Definition of physical security for hardware and networkImplementation of security monitors and controlsAssessment of access monitors and controls
Security Management
Definition of failure and recovery plansBack-ups and disaster recovery strategiesTesting Plan for failure and disaster recoveryAssessment of disaster recovery plans
Disaster Recovery
Definition of availability required (tolerance levels)Definition of Systems architectureMaintenance/Renewal Strategy of hardware, software and networkAvailability Testing Plan and ScheduleRecovery Testing Plan and ScheduleAssessment of system’s availability
Availability Management
Needs Assessment (hardware, software and network requirements) Procurement Plan for Hardware, software and network Monitoring capacity, availability and securityAssessment of performance and capacityRe-evaluation of hardware, software and network sizes
Capacity Planning
OutputsFunctions
17
MIS Operational Risks Framework
Separation of duties
Security Management
Quality ControlData quality Records Management
Fabrication: false transactions
Separation of duties
Security Management
Quality ControlData qualityRecords Management
Modification: unauthorized changes
Separation of duties
Security Management
Quality ControlData qualityRecords Management
Interception: unauthorized access
Organizational StructureStaff DevelopmentExternal expertise
Capacity planningAvailability managementDisaster RecoverySecurity Management
Quality ControlVersion ControlChange Management
Data GovernanceData ArchitectureDatabase ManagementData Warehouse
Interruption: lost, unavailable, unusable
Human CapitalIT InfrastructureSoftware Quality Management
Info. ManagementRisk
18
Country Experience (1)
Program Name: Familias en AcciónAge: 7 years, since 2000 as part of the RAS (Red de Apoyo Social)Type: Conditional Cash Transfer Counterpart: The PresidencyCriteria: “provides cash to poor households in rural areas conditional on school
attendance of school-aged children and visits to health facilities and participation in nutritional programs for younger children and their mothers”
Size: as of January 2007 the program has 600,000 beneficiaries, it is expected to grow to 1.5 million beneficiaries within the next 9 months.
Colombia
Program Design and Implementation:• Decentralized Program with liaisons on each of the participating municipalities. Most
of the decentralized work is paper based • The program has not separated functions to avoid conflict of interest• Little validation of information with external systems• SISBEN, the targeting database, is updated as new municipalities are added to the
program• SIFA leverages the Presidency's investments for the four MIS components• Potential Scalability Problems results of the growth of the program• Complaints is a paper based decentralized process. Very difficult to produce process
indicators
19
Country Experience (2)
Program Name: Jefes y JefasAge: 6 years, 2001-2002 as part of the RAS (Red de Apoyo Social)Type: WorkfareCriteria: “transferred AR$50 per month to participants. The beneficiaries were to: (i) be
unemployed, (ii) be head of a household, (iii) live in a household with at least one minor below the age of 18, or pregnant woman, or handicapped of any age; (iv) work or participate in a training/education activities for 4-6 hours a day.”
Size: in May 2003: ~2 million reduced January 2006: 1.42 million
Argentina
Program Design & Implementation:• Decentralized program where municipalities have access to web base beneficiary
registry. Some information continues to be exchanged via excel sheets.• The program has separated functions. It includes a production team• Beneficiary database is validation with social security (participation in other programs
and taxes paid.) • Jefes y Jefas MIS leverages the investments of the Ministry's investments for the four
MIS components• Scalability • Program leverages the ministry’s central call center for complaints.
20
Country Experience (3)
StrengthsIT team SkillsSystematic Approach to Data ManagementRecords Management Defined (?)Robust InfrastructureEconomies of Scale
WeaknessLarge Amount of Paper Trail Prompt to Error and lack of traceabilityScaling Capacity Not TestedIT Staff Involvement (Structure)Long Payment Cycle (in part due to Manual Processes – 60 days)Interoperability/Information Quality
Colombia
StrengthsScaling CapacitySeparation of Functions and EnvironmentsWeb-based ApplicationRobust InfrastructureData WarehousingCase Management System
WeaknessApplication Version ControlReal Time Validation with Authoritative SourcesDecentralization Challenges
Argentina
21
Considerations/Recommendations
Country ContextProgram Maturity and feedbackPaper and Pencil vs. ElectronicAutomation where it increases efficiencyComplementary Systems ProcurementEconomies of ScaleKnowledge Sharing/Collaboration for IT teamsSystematic System Reviews (supervision)
22
Q & A
