Outline
• Creating Data Demand and Use
• Analyzing and reporting results according to
the target audience
• Data-utilization strategies
• Tools for facilitating use of M&E results for
policy and program development
Information for Decision Making
• Information must be based on quality data in order to be useful
• Information must be communicated effectively in order to be useful
• Information must be used to be good M&E
Information Needs
• Different for different stakeholders
– view activities from different perspectives
– have different degrees of understanding of the program
– need different information
– want different information
– need/want information at different levels of complexity
– have different intensities of interest
Rationale for Data Demand and Utilization
Central to achieving evidence-based decisions are:
• Increasing user demand for quality information
• Increasing availability of data and analysis methods
• Increasing use of data and analysis methods
Components of Strategic Information
Surveillance Population Surveys
Basic/Academic Research
Programs
Routine HMIS Facility Survey
Monitoring, Analyzing and
Triangulating Core Indicators
Program Evaluation Operations Research Targeted Evaluation (Including, outcome evaluations, rapid ethnographies, economic evaluations, case studies)
National Level Response
Coverage Surveys
National Level Impact/Collective
Effectiveness Modeling
Surveillance/Impact M&E
Identifying Needs
Designing & Implementing Intervention
Measuring Outcome
and Impact
Demand for Information
TechnicalTechnicalApproachApproach
SystemsSystemsApproachApproach
IndividualIndividualBehaviorBehavior
Data Demand and Use
Political, Economic, Cultural, Social Environment
Use of data, analyses, and other information for decision making occurs within a political context
Technical Approach
Individual Behavior
OrganizationalApproach
Information for Decision Making
DecisionDecisionmakersmakers DecisionsDecisions
Why
?
Why
?W
ho else?
Who else?
How?How?
What What information?information?
Look at:• Current practices
• Organizational incentives and culture
• Individual attitudes and motivation
Develop a Data Demand and Use Profile:
• specify decision makers at various levels
• data needed to make decisions
• timing of data requirements
• the format for data presentation
Systems Approach to DDUSystems Approach to DDU
Technical ApproachTechnical Approach
Technical resources for transforming data into information
– Data sources
– Tools for data collection
– Methodologies
Focus on the main information the M&E data need to provide?
• Process:– Was the program carried out as planned?– How well was it carried out?
• Results:– Did the expected change occur?– How much change occurred?
• Impact: – Is the change attributable to the program?– Does the change mean program “success”?
Increase availability of data
• Explain indicators – design
– interpretation
– significance
– meaning of changes in value
• Explain data – collection processes– calculation processes– quality
Explain Changes
• Meaning of changes in indicator values – what can explain the change
• What other information should M&E communicate to help use this indicator to make program decisions?
Explaining Contradictory Data and Results
• Results from various data sources – are the sources providing complimentary information?
– immunization rates for DPT-3 in children are very different between official government HMIS figures versus survey with verification of children’s immunization cards
• Results at various times – are there unexpected changes over time?
Clarify Analysis Steps
• Data calculation processes
– Percentage of population reporting any risky sexual behavior in last six months
– Denominator could be (a) all; (b) all sexually active; (c) all within certain age groups; (d) all within defined “high risk” groups
• Which calculation would you recommend? Why? What are key factors to consider?
• How would you explain the significance and meaning of the different options to the program manager(s)?
Data Quality
• Data quality
– a child-immunization survey, verifying with cards, provides data for several Child Survival indicators
– you find out that the non-response rate on many key questions is 40 percent
• What are the implications for data quality of this information?
• How would you explain this to the program manager(s)?
• What options could be recommended?
Triangulation
• “Short-hand term” for data synthesis and integrated data analysis for decision making
• Originally coined by social scientists: basis for validity/truth/credibility
• Process in which multiple data sources (quantitative or qualitative) are brought together and analyzed to better understand the same public health phenomenon
Input Process Output Outcome Impact
Triangulation for HIV Programs Evaluation
HIV/STI Surveillance
Patients’ tracking/ARV resistance surveillance
Programme Monitoring
Second-Generation HIVSurveillance
Second-Generation HIVSurveillance
HMIS in PEPFAR, CDC, MAP, GFATMHMIS in PEPFAR, CDC, MAP, GFATM
Reproductive Health Survey/ CDC, MICS/ UNICEF, MACRO/ DHS and
AIDS Indicators Surveys,Service Availability Mapping (SAM),
Health Facility Surveys (HSPA), Coverage Surveys
Reproductive Health Survey/ CDC, MICS/ UNICEF, MACRO/ DHS and
AIDS Indicators Surveys,Service Availability Mapping (SAM),
Health Facility Surveys (HSPA), Coverage Surveys
Core Consensus Indicators for all Initiatives Based on United Nations General Assembly Special Session (UNGASS) and Recent Meetings
Core Consensus Indicators for all Initiatives Based on United Nations General Assembly Special Session (UNGASS) and Recent Meetings
Household Surveys, e.g. BSSFacility Surveys
Coverage Surveys
Improve Data Use & Communication
To increase use M&E Data must:
– be manageable and timely
– be presented according to the audience’s
• interest
• capacity to understand, analyze
• time, competing demands on time
– have transparent quality
– focus on activities, results of interest
– focus on meaning and direction for action
Communicating M&E information
Often hard to communicate, because:
• Technical issues are important but boring
• Decision makers are busy and distracted
• Program managers have tunnel vision
• Program implementers are too busy
• Data experts/wizards have few communication skills
• Communication experts/wizards have few data skills
Support Services
Information systems
Quality assurance
Community input
Financing
Leadership and management
Logistics
Training and skills maintenance
Supervision
Resources
Staff
Equipment and suppliesDiagnostic
services
Pharmacy
InfrastructureTransport and communication
Client
Care Process
Triage
HistoryCounseling
Examination Treatment
Diagnosis
It’s complex!
Effective communication
• Identify your audience and their needs.
• Focus on what your audience needs to know, not what you know.
• Consider your audience’s technical knowledge, motivations, and interests. Tailor the points accordingly.
• Decide what you hope the audience will do as a result of the presentation.
• Anticipate possible questions.
• Make sure your message source is someone your audience trusts and respects.
Desired Characteristics of Information
• Needs to be simple
• Needs to be timely
• Needs to be compelling
• Needs to be consistent with stakeholder values
Types of Tools and Resources
• Tools – printed documents– computer programs (often on CDs)– Internet-based applications
• Methods/techniques resources include– analytical techniques– case studies and best practices– data resources
Types of Tools
• Policy Development
• Program Development and Management
• Technical Resources
• Best Practices and Case Studies
• HIV/AIDS Programs
• Data Sources
Policy Development
• The Policy Circle
– The Policy Circle is a framework through which the dynamic components of policy development and implementation can be better understood and analyzed.
– The Policy Circle is used to analyze different levels of policy, including national and local policies, and sectoral and operational policies.
Program Development and Management
• The Health Manager’s Toolkit
The Health Manager's Toolkit is an electronic compendium of tools designed to assist health professionals at all levels of an organization to provide accessible high-quality and sustainable health services.
The tools have been developed by organizations working throughout the world to improve the delivery of health services. http://erc.msh.org
Technical Resources
• REPROLINE: Reproductive Health Online
Reproductive Health Online is an Internet resource for health-care professionals, providing technical information and training materials on the full range of reproductive-health topics, including family planning (contraceptive methods and other family planning topics), maternal and neonatal health, cervical cancer, and STDs.
www.reproline.jhu.edu
Technical Resources
• IGWG: Gender Publications and Tools
The Interagency Gender Working Group (IGWG) Gender Publications and Tools. The IGWG promotes gender equity within population, health, and nutrition programs with the goal of improving reproductive health/HIV/AIDS outcomes and fostering sustainable development.
www.igwg.org/pubstools.htm
Best Practices and Case Studies
• Advance Africa: Compendium of Best Practices
The Compendium of Best Practices is a searchable online database of best practices in reproductive health and family planning service delivery. It can be used by program managers who have identified gaps, needs, and opportunities in their programs. By reviewing the lessons learned from other practices, program managers can apply their own practices with greater confidence that they will lead to improved program performance.
www.advanceafrica.org/compendium
Best Practices and Case Studies
• UNAIDS: Best Practice Collection
The UNAIDS Best Practice Collection – hundreds of UNAIDS publications documenting successful approaches to particular HIV/AIDS problems and issues.
www.unaids.org
HIV/AIDS Programs
• Synergy Project: APDIME Toolkit
The Synergy Project APDIME Toolkit is a resource to support program designers and managers in HIV/AIDS prevention, care and support programming. Tools for Assessment, Planning, Design, Implementation Monitoring, and Evaluation are included.
www.synergyaids.com
HIV/AIDS Programs
• HEARD: AIDS Toolkits
The AIDS Toolkits developed by the Health Economics and HIV/AIDS Research Division, University of KwaZulu-Natal encompass a broad range of interventions for HIV/AIDS including assessment of the impact of AIDS on different sectors and ministries.
www.nu.ac.za/heard/toolkits
HIV/AIDS Programs
• UNAIDS: M&E Toolkit for National AIDS Programs
The Guide to Monitoring and Evaluation for National AIDS Programs was developed by UNAIDS in collaboration with MEASURE Evaluation and many other partner organizations. It includes a toolkit providing guidelines and instruments for the development and use of indicators for all components of a national program.
www.cpc.unc.edu/measure/publications/unaids-00.17e
Data Sources
• DHS: Statcompiler
The Demographic and Health Surveys STATcompiler builds customized tables from hundreds of DHS+ surveys and indicators. The HIV/AIDS Survey Indicators Database includes data from DHS+, MICS, BSS, and other surveys.
www.measuredhs.com
Data Sources
• U.S. Census Bureau
International Data Base: The International Data Base (IDB) is a computerized data bank containing statistical tables of demographic and socio-economic data for 227 countries and areas of the world.
www.census.gov/ipc/www/idbnew.html
Data Sources
• U.S. Census Bureau
HIV/AIDS Surveillance Data Base: The HIV/AIDS Surveillance Data Base is a compilation of information from those studies appearing in the medical and scientific literature, presented at international conferences, and appearing in the press.
www.census.gov/ipc/www/hivaidsd.html
Data Sources
About DOLPHN
The Data Online for Population, Health and Nutrition (DOLPHN) system is an online statistical data resource containing selected current and historical country-level demographic and health-indicator data. The DOLPHN system is designed to provide users with quick and easy access to frequently-used statistics and can be helpful as both a reference and analytical tool.
www.phnip.com/dolphn
Top Related