KMC WorkshopGroup E

Monitoring and Evaluation

Clarification of Concepts

• Monitoring: vigilance of a process• Evaluation: assessment, value judgment about

a process and its results.• The group decided that it was going to focus

on monitoring, rather than on the specifics of evaluation

Clarification of Concepts

• Monitoring in KMC can have several meanings:– Vigilance of the implementation of KMC at different

levels• Global (WHO)• Regional• National• Local, etc.

– Vigilance of an ongoing program, again:• Global• Regional, etc.

Clarification of Concepts

• Prior to defining what to monitor during implementation, goals and plans for implementation should be stated.

• Main purposes of monitoring then, would be:– Surveillance of the compliance with

implementation tasks and steps, timetable, etc.– Quantification of performance (e.g. number of

trained health workers, etc.)– Quantification of achieved goals (e.g. mortality

reduction)

Clarification of Concepts• Monitoring of an ongoing program can also have

several purposes and objectives• The group identified monitoring a as a health

care quality assurance tool as a very important purpose.

• Most of the following discussion was centered then on “evidence-based quality assurance” for KMC programs

• The importance of monitoring for implementation was nevertheless acknowledged as well as the need for address it a next step

Objectives

• To identify elements and domains relevant for developing appropriate monitoring tools for quality assessment-improvement (quality assurance) of an ongoing KMC program.

• To generate a series of statements which can help to define good practices for monitoring KMC programs

• To develop the process, a hypothetical program for delivering KMC in a health facility was sometimes used.

• Extrapolation to other settings and to broader scopes can be made.

The EB-Quality improvement cycle• Evaluate current practice, identify problems

– Separation of mother and infant after birth– Low breast feeding rates– Undesired variability in practices and/or in outcomes

• Plan and implement interventions– Set quality standards for practice (e.g. clinical practice guidelines, evidence-

based)– Set quantitative goals– Implement practices

• Monitor – Compliance with requirements– Performance

• Compliance with guidelines• Frequency of selected outcomes

• Adjust performance according to monitoring• Evaluate results (close the cycle, go back to 1st step) and start all over

Delivering KMC implies • A KMC program:– Resources, administration, planning, management, put

together to deliver• The KMC intervention– Set of specific processes (interventions ) for caring for the

health care of newborn infants involving and empowering their mothers-families

– Using a specific method or technique• The KMC method, a complex non-pharmacological

intervention clearly standardized, defined and supported by scientific evidence.– The method is defined as EB recommendations, usually in

the form of structured detailed protocols.

Components of a KMC program as related to health care quality

• Structure• Processes• Outcomes

(Donabedian)

Structure

• Stable part of the Health care system, that provides the support and setting in which health KMC is delivered:– Physical structure– Administrative structure and processess– Norms– Resources• Technical• Human• Capital

Process• Actual specific health care interventions and

procedures employed for providing care– Delivered interventions should be backed by scientific

evidence supporting that they do more good than harm– They should be feasible:

• Available• Affordable• Proficiently performed

– Properly trained personal– Technically appropriate equipment

• Acceptable for– Target population and– Health care personnel

• Ethically appropriate

Outcomes

• Changes in different aspects of health associated with previously delivered interventions– Disease (condition) centered

• Mortality• Morbidity, complications and sequels• Time to event

– Patient centered• Growth and development• Satisfaction• Health related quality of life

Monitoring KMC programs for Quality assurance• Structure:

– Standards should be set• Rate of nurses to patients• Locative facilities for mothers to provide kangaroo position• Etc.

– Verification that minimum acceptable standards are met (Basic part of certification? Accreditation?)

• Process– Recommendations (evidence-based guidelines) have identified

processes known to do more good than harm– Compliance with recommendations is monitored

• Outcomes– Given that there should be evidence showing that recommended

processes do more good than harm not every specific health outcome needs to be monitored

– Nevertheless, given that despite having evidence-based recommendations, thing can go wrong (poor performance, other quality issues) some important and selected health outcomes should be monitored.

Proposed framework to identify what to monitor

Structure Process Condition centered outcome

Patient centered outcome

Method(Technique)Intervention

Program

An exampleElement Structure

MethodDoes the program has Guidelines or Protocols?Explicit, witten?

Clear identification of components?Standardized definitions of components?Identification of therapeutic goals for each component?Evidence-based recommended courses of action?Explicitly defined setting?

Explicitly defined target population?

Clear inclusion / exclusion criteriaClear role definition for health care personnel?

An example (cont.)Element Structure Process OutcomesIntervention Properly trained health care personnel

Compliance withInitiation Mother recruitment processes and inclusion criteria

Infant selection criteriaKP Kangaroo adaptation including monitoring

Kangaroo nutrition strategy initiationKN Apropiate support for breast feeding

Apropiate use of feeding methodsApproriate monitoring of feedingMaintenance of position

Follow up policies Discharge - Follow up proceduresmortalityKMC-related eventsHospital stay Clinic visits

Mother mood depresionBonding indicatorsAttachment indicatorsParents satisfaction

Examples of specific indicators• Performance indicators of a KMC program – LBW Infants in KP first visit to the clinic/total LBW

infants discharged from hospital• Minimum acceptable 90%• Optimal 95% (there are infants discharged after leaving KP)• Measures compliance with KP between discharge and first

visit to out KMC clinic– Hours a day a parent can stay at the NICU

• Minimum acceptable 12 /24• Optimal 24/24• Measures appropriateness of structure:

– Regulations and norms to remove access barriers– Amenities, comfort, space to allow parents to stay

Examples of specific indicators: the “SKIND” score

Item 0 1 2

SSC in 1st hour <50 >50 60 ‘

KP duration in the 1st 6 hours

< 4 h <5.5 >5.5 h

Initiate BF (Self attachment observed)

No 1-2 h 1st hour

Nutrition, Artificial feeding given?

AIF Clear fluids None

Delayed Procedures for 6 h

None bath all

Recommendations• Setting Standards for structure and making sound evidence-based

recommendations which guide processes is indispensable prior to proposing any reasonable plan for monitoring

• Identification of “key indicators” amenable for “universal” use can be attempted afterwards

• A group within INK should work on a guide for developing good monitoring practices for KMC quality assurance

• An inventory of available instruments for monitoring and assessing implementation and for quality assurance on ongoing KMC should be assembled– The group already identified several research and monitoring

instruments both general and specific– Instruments for monitoring the progress of Implementation – Instruments for monitoring quality of care