Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and...

49
New Technologies and Results Based Financing: A discussion paper Prepared for the World Bank RBF Impact Evaluation workshop March 2014

description

A presentation from the 2014 Annual Results and Impact Evaluation Workshop for RBF, held in Buenos Aires, Argentina.

Transcript of Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and...

Page 1: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

New Technologies and Results

Based Financing:

A discussion paper

Prepared for the World Bank RBF Impact Evaluation workshop

March 2014

Page 2: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

Table of Contents Purpose of this discussion paper .................................................................................................................. 1

1. Definitions and introduction to mHealth .............................................................................................. 1

a. What do we mean by new technologies? ......................................................................................... 1

b. The growth of mHealth ..................................................................................................................... 2

c. Applications for mHealth .................................................................................................................. 3

d. mHealth, RBF and Health Systems strengthening ............................................................................ 3

2. mHealth and Results Based Financing (Supply Side and Demand Side) ............................................... 5

3. Where are we now and the challenges ahead ..................................................................................... 8

4. Addressing these challenges ............................................................................................................... 13

a. The role of Governments ................................................................................................................ 13

b. The role of donors ........................................................................................................................... 13

c. The role of implementers of mHealth initiatives ............................................................................ 14

5. The transformative power of mHealth ............................................................................................... 14

6. Conclusion ........................................................................................................................................... 15

Case Study 1: Web-technologies for RBF efficiency, accountability and transparency .............................. 16

Case Study 2: Supporting social accountability and verification in Uganda ............................................... 19

Case Study 3: eVouchers in Ethiopia ........................................................................................................... 20

Case Study 4: Mobile Money for payments in RBF ..................................................................................... 24

Case Study 5: Scaling up mHealth in Tanzania ............................................................................................ 29

Annex 1: Examples of selected mHealth interventions .............................................................................. 31

Glossary of eHealth and mHealth terms: .................................................................................................... 40

References .................................................................................................................................................. 43

Page 3: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

1

Purpose of this discussion paper This discussion paper takes as its theme the application of new technologies to results-based financing

for health. Within this broad field, the focus is on mobile technologies or mHealth. Below we start by

providing an introduction to Information and Communication Technologies (ICT) and mHealth, followed

by an overview of how mobile technologies apply to health systems and health service provision. We

then look specifically at the application of mHealth to key aspects of results-based financing, linked to a

series of case studies that accompany the discussion paper. The challenges of scaling-up mHealth, as

well as suggestions for stakeholders (governments, donors, implementers) in addressing these

challenges, is followed by a brief discussion on the transformative nature of mobile technologies and

how they contribute to the post-MDG agenda and the move towards universal health coverage.

1. Definitions and introduction to mHealth

a. What do we mean by new technologies? For the purpose of this discussion paper, the term new technologies refers principally to Information

and Communication Technologies (ICT) and to their applications in the health sector (eHealth and

mHealth) and more specifically to Results-Based Financing (RBF) approaches in health.

Information and Communication Technologies refer to technologies that provide access to information

through telecommunication which includes the internet, mobile phones, wireless networks and other

communication mediumsi.

eHealth (the “e” stands for electronic) is the use of ICT for the provision of health servicesii. It is an

umbrella term which includes four distinct but related componentsiii:

Mobile Health (mHealth) refers to the provision of health services and information through

mobile and wireless technologies such as mobile phones and tablets.

Telemedicine is the provision of health care services at a distance and includes remote

consultations, inter-professional and patient communications.

Health Information Systems (HIS) are the structures which allow collecting, collating, analysing

and synthetizing data from multiple sources (ranging from commodity management to disease

surveillance and human resources) and to report on health for better planning purposes.

Distance learning (eLearning) refers to education and training in electronic form for health

professionals.

Of these four components, the one which has attracted most attention in recent years is mHealth. There

is no standardised definition of mHealth to date and there is still confusion as to what mHealth means

(e.g. is phoning a fellow health worker on his mobile phone mHealth? The answer may depend on the

purpose of the call) and on how to differentiate eHealth from mHealth (e.g. is a laptop connected to the

internet with a dongle a mobile device?). As technology improves some argue that the distinction

between the two is increasingly blurred and will soon be obsolete.

Page 4: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

2

b. The growth of mHealth The use of ICT in health programming has been rapidly expanding over the past decade (Figure 1) due to

a combination of the proliferation of mobile devices, decreasing prices for both hardware and

connectivity, improvements in the technology (smaller, better, faster) and the expansion of its enabling

infrastructure (e.g. phone masts for network coverage) even in remote areas.

Figure 1: Proportion of health programmes

in low and middle-income countries currently

using ICT by year of launch

mHealth has become an increasingly important aspect of ICT since 2005 and Figure 2 illustrates the

sharp increase in mobile phone subscriptions by region. For example, in Mali mobile phone

subscriptions equalled 32.9% of the population in 2009 and over 98% in 2012 (ITU, 2013v). According to

Lemaire “nearly 90 per cent of the world is now covered by a wireless signal, ensuring that formerly

isolated individuals, such as farmers in rural Kenya or herders in northern Mongolia, have the potential

to access these technologies and communicate globally”iv.

Figure 2: Mobile phone subscriptions by region

Source: ITU, 2013v

0,020,040,060,080,0

100,0120,0140,0160,0180,0

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

*

20

13

*

Per 100 inhabitants

Africa

Arab States

Asia & Pacific

CIS

Europe

The Americas

Page 5: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

3

Examples of RBF web applications include:

Nigeria : www.nphcda.thenewtechs.com

Benin : www.beninfbr.org

Cameroun : http://www.fbrcameroun.org

Burundi : http://www.fbpsanteburundi.bi

Senegal : http://www.fbr.sante.gouv.sn

Zambia : www.rbfzambia.gov.zm

However, despite this rapid expansion there remain a number of challenges linked to infrastructure,

capacity to benefit, and illiteracy (see Section 3).

c. Applications for mHealth The use of mobile devices to provide health services and information can be divided into six broad

categories and examples of each are set out below:

Education and awareness: with generally low costs and high penetration rates, mobiles and

SMS have been used to promote behaviour change and reach populations with health

information.

Data collection: for example population based surveys with instant aggregation of results;

mapping using GPS; and crowdsourcing. There is strong potential for disease tracking as the

geolocation capabilities of modern devices can greatly support epidemiological surveillance.

Point of care support: for example diagnosing and/or treating patients; automated reminders

for visits; and even networked sensors for rapid testing and diagnosis (e.g. smartphones

equipped with microscope add-ons for rapid malaria blood tests).

Remote patient monitoring and support: for example patients reporting conditions to distant

providers; linking patients to networked sensors which automatically send information to a

health professional (e.g. blood glucose or blood pressure monitoring devices connected to a

network); treatment adherence and patient reminders by SMS.

Logistics: this includes everything in the supply chain, from tracking or managing drugs and

commodity stocks to verifying the authenticity of medical supplies.

Training and dissemination of knowledge: provision of training and dissemination of latest

research and best practices to any level of healthcare practitioner to improve quality.

Additional examples for each of these categories are provided in Annex 1.

d. mHealth, RBF and Health Systems strengthening Results-based Financing (RBF) acts in a number of ways on key aspects of the health system, including

governance, efficiency, transparency, management and social accountability.

Governance: In too many countries, getting services to the people who need them, particularly the very

poor, is undermined by widespread corruption and inefficiencies. For example, according to

Transparencia Mexicana’s National Index of Corruption and Good Government, Mexican families living

on less than $5 per day lose nearly a quarter of their

income to corruption in the form of bribes to obtain

essential servicesvi. In Bangladesh, lack of oversight

contributes to a 74 percent absenteeism rate among

public health care doctorsvii. In the Philippines,

corruption leads to lower immunisation rates, delays in

the vaccination of newborns, discourages the use of

public health clinics and reduces satisfaction of

households with public health servicesviii. RBF

addresses corruption through supporting increased

Page 6: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

4

accountability and transparency in the health system. It does this through splitting key functions

(purchasing, providing, financing), and by incorporating rigorous verification, fraud prevention and

enforcement measures.

Efficiency: RBF can contribute to greater efficiencies in the health sector. For example, RBF can improve

allocative efficiency by targeting funds to more cost-effective interventions with greater public health

benefits, and can lead to greater technical efficiency through incentivizing providers to provide better

quality services and managers to improve supervision and support. Transactional efficiency is improved

because resources are allocated directly to providers and households, reducing transaction costs and

bypassing inefficient bureaucracies. mHealth can also support RBF to reach greater numbers of

beneficiaries at a lower cost, thus improving efficiency.

Transparency: Countries such as Nigeria, Benin, Burundi and Zambia share RBF data beyond the health sector. RBF data is shared with the general public through the internet, creating an example of accountability and good governance in countries with low governance indicators. In Benin, all RBF data can be downloaded from the RBF database and the country has also set up an active communication strategy, including local and national radio and television broadcasts on health system performance, Facebook postsix and YouTube videosx.

New technologies are at the heart of RBF systems, and mHealth in particular can enhance these

dimensions of RBF, as will be shown below in Section 2. Web based technologies are used in RBF

programmes in a growing number of countries such as Benin, Cameroon, Chad, Nigeria, Senegal and

Zambia, to manage data flows, including data on health providers, the quantity and quality of health

services, verification, finances and patient satisfaction. Web-based technologies support RBF teams in

the management of RBF interventions, helping to reduce transaction costs and improve the reliability of

systems.

Although interoperability remains a serious challenge for mHealth interventions and RBF data are often

not integrated with HMIS (see Section 3), countries at the forefront of implementing RBF, such as Benin,

are exploring how to share RBF data through enhanced data system interoperability and mobile

technologies: the country plans to interconnect its HMIS and RBF databases so that RBF data can be

exchanged with the HMIS database. The country is further exploring how to share RBF data through

SMS with providers and village health workers, and through emails to district managers, Ministry of

Health (MOH) staff and donors.

Management: Beyond RBF management, data emerging from RBF programmes are of high value for the

stewardship of the wider health system. Verified data on the health system performance, on quality of

care and on patient satisfaction can be used for better decision making at the different levels of the

health system. Currently, countries such as Benin, Burundi, Nigeria, Zambia and Senegal are using web-

based technology for sharing real-time results and financial data on the web, making the data available

to decision makers. Interestingly, Benin displays the ranking of its providers on the internet, creating a

strong incentive effect among these providers to achieve a better

performance.

Social accountability: Social accountability is an essential dimension of RBF

systems. In most supply- and demand-side RBF systems, patients are

In Uganda, mHealth is

supporting social

accountability and

verification (see Case

Study 2)

Page 7: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

5

interviewed to ensure that they have benefited from the services provided. These interviews are often

used to measure patient satisfaction and also to verify the data and results linked to payments. In

Cameroon and Burundi, patient satisfaction is part of the payment formula of the provider. In Benin,

patient satisfaction will be systematically displayed on the web page for each provider. Mobile phones

and social media offer opportunities to reinvent and boost social accountability in health systems and

programmes. Young generations express themselves through mobile phones, and these tools can be

used for improving verification and enhancing accountability.

2. mHealth and Results Based Financing (Supply Side and Demand Side)

Results-Based Financing (RBF) for health has been defined as "a cash payment or non-monetary transfer

made to a national or sub-national government, manager, provider, payer or consumer of health services

after predefined results have been attained and verified” (Musgrove 2011)xi. Results can be defined as

outputs, outcomes or impacts – intended or unintended, negative and/or positive – of a development

interventionxii.

A standard categorisation is to distinguish RBF schemes that tend to revise incentives on the supply side

(supply-side RBF) from those modifying the incentive structure primarily on the household side,

(demand-side RBF), although in practice the boundary between both categories is not clear-cut,

depending on the extent to which the ‘purchasing power’ is given to providers or consumers. Figure 3

provides a typology of the principal supply-side and demand-side approaches in RBF.

Figure 3: Typology of supply and demand-side approaches in results-based financing

Source: adapted from Naimoli (2010)xiii

Page 8: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

6

Figure 3 illustrates that some demand-side financing approaches have a much stronger impact on the

supply-side than others (i.e. vouchers and insurance, where incentives are passed to the providers as

illustrated in Figure 3) and likewise some supply-side approaches include incentives for reaching specific

population groups.

While in supply-side RBF only two main parties are involved (the government or managing agent and the

service provider), in demand-side RBF a third party (the client or client representative) is involved. In

both demand-side and supply-side approaches, the relationship between the government (or managing

agent) and the provider is governed by a contract or Memorandum of Understanding (MOU) and it is

this contract which specifies how outputs or performance are linked to reimbursement payments (see

Figure 4).

Figure 4: Supply and demand-side results-based financing approaches

Demand-side RBF

Health

facilities

Supply-side RBF

Govt/donor funding

Management

Agency

(Govt/non-

Govt)

Clients

HEF cards/

Insurance

cards/

Vouchers

Claims

(vouchers)

Contract

$

Services

Entitlement

(cards/

vouchers)

Govt/donor funding

Contracting

Agency

(Govt/non-

Govt)

Health

facilities/ health

managers

Results

data$

Performance-based

financing

and contracting

In supply-side RBF, incentives are paid to the provider based on a (set of) performance target(s) or

indicator(s), which are mostly linked to the number of beneficiaries or consumers using the service(s),

and increasingly also linked to the quality of the services provided. The key defining feature of all

demand-side RBF (DSF) approaches is that the money (or incentive) follows the client, and may or may

not be passed onto the provider in the form of reimbursements for services provided.

There are many synergies between demand-side RBF approaches (particularly vouchers and health

insurance) on one hand, and supply side RBF approaches on the other (as illustrated in Figure 5).

Page 9: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

7

In Ethiopia, Marie Stopes

International are using

eVouchers for family

planning in 25 villages (see

Case Study 3)

Figure 5: Characteristics of DSF and SSF

Characteristics CCT Vouchers SHI SSF

Enrolment of beneficiaries

Yes Yes (with voucher distribution)

Yes (usually with co-payment)

No

Contracting of providers

No Yes Yes Yes

Quality Assurance No Yes Yes Yes

Distribution of incentives

Yes (cash) Yes (voucher) Yes (insurance card)

To providers

Payments To beneficiaries To providers To providers as premiums

To providers

Claims processing No Yes Yes Yes

Fraud control Yes Yes Yes Yes

Verification Yes Yes Yes Yes

Enrolment of beneficiaries is specific to DSF. Traditionally, health workers or mobilisers will use paper

records to collect beneficiaries’ information (such as age, sex, socio-economic status, date they were

enrolled etc.) which they will send back to the programme’s management team on a regular basis.

mHealth can improve the accuracy of data collection and submit the data almost instantly with

applications such as CommCare (Dimagi) or Magpi (Datadyne).

Contracting of providers refers to the selection of providers who meet the minimum criteria to be part

of the programme (or fall short of the minimum criteria but have developed an improvement plan) and

ensuring they will adhere to the project’s guideline and protocols. Poor performance and misuse of

funds are some examples which would lead to the termination of a contract and the exclusion of a

provider in the scheme. This applies to both DSF (vouchers and health insurance but not conditional

cash transfers CCT)) and supply-side RBF (SSF). mHealth can support monitoring of contracts (see quality

assurance and fraud control below).

Quality assurance is one of the key components to determine continued participation of a provider in a

scheme and to determine the level of incentives they will receive. This is common to vouchers, health

insurance and SSF. Tablets or smartphone applications can improve the accuracy of data collected and

can calculate the results in real time, allowing instant feedback to providers on their performance and

quality.

Distribution of incentives is carried out in both DSF and SSF

schemes. In CCTs, the incentive is cash (or can be another form of

in-kind incentives) distributed to the beneficiary once they have

Page 10: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

8

Mobile money facilitates

payments of a CCT

programme in Kenya and is

used to pay social

franchisees of a voucher

programme in Madagascar

(see Case Study 4)

fulfilled the condition. This can be supported by mHealth through mobile money (see Case Study 3). For

voucher schemes, the incentives are the actual vouchers which have a monetary value and empower

the client. Vouchers can simply be a unique code sent to a phone via SMS or be an eVoucher (see Case

Study 2). Similarly, health insurance providers distribute insurance cards which can be smart cards (a

plastic card with a built-in microprocessor). In SSF, incentives are distributed to the providers based on

their performance and for financial incentives; this can be done through mobile money.

Payments: can be facilitated in both DSF and SSF schemes with mobile money. In CCTs, payments are

made to the beneficiaries (unless using another type of in-kind

incentive) and constitute the incentive for beneficiaries to change

their behaviour. In vouchers, health insurance and SSF, payments

are made to providers based on their performance.

Claims processing: is common to vouchers, health insurance and

SSF schemes. mHealth can support the claims process with

software and apps capable of automatically validating and

verifying the data and instantly triggering payments if the

information submitted appears to be in order. Claims processing is also closely linked to fraud control

and verification.

Fraud control: All DSF and SSF schemes have some degree of fraud control. Software and applications

can support the validation and verification of data and, in its more sophisticated form, algorithms can

spot and flag dubious behaviours to investigate.

Verification is inherent to all DSF and SSF schemes. The process can be facilitated by mHealth for

remote verification (SMS, voice calls etc.). Face to face verification can also be supported by mHealth by

equipping verifiers or field workers with tablet or phone based surveys.

3. Where are we now and the challenges ahead

In 2011, the WHO surveyed all its member states and found that 83% of them reported at least one

mHealth initiative in their country, and the great majority reported four or morexiv. However, many

mHealth initiatives remain at pilot stage, work in silos, and fail to

show their potential success and be scaled-upxv. This has led to

increased dissatisfaction with mHealth pilots and has led to a state of

what has been termed, Pilotitis. For example, in Uganda from 2008 to

2009, 23 mHealth campaigns were not taken forward after pilot

stagexvi. Figure 6 shows the sheer number of pilots in the country. This

led to the Uganda Director General of Health Services issuing a moratorium on all mHealth pilots in

2012. In Ghana an ICT Review Committee has been set up to assess all mHealth projects and it is

expected that many more countries will follow suit.

ICT experts believe that the proliferation of pilot projects is actually an inevitable progression in the

development of innovative approaches and new fields of work (Figure 7). But others argue that mHealth

Pilotitis: a “condition”

characterized by a multitude

of pilot projects which

struggle to reach scale.

Page 11: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

9

has reached its phase of maturity and that by now there should be enough evidence to scale-up pilot

interventions.

A small number of pilots are successfully being brought to scale. For example, in Tanzania the Healthy

Pregnancy, Healthy Baby text messaging service was launched nationwide in 2012 and success has been

attributed to leadership by the Ministry of Health, partnerships

(mHealth partnership) and relevant timing and content. In Rwanda,

UNICEF is working with the Government to put in place a system

where community Health Workers (CHWs) track and report on

pregnant women through the whole continuum of care using

mHealth. Infants’ HIV test results in Malawi and Zambia are being

sent back to clinics via SMS; and in Nigeria the same SMS system is

used to register every birthxvii.

So, why are the majority of mHealth pilots not being scaled up? Some of the key challenges are explored

below.

Figure 6: A severe case of pilotitis in Uganda

Source: UNICEF Uganda, 2012

In Tanzania, the Healthy

Pregnancy Healthy, Baby

mHealth campaign has

successfully been brought

to scale (see Case Study 5)

Page 12: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

10

Figure 7: The progression of mHealth in low and middle income countries

Source: Text to Change

Lack of evidence

The majority of pilots are either not being adequately monitored and evaluated (i.e. lack of or weak

study design) or the results of the evaluation are not being disseminated; the rapid increase in mHealth

initiatives has not been matched with advancements in evaluationiv. Only 12% of WHO member state’s

projects involving mHealth have or are being evaluatedxviii, and implementing a program without

previously evaluating its appropriateness often leads to failureiv. Furthermore, the great majority of

evaluations are limited to the analysis of process indicators such as usage statisticsxix, (e.g. number of

SMS sent) or anecdotal evidence, and have so far not been able to show the impact on health outcomes

that are attributable to mHealth. The authors have only come across a single evaluation framework for

mHealth (see Box 1). In short, little is known on the impact of mHealthxx. This means that even mHealth

programmes which may have the potential to bring about positive change remain unproven or

unknown. There is an urgent need for more research with rigorous designs focused on health

outcomesxxi.

The lack of standards and interoperability

Interoperability is understood as the capacity of operating systems to work together and exchange

information. The potential of mHealth interventions is maximised when health sector actors make their

information systems interoperablexxi. However, most mHealth pilots work in isolation and do not

communicate or collaborate with each other. This leads to inefficiencies and duplications. An Economist

Intelligence report (2012)xxii, which examines the state and potential of mHealth in both developed and

emerging markets, shows that only half of doctors surveyed reported that their mHealth application was

connected to their organisation’s information technology (IT). Even fewer said that their mHealth was

integrated with other parts of the health system. It can be assumed that levels of integration might be

lower in low income countries.

Page 13: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

11

However, interoperability requires common standards to be defined in order for systems to be

compatible with each other. Having clear standards and increased interoperability can lead to increased

efficiency, collaboration and coordination, and simultaneously creates vast pools of data which can be

analysed for planning and policy purposesxx. Better informed planning will support better allocation of

resources and allow for more accurate forecasts, models and simulationsiv.

The lack of sustainable sources of funding

Sustainable funding to bring pilots to scale is essential and needs to be considered during the design

process. For example, scaling from a few smart phones to equipping large numbers of health workers

with both phones and credit is likely to be beyond the budget of most governments. However,

technology is evolving very rapidly and as prices of smartphones continue to drop so too will the cost of

transmitting data and making a call. It should be remembered that developments in ICT are not driven

by the health sector but primarily by communications, commerce and governance considerationsxvi. One

potential solution to sustaining funding is to build strong public private partnerships (PPP) between

governments, network operators, phone providers, local partners and the intended beneficiaries from

the outset.

Lack of capacity

Lack of capacity refers variously to the lack of network capacity or coverage, the lack of health worker

capacity, the lack of IT capacity to develop and/or adapt the technology, and the lack of Monitoring and

Evaluation (M&E) capacity to yield evidence. These are all barriers to scaling-up effective mHealth

interventions. Nonetheless, these challenges are not unique to mHealth interventions and should not

stop governments and international organisations striving to achieve better results through new and

promising approaches.

The lack of policy and strategy

The policy environment is not always favourable to mHealth; government policies can limit the extent to

which the potential of mHealth can be realised, such as in the State of Bihar, India, where the

Government prohibits phone operators from transferring money (see Case Study 3). Furthermore, few

countries to date have developed eHealth strategies which would provide a supportive framework for

mHealth interventionsxxi. Where eHealth strategies are being developed, they are taking a long time

(South Africa) and/or have not been widely communicated, and therefore utilised (Kenya). A

comprehensive strategy focused on building the evidence, ensuring sustainability and developing

standards for interoperability would lead to a great wealth of information and knowledge which would

improve decision making, policy making and the allocation of resources. Governments also need to seek

a balance between enabling innovation while protecting consumer rightsxxi.

Page 14: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

12

Box 1: An mHealth Evaluation Framework

A rapid literature review conducted for the purpose of this paper identified only one evaluation

framework for mHealth: mHealth technologies in developing countries: a feasibility assessment and

a proposed framework, November 2013 (Marshall C., Lewis D. and Whittaker M.). This particular

framework suggests a two phased approach to (i) evaluate the environmental context using an

adapted maturity model (Haux 2006) and (ii) evaluate the project or programme using the Real

Access / Real Impact criteria (Bridges 2010). As a stepping stone in generating more thorough and

systematic evidence on mHealth, the authors of this paper felt it was important to present a brief

summary of this framework here.

Phase 1: The maturity model specifically adapted by Haux (2006) for the development of health

information systems identifies seven stages of growth:

1. A shift from paper-based systems to computer-based processing and storage and increased data processing

2. A shift from local to global information system architectures 3. HIS used by professionals and patients/consumers 4. Data used for patient care and administration, and also increasingly used for healthcare

planning and clinical research 5. A shift of focus from technical HIS problems to change management and strategic

information management 6. A shift from alpha-numerical data to clinical images and data on a molecular level 7. Steady increases in new technologies for continuous monitoring of health status.

It should be noted here that, according to this model, most developing countries are just entering stage two, during which projects that have previously been working in isolation begin to link up, share data and improve efficiency. Phase 2: Bridges’ Real Access / Real Impact framework relies on the assessment of 12 criteria:

1. Physical access to technology 2. Appropriateness of technology 3. Affordability of technology and technology use 4. Human capacity and training 5. Locally relevant content, applications and services 6. Integration into daily routines 7. Socio-cultural factors 8. Trust in technology 9. Local economic environment 10. Macro-economic environment 11. Legal and regulatory framework 12. Political will and public support

Whilst this is a good beginning, the authors believe this framework needs to be developed to incorporate health outcome measures.

Page 15: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

13

4. Addressing these challenges

Governments, donors and implementers all play a key role in addressing these challenges. With better

collaboration, research and strategies many of the barriers which prevent mHealth initiatives being

brought to scale could be overcome.

a. The role of Governments Governments should start by carrying out a thorough assessment of the mHealth situation in their

countries. This would include:

A stakeholder analysis of the relevant actors involved in mHealth initiatives;

Mapping of the different mHealth initiatives through consultations with implementers, donors

and network operators (e.g. who is doing what, where, how, with which device, which software,

what are the lessons learned, what results have they achieved?);

An assessment of the ICT infrastructure (cell phone penetration rates, network coverage,

mapping of phone masts etc.) and use of mobile phones (who has access to a phone or not and

implications for targeting and equity);

An assessment of the policy environment which may enable or impede mHealth initiatives (e.g.

what policies are in place to protect patients’ rights and how does this fit in with mHealth; how

much control does Government have or not over network operators etc.).

The next step is for governments to develop an eHealth strategy informed by the results of the

assessment. Governments should lead this process while consulting with donors, implementers,

network operators and communities. These strategies should take into consideration:

How collaboration should be organised (sharing of information, development of standards, a

push for interoperability, creation of efficiencies);

How research should be conducted (should evaluation be mandatory? should it focus on impact

or cost effectiveness or both etc.?);

How is this strategy aligned with other Government strategies (not only in the health sector);

How the implementation of this strategy will be steered and monitored (setting up inter-

ministerial committees, setting targets and indicators);

What are the human, structural and financial requirements for implementing (e.g. training of

health workers on mHealth, allocation of a budget to Government mHealth activities etc.);

How this strategy can constitute a framework for developing future policies (or reviewing

current ones) in relation to mHealth and ICT;

How will this strategy be disseminated to all relevant actors (other ministries, implementers,

donors, network operators and communities).

Finally, Governments should disseminate and implement the strategy (Figure 8).

b. The role of donors To date, most mHealth initiatives have been funded by private philanthropists and donorsxvi. As many

donors work in multiple countries, they have an important role in pushing the global agenda on

mHealth. Donors should:

Page 16: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

14

Invest in building local capacity in designing, implementing, monitoring and evaluating mHealth

initiatives;

Ensure their objectives are aligned with government strategies, policies, priorities and

standards;

Support local ownership and promote local accountability for mHealth initiatives;

Promote interoperability amongst mHealth interventions and also with government’s and other

partners’ operating systems;

Push the agenda for stronger evidence in mHealth, contribute to the

development/implementation of robust evaluation frameworks, and invest in other areas of

research (e.g. cost-effectiveness of mHealth interventions);

Contribute to the global agenda and global strategy in mHealth;

Contribute to the building and dissemination of global evidence on mHealth;

Ensure the sharing of information, experiences and resources across countries (South-South

collaboration).

c. The role of implementers of mHealth initiatives Implementers can play an important role in:

Refraining from starting new low-scale mHealth pilots;

Ensuring that project designs include robust monitoring and evaluation;

Designing for scalability and sustainability;

Striving to develop software which uses open-source coding to improve the capacity for

adaptation and scalability;

Collaborating with other organisations and sharing information and results;

Aligning with government and donor strategies and priorities;

Striving to feed information and lessons learned into the local HMIS;

Developing local capacity.

5. The transformative power of mHealth

The adoption of mobile devices has happened at an extremely rapid ratexxiii, particularly in developing

countries, where limited landline infrastructure means more people rely on mobile networksxxiv. With

the proliferation of mobile devices came the emergence of mHealth with its possibilities and challenges.

So where does that leave us and how will this affect the way we work in the health sector?

Better planning and a stronger focus on results

mHealth has the potential to facilitate the collection of more accurate data and real time information.

This can lead to better planning and allocation of resources and a focus on results. Participants in an RBF

initiative can receive immediate feedback on their performance, which can be empowering and increase

their focus on results. As with timely payments of incentives to service providers, timely feedback on

performance (to which incentives are linked) and real time data on results are essential to the success of

Page 17: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

15

an RBF initiative. mHealth can therefore reinforce the culture of ‘results-based’ and help achieve greater

success in RBF approaches.

A focus on accountability and transparency

mHealth can empower both patients and providers. It has the potential to help patients become more

involved with their own care by making information more interactive and available anywhere and at any

time. This is particularly interesting for patients suffering from chronic conditions. Therefore, we are

likely to see increased participation and interest in health systems by patients. In turn, this supports

greater accountability and transparency in the sector.

mHealth can also empower service providers by providing them with diagnostic and treatment support

tools, and building trust and confidence between them and communities. mHealth can increase

communications between providers and patients but also between providers and the rest of the health

system, through remote supervision and training, remote technical support and through the reporting of

data and results. This reinforces a culture of transparency and accountability in the health sector.

Support for human resources in hardship areas

Several initiatives have harnessed the potential of mHealth to improve the retention and satisfaction of

health workers deployed to remote areas through improved linkages to the central health system and to

communities, greater efficiency and enhanced support.

6. Conclusion

As we have seen, mHealth has the potential to overcome geographical barriers, and with the rapid

uptake of mobile technologies we can imagine a world where everyone, including the most marginalised

and remote groups, will have access to a mobile phone. In the context of universal health coverage

(UHC) and a post-MDG agenda mHealth could play an essential role in increasing coverage and

enhancing equity. We have also seen how mHealth can contribute to better quality of care and can

enhance the efficiency of those approaches that offer financial protection.

In the words of Julio Frenk, Dean of the Harvard School of Public Health Faculty, “The use of cellular

phones for health care and public health is one of the most promising developments in the quest to

achieve universal health coverage worldwide, because mobile phones are rapidly becoming the

communication technology of choice—and increasingly so among the poor.”xxv

Contacts:

Nicolas de Borman Nicolas Avril Hajo van Beijma

Blue Square Options Consultancy Services Text to Change

[email protected] [email protected] [email protected]

Page 18: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

16

Case Study 1: Web-technologies for RBF efficiency, accountability and transparency

in Benin and Nigeria Benin and Nigeria1 use web-technologies for managing their RBF systems. Both countries use OpenRBF, an open source software developed within the Africa PBF community of practice2. In both countries, web technologies are used primarily for RBF management. The cloud based software helps RBF teams with their daily data management activities:

Service provider configuration and management Contract management Decentralised data entry Strategic purchasing through subsidies and target management Budget management RBF payment calculations and bank transfer orders Data exports

However, both in Benin and Nigeria, these web-based systems contribute to the broader health system stewardship. In Both countries, RBF data is shared publicly on the Internet. Providers, district teams or ministry of health staff can access the performance of any entity of the health system. In both cases, all RBF data can be downloaded on the Internet. Figure 1: Open RBF Data in Benin

1 www.beninfbr.org, www.nphcda.thenewtechs.com

2 www.openrbf.org.

Page 19: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

17

Immunization provides an example of how RBF data can support the management of the overall health system. In Nigeria and Benin, RBF systems collect data on the (verified) quantity and quality of immunization services provided in each health center. This data is of high value for the management of the national immunization program. In sharing this data on the web, the RBF program improves the quality of the data available for decision-making and contributes to a better immunization program in the country. Figure 2: on-line verified immunization data, Adohon centre de santé, Benin.

Figure 3: On-line heat map for quality, including immunization activities, Adohon centre de santé, Benin.

Page 20: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

18

Both countries use web technologies for benchmarking the performance of health care providers. Nigeria has developed an online benchmarking tool that compares standardised data of any entity of the RBF program3. In Benin, the online benchmarking is used as an instrument to highlight the best performers and create a non-financial incentive among health care providers. Figure 4: Highest quality scores in Benin, November 2013

Finally, in both Benin and Nigeria, web technologies contribute directly to good governance and accountability. In both cases, financial data is available on the Internet. This transparency enhances trust in the RBF instrument and creates an example of good governance in the health sector. Figure 5 : online transparency of RBF payments, Benin

3 Available at https://nphcda.thenewtechs.com/data.html

Page 21: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

19

Case Study 2: Supporting social accountability and verification in Uganda

Maternal mortality is still a major problem in Uganda: only 42% of births are attended by skilled health staff, contributing to a maternal mortality rate of 440 deaths per 100,000 live births (compared to the equivalent rate in the Netherlands of 13 maternal deaths). In 2010 there were some 4,700 preventable maternal deaths, many of which could have been avoided if the right information could have been delivered to the right people. The project: The Uganda National Health Consumers Organisation (UNHCO) together with Cordaid, IICD and Text to Change (TTC) started a campaign aimed at reducing maternal mortality. This mobile phone-based pilot programme was designed to reach 3,000 people in three districts of Uganda (Lyantonde, Kamuli and Luweero) and is focused on the collection of data and on obtaining and responding to feedback on health services. Village Health Team (VHT) members are trained to enroll community members in the text messaging platform. Once the community members are registered they receive messages with information to raise awareness on different aspects of maternal health, for example danger signs during pregnancy, nutrition and patients’ rights. Registered mothers receive periodic reminders to have antenatal check-ups. Importantly, community members also receive health questions via SMS on their experience of receiving services (client satisfaction) which they can respond to using their mobile phones. A combination of mobile technology and other media: In addition to sending out and receiving text messages, the programme also airs radio shows consisting of two talk shows per month, during which community members can discuss their concerns regarding maternal health. The feedback that has been received from registered members via SMS is also discussed. A community based intervention is focused on enabling community dialogue meetings where issues, concerns and feedback are discussed at a community level. The use of combined media has led to significant increases in audience for the campaign. Some results: In 2012, the project reached 254 mothers with SMS on different aspects of maternal health. The messages were received in a local dialect called Luganda. More than 200 feedback messages were received via the SMS platform from people in the three districts and beyond. People commented on the services received from the health centres, the limited number of doctors, their need for maternal health rights information, and requested more information on maternal health and ways to fight malaria. It was observed that using mobile technology provided the respondents with the privacy to express themselves freely on several issues. Via the TTC call centre, over 300 patients were interviewed to assess the level of patient satisfaction with the health centres. In 2013, with continued support from Connect4Change, the programme continued to use SMS as an innovative tool to share maternal health messages.

Page 22: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

20

Case Study 3: eVouchers in Ethiopia

The project: In 2012, MSI Ethiopia engaged in a subsidised voucher programme for Family Planning (FP).

The project aimed at distributing vouchers for free, targeting young (people aged 15 to 29 years) from

poor and marginalised communities and with a focus on increasing the uptake of IUDs. The project was

initially piloted in five towns (Assela, Nekemte, Debre Markos, Shashemene and Dessie) with three

Community Health Worker (CHW) distributing vouchers in each town for 18 weeks between August and

February 2013. Vouchers were valid for 14 days and could be redeemed in any of MSI Ethiopia’s centres

in the five towns. Based on the positive results of this pilot the project has been scaled up to 20 towns

and is currently being rolled out to 25 Blue Star franchises.

The problem with paper vouchers: Most voucher programmes currently use paper vouchers (Figure1).

Although paper vouchers have the added value of being tangible and can feature key information about

the project and key health messages, they are also easy to lose or damage. Paper vouchers are also

associated with high implementation costs. They need to have a rigorous design, be printed, stored and

then distributed. Most vouchers will be printed on secured paper and have a unique voucher code for

fraud control, which adds to the cost of production

Figure 1: Example of a paper voucher for SMH and FP implemented by Options in Yemen

In addition, paper vouchers also require a multi-stage monitoring process to ensure used vouchers are

adequately verified and then reimbursed. This is a process which can be cumbersome and take time,

potentially causing a delay between the moment a provider submits a claim and the time they are paid,

which can have a detrimental effect on providers’ motivation and adherence to the scheme. This can

also lead to inefficiencies and data collection errors as a CHW would usually collect used vouchers and

service data from the providers and return them to a central office where they are manually checked for

authenticity before authorising payments.

The solution: In an aim to address some of the issues, MSI Ethiopia decided to pilot eVouchers; vouchers

that are based on a unique code which can be sent to a client’s mobile phone or written on a piece of

paper if the client has no phone. Codes written on a piece of paper still hold many of the advantages of

an eVoucher as demonstrated below. It was expected that eVouchers would make it easier for youth to

Page 23: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

21

redeem vouchers as they tend to have greater access to mobile phones, and would reduce the cost of

managing a voucher programme whilst also simplifying the monitoring process.

How it works and how new technologies are imbedded in the project: Figure 2 shows where MSI

Ethiopia integrated new technologies in the voucher cycle. An initial step of operating a voucher

programme is the enrolment of beneficiaries through voucher distribution. CHWs distributing the

vouchers in this project were equipped with smart phones configured with an application, XForm,

customised for the project and which allowed collecting beneficiary data, transmit the information to

the online system and issue eVouchers. After receiving training on how to use XForm, CHWs went from

house to house to raise awareness of the programme and enrol potential beneficiaries.

Figure 2: Where MSI Ethiopia has successfully integrated new technologies in the voucher cycle

This is the same principle as how one would enrol participants in a conditional cash transfer (CCT). For

example, in the state of Bihar, India, where a state-wide CCT is implemented to incentivise women to

deliver in a facility and contribute to the cost of transport, community mobilisers are equipped with

phones configured with the CommCare platform to enrol beneficiaries.

Page 24: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

22

In Ethiopia, the smart phone CHWs are equipped with also performed some limited validation of

beneficiary data. If incorrect data is entered (e.g. wrong phone number or potential beneficiary not in

the right age group), the software prevents the CHW moving to the next field. Once all the data from a

beneficiary is collected, the phone transmits the data via SMS or via the internet to the online system. If

no network is available, the phone will store the data and send it as soon as it is connected to a network.

When the online system receives the data, it automatically checks it and sends the CHW an error

message if there are inconsistencies in the data (e.g. wrong CHW ID number is given). If all the data

entered is correct, the beneficiary receives an eVoucher on their phone. If the beneficiary does not have

a phone, the voucher is sent to the CHW who can write down the unique voucher code on a piece of

paper and give it to the beneficiary. The use of mobile technology for enrolment and distribution

minimises erroneous data entries and to some extent constitutes an initial form of fraud control and

verification. It also allows receiving real time data on the voucher, thereby ensuring that a voucher

distributed is immediately activated and can be almost instantly redeemed. This improves the

monitoring system compared to paper vouchers, where there can be delays between distribution of the

voucher and activation of the voucher. This means that if a client uses the voucher almost immediately

the system flags it as fraud or as an error, because the voucher is being redeemed before it is officially

activated.

To redeem the voucher, clients present their voucher code on their phone or on a piece of paper to the

provider, who sends the code via SMS to the online system. The online system checks the code to

ensure it is active, still valid and has not already been redeemed and sends back a confirmation SMS

confirming the authenticity of the voucher. This contributes to the fraud control mechanisms of the

project. If the voucher is authentic, the service provider delivers the services to the client. Once the

service has been provided, the provider sends another SMS to the online system with the voucher

number, the service provided and the provider’s unique ID number, to facilitate monitoring. This last

SMS is also linked to claims processing and payments as it triggers the reimbursement to the provider

for the services delivered, provided there have not been any inconsistencies or errors flagged by the

system. Payments will be discussed in greater detail in the case study on mobile money (see Case Study

3).

Results, challenges and lessons learned: Of the 2,521 eVouchers distributed during the pilot phase,

1,278 (51%) had been redeemed, which compares well with other paper voucher projects. The project

successfully targeted people from the 15 to 29 age group, with 92% of clients redeeming the eVouchers

being of that age group. Based on four proxy indicators, the pilot appears to have also successfully

targeted youth from poor and marginalised groups. However, this data is limited to 18 weeks

implementation and does not identify which results are directly attributable to the eVouchers compared

to paper vouchers. Nonetheless, the project did address many of the administrative inefficiencies of

paper vouchers and based on these results the programme has been scaled to an additional 20 towns

with 18,095 eVouchers distributed and 8,278 (46%) redeemed. Lessons learned from this pilot include

the fact that clear roles and responsibilities should be made explicit. Indeed, some providers

occasionally did not send the confirmation SMS to say an eVoucher was redeemed because of unclear

roles and responsibilities. This meant that some data was sent after the expiration date of a voucher and

could have appeared fraudulent, leading to delays in payment. Guidelines of roles and responsibilities

have been developed since. Furthermore, eVoucher codes had to be written on paper by CHWs more

often than expected as only 24% of the targeted group had a mobile phone. Finally, using eVouchers has

Page 25: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

23

simplified monitoring and having access to real time data has also enabled the management team to

monitor performance of distributors without having to undertake costly and time consuming field visits.

This was particularly helpful when the management team was able to immediately follow up with a

health worker when the system reported a drop in distribution of vouchers.

Some cost implications:

Component Cost

Developing the smart phones and online system $47,000

Monthly payment to IT company for hosting the system

$645 / month

Individual cost of smart phones used (Mini Galaxy) $200

Average cost for distributing one voucher $4.50

Page 26: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

24

Case Study 4: Mobile Money for payments in RBF

What is mobile money (mMoney) and how does it work: the case of M-PESA in Kenya

Mobile money is a way of transferring funds via mobile devices and particularly mobile phones. The first

countries to have implemented it were Japan and Norway on a small scale, but mMoney really took off

when Safaricom (Vodafone) in Kenya developed M-PESA (which stands for mobile ‘pesa’ – money in

Kiswahili). M-PESA was entirely developed by Kenyans and was originally designed as a service which

would allow microfinance borrowers to conveniently

receive and repay their loans using Safaricom’s

network of airtime retailers. However, when the

service was introduced, customers very quickly

started utilising it for a variety of alternative uses,

including sending money to their homes across the

country and making payments.

The way it works is relatively simple but relies on a

strong network of airtime retailers. A customer needs

a simple GSM phone and can register free of charge at

any M-PESA agent in the country. Once registered, the

customer receives an updated menu on their phone

from the network which will allow them to receive

and to make payments. To load money onto their

accounts, customers either hand over cash to an M-

PESA agent, who loads the equivalent amount onto

their phones, or they receive a mobile payment. To

send money, the customer goes to the menu on his or

her phone, enters the amount to be transferred and

the recipient’s phone number and payment is made in

the time it takes for an SMS to be delivered. There are two ways to retrieve cash; either by cashing in a

desired amount at a Safaricom kiosk or, the system can generate a code to be used at an ATM. For each

transaction Safaricom will retain a fee, depending on the total amount transferred and whether the

money is being sent to another mobile customer, being cashed in at a kiosk or cashed in at an ATM. M-

PESA has taken off to a huge extent in Kenya, with 15 million users reported as of May 2012.

Mobile money has opened a world of opportunities, particularly for those without bank accounts. More

than 60 countries have some sort of mobile banking initiative and one out of ten of these initiatives will

have over one million customers.

How mMoney can support CCT projects: improving timely immunisation in rural Kenya

In 2011, the Kenya Medical Research Institute piloted a conditional cash transfer (CCT) to incentivise

women to bring their child for immunisation on time in 30 villages of Siaya district, Western Kenya. The

Page 27: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

25

idea was borne out of the fact that, despite improved coverage due to increased vaccine supplies,

lingering barriers to optimal immunisation were not being successfully addressed by supply-side

interventions. Demand-side barriers to immunisation appeared to be becoming more important. In the

study, only 54% of children had received the third dose of pentavalent vaccine by 24 weeks, even

though this is scheduled to be given at 14 weeks, and only 83% of those aged to 12 to 23 months had

received the third dose in 2010. Kenya’s Expanded Programme on Immunisation guidelines require

primary vaccination to be carried out at 6, 10 and 14 weeks of age. Mothers with children aged 0 to 3

weeks were enrolled in the programme by village reporters who informed them that they would receive

Ksh 150, or the equivalent in airtime, for bringing their child on time for immunisation. The CCT amount

was based on the standard reimbursement cost for transport for a round-trip to the clinic, used by all

studies conducted in the area. At point of enrolment, 72 mothers were asked the name and date of birth

of their child and a mobile number they could be reached on. Village reporters then sent an SMS

message with that information to a server which automatically scheduled an SMS to be sent in advance

to remind the mother of the immunisation due date for their child (Figure 1). A first reminder was sent 3

days before immunisation was due and a second reminder was sent on the day of immunisation.

Figure 1: CCT using mMoney to improve timely immunisation in rural Kenya

When mothers presented themselves with their child at the clinic for immunisation, the provider sent an

SMS to notify the server of the child’s visit. The server would than automatically send a message to the

Page 28: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

26

mother congratulating her for vaccinating her child and telling her that she would receive Ksh 150 if she

had brought the child for vaccination on time. Another SMS was simultaneously sent to the study co-

ordinator informing her as to whether the child had been immunised on time or not. Mothers who had

taken their child to vaccination on time were immediately sent Ksh 150 to their registered mobile

money account, or the equivalent amount of airtime was sent to their registered phone. Of those

receiving mMoney, 62% received the cash on the same day, 27% within 3 days and 11% more than a

week later. Six mothers reported never receiving the transfer.

How mMoney can support payments to providers:

In 2010, Marie Stopes Madagascar (MSM) implemented a family planning voucher programme for

family planning services in two rural regions; Itasy and Bongolava. Vouchers could be redeemed in any

of MSM’s 42 social franchises in those regions. However, payments to providers became an issue as

several did not have a bank account or access to a bank. As of July 2011, only three bank branches were

open in the two project regions. This meant that MSM staff would have needed to travel across an area

totalling 23,000km2, with large sums of cash, to reimburse the 42 service providers. This was not only

inefficient but posed a security and fraud risk. However, the three mobile network operators in

Madagascar all offer mobile money payments to their clients and in July 2011 there were twice as many

mMoney kiosks than banks across the country. This was perceived as the perfect opportunity to test

mMoney for reimbursing service providers.

When voucher clients came to redeem their family planning voucher, the provider would send an SMS

to MSM’s online database, providing a unique code printed on the voucher and details of the service

provided. The system verified the provider’s mobile phone number and the validity and authenticity of

the voucher before automatically reimbursing the provider via mobile money. Not only did this method

avoid the need to travel long distances with large sums of cash but it also sped up the reimbursement of

providers. Of the 1,737 claims for reimbursement sent by social franchisees, 35% were paid within 48

hours. The system was also easily adopted by providers. However, one problem encountered by the

programme was that, out of the three network providers, one had a policy which only allowed payees to

cash-in their mobile payment at a kiosk within a week of the payment being made, after which the

company kept the total amount.

In Nigeria a slightly different model (Figure1) was used to pay providers. Service providers, in this

instance transporters who took pregnant women to a facility in exchange for a voucher, sent a voucher

code to a server with their unique driver ID and received a PIN in exchange. The transporter could than

go to an ATM with his PIN and his driver ID and receive payment in cash.

Page 29: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

27

Figure1: A variant of mobile money

The importance of context to mMoney:

Mobile money has incredible potential and is one of the fastest growing areas in mobile technologies.

However, close attention should be given to context to assess the feasibility of using mMoney. The

following section outlines examples of how context can affect the use of mobile money.

mMoney in Mozambique: Mozambique has seen the introduction of mMoney in recent years and

currently has at least two mobile money providers; M-PESA and M-KESH. However, mMoney has been

very slow to pick up in the country, which is linked to macro-economic issues. There is little liquidity in

the country, which means that when a customer wishes to retrieve money, very few kiosks hold enough

funds to make the payment. There are also a limited number of kiosks where money can be uploaded or

retrieved, contributing to a lack of trust in the system.

mMoney in the Bihar CCT: The safe motherhood conditional cash transfer (CCT) in Bihar originally

planned to use mMoney to pay beneficiaries, but the policy environment did not allow it. When the

suggestion was made to use mobile money, the Government of Bihar insisted that phone operators

were not banks and each had distinct roles: mobile operators transfer calls, and banks transfer money.

mMoney for people who do not have phones: In Pakistan, a joint venture between a bank and a phone

operator led to the development of Easypaisa which allows people without phones to transfer mobile

money. The principle is very much the same as M-PESA, except customers visit a kiosk with cash and the

Page 30: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

28

kiosk owner takes the money and sends the equivalent (less a fee) via his mobile phone to the desired

kiosk. The transfer recipient then visits this kiosk with a secret PIN communicated to him by the sender

and retrieves the money, without using a phone.

These examples show how important the economic and policy environment can be to the success of a

project using mobile money. However, the Pakistan example also shows that mobile money does not

necessarily require all of its users to have a phone.

Page 31: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

29

Case Study 5: Scaling up mHealth in Tanzania

At the request of the Ministry of Health and Social Welfare (MOH) in Tanzania, Text to Change (TTC),

together with the Tanzania mHealth partnership, supported the launch of a nationwide Healthy

Pregnancy, Healthy Baby text messaging service In December 2012. This service targets pregnant

women, mothers with newborn babies, their partners and general information seekers. The service

supports the multimedia nationwide healthy pregnancy campaign 'Wazazi Nipendeni' (Swahili for

Parents Love Me) which aims to empower pregnant women and their partners to take the necessary

steps for a healthy pregnancy and safe delivery.

The SMS platform that TTC set up enables pregnant women, birth supporters and others interested in

safe motherhood to receive information and reminders about healthy pregnancy via their mobile

phones. The target beneficiaries can send the word “mtoto” (‘child’ in Swahili) to a specified toll free

number, which is free of charge on all mobile networks. After registering, users receive a text message

covering a comprehensive range of safe pregnancy and early child care information. The Ministry of

Health and Social Welfare approved all messages and schedules them to specific weeks or months of the

pregnancy or age of the baby.

The text messaging service integrates key safe motherhood health topics, including early and complete

antenatal clinic (ANC) attendance, malaria prevention, individual birth planning, and prevention of

mother to child transmission of HIV/AIDS (PMTCT). Although maternal, newborn and child health has

improved in Tanzania over the last few decades, Tanzanian women still face an unacceptably high risk of

preventable morbidity and mortality during their reproductive years.

Scalability of the TTC platform: TTC has been the lead technology partner in the mHealth Tanzania

Partnership and designed and implemented the technology involved. The text messaging service uses

an open-source platform called Vusion which is developed in a partnership between the Praekelt

Foundation and Text to Change. Its scalability and flexibility make it a unique platform which can be

operated in all countries around the world. The platform has been designed to enable campaign

managers to configure complex SMS programs without the requirement for code changes or

development support. This means campaigns can be set up within hours or days, instead of months or

years.

The Healthy Pregnancy, Healthy Baby text messaging services is a good example of an mHealth

campaign that went to scale due to strong leadership by the Ministry of Health (MOH) from the outset,

and a combination of the right partnerships, right timing and relevant content. “I’m really proud of the

TTC technical staff and our state-of-the-art mobile platform Vusion, which makes it possible to set up

large-scale campaigns like this, in a relatively easy manner” says Bas Hoefman, founder and Director of

Text to Change. Wide participation by key stakeholders, including potential beneficiaries, informed the

design of the content of the messages and the service was also tested and refined many times over until

the respondents (women in Tanzania with a cellphone) could understood perfectly and were able to

respond to the content.

Some results: The Wazazi Nipendeni Text Message Service reached 100,000 subscribers in a period of

15 weeks. From the launch in December 2012 to March 2013, the service had reached 100,000 active

Page 32: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

30

registrants, and more than 4 million text messages had been sent to those who signed up for the free

healthy pregnancy and safe motherhood information.

Page 33: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

31

Annex 1: Examples of selected mHealth interventions

These projects have been showcased for a variety of reasons including that they have had successful outcomes, been scaled-up or are scaling-up,

or demonstrate the diversity of projects being implemented with mobile technologies. Nonetheless, by showcasing the projects below, we are

by no means ignoring the fact that there may be more successful and larger projects.

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

Education and Awareness

Mobile Technology for Community Health (MoTeCH) Country: Ghana Status: Scaling up References: iv, xxvi, xxvii

Launched in 2010, MoTeCH aims to use mobile phones to increase the quantity and quality of prenatal and neonatal care in rural Ghana. To do so, two interrelated mobile health services were developed: 1) The ‘Mobile Midwife’ application: sends informational messages via SMS or pre-recorded voice messages to pregnant women or families according to their gestation periods in their own language. 2) The ‘Nurse Application’: enables Community Health Nurses to electronically record on a mobile phone care given to patients and identify women and newborns in their area that are due for care. The two services are linked so that if a patient has missed treatment, the Mobile Midwife service sends a message to remind the patient to go to the clinic for that particular service and the nurse is also informed that the patient is due for treatment.

Achievements: As part of the Mobile Midwife programme in Ghana, MOTECH covers 46 facilities and over 1,100 active mobile midwives, and has more than 21,000 people registered. It has sent nearly 58,000 SMS and voice messages and uploaded over 134,000 patient records. Challenges:

Lack of “senior experience” from local software developers meant that they struggled to meet deliverables on time and had issues with the quality of the code.

The need to connect to multiple telecommunications’ servers delayed establishment of connection between servers by a couple of months.

Interviews with 90 clients of mobile midwife found that 72% found it difficult to access the MoTeCH system via IVR (Interactive Voice Response).

Using low cost Java enabled phones were more suitable than those phones only compatible for SMS. This was because they had greater capacity to enter data and it was cheaper to transmit data.

Content needs to be relevant to the local context, tailored to the interests of the mothers, and delivered in the local language in order to have more impact, particularly in rural settings.

It is advised to integrate the project into the health system.

Due to poor network reliability it is advised that phones which can store data until it is appropriate to send are used.

For software development it is advised to build up local expertise by partnering local software developers.

Page 34: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

32

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

Education and Awareness

Text to Change Country: Uganda Status: Scaled up References:iv, xxviii

Text to Change (TTC) is a health promotion campaign that uses SMS quizzes to test mobile phone users on their HIV/AIDS knowledge, refer users to facilities where they can be tested for HIV, and collect sex, age and location data of the caller.

Achievements: TTC ran a pilot in 2009 in Uganda with Health Initiatives for the Private Sector. Following the success of the pilot, TTC implemented the second phase of the programme with Kakira Sugar in Uganda. The aim of which was to improve Kakira Sugar’s communication with employees and increase access to health information and services for their employees and their families. The programme targeted 3,000 Kakira Sugar employeers and 7,500 farmers who supply sugar cane. The results of the programme included:

At least 200 people responded to HIV testing

Over 100 people sought other health services after the first awareness campaign.

Adapt content to local contexts, such as

language, for successful uptake.

Ensure there is stable funding.

Establishing partnerships with likeminded

organisations focused on ICTs for

development is important for a successful

outcome.

Data collection

Ushahidi (meaning testimony) Countries: Kenya, Haiti, Uganda, Malawi and Zambia Status: Unknown References: xxix, xxx, xxxi, xxxii

Ushahidi provides an open-source platform for collecting data via SMS, individual reports from users, the web, and email and provides tools for translating, classifying, and georeferencing these reports. Submission could also be via voice message—essential for illiterate users. The information is aggregated and presented on a map-based interface accessible via web and mobile phone (google maps) i.e. over the internet. The Ushahidi platform has been used worldwide, examples include:

Wildlife tracking (Kenya).

Tracking medical supply stockouts: Kenya, Uganda, Malawi and Zambia.

Achievements of the Ushahidi Haiti Project (UHP):

The UHP addressed key information gaps such as situational awareness and information with a relatively high geographic precision.

An estimated 40,000 to 60,000 reports were processed through UHP and 3,584 events were mapped in Haiti.

Challenges:

It was difficult to verify reports from the platform. For example only 202 out of the more than 3,500 messages published on the crisis map were tagged as “verified” and that was mostly based on online media reports.

Technological limitations were experienced such as outdated computers, difficulties accessing the UHP website and data streams due to browsers and internet security policies, and limited bandwidth.

To address the challenge of verifying information, it is recommended that a system is established to rapidly identify inaccurate information from crowdsourced reports. Some examples of these systems include: 1) “Bounded crowdsourcing”: This refers to

the use of reports only from a network of trusted participants.

2) Hybrid model: this is where reports are accepted from trusted participants, as well as the general public.

3) Swift River: a tool being developed by Ushahidi. It organises incoming reports on a platform where users rate the accuracy of information.

Page 35: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

33

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

Disaster response: 4636 project following Port au-Prince earthquake in Haiti in January 2010.

Data collection

Commcare for antenatal care services (ANC) Countries: Nigeria Status: Unknown References: iii

Since November 2012 Commcare has been used by Pathfinder in Abuja and Nasarawa State in Nigeria to support Community Health Extension Workers (CHEWs) to record patient data, track ANC clients over time, and use multimedia audio counselling clips during group health talks. Created by Dimagi Inc, Commcare is an easily customised mHealth platform for health workers. Commcare replaces paper based registers and patient education flipcharts, by enabling CHWs to register patients on customised electronic forms on midrange open-source software. Commcare then allows CHWs to automatically submit the data collected to a central cloud server called CommCareHQ. This can be done in ‘real time’. This data can then be accessed by supervisors and programme managers and is securely backed-up and private.

Achievements:

In Nigeria more than 150 CHEWS from 20 primary health centres are using Commcare to track ANC clients: more than 2,400 pregnant women have been registered and tracked via Commcare since December 2012.

Findings of work conducted worldwide (14 published papers, six grey literature reviews, and four papers on related systems) demonstrate that Commcare can is able to increase the timeliness, accuracy and relevance of information delivered to clients..

In order to encourage ownership, it is important to engage the Ministry of Health from the start of the project.

Be aware that the implementation of the platform is required to be tailored to the needs of smaller and bigger sites.

Must remember that creating a one solution for high and low performing frontline health workers is complex.

The design of the programme needs to be evidence based.

Point of care support

Using Mobile Phones to Strengthen Microscopy-Based Diagnostic Services

A feasibility study undertaken in Uganda tested the use of images of microscopy samples taken with mobile phone cameras fixed with a prototype connector to a microscope. Microscopy samples tested included Malaria,

Achievements of the pilot:

Clear images were taken using mobile phone cameras which ranged from having 2 to 5 megapixels e.g. Nokia 6300 and Samsung E900.

Malaria parasites were clearly visible on the mobile phone screen and to the extent that

In order to address the problem of laboratory staff not having mobile camera phones with java and internet subscription, it is advised that a camera phone is supplied to the laboratories through programme funding or government system. Phone restrictions

Page 36: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

34

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

Country: Uganda Status: Pilot phase References: xxxiii

Tuberculosis, Common Stool Parasites, and Bacterial Vaginosis.The images were then sent via an information technology platform designed to transfer the data from the mobile phone to a website, as well as send feedback by text message to the end user.

the parasite stage could be identified.

Forms detailing the ID, sample type, health centre data and images taken were successfully filled in and sent by 3G mobile internet to the central platform accessible by a computer connected to the server.

Challenges:

Mobile phones with a camara lower than 2 megapixels e.g. Sony Ericsson K320i did not produce a clear image.

Sending data via Bluetooth and multi-media messaging services (MMS) proved difficult. Bluetooth could only send information 10 metres away and MMS did not work at all in rural settings, and even in Kampala it was slow, unreliable and expensive.

Some diseases were more difficult to interptret, such as tuberculosis bacterium which was difficult to interpret due to their small size and, specifically in Uganda, the poor connection.

should be put in place to prevent excessive and inappropriate use).

If the mobile phone’s zoomed images are used for data transfer there resolution would be limited, and therefore it is not recommended.

Point of care support

CommCare for Home-Based Care (HBC) Country: Tanzania Status: Unknown Reference:xxvii

Commcare has been used by Pathfinder in Tanzania to assist home-based care providers (HBCP) to screen for common problems, manage their visits and referrals, collect data and report on activities and outcomes. Commcare is a phone-based tool that helps to facilitate the collection and transfer of data (as described above), as well as provide decision making support at the point of care. In this example, Commcare uses guidelines for care and data collection that

Achievements:

The project began in October 2008 in two districts in Dar es Salaam with more than 300 HBCPs.

Findings from qualitative interviews during the early phases of implementation wer positive of the phone-based system. They were seen as better for maintaining privacy and were less vulnerable to being destroyed compared to paper based records.

The system improved reporting time compared to the paper based system.

The use of SMS reminders to HBCPs and their supervisors led to a reduction of 86% in the days that client visits were overdue (9.7 to 1.4

Ensure that the mobile application is simple in order for quick and easy use.

Limit the number of SMS messages sent to a user in order to ensure that they are not overwhelmed.

SMS reminders are more effective when supplemented by calls from supervisors.

In order to reduce duplicating reporting ensure that a plan is agreed with the government and partners early on.

Field capabilities can be improved with the use of local champions or ‘superusers’.

Page 37: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

35

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

HBCPs are trained in to support the delivery and management of their work e.g. providing HBC for those living with HIV/AIDS; counselling and testing; screening for TB in the community; and Family Planning counselling. The application provides this support by:

Offering HBCPs a checklist of activities to be performed during home visits and reminders of appointments;

Giving HBCPs access to electronic registration forms for new clients, forms for existing clients, referral options, and reminder options for pending referrals and visits to patients;

Proving HBCPs with a list of steps for counselling and information on the benefits, side effects, and effectiveness of family planning methods.

The tool runs off-line on the HBCPs java-enabled phone. The HBCPs send their data to the server on a daily basis using general packet radio service (GPRS).

days).

Patient monitoring and support

InStrat’s One World Doctor (OWD) remote patient monitoring system

The system enables clients to use SMS technology to monitor patients with chronic conditions remotely and to deliver appropriate health management and wellness messages. Enrolled patients text via SMS

Achievements of pilot study (March 2011 – September 2011):

A leading client was able to get approximately one third of moderate to severe hypertensive patients under control.

Direct network negotiation may have offered a more stable platform versus an intermediary provider.

Customers are slow to warm up to the service but once they recognize the benefits, they are committed to using it.

Page 38: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

36

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

Country: Nigeria Status: Scaling up References: xxxiv, xxxv

readings and vitals to a provided number. InStrat then monitors recorded trends and alerts patients’ doctors and caregivers (via SMS) if readings go beyond thresholds. In addition, doctors, care givers or even patients can retrieve historical readings by SMS or via the web- based portal, which is useful to inform management decisions. Also, nurses or doctors can be alerted when patients fail to send in their readings at the appointed time. Conditions covered include hypertension, diabetes, asthma, and pregnancy management.

Challenges:

The unstable mobile network operator business models: changes are often made to network operations that disrupt the service without any prior notice. This makes it seem like the platform is unstable.

Patient monitoring and support

Project Mwana Country: Zambia Status: Scaling up References:iv, xxxvi, xxxvii,xxxviii

Project Mwana aims to use SMS technology to send early Infant HIV diagnosis results to community health workers. The project used RapidSMS to develop two applications ‘Results 160’ and ‘RemindMi’. ‘Result 160’ allows HIV test results to be communicated from a laboratory in a timely, efficient way through SMS to health facilities which collect the samples. ‘RemindMi’ is used to improve the rate of postnatal follow-up by reminding mothers through SMS to return for their six-day, six-week and six-month postnatal visits as required

Achievements: A pilot study that took place in Zambia’s Southern Province between 2010 and 2011 compared baseline data from 10 study facilities of turnaround times for the postal notification of results to turnaround times of results sent via the ‘Results 160’ SMS system. The results include:

The mean turnaround time to receive the result notification at a health facility decreased from 44.2 days pre-implementation to 26.7 days post-implementation.

A mere 0.5% of the investigated reports sent via SMS differed from the corresponding paper reports.

Challenges:

Problems with servers crashing, which meant that there was a delay in result delivery.

It is important that the needs and results to be achieved are agreed when establishing partnerships. In order to address challenges of ownership and coordination among the partners, it is advised to convene partners on a regular basis to develop the project strategy, and make sure that there was only one coordinating body and one technical working group led by the Ministry of Health.

Page 39: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

37

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

in Zambia. The application also enables community health workers to register births.

The web-based interface that used to manage the system was slow to being adopted.

Lack of appropriate system management tool meant that it was hard to support facility staff and resolve any problems during the pilot.

Limited continuity and consistency of management personnel.

Delays in data entry at laboratories which led to delay in results being delivered.

Two out of ten facility phones were lost in the pilot. But resolved by switching to personal phones.

Logistics SMS for life Countries: Tanzania, Ghana, Kenya, DRC, Chad, and Cameroon Status: Scaled up References:iv xxxix, xl, xli, xlii

SMS for Life is a combination of mobile phones, SMS messages, internet and mapping technology to monitor and manage weekly stock levels of artemisinin-based combination therapy and quinine levels. In 2009, a 21 week pilot of SMS for Life was implemented in three districts of Tanzania: Ulanga, Kigoma Rural and Lindi Rural covering 129 health facilities and 226 villages. SMS for Life has now been extended to all 5099 health facilities in Tanzania and pilots are taking place or have taken place in Ghana, Kenya, DRC, Chad, and Cameroon.

Achievements:

Results from the pilot conducted in Tanzania showed that the system reduced the likelihood of stock-outs and access to drugs increased for patients. For example, when first launched in Tanzania in 2009, 78% of participating facilities had a stock-out of one or more of the five anti-malarials. By the end of the pilot, this had fallen to 26 % of participating facilities.

Having access to real time data on stock levels greatly helped improve drug management at the district level.

Accurate information on stock levels can be collected from health facilities in rural areas using simple SMS technology. For example, the pilot had an average weekly response rate across all three pilot districts of 95 % and the message formatting errors came to an average of 7.5 % per week with almost all being fixed and re-sent within the time limits.

Local ministries of health should commit early on in the development of the project to taking over the system once it has been rolled out and running it for a period of time until it becomes integrated within the mainstream health system.

Although it can be challenging and time-consuming, establishing public-private partnerships between ministries of health and mobile phone companies will be essential to the implementation and roll out of the project.

In country presence is important and therefore discussions must be had early on concerning funding and funding sources.

Prior to implementing the pilot, it is important to review evidence-based solutions from publications and other pilots, as well as conduct a cost analysis of full country scale up.

Page 40: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

38

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

Logistics mPedigree Countries: Ghana, Nigeria and Kenya Status: Scaling up References:iv, xliii, xliv

mPedigree allows users to instantly verify the safety and efficacy of the drugs that they buy using a mobile phone. Users send via SMS a scratch card code on the drug package to a database at no cost. A reply is then sent to the phone of the user to indicate the authenticity of the drug.

It has been successfully piloted in 6 African countries and has contributed to the steady recovery of more than $200 million that pharmaceutical companies lose daily.

Support was obtained by identifying collective interests, such as loss of revenue for pharmaceutical companies and governments.

Establishing partnerships between drug manufacturers, pharmacists, marketers and regulators is important to creating a sound marketplace.

Training and dissemination of knowledge

Mobiles for Quality Improvement (m4QI) Country: Uganda Status: Completed References: xlv, xxvii, xlvi

The m4QI pilot was completed in July 2011. It aimed to develop and test how technology could be used to support performance improvement by helping identify performance gaps in adherence to clinical protocols, create a platform to manage and automate the delivery of text message reminders and quizzes to address the gaps, and also produce data to improve effectiveness of supportive supervision and follow up. The platform created was Frontline SMS: Learn. This is an adapted version of Frontline SMS chosen for its ability to operate without internet. Frontline SMS: Learn maintains a databank of messages which enables the delivery and receipt of text messages of instructions, tips, and quizzes on infection prevention, client care, and adherence to standards and guidelines.

Achievements:

Of 3,449 messages sent to project participants 86.5 percent were received.

A total of 251 messages were received from participants in response to questions sent.

Post-pilot interviews with participants highlighted high acceptability of text messages for performance improvement.

Though response rate was low (at 19%), evidence from structured interviews at the end of the pilot indicated that participants felt motivated by the reminders to adhere to hand washing guidance; refer to training manuals when they received a quiz about protocols; re-learned processes of instrument sterilization; and used advice on pain management.

Challenges:

Feedback from participants highlighted lack of message clarity and technical problems.

Problems with the modem meant that response rates from participants to questions they’d been sent decreased.

When introducing technology supported interventions there is a need to dedicate human resources for the provision of on-going IT support and problem solving.

Stakeholders should be involved in the planning and budgeting of mobile learning initiative.

Training is important in technology programmes in order to ensure that responses conform to formatting procedures.

When responding to messages is at a cost it is more appropriate if prepaid airtime is provided rather than reimbursing respondents after they have used their own credit to send responses.

Training and

Electronic Integrated

A study conducted in 18 health facilities between May 2008 and

Achievements:

For all ten areas assessed in both systems, the

Unknown

Page 41: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

39

Category Name of project, country, status & references

How does it work Key achievements and challenges (where information is available)

Key lessons learned

dissemination of knowledge

Management of Childhood Illness (eIMCI) protocols Country: Tanzania Status: Unknown References: xlvii

December 2009 in four districts in Tanzania evaluated the impact of electronic technology on the adherence of health workers to the Integrated Management of Childhood Illness (IMCI) protocols compared to paper-based protocols. To undertake this study, an electronic version of IMCI protocols was developed (eIMCI) and used on a Personal Digital Assistant (PDA). The eIMCI protocol followed the same protocol as a paper system (pIMCI): health workers were guided through the child’s assessment classification, treatment, and communication of instructions to the care provider.

level of adherence was greater for the eIMCI compared to the pIMCI: adherence for pIMCI ranged from 61% to 98% compared to 92% to 100% using the eIMCI.

Greater consistency in care using the eIMCI protocols was also observed compared to the pIMCI system.

Qualitative research indicated that the health workers preferred using the PDA eIMCI system rather than the pIMCI system because the former was easier to use and seemed that the children and their care providers had greater confidence in it.

Page 42: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

40

Glossary of eHealth and mHealth terms:

1G: refers to the first generation of analogue mobile telecommunication technologies introduced in the

1980s. It provides basic voice servicesxlviii.

2G: refers to the second generation of digital mobile telecommunication technologies, which introduced data

services for mobile phones and enables the following services: SMS text messages, picture messages and

MMS (multimedia messages)xlix.

3G: refers to the third generation of mobile telecommunication technology. It supports a data transfer rate

of at least 200 kilobytes per second. This means that in addition to the services provided by 2G (SMS text

messages, picture messages and MMS), 3G can provide wireless voice telephony, mobile and fixed wireless

internet access, video calls and mobile TVl.

4G: refers to the fourth generation of mobile telecommunication technology. It is an improvement from 3G

in that it provides ‘mobile ultra-broadband’. This means that in addition to the services possible with 3G,

other services including gaming, 3D television and cloud computing can be usedli.

Crowdsourcing: coined by Jeff Howes in his The Rise of Crowdsourcing, the term refers to the outsourcing of

a task to an undefined, but large group of people i.e. a crowd, rather than the more traditional method of

designating it to an employee or a contractorlii. In recent years there has been an ever increasing use of

crowdsourcing, made more possible with the rise in the adoption of mobile phones and the internetliii, liv. It

has thus been used in a number of innovative ways, ranging from tracking, reporting on, or co-ordinating

relief efforts (e.g. Ushahidi in Haitian earthquakelv) to the tracking of infectious diseases (e.g. Health Maplvi).

eHealth: (the “e” stands for electronic) is the use of ICT for health services and informationlvii. Electronic payments systems: refers to a technology based on an electronic process for moving stored value from an agency account to an account of the recipient (service provider) via mobile networkslviii. Electronic registration systems: refers to systems that register and verify the participation of a recipient to a programme electronically rather than through paper-based means. This could involve enlisting a recipient by placing their information for the programme onto a smart card or plastic card with a barcode that can be scanned.

Feature phone: a term used to refer to mobile phones with basic functionality for sending and/or receiving SMS messages and phone calls.

Global Positioning System: refers to a space-based satellite navigation system that provides location and time informationlix.

Information and Communication Technologies (ICT): refers to technologies that provide access to information through telecommunication (transmission of signals over long distances). This includes internet, mobile phones, wireless networks, and other communication mediumslx

Page 43: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

41

Interoperability: is the capacity of different operating systems (such as software and applications) to

communicate with each other, work with each other and to be integrated.

Java: refers to a programming language that allows the computer or mobile phone to interpret the data that is entered into itlxi. mHealth (or Mobile Health): to date, no standardised definition of mHealth has been established. Following from the Global Observatory for eHealth, mHealth can be defined as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants and other wireless devices”lxii.

Mobile Network Operator (MNO): a provider of wireless communication services. They own or control the means to sell and deliver services to an end user, such as wireless network infrastructure, billing, customer care, provision of computer systems etc. Two key features of an MNO is that they own or control access to a radio spectrum license from a regulatory or government body and own or control the parts of the network infrastructure that provide services to subscriberslxiii. Mobile technology: refers to portable devices that are able to transmit information through a range of communication technologies including wireless fidelity (Wi-Fi); bluetooth - connects mobile devices wirelessly; 'third generation' (3G); global system for mobile communications (GSM); and general packet radio service (GPRS). The devices that use mobile technology include, very basic feature mobile phones, smart phones, personal digital assistants, and wireless credit/debit card payment terminalslxiv Mobile money transfer (MMT): Refers to “any payment or fund that is transferred from one person or organisation’s ‘mobile wallet’ or bank account to another through mobile phones”lxv. Consequently, in following Smith et al’s definition, there are two forms of MMT – one that requires users to have access to formal financial services i.e. a bank account, and the other that does not require users to use a bank account to transfer money. The latter can be described as ‘mobile money for the unbanked’lxvi.

Mobile network coverage: refers to the geographic distribution of a wireless network used by mobile devices. The wireless network is distributed via so-called ‘cells’ that are served by a fixed location transceiver called cell sites or base stations. When these cells are joined together they provide radio coverage over a geographic area that enables mobile devices to communicate together and with fixed transceivers and telephones. How good the coverage is depends on certain factors such as geographic obstacles (e.g. mountains and buildings), technology, radio frequency and the sensitivity and transmit efficiency of the consumer equipmentlxvii lxviii

.

Personal Digital Assistant (PDA): refers to a handheld device with computing and organising functions. Most PDAs can access the internet and can be used as portable media players. They normally run standardised operating systems, such as Windows Mobile for Pocket PC or Palm OS. They differ from smartphones in that they don’t offer telephony (the transmission of voice, fax or data). Consequently, PDAs have nearly become obselete as smartphones become more advanced and widely adoptedlxixlxx.

Pilotitis: a “condition” characterised by a multitude of pilot projects which struggle to reach scale.

Short Message Service (SMS): a text message service that enables users to send short messages (160 characters)

to other userslxxi.

Page 44: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

42

Smart phone: a mobile phone built on a mobile computing platform that includes advanced functionality beyond sending SMS messages and making phone calls. Some of the most basic features of a smart phone include displaying photos, playing games, checking and sending emails, and surfing the web. The more advanced smart phones such as Android and iPhone based phones can run third party applications and thus have limitless functionalitylxxii. Telemedicine or telehealth: a term that comes under the rubric of eHealth. Telemedicine was coined in the 1970s, referring to ‘healing from a distance’lxxiii. There is no one definitive definition of telemedicine, as Sood et al’s (2007) study demonstrated, locating 104 peer-reviewed definitions of the wordlxxiv. However, in line with the World Health Organization, a broad description can be adopted:

“The delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies for the exchange of valid information for diagnosis, treatment and prevention

of disease and injuries, research and evaluation, and for the continuing education of health care providers, all in the interests of advancing the health

of individuals and their communities”.lxxv

Page 45: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

43

References

i Tech Term.com (2010) [accessed 21.02.14]. Available from: http://www.techterms.com/definition/ict. ii Vital Wave Consulting (2009) mHealth for Development: The Opportunity of Mobile Technology for Healthcare in the Developing World. Washington, D.C and Berkshire. iii Mendoza G, Okoko L, Morgan G, Konopka S. (2013), mHealth Compendium, Volume Two. USAID. Arlington, VA. iv Marshall C., Lewis D., Whittaker M. (2013) mHealth technologies in developing countries: a feasibility assessment and a proposed framework. Health Information Series Knowledge Hub, Working paper series, Number 25, March 2013. University of Queensland. v ITU (2013) [accessed 09.03.2014]. Available from: http://www.itu.int/ITU-D/ict/statistics/explorer/index.html vi Indice Nacional de Corrupcion y Buen Gobierno 2007

vii Chaudhury N. & Hammer JS (2003) Absenteeism in Bangladeshi Health Facilities. World Bank Policy Research

Working Paper 3065. viii

Azfar O. & Gurgur T. (2005) Does corruption affect health and education outcomes in the Philippines? Available at SSRN: http://ssrn.com/abstract=723702 or http://dx.doi.org/10.2139/ssrn.723702 ix https://www.facebook.com/PRPSS.bj

x http://www.youtube.com/watch?v=v85m82_cwYA xi Musgrove P: Financial and Other Rewards for Good Governance or Results: A Guided Tour of Concepts and Terms and a Short Glossary. Washington: World Bank; 2011. xii

OECD/DAC Glossary of Key Terms in Evaluation and Results-Based Management. xiii Naimoli J: Building Strong Health Systems: The promise and reality of Performance-Based Finanicng. Global Health Initiative Launch Team, USAID, Africa Bureau's Population, Health and Nutrition Officers; state of the art meeting; 2010. xiv WHO (2011) mHealth: New horizons for health through mobile technologies. Geneva. xv Kuipers P. et al., (2008), Collaborative review of pilot projects to inform policy: A methodological remedy for pilotitits? Australia and New Zealand Health Policy 2008, 5:17 xvi Lemaire J. (2011) Scaling up Mobile Health: Elements necessary for the successful scale up of mHealth in developing countries. White paper commissioned by Advanced Development for Africa, Switzerland. xvii Kochi, E. (2013), mHealth: moving beyond pilots to scale and impact. UNICEF xviii WHO (2011) mHealth: New horizons for health though mobile technologies. Global Observatory for eHealth series – Volume 3. Geneva, xix Shaw V. (2012), Measuring the impact of e-Health Bulletin of the World Health Organisation, vol. 90, pp. 326-27. xx Tomlinson M et. Al. (2009) The use of mobile phones as a data collection tool: A report from a household survey in Siouh Africa. BMC Medical Informatics and Decision Making, vol. 9, p.51. xxi Mendoza G. et. Al. (2013) mHealth Compendium, Volume Three. USAID. Arlington, VA. xxii Economist Intelligence Unit, (2012) Emerging mHealth: Paths for growth. xxiii Curioso, WH & Mechael, PN (2010) Enhancing ‘M-Health’ with South to South collaborations. Health Affairs, vol.29, pp. 264-67.

Page 46: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

44

xxiv Mechael, PN (2009) The case for mHealth in developing countries. Innovations: Technology, Governance, Globalizaion, vol. 4, pp. 103-18. xxv Harvard website [Accessed on 09.03.2014] Available from: http://www.hsph.harvard.edu/news/magazine/mobilizing-a-revolution/ xxvi Grameen Foundation. (2011). Mobile Technology for Community Development in Ghana. What Grameen Foundation has learned so far. Grameen Foundation: Available from: http://courses.cs.washington.edu/courses/cse490d/12sp/docs/MOTECH.pdf xxvii McQueen, Suzzane, Sarah Konopka, Nicole Palmer, Gwendolyn Morgan, Sunday Bitrus, and Lungi Okoko. (2012). mHealth Compendium, Edition One. Arlington, VA: African Strategies for Health project, Management Sciences for Health. xxviii Lemaire, J. (2011). Scaling up mobile health: elements necessary for successful scale up of mHealth in developing countries. Geneva: Advanced Development for Africa (ADA). xxix Freifeld CC, Chunara R, Mekaru SR, Chan EH, Kass-Hout T, et al. 2010). Participatory Epidemiology: Use of Mobile Phones for Community-Based Health Reporting. PLoS Med 7(12): e1000376. http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000376 [accessed 03/09/12] xxx Ushahidi. About us [webage]. Available at: http://ushahidi.com/about-us [accessed 05/09/12] xxxi Morrow, N., Mock, N., Papendieck, A., Kocmich, N.(2011). Independent Evaluation of the Ushahidi Haiti Project. Development Information Systems International Ushahidi Haiti Project xxxii Heinzelman, J and Waters, C. (2010). Special Report: Crowdsourcing Crises Information in disaster affected Haiti. United States Institute of Peace. Accessed at: http://www.usip.org/files/resources/SR252%20-%20Crowdsourcing%20Crisis%20Information%20in%20Disaster-Affected%20Haiti.pdf [05/09/12] xxxiii Tuijn CJ, Hoefman BJ, van Beijma H, Oskam L, Chevrollier N. (2011). Data and Image Transfer Using Mobile Phones to Strengthen Microscopy-Based Diagnostic Services in Low and Middle Income Country Laboratories. PLoS ONE 6(12): e28348. doi:10.1371/journal.pone.0028348 xxxiv InStrat. Global Health Solutions [webpage]. Available here: http://instratghs.com/sms.html (Accessed 06.03.14). xxxv Okuzu, O. ([email protected]). Remote patient monitoring. Email to: le May, V. ([email protected]) on 5th March 2014. xxxvi Unicef Zambia. (2012). Project Mwana: Using mobile technology to improve early infant diagnosis of HIV. Available from: http://www.unicef.org/partners/Partnership_profile_2012_Mwana_Zambia_V2_approved.pdf. xxxvii Phil Seidenberg et al, ‘Early infant diagnosis of HIV infection in Zambia through mobile phone texting of blood test results’, Bulletin of the World Health Organization, vol. 90, no. 5, May 2012, pp. 348–356. xxxviii UNICEF & Ministry of Health (Zambia). (2011). Project Mwana System Overview and M&E (Powerpoint presentation). Available here: http://unicefinnovation.org/sites/unicef.jjcdev2.com/files/Project_Mwana_Zambia_Evaluation_0.pdf xxxix Barrington J, Wereko-Brobby O, Ziegler R. (2010). SMS for life Tanzania pilot project report: Roll Back Malaria. Available from: http://www.rollbackmalaria.org/docs/SMSdetailReport.pdf xl Novartis. (2012). Innovation: SMS for life. Available from: http://malaria.novartis.com/innovation/sms-for-life/index.shtml. [accessed 18th December 2013]. xli Health Unbound. SMS for Life program [webpage]. Available from: http://www.healthunbound.org/content/sms-life-0. [accessed 25th February 2013].

Page 47: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

45

xlii Gayle Mendoza, Lungi Okoko, Sarah Konopka and Edna Jonas. (2013). mHealth Compendium, Volume Three. Arlington, VA: African Strategies for Health project, Management Sciences for Health. xliii mPedigree. We are the mPedigree Network [webpage]. Available from:

http://mpedigree.net/mpedigree/index.php. [accessed 20th December 2013].

xliv Gayle Mendoza, Lungi Okoko, Gwendolyn Morgan, and Sarah Konopka. (2013). mHealth Compendium, Volume Two. Arlington, VA: African Strategies for Health project, Management Sciences for Health. xlv Health Unbound. (2012). Mobiles for Quality Improvement (m4QI) [webpage]. Available here: http://www.healthunbound.org/node/1915 (Accessed 06/03/14) xlvi Riley P, BonTempo J. (2011). Mobiles for Quality Improvement Pilot in Uganda: Strengthening Health Outcomes through the Private Sector Project (SHOPS). Abt Associates Inc. Available from: http://www.shopsproject.org/sites/default/files/resources/SHOPS%20m4QI%20Uganda%20Pilot%20Report.pdf. xlvii Mitchell M, Hedt-Gauthier B, Msellemu D, Nkaka M, Lesh N. (2013) Using electronic technology to improve clinical care - results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania. BMC medical informatics and decision making 13(1):95 xlviii http://en.wikipedia.org/wiki/1G xlix http://en.wikipedia.org/wiki/2G l http://en.wikipedia.org/wiki/3G li http://en.wikipedia.org/wiki/4G lii As defined in Jeff Howe’s blog - Crowdsourcing: why the power of the crowd is driving the future of business. Accessed at: http://www.crowdsourcing.com/ [accessed 05/09/12] liii Freifeld CC, Chunara R, Mekaru SR, Chan EH, Kass-Hout T, et al. 2010. Participatory Epidemiology: Use of Mobile Phones for Community-Based Health Reporting. PLoS Med 7(12): e1000376. doi:10.1371/ http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000376 [accessed 03/09/12] liv Bott, M and Young, G. The role of crowdsourcing for better governance in International Development. The Fletcher Journal of Human Security. Accessed at: http://fletcher.tufts.edu/praxis/~/media/Fletcher/Microsites/praxis/xxvii/4BottYoungCrowdsourcing.pdf [accessed 05/09/12] lv Morrow, N., Mock, N., Papendieck, A., Kocmich, N. 2011. Independent Evaluation of the Haiti Ushahidi Project. DISI: Development Information Systems International. lvi Freifeld, C and Brownstein, J. 2007. About Health Map. http://www.healthmap.org/about/ [accessed 03/09/12] lvii Vital Wave Consulting. mHealth for Development: The Opportunity of Mobile Technology for Healthcare in the Developing World. Washington, D.C and Berkshire, UK: UN Foundation-Vodafone Foundation Partnership, 2009. lviii Smith, G., MacAuslan, I., Butters, S. and Tromme, M. (2011) New Technology Enhancing Humanitarian Cash and Voucher Programming, a Research Report commissioned by CaL. Accessed at: http://www.cashlearning.org/downloads/resources/calp/CaLP_New_Technologies.pdf [accessed 03/10/12] (page 4) lix http://en.wikipedia.org/wiki/Global_Positioning_System lx Tech Term. ICT. http://www.techterms.com/definition/ict lxi http://www.knowyourmobile.com/glossary/java

Page 48: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

46

lxii Kay, M., Santos, J., Takane, M. 2011. Global Observatory for eHealth Series – Volume 3: mHealth – new horizons for health through mobile technologies. World Health Organisation lxiii http://en.wikipedia.org/wiki/Mobile_network_operator lxiv http://www.businesslink.gov.uk/bdotg/action/detail?itemId=1074298219&type=RESOURCES [accessed 12/10/12] lxv Smith, G., MacAuslan, I., Butters, S. and Tromme, M. (2011) New Technology Enhancing Humanitarian Cash and Voucher Programming, a Research Report commissioned by CaL. Accessed at: http://www.cashlearning.org/downloads/resources/calp/CaLP_New_Technologies.pdf [accessed 03/10/12] (p6) lxvi GSMA. MMU Programme Overview: the case for mobile money. http://www.gsma.com/mobilefordevelopment/programmes/mobile-money-for-the-unbanked/programme-overview/ [accessed 11/10/12] lxvii http://en.wikipedia.org/wiki/Coverage_%28telecommunication%29 lxviii http://en.wikipedia.org/wiki/Cellular_network lxix http://en.wikipedia.org/wiki/Personal_digital_assistant lxx http://www.gsmarena.com/glossary.php3?term=pda lxxi http://www.gsma.com/aboutus/gsm-technology/glossary/ lxxii Tech Term. Smartphone. http://www.techterms.com/definition/smartphone (accessed 07/08/12) lxxiii Strehle EM, Shabde N. One hundred years of telemedicine: does this new technology have a place in paediatrics? Archives of Disease in Childhood, 2006, 91(12):956–959. http://www.globalproblems-globalsolutions-files.org/unf_website/assets/publications/technology/mhealth/mHealth_foreword_ack.pdf [accessed 21/08/12] lxxiv Sood SP, et al. Differences in public and private sector adoption of telemedicine: Indian case study for sectoral adoption. Studies in Health Technology and Informatics, 2007, 130:257–268. lxxv WHO. A health telematics policy in support of WHO’s Health-For-All strategy for global health development: report of the WHO group consultation on health telematics, 11–16 December, Geneva, 1997. Geneva, World Health Organization, 1998.

Page 49: Annual Results and Impact Evaluation Workshop for RBF - Day Five - New Technologies and Results-Based Financing - A Discussion Paper

47