Lessons from Country Programs

Implementing the Mobile

Alliance for Maternal Action
Programs in Bangladesh, South
Africa, India and Nigeria,
2010–2016
We gratefully appreciate and acknowledge the developers of the mHealth Assessment and Planning for Scale (MAPS)
Toolkit: the World Health Organization Department of Reproductive Health and Research (WHO RHR/HRP). We
used the MAPS Toolkit to provide a framework for the lessons learned discussions and received permission to use
the graphics for the axis icons.

Cover photo credit: MAMA/Karin Schebrucker.

Table of Contents

v Acknowledgments

vi Abbreviations

vii Executive Summary

1 Introduction

5 Country Profiles

7 Lessons Learned by Topic Axis

9 Groundwork

13 Partnerships

19 Financial Health

22 Content Creation

25 Technology & Architecture

29 Operations

35 Monitoring & Evaluation

38 Overall Achievements

39 Future Considerations

41 Conclusion

42 Appendix A: Streamlined mHealth Assessment and Planning for Scale Template

44 Appendix B: Lessons Learned: Full List, Ranked by Priority

47 Appendix C:Additional Resources

50 Endnotes

0iii
0
iv
Acknowledgments

The country teams provided invaluable insight into their implementation processes and feedback for this
document to describe the lessons they learned as they used the Mobile Alliance for Maternal Action (MAMA)
approach to guide their work.
• Bangladesh—Tahsin Ifnoor Sayeed, Ananya Raihan (Dnet)
• South Africa—Debbie Rogers, Ambika Howard (Praekelt)
• India—Aakash Ganju, Aparna Hegde (Advancing Reduction in Mortality and Morbidity of Mothers,
Children and Neonates)
• Nigeria—Emmanuel Atuma (Jhpiego Nigeria), Ayomipo Edinger, Farouk Jega, and Justin Maly (Pathfinder
International)

Pamela Riley (Abt Associates) and Marion McNabb (HealthEnabled) reviewed country reports and synthesized
them into initial lessons for discussion. They also led the workshop to further describe the experiences that
resulted in the lessons learned.

Susan Rae Ross (United States Agency for International Development, USAID) lent significant time and expertise to
reviewing and developing this document to reflect future considerations for implementation of the MAMA approach.

Numerous partners reviewed this document, lending their historical knowledge of the program, and helped to
shepherd it through rounds of review to ensure it reflected the collective knowledge of individuals involved in
implementation of the MAMA approach in the four countries.
1. Johnson & Johnson—Tommy Lobben, Aakash Ganju, Joanne Peter
2. BabyCenter—Colleen Hancock, Lindsay Dills, Megan Preovolos
3. Maternal and Child Survival Program (MCSP)—Danielle Nielsen, Alice Liu

Additional attendees contributed to the discussions of lessons learned during the December 2016 workshop,
helping to clarify and add to the lessons learned.
1. HealthEnabled—Emeka Chukwu and Patricia Mechael
2. Praekelt Foundation—Brooke Cutler
3. United Nations Foundation—Kate Dodson
4. USAID—Holly O’Hara and Peggy D’Adamo
5. MCSP—Geoff Prall and Alishea Galvin

Radha Rajan (Johns Hopkins Bloomberg School of Public Health) drafted this document using the country
reports, workshop discussions, and the feedback of all the document reviewers. Alice Liu and Steve Ollis edited
the document. Erin Sullivan (Jhpiego) provided the mHealth landscape data for Table 4.

We gratefully appreciate and acknowledge the developers of the mHealth Assessment and Planning for Scale (MAPS)
Toolkit: the World Health Organization Department of Reproductive Health and Research (WHO RHR/HRP), the
United Nations Foundation (UN Foundation), and the Johns Hopkins University Global mHealth Initiative. We used
the MAPS Toolkit to provide a framework for the lessons learned discussions and received permission to use the
graphics for the axis icons.

0v
Abbreviations

ANC antenatal care

ARMMAN Advancing Reduction in Mortality and Morbidity of Mothers, Children and Neonates

CHW community health worker

FMOH Federal Ministry of Health (Nigeria)

ICDDR,B International Center for Diarrheal Disease Research, Bangladesh

ICT information and communication technology

IVR interactive voice response

IWG Innovation Working Group

J&J Johnson & Johnson

M&E monitoring and evaluation

MAMA Mobile Alliance for Maternal Action

MAPS mHealth Assessment and Planning for Scale

MCHIP Maternal and Child Health Integrated Program

MCSP Maternal and Child Survival Program

MNO mobile network operator

MoHFW Ministry of Health and Family Welfare (Bangladesh)

MOU memorandum of understanding

NGO nongovernmental organization

OBD outbound dialing

PEPFAR President’s Emergency Plan for AIDS Relief

PMTCT prevention of mother-to-child transmission of HIV

SMS Short Message Service, commonly referred to as text messages

USAID United States Agency for International Development

USSD Unstructured Supplementary Service Data

WRHI Wits Reproductive Health and HIV Institute

0
vi
Executive Summary

This document highlights key operational lessons learned from four country programs—Bangladesh, South Africa,
India and Nigeria—that implemented the Mobile Alliance for Maternal Action (MAMA) approach. The MAMA
approach uses age- and stage-based messaging directed toward pregnant women, new mothers and families to foster
behavior change and improve maternal and child health outcomes. This report aims to share operational lessons that
country program implementers learned and the strategies they used to overcome implementation challenges.

Methodology
A modified usage of the mHealth Assessment and Planning for Scale (MAPS) toolkit1 provided a structure
for a “lessons learned” meeting on December 15–16, 2016. Twenty-five representatives from the four country
programs and core partner organizations participated in the meeting. In advance, the four country programs
prepared reports using the modified MAPS template. Meeting facilitators analyzed the templates and developed
common program implementation lessons based on broad themes identified in the country reports. Participants
discussed lessons for each axis: Groundwork, Partnership, Financial Health, Technology, Operations,
Monitoring and Evaluation and an additional axis called Content Creation. They then ranked the lessons they
thought were most important for successful program implementation and discussed some overarching lessons.

Country Profiles
The four country programs began implementing the MAMA approach successively, learning from each other and
adapting content to each unique country context. Each program was unique in its leadership, approach and long-term
strategy.

Bangladesh: Aponjon South Africa: MAMA South Africa

Lead coordinator: Dnet Lead coordinator: Praekelt
Length of implementation: 6 years Length of implementation: 3 years
Scale: National Scale: National
Total subscribers: 1.9 million Total subscribers: 500,000
Technology: Outbound dialing and text messaging. Technology: Text messaging, mobile website, social
More recently launched a 24/7 Doctor’s Line and two network platform and Unstructured Supplementary
mobile applications. Service Data (USSD)
Long-term goal: Commercial adoption Long-term goal: Government adoption (achieved)*
India: mMitra Nigeria: HelloMama
Lead coordinator: Advancing Reduction of Mortality and Lead coordinator: Maternal and Child Survival
Morbidity of Mothers, Children and Neonates (ARMMAN) Program (MCSP)
Length of implementation: 3 years Length of implementation: 1 year
Scale: 3 states Scale: 2 states, pilot phase
Total subscribers: 600,000 Total subscribers: 4,609
Technology: Outbound dialing Technology: Outbound dialing and text messaging
Long-term goal: Government adoption Long-term goal: Government adoption
*In 2014, the South African National Department of Health launched a service called MomConnect that built upon content, technology and
partnerships developed for MAMA South Africa. In 3 years, MomConnect has reached 1.7 million mothers in 95% of public health clinics.

0 vii
Lessons Learned
The table below summarizes the key lessons learned.
Groundwork Axis
Key Lesson 1: Initial mobile channel selection was influenced by country context factors such as
literacy level, mobile phone ownership, and usage patterns.
Key Lesson 2: Formative research was essential to making other programmatic design decisions.

Partnerships Axis
Key Lesson 3: Partnership structures needed more clearly articulated roles and responsibilities
from the start of the program.
Key Lesson 4: Unanticipated changes in leadership, strategy, and personnel undermined the
effectiveness of mHealth partnerships.

Financial Health Axis

Key Lesson 5: There is no “right” funding model. Country programs developed diverse funding
models, based on their context and goals for long-term viability.
Key Lesson 6: Cost drivers varied greatly due to country information and communication technology
(ICT) regulations, mobile network operator (MNO) business models, and ICT technical structures.

Content Creation Axis

Key Lesson 7: Content needed to be hyper-localized, with involvement of local stakeholders.

Technology & Architecture Axis

Key Lesson 8: Evolving program requirements required in-house technology expertise to translate
customized needs to technology partners and to exert greater control over the service.
Key Lesson 9: Working with external technology companies, especially aggregators and MNOs,
accelerated time to market but was complex to manage.

Operations Axis
Key Lesson 10: Customer enrollment required multiple partners and approaches with “boots on
the ground” to be successful. However, aligning partner motivations, training, and supervision were
key challenges to enrollment at scale.
Key Lesson 11: Ensuring that women received messages required specific strategies, such as
selecting preferred timeslots, creating jingles, and returning missed calls.

Monitoring & Evaluation Axis

Key Lesson 12: The lack of a clear understanding of data needs, reports, and partners’ privacy
policies led to delays in harnessing data for service improvements and actionable insights.
Key Lesson 13: Impact evaluation was underfunded. Country programs made trade-offs between
routine monitoring and impact assessment.

0 viii
Conclusions
The country programs found it valuable to share implementation lessons with one another, and learned that
despite their different country contexts, there were many lessons that have wide applicability.

All four programs have achieved successes in reaching pregnant women, new mothers and their families with
vital information about how to take better care of themselves and their children. Programs have also developed
public-private partnership networks, particularly engaging governments in supporting mHealth efforts. Overall,
the country programs think the MAMA approach works: the messages are highly valued by those receiving
them, and the program is a worthwhile investment for improving knowledge and attitudes. However, questions
remain about the most sustainable business models for these programs and the extent of the program’s
behavioral and service utilization impact.

Over the last 5 years, mHealth has evolved significantly. There are fewer pilots, and more programs are working
toward or reaching scale with a host of partners. But mHealth is still an emerging area, and it is essential for
implementers to continue to share their experiences with cost structures and business models, and contribute to
the evidence base, particularly regarding the impact on behavior change, service utilization and health outcomes.

The country programs have found it valuable to have an informal learning network where they can support
each other and share experiences that often have broad applicability, while understanding the differing context
across their countries.

0ix
0
x
Introduction

The Mobile Alliance for Maternal Action (MAMA) was launched with the goal of catalyzing a global
community to deliver vital health information to new and expectant mothers and their families through mobile
phones. MAMA’s theory of change was based on evidence that when a woman has appropriate information
about her health, her child’s health and services that she should use, then she is more likely to adopt health
behaviors and use services that will lead to improved health outcomes.

Report Purpose and Approach

On December 15–16, 2016, the four country programs—Bangladesh, South Africa, India, and Nigeria—that
have been implementing MAMA met in Washington, DC, USA to review critical lessons from their programs.
An adaptation of the mHealth Assessment and Planning for Scale (MAPS) toolkit,2 which includes assessment
questions in seven areas or “axes”—Groundwork, Partnership, Financial Health, Technology, Operations,
Monitoring & Evaluation (M&E), and an additional axis called Content Creation—was used to provide a
structure for the lessons learned discussion (See Appendix A). The countries used the template to facilitate in-
country meetings with relevant partners and stakeholders to identify country lessons. The completed templates
were shared with the other three countries and analyzed for broad themes and to identify common lessons in
program implementation. Twenty-five people participated in the two-day interactive workshop to give input on
the summarized findings and lessons learned, to add to, solidify and prioritize them based on importance. A full
list of the discussed and prioritized lessons learned is available in Appendix B.

This document, the result of the December workshop, highlights key operational lessons learned from the
four countries’ experiences, with the aim of providing useful information to other stakeholders (e.g., ministries,
donors, implementers) interested in implementing mHealth initiatives.

This document does not evaluate the impact of country programs, nor does it include an analysis of program
costs or the value of the program for the funds invested. The initial seed funding varied for the four programs
detailed in this report, and those comparative cost data have not yet been captured for analysis and application.

Public Health Justification for MAMA Program

Millions of pregnant women cannot access the basic health information they need to have healthy pregnancies
and healthy babies. In 2010, the average global maternal mortality ratio was 246 maternal deaths per 100,000
live births; with maternal mortality ratios of the four MAMA country programs ranging from a low of 154 in
South Africa to a high of 867 in Nigeria.3 MAMA was designed to address this key barrier facing women and
their families to accessing health information by harnessing the growing ubiquity of mobile phones.

History of Country Programs and MAMA Secretariat

In 2010, Dnet, a Bangladeshi social enterprise, began to develop Aponjon: a text and voice-based system to
provide age- and stage-based information to pregnant women and their family members, supported through
USAID’s Strengthening Health Outcomes through the Private Sector project. In 2011, South Africa continued an
mHealth partnership to start MAMA South Africa reaching pregnant women and their partners primarily through
a mobisite, a website optimized for viewing on mobile phones, and text messages, supported by the President’s
Emergency Plan for AIDS Relief (PEPFAR), Johnson & Johnson (J&J), Vodacom Foundation and the United
Nations’ Innovation Working Group (IWG). As a result of these two promising examples, the MAMA Global
Secretariat was established in 2012 on behalf of the founding partners—USAID, J&J, BabyCenter (a for-profit

01
J&J Company), with the mHealth Alliance/United Nations Foundation serving as the Secretariat’s organizational
host. The key objectives of the MAMA partnership were to:
1. Galvanize the mHealth efforts started in Bangladesh and South Africa to develop voice and/or text
messages for low-income pregnant women, mothers and their families to expand to other countries;
2. Create high quality age- and stage-based messages for low-income pregnant women, new mothers and their
families that could be adapted to meet the needs of the local context;
3. Better understand the role that mHealth can play in behavior change and conduct research to demonstrate
changes in key maternal and child health behaviors in the home as well as increased service utilization.

In 2013, mMitra was launched in urban slums in Mumbai with support from J&J, and, in 2015, Hello MAMA
began in Nigeria with support from USAID/Nigeria and J&J.

Once the global goals were achieved, the MAMA Global Secretariat was dissolved (December 2015). The work
in the four country programs continues to expand with additional donors now providing support.

Table 1 provides a timeline of the country programs and MAMA Global Secretariat.

A woman enrolling for Aponjon service

over SMS.
Photo credit: Naimul Haque Joarder Titu,
Aponjon.

2
Table 1.Timeline of the country programs and MAMA Global Secretariat

Date Activity Implementing Comments

Partners

Supported by USAID/
Bangladesh and J&J through
Aponjon Bangladesh
Strengthening Health
2010 launched pilot, with Dnet
Outcomes through the
text and voice service
Private Sector and then
MCHIP starting in 2012

Wits Reproductive
Health and HIV
Institute (WRHI), Supported by PEPFAR/
2011
SOUTHMAMA South Africa Praekelt Foundation, USAID, J&J, IWG,Vodacom
...... AFRICAlaunched
~A""""'"'""'""''A«<>n Cell-Life (Follow- Foundation
on from previous
project)

MAMA Global Housed at United Supported by USAID and

2012
Secretariat established Nations Foundation J&J and BabyCenter

Based on demand
from subscribers, a Supported by USAID/
Dnet
24/7 Doctor’s line Bangladesh through MCHIP
launched
2013

mMitra in India
mMitra launched
ARMMAN Supported by J&J

2014 SOUTH
MAMA South Africa transformed from solely demand generation to
MomConnect, which also addressed supply side issues.
AFRICA
-•A""""'" ...."'""'A<bOn

0 3
Date Activity Implementing Comments
Partners
Dnet
Aponjon Shogorva
mobile app targeting In association
expecting mothers with International
launched Center for USAID/Bangladesh through
Diarrheal MCHIP
Aponjon Koishor Disease Research,
mobile app for Bangladesh
adolescents launched (ICDDR,B) and
2015 BCCP

- .i - Maternal and Child

........
,...,.
Hel.l.o " HelloMama in Nigeria Survival Program Supported by USAID/
began (MCSP), Pathfinder, Nigeria and J&J
Praekelt Foundation

MAMA Global Country programs

—
Secretariat dissolved continued

Hel.l.o HelloMama launched

2016 Dnet incorporated

a wholly-owned
subsidiary LifeChord
Dnet USAID and Dnet
for taking over
Aponjon after funding
cycle ends

0 4
Country Profiles

The MAMA approach broadly delivers stage-based maternal and child health (MCH) information using digital
technology, yet the four country programs varied in many aspects based on the country context and desired health
outcomes (Table 2). For example, because of low levels of female literacy, Nigeria opted primarily for voice calls
over text messaging. In India, the use of voice calls was linked more to limited mobile consumption patterns
than to female literacy. ARMMAN’s earlier research indicated poor uptake of text even in literate populations.
Bangladesh started with short message service (SMS) (before the national launch) and learned through field studies
and content surveys that there was an unmet demand for voice calls in rural areas and among people of the
lowest socioeconomic status. South Africa focused on multiple channels, or communication methods, to send and
receive information (e.g., text message, website, voice call, reflecting the diversity among its mobile subscribers. In
addition, South Africa had a much greater focus on preventing mother-to-child transmission (PMTCT) of HIV
because of the high rates of HIV/AIDS, but Bangladesh focused more on safe childbirth and newborn care.

Table 2. Summary of Country Programs

Bangladesh South Africa India Nigeria

Program 6 years, national scale 3 years, multiple channels, 3 years, three 1 year, pilot phase
status national scale states

t
Desired Improvements in Improvements in maternal Improvements in Improvements
health maternal health, health, particularly PMTCT maternal health, in maternal
outcomes increase use of increase use of health,increase use
antenatal care ANC, effective of ANC and facility
(ANC) and facility treatment of deliveries, newborn
deliveries, newborn anemia, newborn care, increase in
care, increase in care, increase in immunization
immunization immunization
t

Business Cross-subsidized user Free to all subscribers, Free to all Free to all subscribers
model fees supported by donor funds subscribers supported by donor
supported by funds
Free to 20% of Aimed for advertising donors, corporate
subscribers in lowest revenue or adoption by and individuals Aimed for
socioeconomic status National Government government adoption
category. Others Transitioned to Aimed for
charged USD $0.058/ MomConnect; government government
message and donor funded adoption

Aimed for commercial

adoption
t

Message Voice/Outbound Text/SMS (3%) Voice/OBD (100%) Voice/OBD (43%)

delivery dialing (OBD) (46.3%) Mobisite (72%) Text/SMS (57%)
channels Text SMS (53.7%) MXit Social network (19%)
USSD (6%)
t

Cumulative 1,902,417 as of Dec 31, 500,000 (MAMA women 600,000 4,609 (Dec 2016)
subscribers 2016 only)
(women and
gatekeepers) 1.3 million (MomConnect)
t

Current 370,595 on Dec 31, 700,000 (MomConnect) 400,000 4,609 (Dec 2016)
subscribers 2016

5
0
 Bangladesh South Africa India Nigeria
Languages Standard Rural/Urban English, Afrikaans, Zulu, Hindi, Marathi Igbo, Pidgin English
and dialects Bangla, Chittagong Xhosa, Sotho, Tswana
dialect, Sylhet dialect (MAMA)
MomConnect expanded to
all 11 official languages
...
Founding Dnet, MOHFW, Praekelt Foundation, WRHI, ARMMAN, J&J Praekelt, Pathfinder,
partners USAID Cell-Life, PEPFAR/USAID, MCSP, J&J, USAID
J&J,Vodacom

Target Audience: MAMA programs targeted slightly different population segments

The country programs initially targeted pregnant women and new mothers with children up to age 1 year. All the
countries except South Africa included a secondary target audience called “gatekeepers” comprised of husbands,
mothers and mothers-in-law, who play an influential role on women’s health decisions. South Africa did not include
gatekeepers because they found 78% of women from disadvantaged backgrounds did not live with the father of
their children, and men had very little influence over the health of the mother and child. Instead, addressing single
mothers became a context-specific adjustment to the program. Bangladesh expanded their audience over time to
include mothers of children up to age 5 years, allowing women to access information for a longer period.

0 6
Lessons Learned by
Topic Axis
Methodology
As previously mentioned, a modified MAPS tool was used to provide a structure for the lessons learned
discussion4. The meeting facilitators analyzed the templates for broad themes and to identify common program
implementation lessons. Based on this analysis, lessons for each axis—Groundwork, Partnership, Financial
Health, Content Creation, Technology, Operations and M&E/Research—were discussed and the country teams
voted on the lessons they thought were the most important for successful program implementation.

The country teams identified a total of 13 lessons through this process.

Table 3 summarizes the key questions reviewed by axis and the major lessons learned for each.

Table 3. Summary of Key Lessons

Groundwork
• What was the contextual
environment? Lesson 1: Initial mobile channel selection was influenced by
country context factors such as literacy level, mobile phone
• What was the state of digital
ownership and usage patterns.
health?
• What socioeconomic factors
did you consider in your Lesson 2: Formative research was essential to making other
implementation? programmatic design decisions.

Partnerships
Lesson 3: Partnership structures needed more clearly
articulated roles and responsibilities from the start of the
• Who did you engage with to
program.
start-up the project?
• How did those partnerships Lesson 4: Unanticipated changes in leadership, strategy,
evolve over time? and personnel undermined the effectiveness of mHealth
partnerships.

Financial Health
Lesson 5: There is no “right” funding model. Country
• What was your financial programs developed diverse funding models, based on their
model? context and goals for long-term viability.
• What were the main cost Lesson 6: Cost drivers varied greatly because of country
drivers for your program? ICT regulations, MNO business models and ICT technical
structures.

Content Creation

• How did your project utilize

@ the core MAMA messages?

• How did you adapt the
messages?
Lesson 7: Content needed to be hyper-localized with
involvement of local stakeholders.

0 7
 Technology & Architecture
• Describe your technology. Lesson 8: Evolving program requirements required in-
• What types of data did your house technology expertise to translate customized needs
platform capture? to technology partners and to exert greater control over the
service.
• How easy/difficult was it to
make changes to the platform?
Lesson 9: Working with external technology companies,
• Describe your experiences especially aggregators and MNOs, accelerated time to market
with MNOs and/or but was complex to manage.
aggregators.

Operations
• Describe the registration Lesson 10: Customer enrollment required multiple
process. partners and approaches with “boots on the ground” to be
successful. However, aligning partner motivations, training and
• What were the successes supervision were key challenges to enrollment at scale.
and challenges in enrolling
women? Lesson 11: Ensuring that women received messages required
specific strategies, such as selecting preferred timeslots,
creating jingles, and returning missed calls.

Monitoring & Evaluation

• How did your project Lesson 12: The lack of clear understanding of data needs,
monitor implementation? reports and partners’ privacy policies led to delays in harnessing
• Did your project perform data for service improvements and actionable insights.
an evaluation? What were Lesson 13: Impact evaluation was underfunded. Country
lessons learned in that programs made trade-offs between routine monitoring and
process? impact assessment.

0 8
Groundwork

Groundwork describes key areas required for designing mHealth programs: including developing goals (e.g.,
specific behaviors and health outcomes) and understanding the contextual environment (e.g., female literacy,
network coverage) to inform service design.

The country programs had a common goal: to increase ANC visits and facility deliveries and ultimately to
improve health outcomes for pregnant women, newborns, children up to a certain age and their families.
Countries set several specific goals for knowledge, behavior change and increased service utilization, such
as increased knowledge and practice of exclusive breastfeeding, increased knowledge of the need for iron
supplementation during pregnancy and increase in postnatal care visits as well as ANC.

An important element of groundwork is to document the service delivery points, gaps, and potential linkages to the
MAMA demand side intervention. The program’s goal for geographic scale, whether targeting urban slums (India),
states within a country (Nigeria), or national (Bangladesh and South Africa), was one driver for determining the
target populations, languages and service delivery areas. Mobile network coverage and prevalence of mobile phone
ownership was another goal. This section explores these factors that helped lay the groundwork for the programs.

Lesson 1: Initial mobile channel selection was influenced

by country context factors such as literacy level, mobile
phone ownership and usage patterns.

Family checking maternal health SMS

messages on their mobile.
Photo credit: MAMA/Karin Schebrucker.

9
Understanding the Local Context: All country programs used formative research in their initial program
design and they incorporated audience feedback to make decisions about program delivery. Countries analyzed
primary and secondary quantitative and qualitative research to understand the:
• Current levels of MCH knowledge, behaviors and service utilization;
• Key women’s issues (e.g., female literacy, employment, mobility); and
• ICT landscape in terms of policies, coverage, mobile phone usage, female phone ownership, networks and
the type of content that would be most relevant to women, husbands and other gatekeepers.

Table 4 presents some of the key situational factors that influence mHealth country programs.

Table 4. Situational analysis of mHealth landscape in program countries

Bangladesh South Africa India Nigeria

(2010) (2011) (2013) (2015)
Mobile cellular 38%6 123% 71% 82%
subscription rate (per 100
inhabitants)5
Female Literacy 56%7 92%8 65%9 50%10
+-
Female Phone Ownership 44%11 83%12 31%8 80%8
Network Coverage 99%14 No data found 75% of population; 40% 75% of population;
of land area by 2007 40% of land area by
2013
Number of MNOs 615 416 1117 418
operational I
Channels Voice/OBD & SMS, Mobisite, OBD OBD & SMS/USSD
SMS MXit Social
network, USSD
+
User interaction Two-way One-way One-way One-way
communication, communication, communication, pull communication,
pull messaging push messaging messaging push messaging

Table 5. Channel Selection for MAMA program countries

Bangladesh South Africa India Nigeria

(2010) (2011) (2013) (2015)
Channels Voice/OBD & SMS, Mobisite, OBD OBD & SMS/USSD
SMS MXit Social
network, USSD
+-
User interaction Two-way One-way One-way One-way
communication, communication, communication, pull communication,
pull messaging push messaging messaging push messaging

Channel Selection: Formative research helped to determine the optimal channels to use in an mHealth
program. The 2011 Bangladesh survey found that women shared their phones with their husbands, who took
the phone with them while they were out for work. The survey also revealed that women preferred push

0 10
messaging (via voice) over pull messages where they would call a short code to get the message. A short code
is a short number sequence of four to six digits to send and receive text and multimedia messages. Short codes
are used because they are easier to remember than a standard long phone number. With these preferences in
mind, Dnet opted for voice messaging as their main channel for message delivery.

In South Africa, mHealth was a new approach, so there was little evidence to make a decision on which channel
would be best for achieving scale, sustainability and impact. However, South Africa’s more mature mobile industry
provided more options than the other countries for reaching large segments of the population. Therefore, South
Africa tested four communication channels (Table 6). It should be noted that a fifth channel, voice messages, was
considered, however, the program decided not to pursue it because the costs would be prohibitive at scale.

Table 6. Comparing communication channels in South Africa

Indicator Channel 1: Channel 2: Channel 3: Channel 4:

Mobisite Targeted SMS USSD MXit Social
network
Target market Middle living standard, urban Low to middle living Low to middle living Middle living
standard standard standard,
urban
Marketing 1. Banners Text ads (limited) Sponsorship Text ads, Ad
opportunities 2. Text ads Sponsorship Syndication sales support,
3. Sponsored and syndicated Revenue
content sharing
Sponsorship
Cost per user Decreases as user base Fixed cost per user; Decreases as user Decreases
increases ever-increasing base increases, but as user base
there are some fixed increases
costs per user

Given the range of advantages and drawbacks each communication channel presented, the South Africa
country program decided a mix of channels would be the best strategy for success.

Usage patterns: Technology platforms and message delivery decisions included estimating the proportion of
text and voice/OBD anticipated to be used by subscribers, the time when messages should be delivered, the
frequency of messages, and the voice or persona to use in delivering messages. In Bangladesh, a 2011 needs
assessment prompted the program to focus on voice calls. Their research found: 1) a need for more accuracy
and comprehensibility in the translated Bangla SMS, and 2) the importance of offering a preferred time slot for
women because otherwise their husbands, who had the phone, would be at work and women would miss the call.

Lesson 2: Formative research was essential to making

other programmatic design decisions.
Nigeria applied formative research toward their brand development. Several rounds of testing were conducted
in Nigeria to develop their brand identity, which changed from MAMA Nigeria to HelloMama. Using different
pictures, colors and names, the project team elicited opinions about name options. Although Nigerian
sounding program names were most popular with users, several partners were concerned that the names

011
were not descriptive enough of the service. So, formative research participants were asked to suggest names,
and they predominantly suggested English names. After several rounds of discussion, three options were
shared with partners, including government agencies, leading to the final brand of HelloMama. This name
resonated because it conveyed the mobile phone component of the program and the target audience. When
anyone picks up the phone, the first thing said is “Hello,” and Mama is the most widely accepted name for
mothers in Nigeria. The program now uses HelloMama branding on all of their information, education and
communication and promotional materials.

012
Partnerships

The country programs formed partnerships to support the process of designing, financing, launching,
implementing and scaling up MAMA. Strong, sustainable partnerships are essential for successful mHealth
programs and services. Because mHealth represents the intersection of the health, technology and business
sectors, efforts require involvement of a wide range of groups. Successful partnerships bring together diverse
skills, services, strategies, lessons learned, audiences and ideas with a common goal in mind. However, the
process of achieving and maintaining dedicated, productive collaborations is no easy task. Strategic decision-
making is required at many implementation stages. Selections of partners and establishing effective governance
structures are key to successful mHealth implementation.19

The MAPS toolkit outlines three levels of partners in mHealth programs:

• Founding partners are crucial to pursuing the chosen long-term strategy (including lead implementing
partners, donors); they make decisions about how funds will be spent and the partnership will be structured.
• Resource partners support the program’s success (e.g., policy, enrollment, research, corporate sponsors);
they contribute their expertise.
• Government partners are stewards in launching and sustaining a mHealth program; they provide linkages to the
broader health and technology infrastructure, systems and policies and enhance legitimacy of the partnership.

Table 7 lists partner roles in program implementation in each country.

Table 7.Types of Partners by Country

Bangladesh South Africa India Nigeria

Founding • Ministry of Health • Cell-Life • ARMMAN • Praekelt
Partners Family Welfare • Vodacom • Inscripts • Pathfinder
(MoHFW)
• WRHI • Government of • MCSP
• Dnet Mumbai
• Praekelt • USAID
• USAID • J&J
• J&J • J&J
• J&J
• PEPFAR/USAID
Resource • ICDDR, B • Always Active • Local • Federal Ministry of
Partners • BRAC Technologies nongovernmental Health (FMOH)
organizations • NCC
• National Health Service (NGOs)
Delivery Project • NPHADC
• Municipal/State
• MaMoni government • SMOH
• Infolady • Vas2Net
I· I· I·
Lead • Dnet • Cell-Life • ARMMAN • MCSP
Implementing • WRHI • Inscripts • Praekelt
Partners
• Praekelt • Pathfinder
+ + +
MNOs • SSD Tech • Vodacom • IMI Mobile • Glo
• Grameenphone • MTN (Interactive Voice • Etisalat
Response aggregator,
• Banglalink;Airtel • CellC reached all MNOs • Airtel
• Robi; TeleTalk; Citycell • Telkom Mobile in-country)

0
13
 Bangladesh South Africa India Nigeria
Additional • J&J/BabyCenter • United Nations
Funding • Bangladeshi American Foundation/
Organizations Charitable Organization IWG

• Beximco Group • Vodacom

Multimode Group Foundation

MAMA, J&J and USAID have been founding partners for Aponjon on the global front while Dnet explored
partnership opportunities within the country as well. A partnership strategy was developed for leveraging
financial and technical resources from the private sector. Beximco Pharma was the major in-country founding
partner with a contribution of 250,000 USD.

Government Engagement
All four of the country programs worked with the government, however, the nature of their
partnerships varied by context and desired long-term strategy. Bangladesh and Nigeria had the
closest relationships with government agencies. India had moderate engagement, and South
Africa was largely an information-sharing relationship.
• In Bangladesh, the Office of the Prime Minister and MoHFW were crucial founding partners,
brokering telecom negotiations through regulatory agency, approving program content, and
promoting the program through service delivery points. Dnet emphasized the importance
of engaging the public sector champions early
• In Nigeria, finding the right government partners was a key challenge in establishing the
partnership, given that there are two key health ministries, state ministries and the ICT ministry.
Engaging the government in meaningful ways caused delays in the onset of the partnership and
the program. However, taking the time to secure the right agencies and key decision-makers
in the partnerships—the Nigeria FMOH, National Communications Commission, National
Primary Health Care Development Agency and State Ministries of Health—has greatly benefited
the program.The Minister of Health himself has provided support to the program and the
government now sees themselves as key stakeholders in the program’s success.
• In India,ARMMAN worked with the Mumbai Municipality who facilitated the mMitra
enrollment at their hospitals and ANC clinics.As a result, this is the standard blueprint
used by mMitra to expand to other slums.As of December 2016, they were doing facility
enrollment in 77 government hospitals and community enrollment in over 50 slums.The
project also received permission from State Health Authorities in Mumbai and elsewhere,
thus enabling them to partner with state government hospitals.
• In South Africa, the government was kept informed of the program activities, but they were not
a key partner until transitioning to MomConnect.
14
Lesson 3: Partnership structures needed more clearly
articulated roles and responsibilities from the start of
the program.
Once partnerships have been established, it is critical that teams establish mechanisms to help sustain the
partnership over the long term. Establishment of effective governance structures are critical for the partners
to maintain productivity. Structures will vary by context and the constellation of partners, but key elements
of effective governance include fostering a sense of ownership, trust and respect among the partners. Formal
agreements, such as memoranda of understanding (MOUs), can help define the distribution of roles and
responsibilities and reinforce accountability among the partners.

Establishing the Partnership

The time needed to develop the partnerships varied greatly among the countries based on the lead implementing
partner, relationships with government and other resource partners, and the size of the program. South Africa’s
partnership formed most rapidly because the implementing partners and donors had existing relationships that they
could build upon. In contrast, Nigeria’s partnership took the longest to develop (over 1.5 years) because there was
confusion on roles and responsibilities, unclear government involvement and changes in partners. In November
2015, the partnership was redesigned to clarify roles and responsibilities of the founding partners and to articulate
a collaboration mechanism between the implementation partners and with various government agencies and other
resource partners. Jhpiego, through the MCSP took on the coordination role for the partnership.

Health workers practicing how to register

a client for HelloMama at General Hospital,
Ugep, Cross River, Nigeria. Photo credit:
Ayomipo Edinger/Pathfinder International.

15
Partnership Incentives
Programs need to consider the motivations and incentives of each partner when creating the partnership to ensure
that all partners are aligned to achieve the program goals. In the competitive MNO market, some MNOs viewed
working with development agencies and establishing an mHealth service as a competitive advantage, as long as
they were the exclusive provider of the mHealth service. If there was a dominant MNO, certain negotiations, such
as discounted call rates, could be difficult because of the MNO’s large market share. In HelloMama’s case, MNOs
with lower market share were more reluctant to integrate with HelloMama than the higher market share MNOs.
As a result, the HelloMama pilot integrated technology with just two of the four MNOs.

Roles and Responsibilities

Clearly articulated roles and responsibilities are vital to effective partnership to avoid duplication and ensure key
objectives are met. As previously described in the Nigeria partnership, there was confusion among the partners’
roles. As a result, some partners were responsible for key tasks but had insufficient budget to complete the tasks.
In November 2015, several partners exited the partnership. This provided an opportunity for the partners to
review and revise their roles and responsibilities. These changes were incorporated in their MOUs. Although the
partners had previously established MOUs, the government had never signed them so their validity was not clear.

As the programs evolved, some of the roles and importance of different entities also evolved. For example,
India found that they needed to work more closely with telecom companies as time went on because call costs
were a major cost component of the program that limited the ability to rapidly scale up the program. Similarly,
the program needed a closer collaboration with government regulators to accommodate restrictions on call
times and “Do Not Disturb” policies that limit successful and convenient delivery of messages to subscribers.

Partnership Structures
Governance structures are critical to ensure that partners are well aligned and the objectives of the partnership can be
achieved. To be effective, the countries recommend that key partnership structures should consist of no more than
10 entities. It may be necessary to involve other partners during specific points of the initiation and implementation
but if too many organizations are represented, decision-making becomes very cumbersome and ineffective.
Aponjon developed an advisory committee chaired by the Secretary of MoHFW with representatives from other
relevant government agencies, MNOs, bilateral and multilateral agencies, corporate partners, and international NGOs.

At the outset of the Nigeria program, a standalone MAMA Steering Committee was established with the
implementing partners, donors and the government, in name only. In November 2015, the government asked
MAMA to join the Core Technical Committee, at the national and the state levels, that includes all health
projects so that MAMA could be better coordinated with other development activities in the country, rather
than having a separate committee.

016
 The hospital supervisor is explaining the
importance of mMitra voice call messages to
the pregnant women at the ANC outpatient
department.

Decision-Making
Managing expectations of partners is a challenge and is resource intensive. Creating processes for effective,
streamlined decision-making is also important. In Bangladesh, they accomplished this by dividing the advisory
committee into groups with responsibility for specific topics. For example, if a decision about the technology
platform was required, only the technology partners were engaged to make the decision. In Dnet’s case, if
an issue emerged, Dnet would first approach the most conducive partner for their consent, then they would
proceed to the next partner to obtain their consent, referencing the consent they had previously obtained.
This approach worked well unless there were conflicting views. In that situation, only partners with opposing
viewpoints were brought together to reach a middle ground.

Lesson 4: Unanticipated changes in leadership, strategy,

and personnel undermined the effectiveness of mHealth
partnerships.
Public and private partners have been critical to the establishment of the MAMA programs. Partnership success
requires dedicated efforts to maintaining relationships and adapting to changes in partners’ organizational
strategies and personnel. As previously mentioned, partnerships rely on trust and respect among the partners.
During the initial phases of a partnership, these traits are honed through personal relationships until the
partnership becomes more mature and can be expanded throughout the institutions. Thus, changes, particularly
in the early phases of a partnership, can severely challenge a partnership.

South Africa needed to adapt when their partner Cell-Life, which had been involved primarily in localizing
content for the program, closed down. To address this gap, they hired an employee to spearhead the content
adaptation process rather than partnering with another organization. This position was shared with other
programs with a similar need for content development. Later on, the program experienced staff turnover
with their research partner, WRHI. On reflection, the program realized that they had relied too heavily on
relationships with individuals, establishing verbal agreements and allowing their legal relationships to lapse.

0 17
South Africa believes that it is important to legally formalize relationships with partners (e.g., through MOUs)
to reflect verbal agreements for continuity of responsibilities throughout the lifetime of a program, and changes
in partnership relationships must be legally documented.

Initially, Nigeria worked with an ad-hoc government agency, but after almost a year of initiating the partnership,
there was an election and the new President dissolved the agency. As a result, Nigeria had to largely start over to
identify the appropriate government counterpart and ways to best work with them.

018
Financial Health

Financial health concerns the initial and ongoing investments required by the programs, as well as the evolution
of their financial models. Country programs reflect different philosophical perspectives that informed their
ultimate financial models. Requiring users to pay for the service is one philosophical difference, but commercially
sustainable services supported by advertising can be free, and free services can rely on philanthropy or government
budgets to sustain. Ultimately, every service needs a payer, whether the payer is consumer, government,
corporation or charity. The four MAMA countries present an interesting mix of approaches to funding.

Lesson 5:There is no “right” funding model. Country

programs developed diverse funding models, based on
their context and goals for long-term viability.
The long-term approach to sustainability—commercial adoption, government adoption or hybrid—is not
uniform across country programs (Table 2). The financial models implemented varied based on the country
context, the constellation of strategic partners, current donors, and the level of government interest in
integrating the program into existing health services and ICT infrastructure.

Bangladesh is the only country that

Promotional material featuring the Sponsor
charges for the text or voice messages. a Mother Campaign of Aponjon. Photo credit:
Communications, Aponjon.
All the other countries have provided the
messages free to the subscriber, with a
combination of donor, philanthropic and
corporate funding sources.

Bangladesh designed its program from

the start for commercial adoption,
and thus developed a segmented
payment model. Aponjon conducted
‘willingness to pay’ studies to
determine an appropriate price point.
Of the nearly 1.9 million cumulative
subscribers to Aponjon, 96% of
subscribers are charged. User fees
subsidize a small portion of costs to
provide the service for free to the
poorest subscribers, but they do not generate sufficient revenue to cover all program costs. Dnet explored other
strategies including corporate message sponsorship, individual philanthropy (“sponsor a mother” campaign)
and smartphone applications.

South Africa found a low willingness to pay for mobile information. A history of fraudulent mobile phone
subscription programs had eroded public trust in fee-based programs. In addition, the program saw a 95%
drop-off in registrations when they required a nominal Rand 0.20 network fee for enrollment (USD 0.015).

The team tried to enlist corporate sponsors to advertise on their mobile platform, but found this required a
dedicated team for media sales and posed challenges when sponsors were business competitors with one of

0 19
their donors. Though sponsorship provided some money, the program team and donors agreed it was not
worth the effort. The MAMA South Africa program successfully transitioned to government ownership,
becoming MomConnect.

The transition of the MAMA program to MomConnect was significant for the South Africa team. It is difficult
to attribute the successful transition to one key factor, but the following elements were likely important:
a. Constant communication with the appropriate government officials and other influential partners as to
the success of the MAMA program meant that the MAMA South Africa program was top of mind when
considering a mobile maternal program roll out.
b. The scale of the program was particularly significant in terms of setting it apart from other implementations
of mHealth programs. Although many other programs managed to reach thousands of users, none had
shown the scale of the MAMA South Africa program.
c. Creation of multiple channel technology platforms meant that South Africa could respond quickly to
the request from the government to roll out a national scale program with very little adaptation of the
technology needed or the need to develop new platforms.
d. MAMA South Africa content was already adapted to local context and also translated into four languages,
and hence well-positioned to roll out very quickly as a national program that did not discriminate on the
basis of language.

India secured corporate social responsibility funding and support from private philanthropists as well as donors.
Corporate funding has provided very flexible funding, which has helped mMitra rapidly scale. To date, the
corporate social responsibility funding has not required corporate branding, which has been beneficial because
hospitals do not want corporate branding in their facilities. Also, the initial feedback from government facilities
was that they would hesitate to partner if the program charged women, especially because ANC services were
free for the mothers.

Nigeria is working toward government adoption to support scale. They foresee user fees as a barrier to uptake
of the service because women may not yet see its value. The Nigeria FMOH assisted HelloMama in securing
the approval for a zero-rated short code, which enables users to send messages to that short code at no charge
by the MNO. Users will not be charged for calls to this shortened telephone number, allowing the service to be
free to users and securing government support for the program.

Lesson 6: Cost drivers varied greatly because of country

ICT regulations, MNO business models and ICT
technical structures.
Costs varied across the country programs, and the highest cost factors varied by country as a function of the
spectrum of core and strategic partners, delivery channels used, and other program differentiators. However, all
programs cited subscriber recruitment and MNO charges as some of the top cost centers for their programs.
Detailed cost data has not been collected from country programs for this report and a detailed cost analysis was
not within the scope of this report but cost issues were shared and discussed.

020
 The hospital supervisor is explaining at the
ANC outpatient department about the
various types of cancers and precautions the
pregnant women should take. Photo credit:
Sunita Jadhav.

Some subscriber recruitment costs are one-time investments such as producing training materials for outreach
partners, developing registration software, and integration with MNOs. However, many are recurring costs such as
ongoing training for subscriber enrollment trainings, incentives for enrollment agents and mass media campaigns
to build stakeholder and customer awareness. As programs scale up, these costs increase so it is important to
budget for them throughout the life of the project, and budget for the likely increase as the program expands.

In terms of reaching scale, the mobisite for South Africa was the most cost effective channel and continued to
attract users well after MAMA South Africa ended in 2013, particularly once it was integrated into the Vodafone
Live! Operator deck. The operator deck is a mobile site/portal that is accessible only to subscribers of a
particular MNO and is free for that subscriber to access and browse. On this portal, MNOs typically generate
revenue by selling items such as ringtones and wallpapers. In some cases, the operator deck is programmed to
be the standard landing page whenever a subscriber opens their web browser.

However, the push nature of the SMS platform was the most effective in ensuring messages were regularly
delivered to subscribers, and that subscribers remained enrolled for a long period of time. Users of the mobisite
had to be motivated to access the site on a regular basis, so there was a tradeoff between cost and subscriber
engagement. An important point is that a mobisite is expensive to create, but the per user cost decreases as the
site attracts users. SMS, in contrast, has a linear cost profile with no economies of scale (the per user cost remains
static). It is cheap when the program starts, but quickly becomes very expensive as the program grows in size.

Aggregator and MNO costs for voice and text messages varied by country and channel, and speak to the
importance of high-level partnerships to negotiate lower rates for placing calls and sending SMS. Ministries
of Health can work with regulators, ICT Ministry and MNOs to sensitize them to health issues and serve as a
gateway to getting their approval to offer discounted rates. However, even with discounted rates, costs can rise
as programs scale.

021
Content Creation

Content creation was a major effort for country programs, and thus, was added as its own separate lessons
learned axis. The Global MAMA partnership, through BabyCenter, created a global repository of general age
and stage messages for pregnant women and new mothers with children under the age of one. These messages
provided a starting point for adaptation with local stakeholders. Except for Bangladesh, all the countries used
BabyCenter-provided topic maps, guidelines for audio scripts and guidance to the country teams during the
adaptation process. Bangladesh developed their messages prior to the Global MAMA partnership so the
BabyCenter materials were not available. However, BabyCenter helped Bangladesh refine and refresh their
messages. A central tenet of all MAMA content was that it needed to be written with the mother in mind—
to be stage-based (targeted by gestational age/age of the child), warm and relatable, and to build an emotional
connection with the mother by including information on fetal or child development in addition to more
‘clinical’ health messages. The underlying hypothesis was that only by building an emotional connection first
would the content be successful in fostering trust and driving behavior change.

Over the course of implementation, stakeholders supported the continuous improvement of program content.
In India and South Africa, subscribers voiced an interest in nutrition and HIV/AIDS information. As a result,
BabyCenter developed new topic maps to address these areas of interest, and in-country stakeholders further
tailored the content to the local context. India launched the nutrition messages in May 2017.

Lesson 7: Content needed to be hyper-localized with

involvement of local stakeholders.
All the countries invested significant time and effort in working with local stakeholders to develop content that
would be relevant to the local context and in pretesting the messages with the intended audiences.

Content Review Committees

In all four countries, mHealth was novel, so in-country stakeholder involvement was crucial. Government,
NGOs, faith-based organizations, professional associations and other experts contributed substantially to
ensuring that messages reflected the needs, beliefs, local practices, home remedies and barriers of the target
population as well as an understanding of the broader health and societal ecosystem. They improved the
content by contributing to translation efforts and ensuring it met national standards.

BabyCenter provided resources to inform the structure and process for forming content review committees,
which were usually comprised of 8–10 local stakeholders from government, professional associations, NGOs
and other subject matter experts. These review committees took the BabyCenter topic maps and guides,
compared them to national policies and programs, and supported the process of developing and reviewing
messages from behavior change, cultural appropriateness and medical accuracy perspectives. It took countries
between 4 and 9 months to complete this process.

In India, the program had a diverse content review committee, representing multiple viewpoints in the country. This
included doctors from national medical bodies who were familiar with the information delivered to women when
they received care at medical facilities, as well as nurses, midwives, MOH and community members/beneficiaries of
the program. The provider community’s sense of ownership for the quality of messages was strong.

Nigeria’s HelloMama program benefitted from a national-level partnership to ensure message content met

022
national standards. A representative of the Nigeria FMOH actively participated on the content review
committee to create the content as well as the translations for the voice messages. By working with federal and
state-level stakeholders to localize content, HelloMama fostered a sense of joint ownership for the program.

Pretesting
After the content review committee finalized the full set of messages, they selected a subset of messages to
pretest. Pretesting the content (wording, tone and comprehension) and ongoing research was a vital final step
to ensure acceptability and comprehension of the messages as the programs were implemented.

All the countries extensively pretested both voice and SMS messaging with pregnant women, new mothers and
potential gatekeepers.
• In Bangladesh, women’s preferences regarding the types of messages varied; urban women preferred
directional messages, whereas rural women appreciated dramatized content. In some places content had
to be hyper-localized. For example, in some regions the use of oxytocin to stimulate labor was creating high
rates of stillbirths. As a result, specific messages on stillbirth were developed and incorporated locally.
• In Nigeria, the voice persona used in voice OBD messages is a female doctor for messages directed to
women, and a male doctor for messages targeting gatekeepers. These personas were adopted through
feedback from potential subscribers. Most people said they would like to hear from a doctor who also had
experience as a parent, however, men preferred a male doctor, but women preferred a female doctor. Three
voices were tested, and the preferred voices were used for messages.

Focus group sessions with gatekeepers (male)

in Ebonyi,Abuja, Nigeria. Photo credit:Ayomipo
Edinger/Pathfinder International.

23
• Nigeria also changed some phrasing to adapt to local terminology and understanding. One message
originally stated, “Fever, shivering, lack of appetite and drowsiness are signs of malaria. Go to the clinic
for treatment if your baby has these signs.” Through pre-testing, they found that their audience did not
understand the word shivering; ‘shaking’ is more commonly used to explain symptoms of fever. The
message was subsequently changed to reflect local language.
• India realized that the tone and the type of voice were just as important as the actual message. Current
messaging is an optimal mix of medical and emotional content. Through pre-testing, the program also
found subscribers appreciated content most when it was delivered by a warm, sisterly voice rather than by
a person with authority, such as a doctor or nurse.

Content Management
Contrary to what many might believe, content is not static. It requires regular updates as well as reassessment
for new target groups and geographic locations. Content development for new languages or new regions has a
ripple effect on resources required; programs found that expansion was an involved process. As messages were
translated into new languages, programs incurred the costs for tailoring, translation and back-translation to
ensure that the messages were appropriate. The programs had to budget for voice talent when developing
pre-recorded message content, realizing that expansion into each new language would require an additional
session of voice talent recording.
• In India, language, culture and habits vary significantly enough from one location to another that mMitra
tailored message content with every expansion.
• Working with local speakers during translation, HelloMama found differences in community dialects
that required changes in the messages. For example, the word “vaccination” was not understood when
translated, so the program used the more common word “immunization” when translating to other dialects.
Words like “cesarean” could only be translated to the more general “operation” to be understood. Food and
nutrition-related words were especially contextual, with many localized terms incorporated into the Igbo
and Pidgin English content.

0 24
Technology & Architecture

The technology platform is central to any mHealth program. Today, there are more technology options to
choose from, but during the early period for each program (2010–2013), fewer options were available with the
requisite functionality. Country programs needed to decide whether to use existing or build new platforms. The
former may be faster to implement but not provide all the needed functionality, and the latter could take longer
to implement but match all the needs. Neither choice is guaranteed to result in a perfect system. Regardless
of their decision, all programs refined their technology platforms over the course of implementation, with
implications for time, effort, cost and access to data for platform monitoring. This section also reflects
on lessons learned about collaborating with MNOs and mobile aggregators, essential relationships that
simultaneously posed challenges for program sustainability.

Service
'
®)
.
t. t.
g [B / l
Provider
CICICICI
CICICICI

gg~g
71. / t t .
~/
/ Mobile MNO~ ,,, t Subscdbe, t
g1

-
n
,_Co : e ~~

1~1 w
Technology Service
Aggregator

'
CD
,' -- i\
•
t }
~ t
Provider

f Message,
Te
t t
Lesson 8: Evolving program requirements require
in-house technology expertise to translate customized
needs to technology partners and to exert greater
control over the service.
Country programs discussed the importance of forming collaborative relationships with their technology
partners to navigate the trial and error process of developing and/or extensively customizing a technology
platform. As needs arose, modifications to the platforms were often required to improve the program’s
operations. Although technologists may have been aware of the constraints of a technology platform’s
architecture, country programs were often surprised at the rigidity of certain aspects of the platforms, or that
some changes required extensive technology changes.

In Bangladesh and India, the programs decided to develop in-house technology teams who provided better
insight on how to customize the platform and increased the program’s control over the process. The lesson is
that programs should consider including at least a senior level technologist on their team at the beginning of

0 25
 Technology Partnerships
Aponjon in Bangladesh initially used a technology vendor’s existing technology platform in
lieu of a heavy upfront investment to design a platform from scratch. SSD Tech, a leading
aggregator that integrated with all MNOs in the country, adapted their platform to meet
Aponjon’s business needs. However, although the platform facilitated creation of health
worker and agent IDs, all registration was done on paper forms.This limited the ability of
the Aponjon team to monitor agent productivity and subscriber registration in a timely
manner.They determined that they needed a system that incorporated both, essentially
a sales force management system as it would be called in the IT industry.They assembled
an internal team to implement this system. Later they integrated a customer relationship
management database to monitor call center and counseling line performance, which led
to increased customer satisfaction and improved registration data quality.

India’s technology partner, Inscripts, developed mMitra’s system from scratch on a pro bono
basis.The platform connects with IMI Mobile, an interactive voice response (IVR) aggregator
that had partnerships with many MNOs in the country.As the mMitra program grew, the
platform required upgrades and the program needed support in negotiating relationships
with their technology resource partners. mMitra hired a senior-level Chief Technology Officer
and created an in-house technology team to oversee project management with multiple
stakeholders and offer insights about program analytics.This team has helped mMitra increase
the technology partners’ responsiveness to their needs, allowing them to navigate platform
customization needs more quickly.This team also facilitated optimal integration of the
technology platform with the IVR aggregator, directed platform upgrades to support a large
subscriber base, established frameworks for communication between the technology and IVR
aggregator teams, and processes to streamline capture of platform analytics.

the program who can help plan and “translate” between the program team and technology partners and more
directly oversee the technology implementation. Both program and technology teams need to plan ahead (e.g.,
develop a two- to three-year roadmap) and communicate and educate each other about the service requirements
and the technology and architecture implications.

In contrast, in South Africa the lead partner, Praekelt.org, is itself a technology organization and built the
technology platform using their in-house team (rather than contracting the work out to a software vendor).

Depending on the changes required, the technology changes required could be minor or quite extensive and costly.
For example, incorporation of new messages could often be accommodated without much difficulty since the
system was designed to allow this. In contrast, introduction of a new message channel or additional languages
may require changes to the technology platform itself if these were not planned for and incorporated into the
technology architecture ahead of time. For example, Bangladesh recognized that the preferred time slots for
message delivery would be an important feature to offer to their subscribers, and thus added this specification to

026
their technology platform development. However, it was complicated and expensive to design this element in the
technology platform. Because the preferred time slots drastically increased the rate of listening to messages and
improved ease of use for subscribers, Dnet deemed the extra time and cost of the specification worthwhile.

Lesson 9:Working with external technology companies,

especially aggregators and MNOs, accelerated time to
market but was complex to manage.
Working with aggregators that already had relationships with MNOs helped accelerate the launch and get
country programs to scale more quickly than negotiating with individual MNOs.

Bangladesh and Nigeria all began working through aggregators:

a. In Bangladesh, the team transitioned from reliance on its software and aggregator partner SSD Tech to
its in-house technical team to increase responsiveness to program needs and lower costs. The government
helped to negotiate discounted rates with all MNOs (6), working through an aggregator.
b. Building local ICT capacity in Nigeria was a major objective of the government officials. Thus, a local
Nigerian aggregator was chosen with the recognition that they may not have as much experience as
international firms and therefore would require more time and effort to work with them. Even though the
aggregator had existing relationships with the MNOs, HelloMama’s platform needed to be integrated with
the aggregator and each of the MNOs.

Maternal health SMS messages on a

mobile handset.
Photo credit: MAMA/Karin Schebrucker.

27
South Africa encountered the complexity of using multiple message channels to address a broader range
of population segments. This increased complexity also increased costs of troubleshooting, especially with
SMS and voice systems (eliminating voice early on because of cost at scale), because of the multiple partner
systems involved. South Africa experimented with several message channels over the course of implementation
requiring coordination of multiple partners.

With multiple systems linked together, it becomes exponentially harder to isolate and fix errors, compared to
working with a single system because the root cause could be at any point in the message delivery chain: at the
registration point, in the subscription code, at the SMS gateway to the aggregator or between the aggregator and
the MNO. It could also be because of failures in the mobile network itself. Management of system configurations
(versions of software, etc.) needed to be coordinated among the partners, because of small changes in one system
potentially having unexpected downstream impacts on other systems, leading to system failures.

0 28
 Operations

The Operations area describes decisions made during the implementation process. Country programs
learned valuable programmatic lessons throughout implementation, including effective enrollment strategies,
collaboration with and training of community outreach partners and ways to improve program fidelity.

Lesson 10: Customer enrollment required multiple

partners and approaches with “boots on the ground”
to be successful. However, aligning partner motivations,
training and supervision were key challenges to
enrollment at scale.
Countries that enrolled women during ANC in the health facilities were aware that facility registration limits
the number of women that can be reached, so they addressed this limitation in various ways. They used a
combination of approaches to enroll subscribers including community agents, facility-based workers and self-
enrollment. Programs also had media campaigns to raise the awareness about the options for enrollment and
the benefits of the program. The key ingredients for reliable registration, linkage to care and inclusion of the
hard-to-reach require a mix of facility partners, NGOs with field staff and recruitment of MAMA sales agents.

A health worker registering a client for

HelloMama service using information

§4:t AnrenG' o.Ci;,b}?-?!k.. from the ANC card at St. Joseph Hospital,
Akpabuyo, Cross River, Nigeria. Photo credit:
·· ··:· · - •·· •··•• .. Age L..!c{~. . . O. e.ll>.J
1..[, !>
....:... Ayomipo Edinger/Pathfinder International.

1·················--···· •me•llll ·cldress

of siland....... ' ....

.....s.~·······~'\1······)! ·····
·········
········
······..······....··••·· .....········•·
............ ............... ·•·•··••···.
··•····... ··•··•··....... ·····

ofL. :; ~·h ·t :::········

M ·······
·· •-. •. ··· ir
·;j..············
·'.\;,
............
..........
Es ma1eoda e of deh •rt .....J_(j

ilar kept? (If so, lry lop ocu,e it an affix lo lhis recom ).............. ·····················
r:y
Is (T.B.
for co fi ernent: /\ home ·································· ...... .io ~, 1a .............. ..

to (R els, t bercle , meumatism, so e

, tro ble. opera · ns etc.

~sent pregnancy: V mi ng .......... ___________________________

_
........ ____
. ...... :..... __.... Const1pat1on ..

HISTORY
OFPREVIOUS
PREGN 29
ce
 Enrollment in Health Facilities
All the countries had enrollment at the
health facilities with the exception of
Bangladesh. All countries had a paper 6 D
l!£!l Facility-based
registration back up system when the enrollment
network was not functional or to hasten
enrollment during busy times when it • •
was easier to write down subscribers’ tt
information and complete actual
enrollment later. Many of the health
providers complained about this being CHW enrollment
Cloud
additional workload. server

In South Africa, WRHI provided a cadre

of enrollment agents. Users used their
own phone to dial the USSD number
and subscribe with the assistance of an
ambassador who would walk them through the process. However, there were still challenges. Users were willing
to pay for the USSD text message but a large proportion did not have airtime. Reverse billing was not possible,
and it was not economically viable for the program to cover the cost because the lowest denomination of airtime
vouchers that could be purchased was R5.00 (USD 0.39), much larger than the R0.20 (USD 0.015) required for

A CHW filling out the Aponjon registration

form for a new mother. Photo credit: Naimul
Haque Joarder Titu,Aponjon.

30
 ·-

A mMitra training session for ASHAs

conducted at NGO partner Karnal Vikas
Nidhi, Haryana. Photo credit: Shyamsundar
Mochemadkar, ARMANN.

enrollment. That led South Africa to implement a paper-based enrollment process. User registrations increased
as a result but also led to an increased administrative burden and data errors. MomConnect later successfully
negotiated reverse billing for USSD to solve this issue.

India’s program began enrolling women in health facilities and in the community at the same time but they were
not reaching poor women who did not attend ANC. These women were the target population, so they began
enrolling women in the slums through CHWs (further elaboration below).

Enrollment by Community Workers

Bangladesh worked with existing organizations—BRAC, Grameenphone, Smiling Sun—that had a large
network of existing CHWs. These partners brought trust and credibility to a new service. However, the CHWs
had multiple responsibilities and challenges with some of the enrollment processes such as assessing the
economic status of the women. Aponjon gradually experienced a decline in retention rates and acquisition
quality, making the number of refresher trainings and follow-ups with their agents untenable. Their solution was
to establish an in-house quality control unit to verify each paper-based and electronic registration.

In addition, Dnet tapped their own Infoladies,20 a network of last-mile entrepreneurs. Because the catchment area
for an Infolady is somewhat smaller than that of a typical CHW, they have more familiarity with the community.
Moreover, unlike CHWs, they are entrepreneurs seasoned at creating, managing and closing sales prospects. The
Infoladies served as a dedicated sales team for Aponjon, offering advantages over the CHWs of existing organizations
such as lower cost of customer acquisition, higher rates of customer referrals in their communities, and better control
over their performance. They ultimately ended outreach through CHWs working under local NGOs, but continued
their partnership with BRAC as a means of enrolling subscribers and linking them with health services.

Dnet allocated budget and human resources and established enrollment targets to make acquisition of new
subscriptions a priority. CHWs and Infoladies are paid for each successful registration where the subscriber
remains enrolled for at least 2 months and receives a minimum of eight messages. As of December 2016, the field
forces of local NGOs and agencies and Infoladies were responsible for 96% of Aponjon subscriber registrations.
About 4500 agents from various partner organizations have worked on subscriber acquisition since 2012.

0 31
ARMMAN in India worked with a network of NGOs in urban slums, establishing Sakhis (health friends),
community women that enrolled mMitra subscribers. Initially, ARMMAN used a grant model with formal
sub-grants to the NGOs to support a project officer and field supervisors who recruited the Sakhis. However,
this approach created too many layers of administration, inhibiting feedback.

So ARMMAN transitioned to a ‘brand ambassador’ model similar to Dnet in which they paid Sakhis directly
for each woman enrolled, thus reducing the cost of using NGOs as a pass-through payment mechanism. Sakhis
worked across all mMitra program locations, receiving a one-day training in their communities. ARMMAN also
conducted semi-annual lessons learned workshops with Sakhis in each community. To limit human resources
and costs, ARMMAN used a train-the-trainer model to train NGO staff so that they could provide Sakhis with
refresher trainings as needed.

Self-Enrollment
Bangladesh has a self-registration option, but as cited above, the vast majority of enrollments were through
the field forces of a variety of partners including Dnet’s own Infoladies. Mass media was used to promote the
service in Bangladesh, but self-enrollment was underutilized, allowed for duplicative subscriptions, and women
who were not pregnant were able to register for the service.

A junior project officer is completing the

mMitra enrollment form of a lactating
mother from the nearby area.
Photo credit: Amit Nagwekar, ARRMAN.

32
In the case of South Africa, self-subscription to the USSD and mobile website was very successful and the
mobisite constituted the vast majority of the registrations. The problem with these platforms was that they
were “pull,” not “push” platforms, meaning that constant marketing was required to raise awareness and drive
traffic to the mobile website (e.g., digital media such as “Please Call Me” and banner advertising). This strategy
drove a large number of users to the site, but mobile marketing was very expensive. To mitigate costs, the
program placed their mobisite on the Vodacom Operator Deck. This strategy successfully directed the majority
of traffic to the mobisite, ensured free access for Vodacom customers, and was free for the program. Once the
program moved the site to the Vodafone Live! platform where there was constant free traffic, engagement and
usage went up significantly. However, the quality of engagement declined, with some users appearing to use the
platform as a free chat service rather than a source of pregnancy information.

However, when the program was subsumed into MomConnect, the SMS channel was actually chosen for scale,
meaning that all users would then be registered via health workers using the USSD system. This added quite
a burden on to the health workers, but ensured that only women with confirmed pregnancies were registered
(resulting in a clean user registry), whereas the previous approach of self-subscription to the mobisite did not
allow verification that a user was actually a female or pregnant. This was important because it made the user
database much more valuable to the Department of Health.

Lesson 11: Ensuring that women received messages

required specific strategies, such as selecting preferred
timeslots, creating jingles, and returning missed calls.
Countries placed a high value on process monitoring to ensure that the program was being implemented and
received as intended. Strategies to ensure message delivery were based on operational decisions about program
design, regulatory issues, and findings from target audiences about personal preferences for when calls should
be delivered and how often messages should be repeated.

India and Bangladesh used a responsive design process to develop several solutions based on user feedback to
ensure that women received the full set of messages (e.g., two voice calls a week) and talked to a live doctor
(Bangladesh only).

Key strategies included: selection of time of day to receive the message, playing a jingle before the start of
messages and a call back system for missed or dropped calls.
• Bangladesh, Nigeria and India allowed subscribers to select a preferred call time. Both India and Nigeria
have regulations that curtail evening hours for calls that may be more convenient for the subscribers, thus
presenting a significant challenge for the programs.
• Bangladesh faced challenges with subscribers answering voice/OBD calls. Most subscribers had a pre-
paid mobile subscription, so they would not answer the call when their airtime credit was low, reducing the
delivery of the MAMA message.
• India and Nigeria used a jingle to build in time during calls to allow the phone to be passed from the male
subscriber to the woman, and allow time to prepare to receive the message.

033
• India, Bangladesh and Nigeria repeated calls daily for up to three days. In addition a ‘missed call’ option was
available to enable women to hear messages they missed at their convenience. Women could place a flash
call (call that is not answered), which would prompt the system to call the subscriber back at no charge so
that she could listen to the latest program messages.
• During implementation, the South Africa program found that USSD users did not sustain their engagement
with the stage-based communication, even with reminder messages encouraging them to return to the
program. So, this channel was suspended for the remainder of the MAMA South Africa program.

034
Monitoring & Evaluation

Monitoring and evaluation (M&E) is critical to benchmarking progress of any program. This section describes
how the country programs monitored their programs to improve implementation. Although not discussed
in this report, quantitative and qualitative data were collected in Bangladesh, South Africa and India to assess
program impact with respect to changes in knowledge, attitudes, behaviors and service utilization.

Multiple data sources were needed to be able to fully monitor the progress of the mHealth programs. Each
country developed an M&E plan that clearly outlined indicators to be collected, their data source, and frequency
of collection. A framework of global indicators is available at
http://www.mhealthknowledge.org/resources/mama-global-monitoring-and-evaluation-framework.

Lesson 12:The lack of clear understanding of data

needs, reports and partners’ privacy policies led to
delays in harnessing data for service improvements and
actionable insights.
All the countries developed a dashboard as part of their technology platform to track basic information
on subscribers and the enrollment agents (e.g., CHWs, NGO agents, health workers) as well as progress in
delivering the messages via OBD and/or text (e.g., dropped calls, service issues). However, much of the
information for the dashboard had to be provided by the MNOs. Challenges arose in translating the

A new mother reading an SMS from Aponjon.

Photo credit: Unknown,Aponjon.

35
information needed for the dashboard into specific requirements for the MNOs to conduct data collection,
extraction and analysis. As a result, data from MNOs were not always readily available to the country programs.

Bangladesh had trouble accessing MNO service delivery data through their aggregator. The aggregator was
resistant to integrating Aponjon’s internal monitoring platform with their real-time content delivery platform
because they had a fixed reporting infrastructure. The initial agreement between Aponjon and the aggregator
did not explicitly articulate the data and reporting needs/requirements and Aponjon’s limited core expertise in
this area were also a contributing factor.

As the program matured, changes to the reporting architecture became essential. A stopgap solution was tried,
but it never met the needs of the program. Therefore, Dnet built an in-house platform to directly connect
with the MNOs. This required intensive rounds of on-site work, hardware installation and reconfiguration to
connect directly with the MNOs, but in the end, the change provided Dnet more control over the message
delivery process, real-time monitoring of “leakage” (messages lost somewhere during transmission), and access
to all log files, achieving more complete monitoring data.

Even with sufficient service delivery data, mHealth programs needed expertise to make service changes
based on the information. The mMitra program used Tableau to generate service delivery dashboard reports.
However, they felt they would have benefitted from an expert in database design and operations to mine the
vast quantities of service delivery data generated, enabling them to turn the data into actionable insights for
program revision.

Between the baseline and endline, many lines can be measured through routine
monitoring.This is now a customer service operation and needs to be managed like one,
with routine processes to analyze reports and data on a daily and weekly basis for day-to-
day operations and on a monthly, quarterly, and yearly basis for trend and pattern analysis.

Lesson 13: Impact evaluation was underfunded. Country

programs made trade-offs between routine monitoring
and impact assessment.
Because of budget constraints, programs did not have the resources to sufficiently collect, analyze and utilize
evidence regarding impact or cost effectiveness. Although it is recommended that programs dedicate 5% to
10% of their budget for performance management,21 most countries were only able to dedicate 3–5% of their
budget to M&E activities.

Given limited funding for M&E, Bangladesh and South Africa prioritized routine monitoring over impact
evaluation. Bangladesh uses routine monitoring to make decisions about program adjustment and fed into
Aponjon’s responsive design process. They regularly collect, track and analyze data from service delivery
databases, phone surveys and annual field surveys. The major areas under their reporting surveillance are

036
customer acquisitions/registrations, content delivery status, customer satisfaction and impact on knowledge and
behavior of users. Aponjon performs regular bi-annual phone surveys to track the status of 11 health milestone
indicators, including ANC and postnatal care visits, exclusive breastfeeding, and BCG (TB) vaccination, and to
gauge customer satisfaction and loyalty through net promoter scores. The outcomes of these health indicators
are then compared to national level values. In addition, their bi-annual content survey collects user feedback
on the messages to ensure that additional requirements are reflected in the new versions of content. They also
conduct periodic surveys to gather feedback for further modification of the service.

From the beginning, ARMMAN budgeted for dipstick, surveys (surveys which ask open ended questions)
to be integrated into their program design. These surveys were conducted over the phone and collected data
including demographics, phone ownership and habits, and questions on engagement with an intervention.

In addition to monitoring, programs require the capacity to make changes in response to findings. The budgetary
scope of programmatic changes may be hard to estimate in advance, when program budgets are developed.

Several country evaluations were conducted by external entities to increase objectivity regarding program impact,
successes and challenges. Rigorous impact evaluations often include baseline studies before the launch of a
program, and potentially include a control group for comparison, to better attribute changes to the health program.
• Bangladesh: Through a USAID/Bangladesh project, Bangladesh contracted with ICDDR,B to undertake an
independent evaluation of the Aponjon program and thus did not divert funds from Aponjon’s budget for
the program’s implementation and monitoring. No baseline was conducted at the outset of Aponjon so a
retrospective study was done. Findings from the evaluation can be found at http://www.tractionproject.org/
resources/key-findings-mama-study-traction-supprted-study-conducted-international-center-diarrheal
• South Africa: An impact study around PMTCT summarized its findings as follows: “Although the
intervention group was more likely to attend polymerase chain reaction testing within the recommended
time, this result was not statistically significant. However, a statistically significant increase in average number
of ANC visits attended was found, along with an increased likelihood of attending at least four ANC visits,
and improved birth outcomes for pregnant women and newborns. Based on these findings, national maternal
mHealth programs, such as MomConnect, have potential to improve pregnancy outcomes and impact should
be regularly evaluated.” See http://dx.doi.org/10.1080/09540121.2017.1280126
• India: ARMMAN prioritized the importance of generating impact evidence at the outset of the program.
Given ARMMAN’s expertise, they were able to conduct pre-post intervention studies with in-house
staff. India has also been successful in securing external funding to conduct two larger impact studies: a
randomized clinical trial funded by the United Kingdom Department for International Development and
a cost-benefit study funded by the Gates Millennium Scholars Project. An independent research body,
Foundation for Research in Health Systems, is also evaluating mMitra.
• Nigeria is planning to conduct a quasi-experimental research study, with control and intervention sites in
two states after one year of implementation.

037
Overall Achievements

The lessons learned provide valuable insight into the implementation processes of the Bangladesh, South
Africa, India and Nigeria programs. These countries started implementation sequentially and are at varying
stages of implementation and program scale-up. The country programs have found it valuable to share
implementation lessons with one another, and have found that despite their different country contexts, there are
many lessons that have wide applicability.

The country programs have achieved successes in reaching pregnant women, new mothers and their families
with vital information about how to take better care of themselves and their children. Programs have also
developed public-private partnership networks, particularly engaging governments in supporting mHealth
efforts. Overall, the country programs think the MAMA approach works: the messages are highly valued by
those receiving them, and the program is a worthwhile investment for improving knowledge and attitudes.
However, questions remain about the most sustainable business models and the extent of the program’s
behavioral and service utilization impact.

Aponjon has developed a clear value proposition, as evidenced by the 2 million subscribers who have bought
the mHealth service, representing 96% of the program’s total subscribers. In addition, MAMA South Africa
was an essential catalyst to starting the national program MomConnect, designed by the South African
Department of Health. Many of the critical lessons learned through the implementation of MAMA South
Africa were invaluable to MomConnect, and they facilitated its rapid national roll out. Currently, 60% of all
pregnant women in South Africa who gave birth between August 2014 and April 2017 are registered with
MomConnect and 95% of facilities in the country are registering users.

Mother checking maternal health SMS

messages with her baby.
Photo credit: Karin Schebrucker/MAMA.

38
Future Considerations

Future Technology Advances

With the rapid pace of technology advances compared to program implementation, country programs
anticipate that their technical platforms will need to be redesigned and replaced at some point in the not-so-
distant future. Programs must periodically re-evaluate technology requirements such as the need for higher
network capacity and increased electricity coverage, to assess whether these contextual factors have changed
enough to justify new communication channels.

Countries that have been implementing for several years may need a fresh start with a newly designed platform
to take advantage of advances in technology, enable integration of new messaging tools and accommodate
changes in user habits. Older systems can actually be more costly to maintain and update than new systems.
They become more fragile as more patches, workarounds, and retrofits are applied to keep up with new demands
that the system was not originally designed to meet. In the past 5 years, the user bases in Bangladesh and South
Africa particularly have gained greater accessibility to information through applications such as WhatsApp and
Facebook Messenger. Such free technology platforms can be explored as options to drive down program costs,
eliminating the need to pay high rates to MNOs and mobile aggregators for per-message delivery.

Programs are also moving to two-way interactive messaging platforms. A move away from one-way push
messaging toward two-way conversational messaging allows users to provide real-time feedback to the messaging
service, such as to report the quality of care received, user attitudes and self-reported behaviors. Two-way
messaging enables ongoing engagement with a user to occur, creating a user history and providing a longitudinal
source of data that is cheaper to collect and more meaningful than data collected via separate surveys.

Messaging platforms now also offer “chatbot” functionality, enabling an almost human-like text or voice
conversation based on natural language processing and artificial intelligence software.

Technology providers and software developers are also beginning to integrate messaging platforms with the
health system. For example, MomConnect now includes functionality that captures a facility code at time of
registration to note where the user is receiving care, and a help desk service for compliments and complaints.
In this way, the messaging service now serves a useful feedback function. This allows messaging services to
achieve health systems strengthening objectives in addition to behavior change objectives.

Unfortunately, there is limited understanding among donors and government that technology is a very
competitive sector so the life cycle of a technology can be short as competitors produce new, more advanced
and more efficient technologies. However, donors and government have a limited appetite to invest in new
technology. Country programs will need to consider the best use case for these new tools, understand if these
new tools are emerging as leaders and gaining broader acceptance among technologists, and evaluate if they are
a good fit for their program purpose, which is to reach targeted populations with timely and valued information
about safe pregnancy and delivery.

The cost of cell phones is also rapidly declining, and, in particular, smartphones are becoming more affordable
for the middle-class segment of the population because of the proliferation of lower-cost Android phones.

039
Some people already believe that because of declining costs, mobile phones have become a commodity.22 As more
people acquire smartphones to access social media and other apps, the share of revenues from mobile data use
will overtake the share from voice. This improves the business case for MNOs to offer “value-added services”
(likely for a fee), such as digital financial services (mobile money/mobile banking, mobile remittances), mHealth
applications including telemedicine (“dial a doctor”), mAgriculture (market price and trading systems) and other
applications. This will dramatically change how MNOs charge and how the regulators oversee the market.

Future Regulatory Challenges

Country programs anticipate increased challenges with data ownership because policies are becoming
increasingly restrictive on accessing end-user data and obtaining consent. These issues will need to be
considered more thoroughly as the mHealth legal environment evolves.
• In India, the government requires two forms of consent for enrollment into the program, one of which
must be signed in-person.
• In Nigeria, there are cyber security and anti-spam message laws in the pipeline that could affect cloud-
hosted data and the way the country program acquires consent from new subscribers to their service.

040
Conclusion

The field of mHealth has evolved significantly over the last 5 years. There are fewer pilots and more programs
are working toward or reaching scale with a host of partners. Governments are designing digital health
strategies so that those individual programs do not have to address large system issues such as interoperability.
Phones are becoming cheaper and networks are continuously evolving to accommodate more voice and data
volumes at higher speeds, as well as expanding to reach every last kilometer. But mHealth is still an emerging
area, and it is essential for implementers to continue to share their experiences with cost structures and business
models, and contribute to the evidence base, particularly regarding the impact of mHealth on behavior change,
service utilization and health outcomes. These areas of exploration are critical for mHealth interventions to be
designed and delivered in an effective manner.

Given the rapid trajectory of mHealth efforts globally, the country programs believe that collaboration, both
within countries and among countries, is vital. The country programs have found it valuable to have an informal
learning network where they can support each other and share experiences that often have broad applicability,
while understanding the differing context across their countries. This report was developed to share both the
successes and challenges of implementation, and to enhance the work of our fellow mHealth colleagues.

041
Appendix A: Streamlined
MAPS Template
The Big Picture: Overall Feedback on the MAMA Approach
Does it work? Is it worth it? Be honest: given your experiences as a MAMA implementer, and given the evidence you
have seen so far, what do you think of the MAMA approach as a public health investment? Should further evaluation be
pursued? Do you think this approach can improve, and how?

Groundwork & Context

What was the contextual environment (country, government, telecommunications) in which you implemented your MAMA
project? What was the state of digital health in the area where you implemented the project? Were there obvious barriers
to implementation, or did the context seem ripe for a digital maternal health demand generation program? Beyond ICT
infrastructure, what were socio-economic factors that you considered (or didn’t) in your implementation, such as literacy,
access to resources, and gender roles?

Partnerships
Strategic engagement: With whom did your team work to engage to fund and/or support startup of the project? From
whom did your project seek buy-in or formal partnerships? How did those partnerships continue through time, and how
did they evolve and or change?

Content Development,Adaptation, & Management

• The MAMA content repository: One of the value-adds of the MAMA approach was the adaptable messages
created by BabyCenter available through the MAMA website. Did your project utilize the adaptable messages, or
did BabyCenter work with local partners to develop the content, or a different approach? If you used the adaptable
messages, how much adaptation did they require? How did you obtain buy-in to these globally produced messages at
the country level?

• Content development process: What process did you use to create, localize and translate the content? What
worked and what areas do you see for improvement?

• Adapting and pre-testing content: What types of processes did your project use for adapting and/or pre-testing
content? How did you incorporate these learnings?

• Managing content throughout the life of project: Where were your adapted messages stored? Did the
messages change at all over the course of the project in response to local needs or understanding? How were these
changes managed and approved?

• Ongoing updates to messages: BabyCenter recommends updating the content every 2 years to reflect the latest
health guidelines and incorporate any user feedback collected. Do you have a plan in place to update the content in
your program? How will this work be funded and managed on an ongoing basis?

• Content and program design implications: How did you tie content development to your M&E plan? How
did financial and technology decisions impact the content development process?

Financial Health
Financial model: Did your project consider a financial model from the start, or was the plan to define the financial
model during the period in which the project was donor funded? How did the financial model evolve over time? Who led
the development of this model?

0 42
Initial investments versus ongoing costs: What were your experiences in terms of the upfront investment required
to start the project, versus ongoing costs? How did these costs affect plans for financial or institutional sustainability?
Ongoing funding of large-scale SMS and IVR projects has proven difficult for many implementers. Do you have any
success stories or advice to share about financing your project?

Technology & Architecture

• Platform: Tell us about the platform that facilitated your communication with clients. Was it built from scratch?
Adapted from another platform? Who was the main technology partner?

• Data: What types of data did your platform capture? Was capturing and managing data a consideration at project
inception? How did the platform facilitate monitoring and learning from the project?

• Adaptability: How easy or difficult was it to make changes to the platform or to the content on the platform once
rolled out? What lessons did your team learn about changing or editing technology mid-course, and what are the
implications of those lessons if you had to lay out the timeline for another MAMA project?

• Engagement with MNOs and aggregators: Describe your experience (if any) with mobile MNOs and/or
aggregators. What role did they plan in your project, and why did you choose (or not choose) to work with them?
What were the pros and cons of each?

• Documentation: Did your project emphasize technology documentation? Is there a way to easily describe the
architecture of the platform to others, or to transfer documentation to another entity at the end of the project? Where
are technology project management documents stored?

Operations
• Enrollment: How did the project begin to enroll clients? Did your strategy evolve over time? What were the factors
in enrollment strategy that allowed you to reach target numbers of enrolled clients? Were there any difficulties in the
enrollment process?

• Training & support: Did your project engage in any training or capacity-building to prepare for registration? Who
were the trainees? Did they require ongoing support, or just a one-time training?

• Outreach & mobilization: How did your project promote MAMA and mobilize communities to register for the
service? What barriers did you encounter to adoption/uptake? How did you address them?

Monitoring, Evaluation, & Evidence

• Process monitoring: How did the project monitor rollout and benchmark success throughout implementation?
Did your project utilize any dashboards or control interfaces that allowed stakeholders to view data? To what metrics
did you compare MAMA enrollment numbers to create meaningful indicators?

• Outcomes evaluation: Did your project perform an evaluation? If so, what were the methods? What were the
summary results? Have teams been able to measure health outcomes, knowledge, attitudes and/or behaviors? If so,
what outcomes have been achieved? How did the MAMA Hierarchy of Evidence help shape the evaluations?

043
Appendix B: Lessons Learned:
Full List, Ranked by Priority
Axis Lessons Votes
Groundwork
Formative research is essential.There is no substitute for informing service design
(including choice of timing and frequency of messages, separate content for gatekeepers, 12
and differential of urban vs. rural services).
Diversity of country contexts reflected in choice of varied mobile channels such as IVR or
OBD (Nigeria, India, Bangladesh), SMS (Bangladesh, South Africa), and web delivery (South 7
Africa), and channel selection evolves over time.
High mobile penetration is a factor justifying investment in the MAMA approach, but
common barriers included spotty network coverage in target geographic areas, gender gap 5
in phone ownership and ICT literacy.
As a novel concept with no precedent MAMA partnership required a long ramp-up
(average 2 years) to crystallize the concept, solidify buy-in and partnerships, and develop 4
work plans for each of the components.
A pre-existing national eHealth strategy was an enabler in several countries, which
0
recognized the need to move beyond pilots to achieve scale.

Partnerships
Partnership structure must be established from the start, with clear roles. 12
MAMA relies on large institutional partners, which undergo continuous change in
8
leadership, strategy, and realignment of government agencies.
Government endorsements are essential to service credibility, authorization, content
5
approval, coordination of agencies and MNO negotiations.
Corporate support or sponsorships require dedicated resources and a particular skill set
1
to develop.
A motivation expressed by technology partners is the ability to learn from the novel MAMA
0
approach, with potential for them to leverage the experience for future products and services.
Partner in-kind contributions are common (pro bono software, NGO registration support)
0
but it is challenging to assign value.

Financial Health
MAMA partners used a variety of business models to sustain funding and cover costs
(government funding for India and South Africa, user fees for Bangladesh), representing
9
different philosophical perspectives on “free for all” vs. segmented “willingness to pay” for
mobile information.
Highest cost factors varied by country, driven by profile of key coordinating partners,
delivery channel, and other differentiators—but all cite subscriber recruitment and telco 5
charges as top cost centers.
Budget requirements for MAMA implementation require continuous adjustment, as there
2
are not precedents for estimating rate of uptake, or need for adaptations.
Intensive resources were needed to attract, manage and maintain complex MAMA
2
partnerships.These resources were not well reflected in budgets and work plans.
Funding from MAMA donors USAID and J&J would benefit from more predictable
timelines and sequencing, with faster J&J grant mechanism for initiating activities and 1
longer-term USAID approved funds to sustain.

0 44
Axis Lessons Votes
Content Creation
MAMA content must be hyper-localized to reflect local practices, myths, home remedies,
12
dialects, terminology, and national standards.
Localization through in-country stakeholders of global content is essential, providing them
9

@ with the capacity to leverage expertise to continuously improve content.

Global processes such as BabyCenter topic maps and guidelines for audio scripts are
valuable and valued, including structure and process for content review committees.
Content changes are costly when they require software revisions (which is common) or to
5

0
maintain fresh messaging for media-rich environments.

Technology & Architecture

Use of mobile aggregators with existing telco relationships and connectivity (if available)
are essential for getting to get to market quickly, but at scale they can be costly, hard to 11
control and ineffective in holding MNOs to commitments.
MAMA partners required either new or extensive customization of existing software in all
countries, and experience costly trial and error in translating business needs into technical 9

@ ,_______
____
requirements, a specialized skill.
MAMA services require base technology that is as simple and flexible as possible—once
architecture is in place, it is costly and time-consuming to make changes.
Due to its novelty, MAMA implementing partners have struggles with making accurate
_ 7

projections on subscribers and usage, resulting in platforms at higher or lower capacity 2

than needed.

Operations
MAMA utilizes human intermediaries as an interface (CHWs, NGO agents, brand
ambassadors) to enroll subscribers, creating major challenges in covering costs of training 8
and incentives payment at scale.
Solutions in several countries to improve message “dose” for those who do not answer
the calls include “missed call” options that enable women to hear messages at their 5
convenience at no charge.
MAMA automated message delivery model needs to be supplemented by a live call
center (for reporting miscarriages, delivery date, subscriber identity module card changes, 2
technical problems).
Facility based enrollment is easier and less costly than community-based outreach, but does
not reach priority demographic not accessing healthcare; has been used to establish proof 2
of concept before investing in community-based outreach.
Quality assurance for NGO partners and agent networks has been problematic: quality
declines over time; solutions include MOUs with targets, verification per subscriber, 1
stipends tied to service retention.

045
 Axis Lessons Votes

I Monitoring & Evaluation

There is a disconnect between the desired quality of rigorous evidence on impact and
8
budget with trade-offs required on cost, data quality and statistical significance.
M&E resources were not sufficiently included in the design process to address the full
range of life cycle evidence including operational research, user satisfaction surveys, 6
program evaluation and value for money.

@ Challenges arose in translating platform needs for data collection, extraction and analysis
requirements with technology partners, compounded by MNO and tech partner policies
on data ownership and privacy.
USAID and J&J provided a consistent expectation for MAMA partners to build in monitoring
4

indicators across the four countries to measure platform metrics related to message delivery, 2
I
message access, duration, and subscriber retention with dashboards accessible.

0 46
Appendix C: Additional
Resources
General:
• Search results for MAMA showing 36 resources: http://www.mhealthknowledge.org/search/site/MAMA
• The MAPS Toolkit: mHealth Assessment and Planning for Scale. 2015. Geneva: The World Health
Organization. http://www.who.int/reproductivehealth/topics/mhealth/maps-toolkit/en/

Bangladesh Country Program: Aponjon:

Aponjon project page: http://www.aponjon.com.bd/

Evaluation Reports:
ICDDR,B Evaluation: http://www.tractionproject.org/resources/key-findings-mama-study-traction-supprted-
study-conducted-international-center-diarrheal

Conference presentations:
• Understanding mHealth impact among Aponjon (Dnet) subscribers through a phone survey in Bangladesh:
https://www.researchgate.net/publication/262157567_Understanding_mHealth_impact_among_
Aponjon_MAMA_Bangladesh_subscribers_through_a_phone_survey_in_Bangladesh
• Assessing the Impact of Mobile Health Messages among Expectant Women and New Mothers—
Case Study Aponjon: http://pdf.usaid.gov/pdf_docs/PA00JTNG.pdf
• Perinatal Deaths: A Verbal Autopsy on Deregistered Users of Aponjon: http://sbccsummit.org/wp-
content/uploads/2015/07/Abstract-Booklet-FINAL-reduced.pdf

Blog posts:
• Health Market Innovations: http://healthmarketinnovations.org/program/aponjon
• MAMA: mobile technology to deliver health information to mothers in Bangladesh: http://
sunbusinessnetwork.org/casestudy/mama-using-mobile-technology-to-deliver-vital-health-information-to-
new-and-expectant-mothers-in-bangladesh/

Additional resources:
• mHealth Compendium, Special Edition 2016 : Reaching Scale: http://www.africanstrategies4health.org/
uploads/1/3/5/3/13538666/2016_mhealth_31may16_final.pdf
• Bangladesh First MAMA Country to Take Mobile Health Messaging Service National: https://www.
healthynewbornnetwork.org/blog/bangladesh-first-mama-country-to-take-mobile-health-messaging-service-
national/
• Bangladeshi mums benefit from improved health communication: http://www.themalaymailonline.com/
features/article/bangladeshi-mums-benefit-from-improved-health-communication

India Country Program: mMitra

http://mmitra.org/

047
Blog posts:
mMitra: Connecting more moms via mobile: https://www.jnj.com/our-giving/mmitra-connecting-more-moms-
via-mobile

South Africa Country Program: MAMA South Africa

http://www.askmama.co.za/

Monitoring and evaluation reports:

• Cell-Life: http://apps.who.int/iris/bitstream/10665/192492/1/WHO_RHR_13.26_eng.pdf
• Reimagining mobile in South Africa—Lessons from year one: http://www.mhealthknowledge.org/sites/
default/files/Reimagining%20Mobile%20in%20South%20Africa.%20Lessons%20from%20Year%20One.
pdf

Journal articles:
• Journal of Mobile Technology in Medicine—Monitoring MAMA: Gauging the impact of MAMA South
Africa: http://articles.journalmtm.com/jmtm.2.4S.7.pdf
• AIDS Care—Effectiveness of an SMS-based maternal mHealth intervention to improve
clinical outcomes of HIV-positive pregnant women: http://www.tandfonline.com/doi/
abs/10.1080/09540121.2017.1280126?journalCode=caic20

Conference presentations:
mHealth for maternal health—bridging the gaps: https://cdn2.sph.harvard.edu/wp-content/uploads/
sites/32/2014/05/Whats-new-in-mHealth-for-maternal-health.pdf

Media coverage:
• South African partnership hopes to prove text messages can save the lives of mothers and children:
https://www.pri.org/stories/2014-09-29/south-african-partnership-hopes-prove-text-messages-can-save-
lives-mothers-and
• Wider mHealth scope on the cards: https://www.itweb.co.za/content/WPmxVEMKwoeMQY85

Blog posts:
• Praekelt mobile platforms support global health: http://blog.praekeltfoundation.org/post/87070937767/
praekelt-mobile-platforms-support-global-health
• MAMA launches healthy family nutrition program: https://www.healthynewbornnetwork.org/blog/
bangladesh-first-mama-country-to-take-mobile-health-messaging-service-national
• MAMA South Africa Delivers and Women Deliver 2013: http://blog.praekeltfoundation.org/
post/52363574552/mama-sa-delivers-at-women-deliver-2013
• MAMA gives SA mamas the power of health: http://blog.praekeltfoundation.org/post/50076994898/
mobile-alliance-for-maternal-action-mama-gives

048
• MAMA launches in South Africa this week: http://blog.praekeltfoundation.org/post/49763266192/mama-
launches-in-south-africa-this-week

mHealth Compendium, Volume 2: http://www.africanstrategies4health.org/uploads/1/3/5/3/13538666/

usaid_mhealth_compendium_vol._2_us_letter_web.pdf

Nigeria Country Program: HelloMama

http://www.mcsprogram.org/where-we-work/nigeria/

049
Endnotes

1
The MAPS Toolkit: mHealth Assessment and Planning for Scale. Geneva:The World Health Organization; 2015.
http://www.who.int/reproductivehealth/topics/mhealth/maps-toolkit/en/.
2
MAPS toolkit developed in partnership by the World Health Organization Department of Reproductive Health and Research (WHO RHR/HRP),
the United Nations Foundation, and the Johns Hopkins University Global mHealth Initiative.
WHO, UNICEF, UNFPA,World Bank Group, and the United Nations Population Division.Trends in Maternal Mortality: 1990 to 2015. Geneva,
3

World Health Organization, 2015.Accessed 12 Feb. 2017 from http://data.worldbank.org/indicator/SH.STA.MMRT.

4
The MAPS model uses six axes to help programs plan for scale. Given the importance of message content development within the MAMA
process, a seventh axis for Content Creation was added for the purposes of the exercise.
https://www.itu.int/en/ITU-D/Statistics/Documents/statistics/2018/Mobile_cellular_2000-2016.xls
5
Mobile-cellular Telephone Subscriptions.Time Series by Country. ITU, 2015.Web. 6 Feb. 2017.
6
The Bangladesh Telecoms Sector: Challenges and Opportunities.Accessed 21 March 2017 at
http://www.basis.org.bd/resource/Telecom-Challenges.pdf.
7
Ages 15 and older.“Report on the Bangladesh Literacy Survey, 2010.” (2011): Industry and Labour Wing Bangladesh Bureau of Statistics Division
Ministry of Planning, June 2011.Web. 6 Feb. 2017.
8
“South Africa - Literacy Rate.” United Nations Educational, Scientific, and Cultural Organization (UNESCO) Institute for Statistics., n.d.Web. 6 Feb.
2017.
9
Data for 2011. Saurabh, Suman, Sonali Sarkar, and Dhruv K. Pandey.“Female Literacy Rate Is a Better Predictor of Birth Rate and Infant Mortality
Rate in India.” Journal of Family Medicine and Primary Care 2.4 (2013): 349–353. PMC.Web. 6 Feb. 2017.
10
Nigeria Action Plan. Rep. Paris: UNESCO, 2012. Nigeria Action Plan. High Level International Round Table on Literacy: Reaching the 2015 Literacy
Target: Delivering on the Promise, 7 Sept. 2012.Web. 7 Feb. 2017.
11
Data for 2013. Qureshi, Zara, and Lucy Kaaria.“Mapping the Gender Gap:Tracking Movement and Identifying Causes for the Male/female Divide
in Financial Inclusion.” Financial Inclusion Insights.Web.Accessed 7 Feb. 2017 from
http://finclusion.org/uploads/file/reports/InterMedia%20FII%20Gender%20Report.pdf.
12
“Cell Phones in Africa: Communication Lifeline.” Pew Research Center’s Global Attitudes Project. Pew Research Center, 15 Apr. 2015.Web.
Accessed 7 Feb. 2017 from http://www.pewglobal.org/2015/04/15/cell-phones-in-africa-communication-lifeline/#.
13
Hatt,Tim, Kenechi Okeleke, and Mike Meloán.“Closing the Coverage Gap — a View from Asia.” GSMA Intelligence, June 2015.Web.Accessed 7
Feb. 2017 from https://www.gsmaintelligence.com/research/?file=e245c423854fcfd38eeae0a918cc91c8&download.
14
Information and Communications for Development 2012: Maximizing Mobile, page-119
15
Islam, Ifty.“Bangladesh Telecoms Sector Challenges and Opportunities.” 76.1 (2010): n.88.Association of Mobile Telecom Operators of Bangladesh.
http://www.amtob.org.bd/resource/Bangladesh-Telecoms-Tiger.pdf, Nov. 2010.Web. 6 Feb. 2017.
16
Gillwald,Allison, Mpho Moyo, and Christoph Stork. Understanding What Is Happening in ICT in South Africa:A Supply- and Demandside Analysis
of the ICT Sector. Rep. ResearchICTafrica.net, 2012.Web. 7 Feb. 2017.
17
“List of Mobile Network Operators of the Asia Pacific Region.” OMICS International, 2014.Web. 07 Feb. 2017.
18
Nigeria Hello Mama Program team. Email. 21 March 2017.
19
The MAPS Toolkit: mHealth Assessment and Planning for Scale. Geneva:The World Health Organization; 2015.
http://www.who.int/reproductivehealth/topics/mhealth/maps-toolkit/en/.
20
Infoladies are part of a different countrywide program run by Dnet, and are the largest network of last-mile women entrepreneurs.They are
mentored by Hub Managers who advise Infoladies on performance, as opposed to CHWs, who are managed by district managers at local NGO
offices.
21
http://usaidprojectstarter.org/content/pmp-budget accessed October 31, 2017.
22
A Google search of “mobile phones will become a commodity” returns numerous results, including articles published as far back as 2008.
https://techcrunch.com/2008/04/28/cellphones-now-treated-as-commodity-one-third-of-people-ditch-landlines/;
https://www.computerworld.com/article/2474258/apple-ios/apple-advantage--as-smartphones-become-commodity--software-and-services-will-
drive-growth.html;
https://beta.theglobeandmail.com/technology/tech-news/commodity-boom-the-smartphones-global-price-war/article12299152/?ref=.

0 50
0
51
0
52
0
53