SEPTEMBER 2019

A guide for selecting

Monitoring
Evaluation &
Tools
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

01
Monitoring &
Evaluation +Learning
Monitoring and evaluation is a combination of data
collection and analysis (monitoring) and assessing
to what extent a program or intervention has, or has
not, met its objectives (evaluation).

Learning

Learning is taking the information and insights

acquired in the first two steps (M&E) to make critical
strategic decisions. These informed decisions can
be at a management level, a program design level, or
anywhere in between.

The point is to use the data and results obtained

from analyses to breed accountability into an
organizations' activities and also into the M&E
process itself.

Learning can take place during Monitoring,

Evaluation, or both. It recognizes that programs need
to be able to respond to the dynamic nature of
impact-seeking activities. By applying data-driven
insights at various intervals, an organization can
better achieve the outcomes it seeks for its
beneficiaries.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

02
Benefits of
M&E
It is different for each stakeholder.

Stakeholders are organizations, individuals or groups that are

concerned about the activities of a business. Stakeholders can be
internal or external and each group has a different interest in the
company.

Executive management
Determining changes to strategic direction becomes much more
data-driven with the ongoing data and analyses from M&E
processes. Adaption ideally becomes more agile. With relevant and
comprehensive data (both process-related and impact-related),
executives can build much more persuasive arguments.

Employees: Workers
M&E can generate more buy-in and trust in the organization's
commitment to the mission. It helps if there is an apparent effort to
not only assess progress but use that assessment to get better at
delivering impact. For employees in contact with beneficiaries (e.g.
"on the ground") conducting an evaluation, assessment can also
generate more trust between those employees and the beneficiary
community
New, often unforeseen, insights can emerge, helping employees
discover new, more effective ways to deliver programs and create
impact.

Funders
Money for impact flows to where the data is, and proper M&E
implementation can open up that flow because it breeds impact
credibility. A more precise understanding of how much impact can be
generated, more investment dollar comes in.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

02
Benefits of
M&E
It is different for each stakeholder

End Beneficiaries: Customers: Community

Feedback processes (data collection) is a sign that the organization
cares about results, and learning from it to make outcomes better.
Data can be used to improve the efficiency of implementation as well
as implementation design.

Government
Government is an external stakeholder but is not directly tied to the
organization. M&E enables government in making correct policy
changes for example carbon emissions. The government decision
affects the operations of any business with increased levels of
carbon.

Environment
The natural environment can be affected by a company's activities
and, through channels such as climate change, can have an effect
upon the company. Climate change combines power, legitimacy,
urgency and proximity damaging the ability of business to operate,
withholding resources, impacting products, market and
infrastructure.

Suppliers
Suppliers are external stakeholders. Data-driven decisions can
provide inputs and affects how an organization buys raw material,
equipment, etc. Demand for supply chain transparency is increasing
with new generation.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

03
Monitoring
Strategy
Ideal questions to be asked:

What key metrics can give us an idea of

the state of program implementation?

Do we have lean data collection and

analysis processes?

How efficiently are we implementing our

program(s)?

Based on the data acquired, do we need

to make any changes to our program(s)?

A monitoring plan usually focuses on the processes

occurring during the implementation of a program.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

04
Creating a Monitoring
and Evaluation Plan
Quantitative Tools of Evaluation vs. Qualitative

Putting quantitative (or qualitative) tools to work

means defining the right indicators to measure.

In the planning stage, define the indicators that will

be used throughout the monitoring and evaluation.
Indicators can be both quantitative and qualitative,
depending on what needs to be measured and in
what ways.

Quantitative M&E Indicators
Primarily output-focused, they help organizations
determine if activities are taking place, when, and to
what extent. By definition, numbers are used to
communicate quantitative measures (percentages,
ratios, $ sums, etc.)

Qualitative M&E Indicators

It involves subjective terms and often outcome-
focused. They can help organizations determine if a
change has occurred by gathering perceptions from
the stakeholders. Data accuracy can often be
difficult to assess given the subjective nature of the
collecting judgements about change (see example
below)
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

05
Determining
Attribution

Using a Combination of Indicators

Let us take an example of a quantitative indicator,

a number of shoes donated. As we can see, a pure
count of shoes donated doesn't tell us what impact
has been generated. By also collecting qualitative,
outcomes-focused data, the organization gets a
better idea of the impacts of those shoes for
people who before did not have them.

The best indicators help organizations make a

clear attribution between the intervention (shoes
given) and the impact(s) generated. In this
example, there are many other variables that could
contribute to an increase in opportunities generated
by reception of shoes. Gathering qualitative data,
(specifically asking to what extent the shoes had to
do with any observed changes in those areas)
would help to increase the level of attribution the
organization might report.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

06
Evaluation

Evaluation Models Depend on Organizational

Context. Determining an evaluation
method depends on the organizational
objectives.

They are determined by asking:

What is the impact being sought?

What is the purpose of the evaluation?

In addition to those objectives, other

variables include the context of the target
beneficiary group and the organizational
resources available (money, skills, tools,
etc.). Of course, the focus should be given to
achieving the overarching objectives.

The following, while not an exhaustive list,

are some examples of the main evaluation
approaches which accomplish unique
organizational objectives.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

07
Types of Evaluations
In project evaluations, examinations are basically conducted in view of all five evaluation

criteria, Relevance, Effectiveness, Efficiency, Impact, and Sustainability. Depending on the

timing of the evaluation study, the perception of each issue is different. Depending on the

evaluation it can have different project cycles: ex-ante evaluation, mid-term evaluation,

terminal evaluation and ex-post evaluation.

A formative evaluation often conducted before a program

Formative
begins to examine both feasibility and determine its
Evaluation
relevance to the strategic objectives of the overall
organization.

A process evaluation is carried out during the

Process
implementation phase of a program. As the name suggests,
Evaluation
it focuses on processes being carried out -- inputs, activities,
outputs, etc. It identifies any issues with the efficiency of
implementation.

An outcome evaluation aims to determine whether overall

Outcome
program objectives met. In that process, practitioners also
Evaluation
identify what might have spurred or limited those changes.
Finally, it helps shed light on unexpected changes in the
target beneficiary population at the end of the intervention or
at the point of evaluation.

An impact evaluation gets to the heart of a program’s actual

Impact
effectiveness by determining attribution. Changes observed
Evaluation
(outcomes) can be causally connected to the activities
carried out during the program period. Timing is an essential
element of any impact evaluation. Conducted too early, the
intervention has not had time to create observable change.
Conducted too late, it could reduce the usefulness of insights
for informing decision-making.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

08
M&E Tools
Some examples of monitoring and evaluation tools

Directed towards the development sector, Delta offers

practitioners tools for both the planning, tracking, and
measuring stages of M&E. They specifically highlight
NGOs, International Development Agencies, and
Universities as some of their key target segments.

DevResults in perhaps the most robust M&E platform

designed based on results framework. With DevResults,
you can set up your program’s results framework , and
define indicators to measure progress. You can then log
your program’s progress in granular detail.

The challenge for this kind of platform is an extended

implementation period and cost.

In addition to offering a multilingual platform with the

necessities for M&E (defining indicators, etc.), they
provide a mobile application integrated with the platform
to facilitate data collection.

An end-to-end management solution for M&E needs,

Impact Cloud, is a cloud-based platform
that enables practitioners to house and manage data,
conduct analyses, create reports, and collaborate across
teams. It is designed to facilitate impact data
management for organizations of all types and sizes.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

08
M&E Tools
Some examples of monitoring and evaluation tools

One of the most significant benefits of this software is

the ability to manage diverse types of data, from text and
audio to video and online surveys. An automated
transcription feature adds to the efficiency-boosting
nature it provides for qualitative data analysts. It is
designed for qualitative research.

TolaData offers real-time data management, combining

built-in data collection features as well as import
capabilities to manage data collected offline. A results-
framework feature complements the analysis tools so
that users can track how they are doing in reaching their
impact goals. It is used at small & midsize NGO level.It is
easy to setup, integrates data feed from a few offline
tools. It has limitation of reliance on results framework,
limited scalability, limited dashboard, basic scorecard &
no professional impact reporting.

While not as complete a solution, Indicata gives impact

practitioners the tools necessary to execute data-driven
processes in the M&E journey. Like the Impact Manager,
Indicata can help with data collection, analytics, and
visualization (through its dashboards features).
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

09
Limitations of M&E
software today
Many M&E Softwares are legacy softwares, and they suffer from:

The rigid framework that does not allow to define appropriate

social impact context required for each unique initiative.

Long and tedious implementation time. Most software

implementation can take a long time to customize and implement.

Cost is significantly high. 

By the time implementation completes, the requirements may

change.  This creates a tremendous change in control risk.

Some of the features such as project management is best as an

external integration solution as many new-age online products are
available. It can save the cost and complexity.

Lack of cause-and-effect
linkages, not easy to
quantify, fear of framework,
and cost.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

10
5 Mistakes to avoid
in M&E System
Selection
1. Many Features Increases Implementation

Organizations needing M&E software often start by hiring a third

party M&E software evaluator who can help them define the RFP for
M&E software selection. The first sin, if an organization begin this
process without key goals or requirements. In the absence of such
clarity, the M&E software evaluator (who may or may not have prior
experience of implementing M&E software) will start with a
commonly known results framework. She/he will look for a
software that meets defined criteria, resulting in a long list of
features which may not be core requirements for the organization.
While a system with all features may look very comprehensive, the
implementation can go beyond 1+ year. It also requires significant
investment and adds considerable execution risk.

2. Results Framework = Tight Coupling

The results framework describes how a program interventions

contribute to the ultimate results through cause-and-effect linkages,
i.e., associating lower-level results that contribute to output or
outcome to the overall goal. However, these frameworks often have
not been able to trace factors that can influence the results.

Most Monitoring & Evaluation softwares have failed to provide a

mechanism to aggregate survey data. The data as a baseline,
midline, and exit line, and represent the results visually to
understand change. While this is not a limitation of the framework,
it is a flaw in M&E software. A limitation of the framework is that, if
program managers are fearful of not reaching the objective, they
feel they might get penalized for reporting the negative outcomes.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

3. Data Collection is NOT Monitoring and Evaluation

While data collection is an important function, in reality, it is only one part of

the big monitoring and evaluation puzzle. A complete monitoring and
evaluation process requires, connecting funder's impact framework with other
supporting organizations and aggregate results from many online and offline
data collection sources. Unfortunately, both data collection tools and most
Monitoring & Evaluation systems are designed based on limited or rigid data
collection approach, and do not solve any data aggregation challenges.

On the other hand, the modern M&E platforms empower downstream

organizations with a better and easy to use data aggregation system so that
they can find faster outcome results with a Lean Data approach.

4.  Monitoring & Evaluation is NOT about Activity Reporting

Traditional M&E systems excessively focus on activity reporting. 

Empower stakeholders to demonstrate their voice so that the program, service,
and provider have a short feedback loop. Reduce the distance between
funders, top-level, and downstream organizations. Even if you implement the
best M&E software, at best they are good at collect project-level reporting.
Decentralized organizations may have a challenge of rolling up the aggregated
result.

5. Customization-based Systems Will Increase Cost & Risk

M&E systems should guide organizations to log in and implement their impact
strategy, metrics, and survey in a short time. This approach puts program
management or evaluation management staff in the driving seat, allowing
them to configure changes over a long time adapting to changing metrics or
reporting requirements.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

11
Data Flow:
distributed, decentralized
and hierarchical
Many international development organizations employ a
distributed, decentralized, and hierarchical approach.  This
creates many challenges for stakeholders, both internal
(teams), and external (partners, funders, etc.).

Challenges:
Inconsistent reporting to funders:
It is difficult to get a clear, transparent picture of what, how,
and how much impact programs are achieving. An added
difficulty is if different standards are used at each level of data
collection, analysis, and reporting.

Inconsistent data collection processes at multiple locations

If team members are using different tools for data
management, it can create huge reliability issues within data
sets. 

Long reporting cycle

Often because of resource limitations, organizations may not
be employing M&E processes from the start and often
enough. This could reduce the visibility of the impact
generated, while also making it near impossible to have it in a
dynamic contextual manner.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

12
Geographical
Challenges

INTERNATIONAL NGOS, DEVELOPMENT FINANCE, AND

TRADITIONAL DEVELOPMENT AGENCIES

Unlike development agencies, International NGOs typically

have a headquarter in western countries such as the US,
Canada, or Europe. They often act as a funder to the program
organizations in different countries. These country offices
often act as decentralized units and follow different
processes. The challenge with that approach is
headquartered often needs much faster visibility of financial,
donor, operational, and social Impact data which they do not
get.

Traditional M&E software does not provide any support

similar to the data warehouse for the country-level offices.
These offices often collect data in the form of paper, offline
mobile data collection, or local IT systems. However, aligning
data requirements between different offices becomes
challenging.

Mid and Large size development finance institutes often are

faced with inherent limitations of results framework based
design. This approach seems to make the process of impact
data aggregation time-consuming.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

13
Future State of
M&E Systems
Most M&E Systems were designed 10-20 years
ago. Organizations continuing to use these
outdated systems will continue to face mounting
challenges because of their lack of nimble
functionality/adaptability.

What should a future M&E solution look like?

Program Data Aggregation as Data Warehouse

Be able to manage all the data in one place
accessible to on the ground workers, directors, and
funders.

Integrated Impact Learning

Better data analyzing capability to extract important
impact insights.

Social Impact & Outcome Management

Create impact strategies, execute, track/measure,
analyze, improve, and report all in one platform.
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

14
Game Changer
Flexibility
Traditional M&E Software and SoPact Impact Cloud® both have similar
goals. However, their approaches are completely different.

SOPACT
ImpactCloud®

A flexible impact framework

Traditional M&E unique to each organization's
unique need
Tightly coupled with results Start your impact journey with
framework the theory of change   
Has only custom indicators Save time with global impact
Limited reporting format indicators
Limited qualitative and quantitative Multiple reporting formats
alignment Unlimited qualitative and
No concept or scope of SDG quantitative alignment
alignment Aligns with sustainable
Based on rigid reporting framework development goals (SDG) and
approach global indicators
Focuses on results aggregation and Simplifies top level portfolio
does not provide any support for
data aggregation
data aggregation at the downstream
Easier alignment with the
level
downstream, decentralized
Downstream impact organization
organizations
often do not learn social impact
through lean data approach
SOCIAL IMPACT WITH M&E TOOL // SEPTEMBER 2019

15
How to have deeper impact
by Listening to every
Stakeholder?
Integrate qualitative and quantitative analytics and simplify data aggregation from
different data sources such as paper, online survey, offline survey and custom database.
Unleash the power of stakeholders voice to improve program effectiveness and
augmented impact capital.

"Fewer people touched deeply may be worth more than

many people hardly affected."
Impact Measurement and Management Made Simple!
www.sopact.com