Session 9:

Monitoring and Evaluation of

Health Promotion Programs
Prof. Flora Kessy
CMT 05105 Health Promotion
29th January 2018

1
 Learning Objectives
By the end of this session, students are expected to
be able to:
• Define monitoring and evaluation in health
promotion
• Explain the importance of monitoring and
evaluation
• Describe methods of collecting data for monitoring
and evaluation
• Describe the process of monitoring and evaluation.

2
 Monitoring: Definition
• A systematic and continuous assessment of the progress
of an activity over time
– It is a part of implementation
• It can be done through the process of collecting,
coordinating, processing, measuring and communicating
information to assist management to make decisions
• It involves follow up of:
– Inputs - funds, personnel and others
– Process - activities/tasks being done according to planned
interventions.

3
 Evaluation: Definition (1/3)
• The systematic collection, analysis, and
reporting of information about a program in
order to improve planning or implementation
of current and future program activities.
• It includes areas of both process and impact to
assess whether the set out objectives have
been achieved. This can be:
– Internal (by implementers)
– External (by outsiders).
4
 Evaluation: Definition (2/3)
• It can be done at three levels:
– Before implementation of the project to set
baseline – assess development needs and
potentials as well as feasibility of the plan
– During implementation (formative evaluation
or mid-line evaluation to identify areas for
changes or modifications)

5
 Evaluation: Definition (3/3)
• End of intervention evaluation (summative
evaluation) aims at assessing project or program
effect, outcomes and aim at obtaining information on:
– Effectiveness of the program in achieving its stated
objectives.
– Relevance - its contribution to developmental goals.
– Efficiency of the program or project on utilization of
resources.
– Sustainability of the project results.
– Whether to continue, modify or terminate the project.

6
 Importance of Monitoring (1/2)
• To make sure that work progresses according to
schedule
• To ensure that agreed upon rules and
regulations are maintained
• Resources are used rationally as planned
• Required information is available and used
• Detection of problems during implementation
and to take corrective action.

7
 Importance of Monitoring (2/2)
• Tools that can be used to monitor health
intervention:
– Plan of action
– Quarterly technical and financial reports
– Supervision reports, and
– Activity progress reports.
• Outputs/outcomes:
– Are the intended objectives and expectations met?.

8
 Importance of Evaluation (1/2)
• To determine performance effectiveness and
efficiency of services/or program which include:
– Decision whether an activity is worth doing
– Determine whether the objective set were achieved
– Identify ways of improving a program, such as ensuring
that all activities are relevant and appropriate to the
health needs of community members, and removing
potential barriers to participation
– Determine (formative evaluation) whether activities
should be continued or not.

9
 Importance of Evaluation (2/2)
– Use of the evaluation information for a range of
purposes, including promotion, fundraising,
attracting potential partners, and lobbying and
advocating for policy changes to create healthier
communities.
– Determine whether the project should be
extended else where.
– Justifying the program to the management.
– Providing evidence of success or the need for
additional resources.

10
 Evaluation in Health Promotion Programs

• Read Handout 9.1: Evaluation in HP Programs

• It provides more details on:
– What is evaluation?
– Why evaluate?
– Types of evaluation
– Definition of desired outcomes
– Why is specifying goals and objectives important?
– Steps to address goals and objectives
– Checklist for goals and objectives.

11
 Methods for Collecting Data for
Evaluation (1/3)
• Depending on the evaluation purpose, evaluations
can use qualitative or quantitative data collection
methods or a combination of both, i.e. use mixed
methods design
• The selection of data collection methods depends
on a number of factors:
– The purpose of your evaluation
– The questions the evaluation is seeking to answer
– Financial resources
– Time and skills
12
 Methods for Collecting Data for
Evaluation (2/3)
• If you want to explore participants’ or
workers’ experiences, it would be more
appropriate to use qualitative methods
– This will allow you to ask “how” and “why”
questions.
• Qualitative methods answer why and how and
usually involve talking to or observing people.

13
 Methods Collecting Data for
Evaluation (3/3)
• Quantitative methods answer who, what,
where, and how much
– They target larger numbers of people and are
more structured and standardized (meaning that
the same procedure is used with each person)
than qualitative methods.

14
 Key Steps In Evaluating Any Health Program (1/7)

• Clarify your program:

– What was the problem?what underlying needs
and resources must be addressed?.
– Goals, what are the goals, target population, and
objectives?
– What should the members of the intended
population think, feel, or do as a result of the
health promotion program in contrast to what
they thought, felt, or did before?

15
 Key Steps In Evaluating Any Health Program (2/7)

• Define program activities and outputs:

– How can these changes be measured? Establish
measurable program indicators
– Input indicators: measures the resources that are
used in the provision of services e.g. human,
materials and finances
– Output indicators: measures the output following a
process resulting from certain inputs e.g.
community trained, number of latrines constructed,
number of health education sessions conducted.

16
 Key Steps In Evaluating Any Health Program (3/7)

– Outcome indicators: measures result following some

activity e.g. number of people came for counselling
and testing services.
– Impact indicators: Show the long term effects of an
intervention e.g. reduced number of malnutrition in
the community, leading to improved health status of
children in a given time
– Engage stakeholders/community members in the
health promotion program
– Understand stakeholder interests and expectations
(needs assessment)
17
 Key Steps In Evaluating Any Health Program (4/7)

• Assess resources for the evaluation

– Determine availability of staff and resources
– Determine amount of money allocated for evaluation
• Conduct a resource mapping and asset assessment
• Examine the community resources and other assets that exist
(or do not exist) to respond to the targeted problem/issue in
the community
• Design the evaluation
– Select type of evaluation to be conducted
– Design evaluation framework
– Consider ethical and confidentiality issues
18
 Key Steps In Evaluating Any Health Program (5/7)

– Develop evaluation questions (based on program goals and

objectives and stakeholder interests and expectations)
– Select a target area to be assessed.
– Gather data to develop a clear ‘picture’ of the nature and
extent of the problems in that geographic area
• Gather data that help describe the nature and causes of the problem.
– Assess the risk and protective factors of participants in the
target area
– Once you have identified a target group, conduct a systematic
assessment of those risk conditions that contribute to the
problem/issues and those protective factors that improve risk
conditions.

19
 Key Steps In Evaluating Any Health Program (6/7)

• Determine appropriate methods of measurement and

procedures
– Select the most appropriate qualitative and quantitative
methods for collecting information about your program.
– Consider qualitative versus quantitative methods; and
– Select your sampling design.
• Develop work plan, budget and timeline for evaluation
• Collect the data using agreed-upon methods and
procedures
– Pilot test
– Determine data collection techniques.

20
 Key Steps In Evaluating Any Health Program (7/7)

• Process and analyze of data

– Prepare the data for analysis
– Analyze the data
• Interpret and disseminate the results
– Interpret the results
– Present the results
– Share the results
– Take action

21
 Selecting Evaluation Type for Health Promotion Program (1/3)

• When selecting types of evaluation for your

Health Promotion program, it is important to
consider:
– The length of time the program has been in operation
– Your program’s stage of development
• What, if any, types of evaluation have already been
conducted?
• The amount of resources you have to devote to evaluation?
• The interests of your program stakeholders

22
 Selecting Evaluation Type for Health Promotion Program (2/3)

– How do you expect change to occur?

• Will it be slow or rapid?
• What measurable intermediate outcomes (steps toward the desired
behaviour) are likely to take place before the behaviour change can
occur?
– How long will the program last? What kinds of changes can
we expect in that time period (e.g., attitudinal, awareness,
behaviour, policy changes)?
• Sometimes, programs will not be in place long enough for objectives
to be met.
– To help ensure that you identify important indicators of
change, decide which changes could reasonably occur from
year to year.

23
 Selecting Evaluation Type for Health
Promotion Program (3/3)
– Which aspects of the outcome evaluation plan best fit
with your organization’s priorities?
• Other factors to consider when deciding on an
evaluation type
– Obtain community members support
– Establish healthy community committee – partnership
with population from all levels should be actively
engaged in the program
– Conduct situational assessment to understand the
needs and preferences of the people
24
 Conducting Outcome Evaluation:
Questions to Ask and Answer
• How can we use outcome evaluation to assess
the effectiveness of our program?
• How do we decide what outcome evaluation
methods to use?
• How should we use our evaluation results?
• How can we determine to what degree we have
achieved our communication objectives?
• How can we make our communication program
more effective?
25
 Conducting Outcome Evaluation: How Well is the Program Working? (1/3)

• Conduct outcome evaluation by following these

steps:
– Determine what information the evaluation must
provide that help to answer important questions, such
as
• Did the program work? Why? Why not?
• Should we continue the program?
– What can be modified to make the program more
effective?
• What evidence proves that funders should continue to spend
their money on this program?

26
 Conducting Outcome Evaluation: How Well is the Program Working? (2/3)

– Define the data to be collected

• Knowledge: What people learn about a subject (e.g. the
short- and long-term health risks of smoking)
• Attitudes: How people feel toward a subject (e.g.
smoking is dangerous to their health)
• Skills: How peoples’ skills and abilities affect a problem
by themselves (e.g. variety of ways to say ‘no’ to
smoking and awareness of smoking cessation classes)
• Behaviors: How people actually change their way of
doing things (e.g. a measurable decrease in participants
who smoke)

27
 Conducting Outcome Evaluation: How Well is the Program Working? (3/3)

• Decide on data collection methods

• Develop and pretest data collection instruments
• Collect data
• Process data
• Analyze data to answer the evaluation questions
• Write an evaluation report
• Disseminate the evaluation report

28
 Conducting Outcome Evaluation

• Read:
– Handout 9.2: Conducting Outcome Evaluation in
Health Promotion Intervention
– Handout 9.3: Evaluation Report

29
Enjoy your community health field
experience!!!

Thanks
www.ttchih.org
[email protected]
30