Project Monitoring & Evaluation (BMPR 3)
Project Monitoring & Evaluation (BMPR 3)
(TIA)
COURSE: BMPR 3 (F/TIME & EVENING 2022/2023)
SUBJECT: PROJECT MANAGEMENT
CODE: BAU01101
PROJECT MONITORING AND EVALUATION
LECTURER: DR. ANICETH KATO MPANJU
However, it is a method often overlooked and only done for the sake of fulfilling the
requirements of a project management plan. But if put into practice,
Project monitoring can help project managers and their teams foresee potential
risks and obstacles that if left unaddressed, could derail the project.
It clarifies the objectives of the project, links the activities to the objectives, sets
the target, reports the progress to the management and keeps the management
aware of the problems which crop up during the implementation of the project.
It supports and motivates the management to complete the project within the
budget and on time.
To simply put, project monitoring is overseeing all tasks and keeping an eye on project
activities to make sure you’re implementing the project as planned.
The process of project monitoring begins during the planning phase of the project.
During this phase, it is important to define how the project success will look like and
how the goals can be measured using KPIs (Key performance indicators).
1
1.2 DESCRIBE EVALUATION
1.2.0 Introduction
When it comes to project management, closing out a project isn’t just a matter of
executing deliverables. Though the process may seem tedious or overly administrative,
a formal closure phase ensures all loose ends are tied up, documentation is signed and
approved, contractors are paid, and everyone is on the same page.
The closing phase also gives you the opportunity to review and evaluate the project’s
success (or failure), which is crucial for planning and executing successful projects in
the future.
1. Pre-project evaluation: What can I expect from the delivery of this project?
What are the obstacles I could face? What potential risks and issues are
present?
2. Ongoing evaluation: Is the project moving forward efficiently and toward the
goals set prior to its initiation? Am I properly using resources? What risks have
arisen now that the project is in motion?
3. Post-project evaluation: What went well? What did not? What lessons can be
learned from any struggles to ensure the next project runs more smoothly?
2
the independence of the evaluators. Their expertise and independence is of major
importance for the process to be successful.
A monitoring and evaluation (M&E) plan is a document that helps to track and assess
the results of the interventions throughout the life of a program. It is a living document
that should be referred to and updated on a regular basis. While the specifics of each
program’s M&E plan will look different, they should all follow the same basic structure
and include the same key elements.
An M&E plan will include some documents that may have been created during the
program planning process, and some that will need to be created new. For example,
elements such as the logic model/logical framework, theory of change,
and monitoring indicators may have already been developed with input from key
stakeholders and/or the program donor. The M&E plan takes those documents and
develops a further plan for their implementation.
An M&E plan should be developed by the research team or staff with research
experience, with inputs from program staff involved in designing and implementing the
program.
An M&E plan should be developed at the beginning of the program when the
interventions are being designed. This will ensure there is a system in place to monitor
the program and evaluate success.
3
1.3.5 A Step by Step Guide to Monitoring & Evaluation
The first step to creating an M&E plan is to identify the program goals and objectives. If
the program already has a logic model or theory of change, then the program goals
are most likely already defined. However, if not, the M&E plan is a great place to start.
Identify the program goals and objectives.
Answering these questions will help identify what the program is expected to do, and
how staff will know whether or not it worked. For example, if the program is starting a
condom distribution program for adolescents, the answers might look like this:
From these answers, it can be seen that the overall program goal is to reduce the rates
of unintended pregnancy and STI transmission in the community.
It is also necessary to develop intermediate outputs and objectives for the program to
help track successful steps on the way to the overall program goal. More information
about identifying these objectives can be found in the logic model guide.
4
Step 2: Define Indicators
Once the program’s goals and objectives are defined, it is time to define indicators for
tracking progress towards achieving those goals. Program indicators should be a mix of
those that measure process, or what is being done in the program, and those that
measure outcomes.
Process indicators track the progress of the program. They help to answer the
question, “Are activities being implemented as planned?” Some examples of process
indicators are:
Number of trainings held with health providers
Number of outreach activities conducted at youth-friendly locations
Number of condoms distributed at youth-friendly locations
Percent of youth reached with condom use messages through the media
These are just a few examples of indicators that can be created to track a program’s
success.
The source of monitoring data depends largely on what each indicator is trying to
measure. The program will likely need multiple data sources to answer all of the
programming questions. Below is a table that represents some examples of what data
can be collected and how.
5
Information to be collected Data source(s)
Reach and success of the program Small surveys with primary audience(s),
intervention within audience such as provider interviews or client exit
subgroups or communities interviews
After all of these questions have been answered, a table like the one below can be
made to include in the M&E plan. This table can be printed out and all staff working on
the program can refer to it so that everyone knows what data is needed and when.
6
Indicator Data source(s) Timing
MoHCDEC or other
Percent of adolescents reporting condom
population-based Annually
use during first intercourse
survey
MoHCDEC or other
Number and percent of new STI
population-based Annually
infections among adolescents
survey
The next element of the M&E plan is a section on roles and responsibilities. It is
important to decide from the early planning stages who is responsible for collecting the
data for each indicator. This will probably be a mix of M&E staff, research staff, and
program staff. Everyone will need to work together to get data collected accurately and
in a timely fashion.
7
Data management roles should be decided with input from all team members so
everyone is on the same page and knows which indicators they are assigned. This way
when it is time for reporting there are no surprises.
An easy way to put this into the M&E plan is to expand the indicators table with
additional columns for who is responsible for each indicator, as shown below.
Data Data
Indicator Timing
source(s) manager
Training
Number of trainings held with health Every 6 Activity
attendance
providers months manager
sheets
Condom
Number of condoms distributed at Every 6 Activity
distribution
youth-friendly locations months manager
sheet
MoHCDEC or
Percent of adolescents reporting other Research
Annually
condom use during first intercourse population- assistant
based survey
8
Data Data
Indicator Timing
source(s) manager
based survey
Once all of the data have been collected, someone will need to compile and analyze it
to fill in a results table for internal review and external reporting. This is likely to be an
in-house M&E manager or research assistant for the program.
The M&E plan should include a section with details about what data will be analyzed
and how the results will be presented. Do research staff need to perform any statistical
tests to get the needed answers? If so, what tests are they and what data will be used
in them? What software program will be used to analyze data and make reporting
tables? Excel? SPSS? These are important considerations.
Another good thing to include in the plan is a blank table for indicator reporting. These
tables should outline the indicators, data, and time period of reporting. They can also
include things like the indicator target, and how far the program has progressed
towards that target. An example of a reporting table is below.
9
Baselin Year Lifetime % of target
Indicator
e 1 target achieved
Percent of adolescents
reporting condom use during 20% 30% 10% 31%
first intercourse
The last element of the M&E plan describes how and to whom data will be
disseminated. Data for data’s sake should not be the ultimate goal of M&E efforts. Data
should always be collected for particular purposes.
10
Consider the following:
How will M&E data be used to inform staff and stakeholders about the success and
progress of the program?
How will it be used to help staff make modifications and course corrections, as
necessary?
How will the data be used to move the field forward and make program practices
more effective?
The M&E plan should include plans for internal dissemination among the program team,
as well as wider dissemination among stakeholders and donors. For example, a
program team may want to review data on a monthly basis to make programmatic
decisions and develop future workplans, while meetings with the donor to review data
and program progress might occur quarterly or annually. Dissemination of printed or
digital materials might occur at more frequent intervals. These options should be
discussed with stakeholders and your team to determine reasonable expectations for
data review and to develop plans for dissemination early in the program. If these plans
are in place from the beginning and become routine for the project, meetings and other
kinds of periodic review have a much better chance of being productive ones that
everyone looks forward to.
Conclusion
After following these 6 steps, the outline of the M&E plan should look something like
this:
1. Introduction to program
Program goals and objectives
Logic model/Logical Framework/Theory of change
2. Indicators
Table with data sources, collection timing, and staff member responsible.
5. Dissemination plan
Description of how and when M&E data will be disseminated internally and
externally
11
1.4 CHALLENGES FOR MONITORING AND EVALUATION
1.4.0 Introduction
So without further ado, let's dive into the eight worst mistakes to avoid when choosing
monitoring and evaluation systems.
In this section, we will describe the most common monitoring and evaluation
challenges. So, let us start this with most challenging obstacles in Monitoring.
1. Limited Knowledge, Skills and Competence
The main challenge is that the knowledge, skills and competence required for those
aspiring and performing duties related to M&E of public projects is limited. Officials fail
to understand the importance of M&E at the local government level of the various
projects. Therefore, they have failed to develop an institutional M&E system (including
M&E plans, indicators and tools). This reveals that although much has been achieved in
terms of providing services to the majority of Tanzanians, much still needs to be done
in terms of training, workshops, dialogue on M&E and how suitable systems can be
implemented at local government level to enhance service delivery. The definition of an
M&E system requires that such a system be established across provinces to attain
effective and efficient service delivery.
12
Inadequate finance is a perennial challenge facing project management in local
governments. Many projects are abandoned in local government due to inadequate
funds. The flow of funds cannot be fully guaranteed, especially because municipalities
have to face a fluctuation in world oil prices, inflation, mismanagement, corruption, and
a failure to explore internal sources of revenue and to use scarce resources.
Mismanagement of funds and corruption hinder successful M&E and hinder the
completion of projects at local government level, which in turn leads to dissatisfaction
among citizens and sometimes to violent service delivery protests.
M&E experts report a lack of usable data. When they are unable to collect sufficient
data, it is very challenging to perform a proper analysis, draw usable conclusions and
make data-driven suggestions to improve development programmes and interventions.
Sometimes aid beneficiaries are located in remote areas that are hard to reach, and are
sometimes unsafe. Apart from dangers for staff and the difficulties reaching the areas,
it is often also expensive to send personnel to perform M&E activities in these locations
and to collect the necessary data with a frequency they require.
Most M&E projects and programmes favour after-the-fact evaluations over near-time or
real-time monitoring. However, learning from evaluations is often a longstanding
process. Much is to gain from shorter learning cycles and implementing lessons learned
from monitoring directly into the development projects while they are still running.
The primary purpose of impact measurement is to drive a better outcome for the
beneficiary. Who are our stakeholders? According to principles of Social Value
International, stakeholders are who affects and are affected by your activities.
However, do you have a system that communicates stakeholder's user experience or
satisfaction or dissatisfaction?
13
Does it make sense to report the number of meals to children while needs often
tend to outstrip any intervention?
The most significant barrier for funders in making a decision is a lack of visibility
between funding and actual impact. Provide the best way to provide impact capital
distribution through a better impact evidence.
1.5.0 Introduction
Risk is inherent in everything we do. Project managers, more than most, know how to
mitigate risk, and use it as a core strategy in project management.
To understand where risk can come into a project, always start with the lens of the triple
constraint. When you are documenting risks, note where impacts to time, cost and
quality are likely to occur. Once you’ve identified risks, you’ll want to work with your
team to develop strategies for addressing them, should they arise.
You cannot manage risks until you first identify them. As you read these notes, perhaps
you thought, “That’s something I need to work on.”
14
Select one or two of these tips and incorporate in one of your projects. Over time,
incrementally take additional steps to help you better identify your project risks.
If we are going to identify project risks, it is critical that we first define what we mean.
When people hear the term “risk,” they often think of negative events. Why? Well,
the dictionary defines risk as “the possibility that something bad may happen.”
Notice the words “possibility” and the phrase “may happen.” Uncertainty is implied.
We are on the right track, but something is missing here. The definition has a singular
focus on bad things that may happen.
A more modern definition of risk includes not only the bad things but recognizes
the possibility of good things that may happen. (Some project managers push
back on the concept of positive risks.)
Whether you support the concept of positive risks or not, wise project managers
employ a method for identifying and seizing opportunities (positive events and
conditions). Project managers who neglect opportunities lose one of the most powerful
ways of achieving their objectives.
The important thing in your projects is that your stakeholders have a common
understanding of the term risk. This is the beginning point of identifying risks.
There may be some things in your project that are non-risks, perhaps risks that have
already occurred. That’s another topic called issues.
15
It is suggested that you think about your risks in a consistent fashion. Here are some
examples of risk statements:
Notice that the causes and the effects are NOT risks. Think of your risks as future
events or conditions that may or may not occur.
Tip: If the Cause is uncertain, you have another risk. Ask the subject matter experts
what is causing the uncertainty. In effect, you should continue to drill down until you
understand the root cause (e.g., a fact or condition that gives rise to the risk).
If someone looked in your risk management toolbox, what tools would s(he) see? Do
you have one or two tools to identify project risks?
Some project managers do the same thing on every project. They have one meeting
with a few subject matter experts, brainstorm the risks, and call it done.
That’s a good start, but consider adding and using more tools and perform periodic risk
reviews to evaluate current risks and identify new risks.
16
How about a root-cause analysis to discover the underlying causes of your threats and
opportunities?
Next, the SWOT analysis can help you and your stakeholders to identify strengths,
weaknesses, opportunities, and threats. This technique may be applied to a business
process, a system, or your project, to name a few.
How many times have you been bitten by wrong assumptions or poorly understood
constraints? Project managers can perform assumption analysis to test assumptions.
The constraint analysis can help you identify limitations that you can work to reduce or
remove.
17
Lastly, consider using a predetermined list of risk categories called a prompt list. For
example, you could facilitate a meeting where you ask stakeholders to identify risks for
the project schedule, cost, quality, and scope.
Schedule risk – schedule events or conditions that, if they occur, will cause a
positive or negative impact to the objectives.
Cost risk – cost events or conditions that, if they occur, will cause a positive or
negative impact to the objectives.
Quality risk – quality events or conditions that, if they occur, will cause a positive
or negative impact to the objectives.
Scope risk – scope events or conditions that, if they occur, will cause a positive
or negative impact to the objectives.
You may have the best facilitation skills in the world, but if you don’t have the right
people in the room, you will likely miss some of the most significant risks.
Do whatever is necessary to get the subject matter experts to help you identify project
risks.
It starts with your stakeholder analysis while you are initiating your projects. Identify
the individuals, groups, and organizations who may impact your project.
Use Stakeholder Register to identify anyone who may be impacted by your project.
What about people outside your organization? Yes, those people too.
As you plan for your risk identification meetings, review your stakeholder register and
select the appropriate stakeholders.
18
5. Look beyond the Obvious Risks
Many project managers make the mistake of only looking for the obvious risks.
We are suggesting that you look around the corners. What are the other potential
risks?
As you identify projects risks, you should capture the risks in a risk register. Make your
job easy. Use one place to store all your risk-related information.
What kind of information should we store in the risk register? Common things include:
Risk ID
Risk Statements
Risk Owners
Risk Triggers
Risk Categories
Probability Risk Ratings
Impact Risk Ratings
Risk Scores
Risk Response Strategies (e.g., mitigate)
Risk Response Plans
Residual Risks
Risk Trends
Keep in mind, risk identification is NOT a one-time event. Start early in your
projects. And perform periodic risk reviews to review and update your current risks as
well as add new risks.
19
How will things be different with this information?
In the above table, failure by suppliers to deliver some components has been rated as a
minor risk. This sort of judgement can only be made on the basis of experience and
within the context of the current project. If the supplier is well known and trusted, then
the likelihood of them delivering late is likely to be low and hence the risk can be
classified as minor.
20
benefits of having a separate risk management officer since it is unlikely a project
manager, however honest, would rate his own scheduling abilities as “high” risk.
REVIEW QUESTIONS
21