Domain VI: Problem Detection and Resolution:

Part 2

This course is based on PMI’s Agile Practice Guide ® The PMI Registered Education Provider logo
is a registered mark of the
PMI and PMI-ACP are registered trademarks of the Project Management Institute Inc. Project Management Institute, Inc.
 Learning Objectives

By the end of this lesson, you will be able to:

Discuss the meaning and benefits of metrics and measures

Analyze the deviation between planned and actuals using
variance and trend analysis techniques
List and explain the four steps of risk management life cycle
Classify risks and risk trends using risk log, risk burndown
chart, risk profile graph, and spike
Identify types of agile failure modes and agile coach failure
modes
Discuss pain points encountered in agile teams and
troubleshooting guidelines to resolve them
Metrics and Measures
 Measures

Measure can be a quantity, proportion, or qualitative comparison.

Quantity Proportion Qualitative Comparison

Example: "There Example: “There Example: "The

are 25 open are 10 percent new version of
defect reports on fewer open the software is
the application defect reports easier to use
as of today." this week than the old
compared to the version."
last week."
 Metrics

Metric refers to a standard for measuring or evaluating something.

Metrics are important for the success of a project. They are beneficial to a project as they:

Help in retrospectives

Provide continuous feedback

Help in maintaining a high-quality code base

Help in timely refactoring of code (code complexity)

Support frequent releases

Identify issues at an early project phase

 Example of Metrics

Some of the most commonly used metrics in projects are as follows:

Business metrics Process metrics Project testing metrics

• Running tested features (RTF) • Impediments cleared per • Acceptance tests per story
• Earned business value (EBV) iteration • Defects count per story
• Net present value (NPV) • Impediments and user stories • Escaped defects per cycle
carried over to the next • Time to run tests
• Return on investment (ROI)
iteration • Tests run per frequency
• Internal rate of return (IRR)
• User stories done per iteration • Time to fix tests
• Defects carried over to the
next iteration
• Team member loading
• Velocity
• Backlog size
 Baseline Metrics

Baseline metrics are calculations that result in the baseline data for a project.
Some of the baseline measures used for comparison are as follows:

Length of each Number of story

Number of planned
sprint in calendar points planned for
sprints in the release
days the release

Budget planned for Start date of the

the release project
 Recommended Practices for Metrics

Some of the best practices for using metrics in projects are:

Measure outcomes, not Measure results, not

outputs activities

Measure work items

Follow trends, not numbers
done, not time spent per
task
Variance and Trend Analysis
 Variance Analysis

Variance analysis is a quantitative analysis of the difference between:

Planned Metrics Actuals

Variance

Normal if it is within Action to be taken if it is

permissible limits. beyond set limits.
 Variance Analysis

Variance analysis is a quantitative analysis of the deviation between the planned and actuals.
Variance can be of two types:

Common Cause Variations

Special Cause Variations
Factors that are inherent in the system or
process and that have minimal impact on Factors that greatly deviate from the plan
the outcome. should be proactively monitored.
Example: If the velocity trend of a team is: Example: If the velocity trend of the team is:
56, 60, 63, 59, 61 56, 30, 60, 35, 25
It can be stated that the team is delivering While the team can deliver 56 story points in
consistently. one sprint, they delivered only half the
velocity in the next sprint.
 Trend Analysis

Trend Analysis is done to analyze the measurements taken over time to gain a better
understanding of the performance of a metric.

Tracking their
trend over time Such observations
Two or more
helps to can be used to
metrics can be
understand the improve the
combined for
influence of one project processes
effectiveness.
metric over the and practices.
other.

If there are 25 risks identified at the beginning of the project, it conveys little meaning. However, if the
number of risks reduces over time, it means that the risk management in the project is highly effective.
Risk Management life cycle
 Risk Management Life Cycle

Risk Identification
Done during daily scrum, includes
retrospection, requirement workshop,
sprint planning, and sprint review

Agile Risk Risk Assessment

Risk Review
Management Captures the likelihood, impact,
Done during retrospective Process and exposure

Risk Response
Decides to avoid, mitigate,
transfer, or accept the risk
 Step 1: Risk Identification

The team identifies project risks and records them in a risk register or spread sheet.
Risk identification is done using information gathering techniques, such as:

Checklists

Risk

Document reviews

Assumption analysis
 Risk Identification Opportunities

Risk identification in a project can happen at the following stages:

Team and the product manager discuss new ideas to

Requirement Gathering consider and adjust requirements in a way that maximizes
value and minimizes risk

Team estimates the relative size of each user story. Rejecting

Planning Poker
user stories that are over a certain size helps reduce risk

Team identifies, assesses, and responds to risk. Team works

Iteration or Sprint Planning with the product manager to maximize output, thereby
reducing the risk of failure
 Risk Identification Opportunities

Team identifies risks and issues. The team members add

Scrum Meeting the risks to the risk board and document the agreed
response plan

Team discusses risks. The team members should

Iteration or Sprint Review highlight successful mitigation or evasion activities and
share them with the stakeholders

Team discusses risks handled successfully, the chances of

Retrospective Meeting reoccurrence of risks, and what will be done differently
for risk mitigation in the subsequent iteration or sprint
 Step 2: Risk Assessment

The risks identified in step 1 need to be assessed based on the probability of failure, impact, or loss generated
by the risk. Classifying risks can serve as a checklist for identifying the risks.

A broad categorization is as follows:

Business risk Technological risk Logistical risk

 Step 2: Risk Assessment

To ensure that a broad range of risks have been identified, the following acronym can be used:

P
Political

E
Environmental

S
Societal

T
Technological

L
Legal

E
Economic
 Risk Assessment: Risk Census

A risk census is a simple framework for analyzing the risk exposure of a project.

Features of risk census are:

It provides an It indicates the It identifies the

It describes each estimate of how impact to the expected
risk likely the risk is schedule if the exposure to the
to occur risk occurs risk
Risk Assessment: Risk Exposure

• The total exposure of a risk can be determined by

multiplying its probability with its impact

• A risk that has a probability of 10% and an impact

of $10,000 would have a risk exposure of $1,000

• Risk exposure can be used to determine how much

slack or buffer is required in the project
 Risk Assessment: Risk Board

Risk board is an information radiator used to make the risks of a project transparent to
the team and stakeholders. A risk board indicates the following:

Risk
Identified response
Probability Impact
risks (if
applicable)

Risk board should be reviewed regularly as part of the daily scrum and/or as part of the
end-of-sprint or iteration retrospective.
 Risk Assessment: Risk Board

An example of a risk board with identified risks, probability, impact, and risk exposure is given here.

Impact
Risk Probability Risk Exposure
(Days of Delay)

Required test bed server and software

30% 10 3
licenses may not arrive on time

Complexity of the APIs used may increase

50% 15 7.5
the effort required

Platform team may not deliver a stable

20% 25 5
version on time

Unexpected absence of team members

40% 5 2
during the flu season
Total Exposure 17.5
 Step 3: Risk Response Strategies

Every risk identified should have a response strategy, such as:

Accept
The team agrees to handle low
impact risks, when they arise.

Avoid
The entire project or part of Transfer
the project is cancelled to Most of the risk is transferred
remove the threat of risk to a third party. The residual
realization. risk is handled using different
strategies.

Mitigate
Reduces the impact of risk if
the event occurs.
 Step 4: Risk Review

The team meets to:

Review outstanding risks

Reassess risk exposure

Gauge the effectiveness of risk responses

 Step 4: Risk Review

The key focus during risk reviews is:

✔ To review the active risks through the daily scrum.
✔ To provide feedback to the team on the risk management process.
✔ To review the risk impact and risk realizations on project and conduct
company objectives.

The aim of risk reviews is to make risks visible, monitor them regularly, prioritize risk issues to elevate
! accountability, encourage action, and track ownership and resolution status.
 Risk Log

The risks identified through the risk management life cycle are captured and tracked in the risk log.
Some of the key aspects of risk logs are:

Risk score is computed for Risks are prioritized according

each identified risk using the to the risk score and the
formula: owners responsible for
Probability of Failure × resolving the risk are captured
Impact in the risk log.

Risk log is one of the key

Risk logs need to be constantly
information radiators that
monitored. If required, high
helps in ensuring
impact risks must be
transparency, accountability,
escalated for a quicker
encouraging actions, and
response.
identifying resolution status.
 Risk Log Sample

Referenc Likelihoo Risk

Description Date Impact Countermeasure Contingency Owner
e d Category
1 The main supplier cannot 21/03/15 Likely High High Include financial Revise project schedule. Annie
deliver on time because of penalties in contract;
other commercial build contingency into
commitments. the schedule; monitor
contractor
performance.
The lead time for the leased 21/03/15 Unlikely Medium Medium Order line earlier than None Jim
2 line exceeds 90 days. necessary; incur
additional rental fees.

Release of the new system is 21/03/15 Very likely High High Build eight-week Employ temporary staff to Mark
delayed because user contingency into free up resources for
acceptance testing schedule. testing; revise project
3
commences after planned schedule.
start.

There is insufficient capacity 18/04/15 Very Medium Low Give advanced notice; Prioritize projects; Jim
to create additional database unlikely monitor SAN usage. temporarily remove
4 instances for data migration alternative development
and testing. instance.

Load testing is done to 20/04/15 Unlikely High Medium Phase introduction of Manage system access; Kent
identity problem prior to new system users. remove high-load
5
system release. features; initiate incident
management process.
Risk Burndown Chart
 Risk Burndown Chart

Risk burndown chart is a simple graphical indicator of the risk trends in the project.

Features of this chart are:

• This chart is created by plotting the sum of the risk exposure or EMV using the risk
census and compare with time or iterations.
• In this chart, there should be a linear drop in risk over the course of the project.
• Each iteration should de-risk the project by delivering a user story that mitigates or
eliminates risk on the risk board.
 Risk Burndown Chart

Here is a sample risk burndown chart.

Risk Exposure (Days)

 Risk Profile Graph

A risk profile graph is a more detailed version of the risk burndown chart similar to the
cumulative flow diagram (CFD).

The graphs are stacked area graphs of risk severity.

The risk severity scores for each risk are plotted one over the other to give a cumulative
severity profile of the project.

These graphs inform stakeholders if the risks are moving in the right direction
(downwards) or if issues and concerns are escalating (upwards).
 Risk Profile Graph

A sample risk profile graph:

Risk 1
Risk 2

Risk 3
Risk 4
Risk 5
Risk 6
Risk 7
Risk 8
Risk 9

Risk 10

Iterations
 Spike

Spike is a time-boxed period designated to reduce uncertainty by learning about a feature, technology,
or process. This helps to better estimate and develop an upcoming feature or fix defects.

Product Backlog with Spike Stories

Stories

Spike

Defects
 Spike

Helps in reducing project risks by envisioning

architecture early

Should require minimal time and effort; not more

than two days

Should be made visible by creating a story for it in

the product backlog

Should be used sparingly as they do not create story

points
Agile Failure Modes
 Agile Failure Modes

In Scrum, the term ScrumBut refers to the proverb, “Agile practices don't fail - rather the
variations on agile adoption fail.”

• Culture does not support the change.

• There is lack of reward plan and existence of a static and prescriptive
standard of work.

• There is lack of retrospectives.

Reasons for failure
of agile projects • Actions that are outcome of the sprint or release retrospection gets
ignored or written-off.

• There is lack of collaboration in planning.

• Scrum master who uses commanding style must speed up the project
progress, but in reality slows things down.
 Agile Coach Failure Modes

Lyssa Adkins has identified seven types of agile coach failure modes. They are:

Spends most of the time observing the team

The Spy to pick up topics for the next retrospective.

The Seagull Criticizes the teams, and does not give

constructive ideas to address problems.

The Gets attached to their opinion and lose the

objectivity needed to help the team have
Opinionator fruitful discussions.
 Agile Coach Failure Modes

The expert is highly involved in the details of the team;

The Expert does not see the broader goals and focuses on the
details of implementing stories.

The butterfly spends just enough time to advice or pose

The Butterfly a question before leaving for another meeting or issue
that needs attention.

The hub acts as a single point of contact for

The Hub communication between team members and for task-
level coordination.

The admin undermines the team ownership by

The Admin becoming an unnecessary middleman to organize
meetings, manage access requests, and other jobs.
 Troubleshooting Guidelines

Pain Point Guidelines

There is unclear purpose, requirements, Use agile chartering to clarify purpose and
team context and agreements. craft and articulate a product vision.

Reduce story sizes by decomposition.

Estimates are off the mark.
Use relative sizing (story pointing) and velocity.

Work assignments or progress are

Visualize using Kanban boards.
unclear.

Use good technical practices.

Defects and technical debt are present.
Clarify definition of done.

Solutions are too complex. Look for simpler solutions.

There is slow process improvement. Use retrospectives effectively.

Teams are working in silos. Self-organize as cross-functional teams.

Ref: Agile Practice Guide

Knowledge Check
Knowledge
Check
Which strategy seeks to reduce the impact of a risk event when it occurs?
1

A. Avoid

B. Transfer

C. Mitigate

D. Accept
 Knowledge
Check
Which strategy seeks to reduce the impact of a risk event when it occurs?
1

A. Avoid

B. Transfer

C. Mitigate

D. Accept

The correct answer is C

Mitigation seeks to reduce the impact of a risk event if it occurs.

Knowledge
Check
What does Spike signify?
2

A. An activity intended to reduce uncertainty in a feature or technology

B. A significant increase in velocity from one iteration to another

C. Adding extra stories to an existing iteration

D. A story that requires an increase in its story points due to uncertainty

 Knowledge
Check
What does Spike signify?
2

A. An activity intended to reduce uncertainty in a feature or technology

B. A significant increase in velocity from one iteration to another

C. Adding extra stories to an existing iteration

D. A story that requires an increase in its story points due to uncertainty

The correct answer is A

Spike is an activity intended to reduce uncertainty by learning about a feature, technology, or process to better
estimate, develop, or fix an upcoming feature or defect.
 Key Takeaways

Metric refers to a standard for measuring or evaluating something

and measure is a quantity, proportion, or a qualitative proportion.
Variance can be of two types: common cause variations and
special cause variations.
Risk management life cycle consists of four steps: risk
identification, risk assessment, risk response, and risk review.
Risk burndown chart is a simple graphical indicator of the risk
trends in the project.
Lyssa Atkins has identified seven types of agile coach failure
modes. They are the Spy, Seagull, Opinionator, Admin, Hub, Butterfly,
and Expert.
Agile practice guide covers common pain points encountered in
agile teams and possible remedial measures.