Unit - III

Test Management
 TEST PLANNING
WHAT IS A TEST PLAN?
-- A test plan is a detailed document that outlines the test strategy,
testing objectives, resources (manpower, software, hardware) required for
testing, test schedule, test estimation and test deliverables(meaning
output/result).

-- The test plan serves as a blueprint( meaning layout / outline) to

conduct software testing activities as a defined process which is minutely
monitored and controlled by the test manager.
IMPORTANCE OF TEST PLAN----
Making Test Plan has multiple benefits--

1. Test Plan helps us to determine the effort needed

2. Help people outside the test team such as developers, business

managers, customers understand the details of testing.

3. Test Plan guides our thinking. It is like a rule book, which needs to be
followed.

4. Important aspects like test estimation, test scope, test strategy are
documented in Test Plan, so it can be reviewed by Management Team
and re-used for other projects.
Test Planning steps:

1. Preparing a test plan

2. Deciding test approach
3. Setting up criterion for testing
4. Identifying responsibilities, Staffing
5. Identifying Resource requirement
6. Identifying Test deliverables
7. Testing tasks
1. PREPARING A TEST PLAN
“Failing to plan is planning to fail. ”

-- Testing should be driven by a plan.

-- The test plan acts as the anchor for the execution, tracking and reporting
of the entire testing project.

-- Test plan covers

1. What needs to be tested--the scope of testing, including clear identification of

what will be tested and what will not be tested.
2. How the testing is going to be performed---breaking down the testing into
small and manageable tasks and identifying the strategies to be used for carrying
out the tasks.

3. What resources are needed for testing--computer as well as human

resources.

4. The time lines by which the testing activities will be performed.

5. Risks that may be faced in all of the above with appropriate mitigation and
contingency plans.
-- One single test plan can be prepared to cover all phases or there can be
separate plans for each phase.

-- For example-- there needs to be plans for unit testing, integration testing,
performance testing, acceptance testing and so on.

-- If there are multiple test plans, there should be one test plan, which covers
the activities common for all plans. This is called the master test plan.
2. DECIDING TEST APPROACH / STRATEGY
1. The test strategy identifies the right type of test for each of the feature.

2. It should be objective criteria for measuring the success of a test.

3. This includes identifying

-- What type of testing would you use for testing the functionality?

-- What are the configuration or scenarios for testing the features?

-- What validations would be needed?

-- What non-functional tests would you need to do?

3. SETTING UP CRITERION FOR TESTING
-- As we have discussed in earlier chapter there must be entry and exit
criterion for different phases of testing.

-- Test strategy determine how these features would be tested.

-- Ideally, tests must be run as early as possible sothat last minute

pressure of running tests is minimized.
-- A test activity willnot be isolated, that can be carried out at one go.

-- It may be suspended at various points of time because it is not possible to

proceed further. When it is possible to proceed further, it will have to be resumed.
-- Some of suspension/suspend criterion include:

1. Encountering more than a certain number of defects, causing frequent

stoppage of testing activity

2. Hitting showstoppers (meaning obstacle) that prevent further progress of

testing for eg.- if a database does not start further tests of query, data
manipulation and so on arenot possible to execute

-- When such conditions are addressed, the tests can resume.

4. IDENTIFYING RESPONSIBILITIES, STAFFING
-- A testing project requires different people to play different roles.

-- There is role definition on the dimensions of the modules being tested.

-- The different role definitions should

1. Ensure there is clear accountability for a given task, sothat each person
knows what he or she has to do

2. Clearly list the responsibilities for various functions to various people,

sothat everyone knows how his/her work fits into the entire project.
 3. Complement / companion each other, ensuring no one steps on an
others toes and

4. Supplement each other, sothat no task is left unassigned.

-- Role definition should not only address technical roles, but also list the
management and reporting responsibilities.

-- This includes frequency, format and recipients of status reports and other
project-tracking mechanism.

-- Staffing is done based on estimation of effort involved and the

availability of time for release.
-- In order to ensure that the right tasks got executed , the feature and tasks are
prioritized the basis of on effort, time and importance.

-- People are assigned to tasks that achieve the best possible fit between the
requirements of the job and skills and experience levels needed to perform
that function.

-- It may not always be possible to find the perfect fit between the
requirements and the skills available.
-- In case there are gaps between the requirements and availability of skills,
they should be addressed with appropriate training programs.

-- It is important to plan for such training programs upfront as they are usually
are deprioritized under project pressures.
5. IDENTIFYING RESOURCE REQUIREMENT

As a part of planning for a testing project, the project / test manager should
provide estimates for the various hardware and software resources required.
Some of the following factors need to be considered.

1. Machine configuration(RAM, processor, disk and so on) needed to run the

product under test.

2. Appropriate number of licenses of all the software

3. Special requirements for running machine-intensive tests such as load tests

and performance tests.

4. The different configurations of the supporting software that must be present.

-- Underestimation of these resources can lead to considerable slowing
down of the testing efforts and this can lead to delayed product release and
to de-motivated testing team.

-- Proper estimation of these resources requires cooperation and

teamwork among different groups--product development team, testing team,
system administration team and senior management.
6. Identifying test deliverables(results/output)

-- The test plan also identifies the deliverables that should come out of the
testing activity. The deliverables include the following:

1. The test plan itself (master test plan, various other test plans for the project)

2. Test case design specification

3. Test cases

4. Test logs

5. Test summary report

6. Defect repository
-- A defect repository gives the status of the defects reported in a product life
cycle

-- Defect repository includes entering new defects in the repository and updating
the status of the defect fixes after verification.
7. Testing tasks: size and effort estimation

-- The scope identified above gives a broad overview of what needs to

be tested.

-- This understanding measured in the estimation step.

-- Estimation happens broadly in three phases

1. Size estimation

2. Effort estimation

3. Schedule estimation
1. Size estimation--

-- Size estimation measures the actual amount of testing that needs to

be done.

-- Size estimate provides an estimate of the actual ground to be covered

for testing.

-- Size estimate acts as a primary input for estimating effort.

-- Several factors contribute to the size estimate of a testing project.

1. Size of the product under test----this determines the amount of testing

that needs to be done. The larger the product, greater is the size of testing
to be done. Some of the measures of the size of product under test are
LOC(Line of Code), function point(FP)

2. Number of platforms and interoperability environments to be tested---if

a particular product is to be tested under several different platforms or
under several different configurations, then the size of testing task
increases.
-- In order to have a better handle on the size estimate, the work to be
done is broken down into smaller and more manageable parts called work
breakdown structure (WBS) units.

-- Size estimate is expressed in terms of the following

1. Number of test cases

2. Number of test scenarios

3. Number of configurations to be tested

2. Effort estimate--
-- Estimating effort is important because effort has a more direct
influence on cost than size.

-- The factors that drive the effort estimate are as follows:

1. Productivity data---

---productivity refers to speed at which the various activities of testing

can be carried out. This is based on historical data available in the
organization.

---productivity data can be further classified into number of test cases

that can be developed per day, the number of test cases that can be
run per day, the number of pages of documentation that can be tested
per day, and so on.

----having this fine grained productivity data enables better planning

and increase confidence level and accuracy of the estimates.
2. Reuse opportunities---

---if the test architecture has been designed keeping reuse in mind,
then the effort required to cover a given size of testing can come
down.

----for example, if tests are designed in such a way that some of the
earlier / prior tests can be reused, then the effort of test development
decreases.
3. Robustness of processes---

---existence of well-defined processes will go a long way in reducing the effort

involved in any activity.

---for example, in an organization the higher levels of processes are likely to be

1. Well-documented standards for writing test specifications, test scripts and so

on

2. Consistent ways of training people and

3. Proven processes for performing functions such as reviews and audits

-- Effort estimate is derived from size estimate by taking the individual
WBS units and classifying them as “reusable”, “modifications” and “new
development”.

-- For example-- if parts of test case can be reused from existing test
cases, then the effort involved in developing these would be close to zero.

-- Effort estimation given in person days, person months, or person

years.

-- The effort estimate is then translated to a schedule estimate.

3. Schedule Estimation--- Schedule estimation translating the
effort required into specific time frame. The following steps make up this
translation

1. Identifying external and internal dependencies among the activities.

External dependencies are:

a. Availability of the product from developers

B. hiring

C. training

D. acquisition of hardware / software required for training

-- Internal dependencies are fully within the control of the person
performing that activity

a. Completing the test specification

B. coding the tests

C. executing the tests.

2. Sequencing the activities, based on the expected duration as well as

on the dependencies.

3. Identifying the time required for each of the WBS activity

4. Monitoring the progress in terms of time and effort

5. Rebalancing schedules and resources as necessary

 Test Management
----In this section, we will look at some of the aspects that should be
taken care of in planning a project

1. Test infrastructure management

2. Test people management
 Test Infrastructure Management
----Testing requires a robust infrastructure to be planned upfront(meaning
in advance).

----This infrastructure is made up of three essential elements.

1. A test case database (TCDB)

2. A defect repository

3. Configuration management repository and tool

 A test case database (TCDB)----

----A test case database captures all the relevant information about the
test cases in an organization.

-----Some of the entities and the attributes in each of the entities in such a
TCDB are given in following table
entity purpose attributes

Test case Records all the information about ● Test case ID

the tests ● Test case name
● Test case owner
● Associated files for
the test case
Test case ● identification of tests for a given ● Test case ID
-- product product feature ● Module ID
---cross
reference
 ● Provides a mapping between
the tests and the
corresponding product
features
Test case run ● Gives history of when a test ● Test case ID
history was run and what was the ● Run date
result ● Time taken
● Run status
(success/fail
ure)
Test case-- ● Gives details of test cases ● Test case ID
defect cross introduced to test certain ● Defect reference #
reference specific defects detected in (points to a record
the product in the defect
repository)
A defect repository---
-- Repository is a central location in which data is stored and managed.

-----A defect repository captures all the relevant details of defects reported
for a product.

----The information that a defect repository includes is given in following

table
entity purpose attributes

Defect Record all ● Defect ID

details the ● Defect priority / severity
information ● Defect description
about the ● Affected product
tests ● Environment information (for eg. OS
version)
● Customer who encountered the problem
(could be reported by the internal testing
team also)
● Date and time of defect occurance
Defect test Provides details of test cases ● Defect ID
details for a given defect. ● Test case ID
Fix details Provides details of fixes for a ● Defect ID
given defect ● Fix details
communicati Capture all the details of the ● Test case
on communication that transpired ID
(means become known) for this ● Defect
defect among the various reference
stakeholders. These could #
include communication ● Details of
between the testing team and communic
development team, customer ation
communication and so on.
Provides insights into
effectiveness of
communication.
Configuration management repository and tool
-- Another infrastructure that is required for a software testing team is a software
configuration management (SCM) repository.

-- An SCM repository also known as CM repository

-- SCM repository keeps track of change control and version control of all the
files/entities.

--
-- change control ensures that

1. Changes to test file are made in a controlled fashion and only with proper
approvals.

2. Changes made by one test engineer are not accidentally lost or overwritten by
other changes.

3. Each change produces a distinct version of the file

4. At any point of time, everyone gets access to only the most recent version of
the test files
-- version control ensures that the test scripts associated with a given release
of a product are baselined along with the product files.

-- baselining is taking a snapshot of the related files of a version, assigning a

unique identifier to this set.

-- In future, when anyone wants to recreate the environment for the given release,
this label would enable him or her to do so.
 Test people management
---- 30%the success of the project comes from technical skills such as
Testing skill, development skills, management skill…. While 70% comes
from the human resource.

---- To manage the human resource, the Test Manager needs the people
skills.

----The most important about management is people skills, means how we

deal with others.
---- For any organisation employees are the biggest asset.

---- Their performance and attitude can result in the success or

failure of your business.

---- The most difficult part of any manager's job is people

management.

---- He or she is required to lead, motivate, train, inspire, and

encourage.
----- On the other hand, he or she is also responsible for hiring,
firing, disciplining, training and evaluating.

----These functions seem to be at odds, but a successful manager

can integrate both the positive and negative aspects of these tasks
to create a positive, productive workforce.
 Following tools helps for effective people management---

1. Understanding the Basics of Management Roles.

2. Effective Recruitment and Induction.
3. Understanding Team Dynamics.
4. Team Effectiveness and Motivation.
5. Developing and Coaching Your Team.
6. Coping With Challenging Management Situations.
1. Understanding the Basics of Management Roles---

--- A good starting point is to find out how good your management
skills are right now

--- asking yourself questions such as, "Can I learn how to motivate
people and earn their respect?" "How will I find time to complete
my own work while managing others?" or "Will I be able to lead
people who were once my peers?"
2. Effective Recruitment and Induction--

---- It's said that the most important part of directing a movie is
casting the right actors. If the actors are right for the roles, there will
be less need for detailed instruction, or correcting mistakes, later.

---- Investing in new talent can be an expensive and

time-consuming process, so it's not a decision to be taken lightly.
3. Understanding Team Dynamics---
--- Well-rounded teams are composed of people with differing
skillsets, personalities and experiences

---- a person willing to go the extra mile can have a hugely positive
effect on your team's productivity and performance, if you place her
in the right position.

---- Understanding their strengths and weaknesses, too, is crucial

when it comes to helping them and their co-workers to be happy
and to maximize their contribution.
4. Team Effectiveness and Motivation---
----To improve team effectiveness, it's important to have relevant
data that you can collect, track, and respond to.

-----Tracking progress requires you to understand how your team is

using its time.

---A motivated team is usually an effective team

----Similarly, an emotionally engaged team tends to produce better

work.
5. Developing and Coaching Your Team--

-----Your people will feel a greater sense of satisfaction at work if

they know that they are growing and developing in their roles.

-----You can improve performance through coaching, delegation,

and talent management, among a host of other strategies.

----aim to introduce a positive team atmosphere

6. Coping With Challenging Management Situations---
----Being a manager can be hugely enjoyable and rewarding when
things are going well. But you discover your real strength as a boss
when things get tough.

---The challenges you face can come from within your team or from
external factors beyond you or your organization's control. For
example, you may be feeling overstretched because your team is
growing too large or it is shrinking or suffering from high turnover.
 TEST PROCESS
1. Putting together and baselining a test plan

2. Test case specification

Putting together and baselining a test plan
-- A test plan combines all the points into a single document.

-- A template of a test plan and a checklist of questions that are useful to arrive
at a test plan.(refer ls 15- test planning appendix A and B)

-- Each testing project puts together a test plan based on the template.
-- Should any changes be required in the template then such a change is made
only after careful deliberation and with appropriate approvals.

-- The test plan is reviewed by a designated set of competent people in the

organization.

-- It then is approved by a competent authority.

-- After this, the test plan is baselined into the configuration management
repository.

-- From then on, the baselined test plan becomes the basis for running the
testing project.

-- Any significant changes in the testing project should thereafter be reflected in

the test plan and changed test plan baselined again in the configuration
management repository.

-- In addition, periodically, any change needed to the test plan templates are
discussed among the different stakeholders and this is kept current and applicable
to the testing teams.
 Test case specification

-- Using the test plan as the basis, the testing team designs test case
specifications, which then becomes the basis for preparing individual test cases.

-- Formally, a test case is nothing but a series of steps executed on a product,

using a pre-defined set of input data, expected to produce a pre-defined set of
outputs, in a given environment.

--
-- Hence a test case specification should clearly identify:

1. The purpose of test-- this lists what feature or part the test is intended for.

2. Items being tested

3. Environment that needs to be set up for running the test case

4. Input data to be used for the test case

5. Steps to be followed to execute the test

6. The expected results that are considered to be “correct results”. These
expected results can be what the user may see in the form of GUI, report and so
on.

7. A step to compare the actual results produced with the expected results.

8. Any relationship between this test and other tests.

 Test Reporting
1. Executing test cases

2. Preparing test summary report

 Executing test cases
-- The appropriate test cases have to be executed at the appropriate times
during a project.

-- For example-- test cases corresponding to smoke tests may be run on a daily
basis. System testing test cases will be run during system testing.

-- As mentioned earlier, the defect repository contains all the information

about defects uncovered by testing (and the defects reported by customer) .
-- As the test cases are executed during a test cycle, the defect repository is
updated with

1. Defects from the earlier test cycle that are fixed in the current build and

2. New defects that get uncovered in the current run of the tests.

-- Defect repository should be the primary vehicle of communication

between the test team and the development team.
-- All the stakeholders should be referring to the defect repository for knowing
the current status of all the defects.

-- As discussed in the test plan, a test may have to be suspended during its run
because of certain show stopper defects. In this case , the suspended test case
should wait till the resumption criteria are satisfied.

-- Likewise a test should be run only when the entry criteria for the test are
satisfied and should be deemed complete only when the exit criteria are satisfied.
-- During test design and execution, the traceability matrix should be kept
current. As and when tests get designed and executed successfully, the
traceability matrix should be updated.

-- The traceability matrix itself should be subject to configuration management,

that is, it should be subject to version control and change control.
 Preparing test summary report
-- At the completion of a test cycle, a test summary report is produced.
This report gives insights to the senior management about the fitness of the
product for release.

-- Test summary report is the final step in a test cycle is to recommend the
suitability of a product for release.

-- A report that summarizes the results of a test cycle is the test summary report.
-- There are two types of test summary reports.

1. Phase-wise test summary, which is produced at the end of every phase

2. Final test summary report, which has all the details of all testing done by all
phases and terms, also called as “release test report”

-- A summary report should present

1. A summary of activities carried out during the test cycle or phase

2. Variance of activities carried out from the activities planned,

This includes:

-- The tests that were planned to be run but could not be run (with reason)

-- Modifications to tests (in this case, the TCDB should be updated)

-- Additional tests that were run (that were not in the original test plan)

-- Differences in effort and time taken between what was planned and what was
executed

-- Any other deviations from plan

3. Summary of results should include

● Tests that failed, with any root causes descriptions and

● Severity of impact of the defects uncovered by the tests.

4. Comprehensive assessment and recommendation for release should include

● “Fit for release” assessment and

● Recommendation of release.