Test Plan

• A Test Plan refers to a detailed document that catalogs the test strategy, objectives, schedule, estimations,
deadlines, and the resources required for completing that project. Think of it as a blueprint for running the
tests needed to ensure the software is working properly – controlled by test managers.
• Test Plan helps us determine the effort needed to validate the quality of the application under test.
• A well-crafted test plan is a dynamic document that changes according to progressions in the project and
always stays current. It is the point of reference, based on which testing activities are executed and
coordinated among a QA team
• The test plan is also shared with Business Analysts, Project Managers, Dev teams, and anyone else associated
with the project, This mainly offers transparency into QA activities so that all stakeholders know how the
software will be tested.
 What is the Importance of Test Plan?

• 1.Help people outside the test team such as developers, business managers, customers understand the details
of testing.

• 2.Test Plan guides our thinking. It is like a rule book, which needs to be followed.

• 3.Important aspects like test estimation, test scope, Test Strategy are documented in Test Plan, so it can be
reviewed by Management Team and re-used for other projects
 Components of a Test Plan

• Scope: Details the objectives of the particular project. Also, it details user scenarios to be used in
tests. If necessary, the scope can specify what scenarios or issues the project will not cover.
• Schedule: Details start dates and deadlines for testers to deliver results.
• Resource Allocation: Details which tester will work on which test.
• Environment: Details the nature, configuration, and availability of the test environment.
• Tools: Details what tools are to be used for testing, bug reporting, and other relevant activities.
• Defect Management: Details how bugs will be reported, to whom and what each bug report
needs to be accompanied by. For example, should bugs be reported with screenshots, text logs,
or videos of their occurrence in the code?
• Risk Management: Details what risks may occur during software testing, and what risks the
software itself may suffer if released without sufficient testing.
• Exit Parameters: Details when testing activities must stop. This part describes the results that are
expected from the QA operations, giving testers a benchmark to compare actual results to.
How to create a Test Plan?

• You already know that making a Test Plan is the most important task of Test Management Process. Follow the
seven steps below to create a test plan as per IEEE 829
1. Analyse the product
2. Design the Test Strategy
3. Define the Test Objectives
4. Define Test Criteria
5. Resource Planning
6. Plan Test Environment
7. Schedule & Estimation

8. Determine Test Deliverables

 1.Analyse the product

• Start with learning more about the product being tested, the client, and the end-
users of similar products. Ideally, this phase should focus on answering the following
questions:

• Who will use the product?

• What is the main purpose of this product?
• How does the product work?
• What are the software and hardware specifications?

In this stage, do the following:

• Interview clients, designers, and developers

• Review product and project documentation
• Perform a product walkthrough
 2.Design the Test Strategy
• Test Strategy is a critical step in making a Test Plan in Software Testing. A Test Strategy document is a high-
level document, which is usually developed by Test Manager. This document defines:
• The project’s testing objectives and the means to achieve them

• Determines testing effort and costs

You should follow steps below:-

• 2.1 Define scope of testing
• 2.2 Identify Testing type
• 2.3 Document Risk & Issues
• 2.4 Create Test Logistics
2.1) Define Scope of Testing
Before the start of any test activity, scope of the testing should be known.
You must think hard about it.
• The components of the system to be tested (hardware, software,
middleware, etc.) are defined as “in scope “
• The components of the system that will not be tested also need to be
clearly defined as being “out of scope.”
Defining the scope of your testing project is very important for all
stakeholders. A precise scope helps you
• Give everyone a confidence & accurate information of the testing you
are doing
• All project members will have a clear understanding about what is
tested and what is not
 2.1
• How do you determine scope your project?
• To determine scope, you must –
• Precise customer requirement

• Project Budget

• Product Specification

• Skills & talent of your test team

• Now should clearly define the “in scope” and “out of scope” of the testing.
• As the software requirement specs, the project Bank only focus on testing all the functions and external
interface of website Bank (in scope testing)

• Non-functional testing such as stress, performance or logical database currently will not be tested. (Out
of scope)
 2.1
• Problem Scenario
• The customer wants you to test his API. But the project budget does not permit to do so. In such a case what
will you do?
• Well, in such case you need to convince the customer that Api Testing is extra work and will consume
significant resources. Give him data supporting your facts. Tell him if Api Testing is included in-scope the
budget will increase by XYZ amount.
• The customer agrees and accordingly the new scopes, out of scope items are
• In-scope items: Functional Testing, Api Testing

• Out of scope items: Database Testing, hardware & any other external interfaces
 2.2 Identify Testing Type

• A Testing Type is a standard test procedure that gives an expected test outcome.
• Each testing type is formulated to identify a specific type of product bugs. But, all Testing Types are aimed at
achieving one common goal “Early detection of all the defects before releasing the product to the customer”
• The commonly used testing types are described as following
• Unit testing, API Testing, Integration Test, System Test and Install/Uninstall Testing
• There are tons of Testing Types for testing software product. Your team cannot have enough efforts to
handle all kind of testing. As Test Manager, you must set priority of the Testing Types

• Which Testing Types should be focused for web application testing?

• Which Testing Types should be ignored for saving cost?

 2.3 Document Risk & Issues
Risk is future’s uncertain event with a probability of occurrence and a potential for loss. When the risk actually happens, it becomes the ‘issue’.

In the QA Test Plan, you will document those risks

Risk Mitigation

Team members lack the required skills for website testing. Plan training course to skill up your members

The project schedule is too tight; it’s hard to complete this project on time Set Test Priority for each of the test activity.

Test Manager has poor management skill Plan leadership training for manager

A lack of cooperation negatively affects your employees’ productivity Encourage each team member in his task, and inspire them to greater efforts.

Establish the scope before beginning work, pay a lot of attention to project planning and
Wrong budget estimate and cost overruns
constantly track and measure the progress
 2.4) Create Test Logistics
• In Test Logistics, the Test Manager should answer the following questions:
• Who will test?

• When will the test occur?

• Who will test?
• You may not know exact names of the tester who will test, but the type of tester can be defined.
• To select the right member for specified task, you must consider if his skill is qualified for the task or not, also
estimate the project budget. Selecting wrong member for the task may cause the project to fail or delay.
• Person having the following skills is most ideal for performing software testing:
• Ability to understand customers point of view

• Strong desire for quality

• Attention to detail

• Good cooperation
• In your project, the member who will take in charge for the test execution is the tester. Base on the project
budget, you can choose in-source or outsource member as the tester.
• on the project budget, you can choose in-source or outsource member as the tester.
2.4 Create Test Logistics
• When will the test occur?
• Test activities must be matched with associated development activities.
• You will start to test when you have all required items shown in following figure
 3. Define Test Objective
• Test Objective is the overall goal and achievement of the test execution. The objective of the testing is finding
as many software defects as possible; ensure that the software under test is bug free before release.
To define the test objectives, you should do 2 following steps
1. List all the software features (functionality, performance, GUI…) which may need to test.

2. Define the target or the goal of the test based on above features

Let’s apply these steps to find the test objective of your Bank testing project
You can choose the ‘TOP-DOWN’ method to find the website’s features which may need to test. In this
method, you break down the application under test to component and sub-component.
In the previous topic, you have already analyzed the requirement specs and walk through the website, so you can
create a Mind-Map to find the website features as following
This figure shows all the features which the bank website may have.
Based on above features, you can define the Test Objective of the project bank as following
Check that whether website bank functionality(Account, Deposit…) is working as expected without any error or bugs in real
business environment
Check that the external interface of the website such as UI is working as expected and & meet the customer need
Verify the usability of the website. Are those functionalities convenient for user or not?
 4) Define Test Criteria
• Test Criteria is a standard or rule on which a test procedure or test judgment can be based. There’re 2 types of test criteria as
following
Suspension Criteria
• Specify the critical suspension criteria for a test. If the suspension criteria are met during testing, the active test cycle will
be suspended until the criteria are resolved.
• Test Plan Example: If your team members report that there are 40% of test cases failed, you should suspend testing until the
development team fixes all the failed cases.
Exit Criteria
• Example: 95% of all critical test cases must pass.
Some methods of defining exit criteria are by specifying a targeted run rate and pass rate.
• Run rate is ratio between number test cases executed/total test cases of test specification. For example, the test specification
has total 120 TCs, but the tester only executed 100 TCs, So the run rate is 100/120 = 0.83 (83%)

• Pass rate is ratio between numbers test cases passed / test cases executed. For example, in above 100 TCs executed, there’re
80 TCs that passed, so the pass rate is 80/100 = 0.8 (80%)
This data can be retrieved in Test Metric documents.
• Run rate is mandatory to be 100% unless a clear reason is given.

• Pass rate is dependent on project scope but achieving high pass rate is a goal.
• Test Plan Example: Your Team has already done the test executions. They report the test result to you, and they want you to
confirm the Exit Criteria.
• In above case, the Run rate is mandatory is 100%, but the test team only completed 90% of test cases. It means the Run rate is not
satisfied, so do NOT confirm the Exit Criteria
Define Test Criteria
5) Resource No. Member Tasks

Planning 1. Test Manager

Manage the whole project
Define project directions
Acquire appropriate resources

Identifying and describing appropriate test

techniques/tools/automation architecture
• Resource plan is a detailed summary of all Verify and assess the Test Approach
types of resources required to complete Execute the tests, Log results, Report the defects.
project task. Resource could be human, 2. Tester Tester could be in-sourced or out-sourced members,
equipment and materials needed to complete a base on the project budget
project For the task which required low skill, I recommend
you choose outsourced members to save project
• The resource planning is important factor of
the test planning because helps cost.
in determining the number of resources
(employee, equipment…) to be used for the 3. Developer in Test Implement the test cases, test program, test suite etc.
project. Therefore, the Test Manager can make
the correct schedule & estimation for the Builds up and ensures Test Environment and assets
project. are managed and maintained
4. Test Administrator
• This section represents the recommended SupportTester to use the test environment for test
resources for your project. execution
Human Resource Take in charge of quality assurance
5. SQA members Check to confirm whether the testing process is
• The following table represents various meeting specified requirements
members in your project team
 No. Resources Descriptions

5) Resource Planning Install the web application under test

1. Server This includes a separate web server, database
server, and application server if applicable

• System Resource
• For testing, a web application, you The testing tool is to automate the testing,
simulate the user operation, generate the test
should plan the resources as following 2. Test tool results

tables: There are tons of test tools you can use for
this project such as Selenium, QTP…etc.

You need a Network include LAN and Internet

3. Network to simulate the real business and user
environment

The PC which users often use to connect the

4. Computer
web server
 6) Plan Test Environment

• What is the Test Environment

A testing environment is a setup of software and hardware on which the testing team is going to execute test
cases. The test environment consists of real business and user environment, as well as physical environments,
such as server, front end running environment.

• How to setup the Test Environment

Back to your project, how do you set up test environment for this banking website?
To finish this task, you need a strong cooperation between Test Team and Development Team
You should ask the developer some questions to understand the web application under test clearly. Here’re some
recommended questions. Of course, you can ask the other questions if you need.
• What is the maximum user connection which this website can handle at the same time?

• What are hardware/software requirements to install this website?

• Does the user’s computer need any particular setting to browse the website?
 7.Schedule & Estimation Task Members Estimate effort

• In the article Test estimation, you already used some techniques to

estimate the effort to complete the project. Now you should include that
estimation as well as the schedule to the Test Planning Create the test
Test Designer 170 man-hour
specification
• In the Test Estimation phase, suppose you break out the whole
project into small tasks and add the estimation for each task as below
• Then you create the schedule to complete these tasks.
• Making schedule is a common term in project management. By
creating a solid schedule in the Test Planning, the Test Manager can use Tester, Test
Perform Test Execution 80 man-hour
it as tool for monitoring the project progress, control the cost overruns. Administrator
• To create the project schedule, the Test Manager needs several types
of input as below:
• Employee and project deadline: The working days, the project Test Report Tester 10 man-hour
deadline, resource availability are the factors which affected to the
schedule
• Project estimation: Base on the estimation, the Test Manager knows Test Delivery 20 man-hour
how long it takes to complete the project. So he can make the
appropriate project schedule
• Project Risk : Understanding the risk helps Test Manager add enough Total 280 man-hour
extra time to the project schedule to deal with the risks
 • Test Deliverables is a list of all the documents, tools and
other components that has to be developed and maintained in
8. Test support of the testing effort.
• There are different test deliverables at every phase of the
Deliverables software development lifecycle
Test Deliverables
• Test deliverables are provided before testing phase. • Test deliverables are provided after the testing cycles is
over.
• Test plans document.
• Test Results/reports
• Test cases documents
• Defect Report
• Test Design specifications.
• Installation/ Test procedures guidelines
• Test deliverables are provided during the testing
• Release notes
• Test Scripts

• Simulators.

• Test Data

• Test Traceability Matrix

• Error logs and execution logs.