Test Plan

1
Outline
• Test Plan
• Test selection
• Writing test cases
• Test Execution

2
What is a Test Plan?

• A test plan is a document that outlines the objectives, scope,

approach, and schedule for testing a software system.
• It provides a roadmap for the testing effort,
• ensuring that all necessary activities are planned and
• executed to achieve the desired level of quality.
Why Do We Need Testing? ​

• Testing is a critical phase in software development that helps

identify defects, ensure system reliability, and improve overall
quality. It helps in:
• Detecting and fixing software defects.
• Ensuring the software meets the requirements.
• Reducing the risk of failures in production.
• Enhancing user satisfaction.
• Improving the software's performance, usability, and security.
Objectives of a Test Plan ​

• The objectives of a test plan include:

• Defining the scope, goals, and strategy of testing.
• Identifying the testing approach and techniques to be used.
• Outlining the test deliverables, schedule, and resources required.
• Establishing the test environment setup and configuration.
• Identifying the risks associated with testing and proposing
mitigation strategies.
• Setting the entry and exit criteria for testing phases.
• Establishing communication and reporting channels for the
testing team.
Components of a Test Plan

1. Scope and Overview

2. Test Strategy
3. Test Deliverables
4. Test Schedule
5. Test Environment
6. Test Execution Approach
7. Test Entry and Exit Criteria
8. Test Risks and Mitigation
9. Test Resources
10. Test Reporting
11.Approval and Sign-Off
1. Scope and Overview ​
• This section provides a high-level description of the system to be
tested, including its purpose, features, and functionalities.
2. Test Strategy ​
• The test strategy outlines the overall approach to be followed
during testing, including the test levels, test types, and test
techniques to be used.
3. Test Deliverables ​
• This section lists the various deliverables that will be produced
during testing, such as test cases, test scripts, test
data, defect reports, and test summary reports.
• 4. Test Schedule ​
• The test schedule defines the timeline for the testing activities,
including start and end dates for each testing phase, milestones,
and dependencies.
• 5. Test Environment ​
• This section describes the hardware, software, and network
setup required for testing, including details on test environments,
test data, and any specific configurations.
• 6. Test Execution Approach ​
• The test execution approach outlines how the test cases will be
executed, including the order, prioritization, and
any automation or manual testing techniques to be used.
• 7. Test Entry and Exit Criteria ​
• Entry criteria- This section defines the criteria that must be met
before testing can begin. Example- such as completion of
development, availability of test environments, or sign-off from
stakeholders.
• Exit Criteria - The criteria for completing testing activities.
Example- such as the percentage of test case execution, defect
closure rate, or stability of the system.
• Identify any specific conditions or criteria that need to be met
before progressing to the next phase of testing.
• Consider dependencies with other project activities, such as user
acceptance testing or deployment.
• 8. Test Risks and Mitigation ​
• Identify the potential risks and challenges associated with testing and propose
strategies to mitigate or address them effectively.
• 9. Test Resources ​
• This section identifies the resources required for testing, including personnel,
hardware, software, testing tools, and any training or skill requirements.
• 10. Test Reporting ​
• Define the reporting mechanisms and formats for test progress, defect
tracking, and test summary reports. Specify the frequency and recipients of
the reports.
• 11. Approval and Sign-Off ​
• Establish the process for obtaining approval and sign-off of the test plan from
the relevant stakeholders.
Advantages of a Test Plan ​

• Provides a structured approach to testing, ensuring thorough

coverage.
• Clearly defines the objectives, scope, and expectations of
testing.
• Facilitates effective communication and collaboration among
team members.
• Helps identify potential risks and plan mitigation strategies.
• Acts as a reference document for the testing team throughout
the project.
• Assists in estimating effort, resources, and timelines for
testing.
Disadvantages of a Test Plan ​

• Test plans can become outdated if not regularly reviewed and

updated.
• A rigid test plan may hinder agility and adaptation to changing
requirements.
• Test plans can be time-consuming to create and maintain.
• Inadequate stakeholder involvement may lead to unrealistic or
incomplete test plans.
Test Plan Tools

• Various tools are available for creating and managing test

plans, such as:
• Test management tools (e.g., TestRail, Zephyr, qTest) for
creating and organizing test plans, test cases, and test
execution results.
• Documentation tools (e.g., Microsoft Word, Google Docs) for
creating and formatting test plan documents.
• Collaboration tools (e.g., Confluence, SharePoint) for sharing
and collaborating on test plans with team members.
Testing
• There is a massive misunderstanding about testing: that it
improves software. It doesn't!
• Weighing yourself doesn't reduce your weight
• Going to the doctor doesn't make you healthy.
• Those things help to identify problems that you might choose
to resolve. Testing does too.
• The testing does not make the product better, even though it's
part of a process that does make a product better.

14
Test selection

15
Test cases
• A test case, is a set of conditions under which a tester will
determine whether an application, software system or one of
its features is working as it was originally established for it to
do.
• Test cases are often referred to as test scripts, particularly
when written – when they are usually collected into test
suites.
• A Test Case is a set of actions executed to verify a particular
feature or functionality of your software application.

16
Test cases
• A test case is a description of a specific interaction that a tester
will have in order to test a single behavior of the software.
• Test cases are very similar to use cases, in that they are step-by-
step narratives which define a specific interaction between the
user and the software.
• A typical test case is laid out in a table, and includes:
• A unique name and number
• A requirement which this test case is exercising
• Preconditions which describe the state of the software before the test case
• Steps that describe the specific steps which make up the interaction
• Expected Results which describe the expected state of the software after
the test case is executed

17
Test cases
• Test cases must be repeatable.
• Good test cases are data-specific, and describe each
interaction necessary to repeat the test exactly.

18
Writing test cases
• First you must understand the language fundamentals
• Sizes and limits of variables, platform specific information
• Second, you must understand the domain
• Read the requirements
• Think like a user – what possible things do they want to do
• Think about possible “mistakes”; i.e. Invalid input
• Think about impossible conditions or input
• What is the testing intended to prove?
• Correct operation – gives correct behavior for correct input
• Robustness – responds to incorrect or invalid input with proper results
• User acceptance – typical user behavior
• Write down the test cases
19
Writing Good Test Cases
• Test Cases need to be simple and transparent
• Create Test Case with end user in mind
• Avoid test case repetition
• Do not Assume
• Stick to the Specification Documents.
• Ensure 100% Coverage
• Test Cases must be identifiable.
• Implement Testing Techniques
• It's not possible to check every possible condition in your software
application
• Testing techniques help you select a few test cases with the maximum
possibility of finding a defect
20
Writing Good Test Cases
• Boundary Value Analysis (BVA)
• testing of boundaries for specified range of values.
• Repeatable and self-standing
• The test case should generate the same results every time no matter
who tests it

21
Writing a test case
While drafting a test case do include the following
information
• The description of what requirement is being tested
• Inputs and outputs or actions and expected results
• Test case must have an expected result.

• Verify the results are correct

Testing Normal Conditions
Testing Unexpected Conditions
Bad (Illegal) Input Values
Boundary Conditions

22
Writing test cases
• Cover all possible valid input
• Try multiple sets of values, not just one set of values
• Permutations of values
• Check boundary conditions
• Check for off-by-one conditions
• Check invalid input
• Illegal sets of value
• Illegal input
• Impossible conditions
• Totally bad input
• Text vs. Numbers, etc.

23
Writing test cases
• Beware of problems with comparisons
• How to compare two floating numbers
• Never do the following:
float a, b;
. . .
if (a == b)
• Is it 4.0000000 or 3.9999999 or 4.0000001 ?
• What is your limit of accuracy?
• In object oriented languages make sure whether you are comparing the
contents of an object or the reference to an object
String a = “Hello world!\n”
String b = “Hello world!\n”
if ( a == b )
vs.
if ( a.equals(b) )

24
Writing test cases
Let’s consider the triangle example
• Cover all possible valid input
• all three possible conditions: equilateral, isosceles, scalene
• Try multiple sets of values, not just one set of values
• Permutations of values {3,4,5}, {4,3,5}, {5,4,3}
• Check boundary conditions
• Check for off-by-one conditions: 0, MAX_INT
• Check invalid input
• Illegal sets of value
• Wrong format: not integer
• Negative numbers
• Illegal input
• Impossible conditions: {2,3,8}, {2,3,5} {definition of a triangle}
• Totally bad input
• Text vs. Numbers, etc.
25
Writing test cases
Let’s consider the triangle example
• How to test the code?
• Have the code read from the standard input
• java Triangle < testcases.txt
or,
• java Triangle
2 3 3
3 4 5
• Have the output print to standard output.
• java Triangle > results.txt
• Combine the two and we have:
• java Triangle <testcases.txt >results.txt
• This assumes we have the code read three numbers on the line, or have
the code read three numbers whether they are on one line or more.
26
Tips for testing
• You cannot test every possible input, parameter value, etc.
• So you must think of a limited set of tests likely to expose bugs.
• Think about boundary cases
• positive; zero; negative numbers; infinity; very small
• right at the edge of an array or collection's size (plus or minus one)
• Think about empty cases and error cases
• 0, -1, null; an empty list or array
• test behavior in combination
• maybe add usually works, but fails after you call remove
• make multiple calls; maybe size fails the second time only

27
Test Cases – Good Example

28
Test Cases – Bad Example

29
Test cases
• Why we write test cases?
• The basic objective of writing test cases is to validate the testing
coverage of the application.
• Keep in mind while writing test cases that all your test cases
should be simple and easy to understand.
• For any application basically you will cover all the types of
test cases including functional, negative and boundary
value test cases.

30
Trustworthy tests
• Test one thing at a time per test method.
• 10 small tests are much better than 1 test 10x as large.
• Each test method should have few (likely 1) assert
statements.
• If you assert many things, the first that fails stops the test.
• You won't know whether a later assertion would have also failed.
• Tests should avoid logic.
• minimize if/else, loops, switch, etc.
• avoid try/catch
• If it's supposed to throw, use expected= ... if not, let JUnit catch it.
• Torture tests are okay, but only in addition to simple tests.

31
Test Execution
• The software testers begin executing the test plan after the developers
deliver the alpha build, or a build that they feel is feature complete.
• The alpha should be of high quality—the developers should feel that it is
ready for release, and as good as they can get it.
• There are typically several iterations of test execution.
• First, focus on new functionality
• Then, regression test to make sure that a change to one area of the software has
not caused any other part of the software
• Regression testing usually involves executing all test cases which have previously
been executed
• There are typically at least two regression tests for any software project

32
Test Execution

• When is testing complete?

• No defects found
• Or defects meet acceptance criteria outlined in test plan

33
Automating Test Execution

• Designing test cases and test suites is creative

• Like any design activity: A demanding intellectual activity, requiring human
judgment
• Executing test cases should be automatic
• Design once, execute many times
• Test automation separates the creative human process from the
mechanical process of test execution

34
From Test Case Specifications to Test Cases

• Test design often yields test case specifications, rather than concrete
data
• Ex: “a large positive number”, not 420023
• Ex: “a sorted sequence, length > 2”, not “Alpha, Beta, Chi, Omega”
• Other details for execution may be omitted
• Generation creates concrete, executable test cases from test case
specifications

35
Scaffolding
• Code produced to support development activities
(especially testing)
• Not part of the “product” as seen by the end user
• May be temporary (like scaffolding in construction of buildings)
• Includes
• Test harnesses, drivers, and stubs
• Example:
• JUnit – test harness
• Eclipse – IDE, scaffolding, JUnit built in

36
Defect Tracking

• The defect tracking system records and tracks defects.

• It routes each defect between testers, developers, the project
manager and others, following a workflow designed to ensure that
the defect is verified and repaired.

37
Smoke Tests

• A smoke test is a subset of the test cases that is typically

representative of the overall test plan.
• Smoke tests are good for verifying proper deployment or other non invasive
changes.
• They are also useful for verifying a build is ready to send to test.
• Smoke tests are not substitute for actual functional testing.

38
Summary

• Goal: Separate creative task of test design from mechanical task of

test execution
• Enable generation and execution of large test suites
• Re-execute test suites frequently (e.g., nightly or after each program change)
• Scaffolding: Code to support development and testing
• Test drivers, stubs, harness, including oracles
• Ranging from individual, hand-written test case drivers to automatic
generation and testing of large test suites
• Capture/replay where human interaction is required

39