Software Testing –

Strategies

2
 Software testing
 Software testing (or “testing”) was the first software quality assurance tool applied to
control the software product’s quality before its shipment or installation at the
customer’s premises.

 At first, testing was confined to the final stage of development, after the entire
package had been completed.

 Later, as the importance of early detection of software defects penetrated quality

assurance concepts, SQA professionals were encouraged to extend testing to the partial
in-process products of coding,
 which led to software module (unit) testing and integration testing.
3
Software testing…
 Testing is certainly not the only type of SQA tool applied to software code.

 Additional tools are code inspections and code walkthroughs, methods

implemented on code printout without actually running the program.
 These procedures, which are similar to those applied in design inspection and
walkthroughs, yield good results in identifying code defects.
 Nevertheless, these tools, because they are based solely on the review of
documents, can never replace testing, which examines the software product’s
functionality in the form actually used by the customer.

4
Code Inspections
The main purpose of code inspection is to find defects and it can also spot any process improvement
if any.
An inspection report lists the findings, which include metrics that can be used to aid improvements
to the process as well as correcting defects in the document under review.
Preparation before the meeting is essential, which includes reading of any source documents to
ensure consistency.
Inspections are often led by a trained moderator, who is not the author of the code.
The inspection process is the most formal type of review based on rules and checklists and makes
use of entry and exit criteria.
It usually involves peer examination of the code and each one has a defined set of roles.
After the meeting, a formal follow-up process is used to ensure that corrective action is completed in
a timely manner.

5
Code Walkthrough
Code Walkthrough is a form of peer review in which a programmer
leads the review process and the other team members ask
questions and spot possible errors against development standards
and other issues.

6
Software testing – strategies
• White box testing
data processing and calculation correctness tests
path coverage vs. line coverage
McCabe’s cyclomatic complexity metrics
software qualification and reusability testing
advantages/disadvantages of white box testing
• Black box testing
Equivalence classes for output correctness tests
Operation / Revision / Transition factor testing classes
advantages/disadvantages of white box testing

7
Definition and objectives

8
9
Software testing strategies
 There are basically two testing strategies:
“Big bang testing”: tests the software as a whole, once the completed
package is available.
“Incremental testing”: tests the software piecemeal –
• software modules are tested as they are completed (unit tests),
• followed by groups of modules composed of tested modules integrated
with newly completed modules (integration tests).
• Once the entire package is completed, it is tested as a whole (system test).
10
Big bang vs. incremental testing.
Unless the program is very small and simple, applying the “big bang”
testing strategy presents severe disadvantages.
Identification of error in the entire software package when perceived as
one “unit” is very difficult and, in spite of the vast resources invested, is not
very effective.
Moreover, performing perfect correction of an error in this context is
frequently laborious.
Obviously, estimates of the required testing resources and testing schedule
tend to be rather fuzzy.
11
Big bang vs. incremental testing.
incremental testing,
In contrast, the advantages of
because it is performed on relatively small
software units, yields higher percentages of
identified errors and facilitates their
correction.
As a result, it is generally accepted that incremental testing should
be preferred.

12
Incremental testing : Bottom-up and top-down.
There are two basic incremental testing strategies: bottom-up and
top-down.

 In top down testing, the first module tested is the main module, the highest
level module in the software structure; the last modules to be tested are the
lowest level modules.

 In bottom-up testing, the order of testing is reversed: the lowest level modules
are tested first, with the main module tested last.
13
 Top-down testing

14
Bottom-up testing

15
 Stubs and Drivers for Incremental
testing
• Stubs and drivers are software replacement simulators required for
modules not available when performing a unit test or an integration test.

• In top-down testing of incomplete Systems : Stubs (often termed a

“Dummy Module“) replaces an unavailable lower level module,
subordinate to the module tested.

• In bottom-up testing until the upper level modules are developed : A

driver is a substitute module but of the upper level module that activates
the module tested. The driver is passing the test data on to the tested
module and accepting the results calculated by it.

16
 Top-Down Vs Bottom-Up
top-down
Advantage : the early stage at which it is possible to demonstrate the program as a whole, a condition
that supports early identification of analysis and design errors.
 prototype system available at a very early stage
 Validation can begin early in the testing process as a demonstrable system can be made available to the
users.
disadvantage : the comparative difficulty of its performance.
Test Conditions are difficult to create

17
Top-Down Vs Bottom-Up
bottom-up
advantage : Test Conditions are easier to create
Do not have to build custom adapters at early phase
disadvantage : the lateness of the stage at which it is possible to
observe the program as a whole.

18
Software test classifications
Classification according to testing concept
 Black box (functionality) testing : identifies bugs only according to
malfunctioning of the software as revealed from its outputs, while disregarding the
internal paths of calculations performed by the software.
 White box (structural) testing : examines the internal paths of calculations in
order to identify bugs.

19
Classification according to requirements
An extension of McCall’s model to the classification of the tests carried out to ensure full coverage of the
respective requirements
 Operation, Revision, Transition

Image from http://testerdeep.blogspot.in/ 20

21
 Test Scenarios
Test scenarios provides high-level description of the functionalities we need to test for.
For example: There is a home page of website and on Home page Login functionality is present
so here test scenarios for Login functionality would be:

22
Image from http://testerdeep.blogspot.in/
 Test cases
Test cases are the conditions which we test for any particular scenarios and test cases provides more
details of functionalities.
For Example: For the scenario “Active user tries to login” following could be the test cases:

23
Image from http://testerdeep.blogspot.in/
 When bugs are found...
Bug/Defect:
• This is most interesting thing for testers.
• This is the output of testing effort .
• Any behaviour of the system which is not according to the
requirement and any behaviour which negatively impacts the
system is a bug/defect.
• Bug can be related to functional behaviour, UI behaviour, Usability
behaviour ,Performance behaviour and Security behaviour of the
application.

24
Points to remember when raising bugs

Image from http://testerdeep.blogspot.in/ 25

Bug/defect life cycle

Images from http://www.techbeamers.com/software-bug-life-cycle/ 26

 White box testing
Realization of the white box testing concept requires verification of every
program statement and comment.

White box testing enables performance of:

 data processing and calculations correctness tests (white box correctness
test),
 software qualification tests,
 maintainability tests and
 reusability tests.

27
Data processing and calculation correctness tests
 Applying the concept of white box testing, which is based on checking the data processing for each test case,
immediately raises the question of coverage of a vast number of possible processing paths and the multitudes of lines
of code.

every computational operation in the sequence of operations created by each test case (“path”) must be
examined.
This type of verification allows us to decide whether the processing operations and their sequences were
programmed correctly for the path in question, but not for other paths.

 Two alternative approaches have emerged:

Path coverage
Line coverage
28
Path coverage
 “Path coverage” is defined as the percentage of possible paths of software
processing activated by the test cases.
Different paths in a software module are created by the choice in
conditional statements, such as IF–THEN–ELSE or DO WHILE or DO
UNTIL.
Ideally one test case for each possible path….. Impractical
Directed mainly to high risk software modules

29
Path Testing
Path testing is a structural testing method that involves using the
source code of a program in order to find every possible
executable path. It helps to determine all faults lying within a piece
of code. This method is designed to execute all or selected
path through a computer program

30
Path Testing
The objective behind basis path in software testing is that it defines the number of independent
paths, thus the number of test cases needed can be defined explicitly (maximizes the coverage
of each test case).

In this example, we can see there are few conditional

statements that is executed depending on what condition it
suffice. Here there are 3 paths or condition that need to be
tested to get the output,
•Path 1: 1,2,3,5,6, 7
•Path 2: 1,2,4,5,6, 7
•Path 3: 1, 6, 7

31
Example :
Taximeter

32
Steps for Path Testing
1. Draw a flow graph or control graph (to determine different
program paths)
2. Calculate Cyclomatic complexity (metrics to determine the
number of independent paths)
3. Find a basic set of paths
4. Generate test cases to exercise each path

33
 McCabe’s cyclomatic complexity metrics
 The cyclomatic complexity metrics developed by McCabe (1976) measures the complexity of a
program or module at the same time as it determines the maximum number of independent paths
needed to achieve full line coverage of the program.
An Independent path is any path on the program flow graph that includes at least one edge that is not included in any
former independent paths.

34
Cyclomatic Complexity

Flow Graph notation for a program depict several nodes

connected through the edges. These Flow diagrams for
statements like if-else, While, until and normal sequence of
flow.
35
 McCabe’s cyclomatic complexity metrics
• The cyclomatic complexity metric V(G)

V(G) = R = E – N + 2 = P + 1
◦ Where, R = the number of regions
◦ E = number of edges
◦ N = number of nodes
◦ P = number of decisions (nodes having more than one leaving edge).

• An empirical studies show that if V(G) < 5 considered simple

• If it is 10 or less considered not too difficult

36
 V(G) = R = E – N + 2 = P + 1

V(G) = E – N + 2
V(G) = 8-7+2= 3

or

V(G) = P + 1
V(G) = 2 + 1 = 3

• An empirical studies show that if V(G) < 5 considered simple

37
Advantages of Path Testing
It helps to reduce the redundant tests
It focuses attention on program logic
Identify independent paths

38
Line coverage (Statement Coverage)
“Line coverage” is defined as the percentage of executed lines of
code examined during the tests.
alternative yet weaker coverage concept.
Requires far fewer test cases but, as expected, leaves most of the
possible paths untested.
The concepts of path testing and line coverage are applicable for
estimating white box testing coverage only.

39
Line Coverage
It is used to calculate and measure the number of statements in the source code which can be
executed given the requirements.

Line Coverage =(Number of statements exercised/Total number of statements)*100

Generally in any software, if we look at the source code, there will be a wide variety of elements
like operators, functions, looping, exceptional handlers, etc. Based on the input to the program,
some of the code statements may not be executed. The goal of Statement coverage is to cover
all the possible path's, line, and statement in the code.

40
Line Coverage Example

◦ If A =3 and B=9

Number of executed statements = 5,

Total number of statements = 7
Line Coverage: 5/7 = 71%

41
Line Coverage Example
If A = -3, B = -9
Number of executed statements = 6
Total number of statements = 7
Line Coverage: 6/7 = 85%

all the statements are being covered by 2nd scenario's

considered. So we can conclude that overall statement
coverage is 100%.

Line Coverage allows the tester to identify

◦ Unused Statements
◦ Dead Code
◦ Unused Branches

42
software qualification
Focus here shifts to the examination of software code (including comments) compliance with
coding standards and work instructions.

Does the code fulfill the code structure instructions and procedures,
such as module size, application of reused code, etc.?
Does the coding style fulfill coding style procedures?
Do the internal program documentation and “help” sections fulfill
coding style procedures?

A popular Code Auditing tool : Sonarqube [https://www.sonarqube.org/]

43
44
Maintainability
Corrective maintenance – Correcting problems. The maintainability
of a system can be measured in terms of the time taken to diagnose
and fix problems identified within that system.
Perfective maintenance – Enhancements. The maintainability of a
system can also be measured in terms of the effort taken to make
required enhancements to that system.
◦ This can be tested by recording the time taken to achieve a new piece of
identifiable functionality such as a change to the database, etc. A number of
similar tests should be run and an average time calculated. The outcome will
be that it is possible to give an average effort required to implement
specified functionality.

45
Maintainability
Adaptive maintenance – Adapting to changes in environment. The
maintainability of a system can also be measured in terms on the
effort required to make required adaptations to that system. This can
be measured in the way described above for perfective
maintainability testing.

Preventive maintenance – Actions to reduce future maintenance

costs. This refers to actions to reduce future maintenance costs.

46
Maintainability Tests
Maintainability tests : refer to special features, such as those installed
for detection of causes of failure, module structures that support
software adaptations and software improvements, etc.

It basically defines that how easy it is to maintain the system.

Maintenance Testing is done on the already deployed software. The

deployed software needs to be enhanced, changed or migrated to
other hardware. The Testing done during this enhancement, change
and migration cycle is known as maintenance testing.

47
Reusability tests
Reusability tests : examine the extent that reused software
is incorporated in the package and the adaptations
performed in order to make parts of the current software
reusable for future software packages

48
 White box testing
advantages :
 It permits direct checking of processing paths and algorithms.
 It provides line coverage follow-up that delivers lists of lines of code that have not yet been
executed.
 It is capable of testing the quality of coding work.

disadvantages :
It requires vast resources, much above those required for black box testing.
It cannot test the performance of software in terms of availability, reliability, stress, etc.
49
Black box testing
Black box (functionality) testing : identifies bugs only according to
malfunctioning of the software as revealed from its outputs, while
disregarding the internal paths of calculations performed by the
software.
 Some testing classes are unique to black box testing.

 Still, due to the special characteristics of each testing strategy and the test classes unique to white box
testing, black box testing cannot automatically substitute for white box testing.

50
Equivalence classes for output
correctness tests
 Output correctness tests are, in most cases, among the tests that consume the greater part of testing
resources.
 Equivalence partitioning or equivalence class partitioning (ECP) is a software testing technique that divides the
input data of a software unit into partitions of equivalent data from which test cases can be derived. In
principle, test cases are designed to cover each partition at least once.
 The output correctness tests apply the concept of test cases.
 Improved choice of test cases can be achieved by the efficient use of equivalence class partitioning.
An equivalence class (EC) is a set of input variable values that produce the same output
results or that are processed identically.
Test cases are defined separately for the valid and invalid ECs.
Importantly, as the equivalence class method is a black box method, equivalence class partitioning
is based on software specification documentation, not on the code.
51
Test cases and boundary values
According to the definition of equivalence classes, one test case
should be sufficient for each class.
However, when equivalence classes cover a range of values (e.g.
monthly income, apartment area), the tester has a special
interest in testing border values when these are considered to be
error prone.
In these cases, the preparation of three test cases – for mid range,
lower boundary and upper boundary values – is recommended.

52
Example 1
Text Box that accepts ages from 18 to 56

Age (accepts 18 to 56)

53
Example 2
Need to enter mobile numbers with 10 digits

Mobile Number (accepts 10 digits)

54
Example 3– the Golden Splash Swimming
Center

55
56
57
Other operation factor testing classes
Apart from output correctness tests, operation factor testing classes include the following
classes of tests:

58
Revision factor testing
Easy revision of software is a fundamental factor assuring a
software package’s successful, long service and its
successful sales to larger user populations.
■ Maintainability tests
■ Flexibility tests
■ Testability tests.

59
Transition factor testing classes
The software characteristics required to be operative, with minor adaptations, in different
environments, and those needed to incorporate reused modules or to permit interfacing with
other software packages are all among the transition features required from a software system,
especially for commercial software packages aimed at a wide range of customers.

(1) Portability tests

(2) Reusability tests
(3) Tests for interoperability requirements:
◦ ■ Software interfacing tests
◦ ■ Equipment interfacing tests.

60
Black box testing
advantages :
For test classes that can be carried out by both white and black box testing, black box
testing requires considerably fewer resources.

disadvantages :
 It allows for identification of coincidental errors as correct.
It lacks control of line coverage.
It lacks possibilities to test the quality of coding work.

61
 Summary
• Software test classifications

 classification according to testing concept

 classification according to requirements
• White box testing

 data processing and calculation correctness tests

 path coverage vs. line coverage
 McCabe’s cyclomatic complexity metrics
 software qualification and reusability testing
 advantages/disadvantages of white box testing
• Black box testing

 Equivalence classes for output correctness tests

 Operation / Revision / Transition factor testing classes
 advantages/disadvantages of black box testing

62