1

Chapter 9.1

Software Testing Strategies

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 2

Chapter 9.1 – Software Testing Strategies

• Definitions and objectives
• Software testing strategies
• Software test classifications
• White box testing
• Data processing and calculation correctness tests
• Correctness tests and path coverage
• Correctness tests and line coverage
• McCabe’s cyclomatic complexity metrics
• Software qualification and reusability testing
• Advantages and disadvantages of white box testing
• Black box testing
• Equivalence classes for output correctness tests
• Other operation factor testing classes
• Revision factor testing classes
• Transition factor testing classes
• Advantages and disadvantages of black box testing

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 3
Introduction
• In the past, testing (other than some unit testing) was
undertaken as the last part of development after the
entire app was developed and documented.
• Later, when discovered problems were discovered too
late, unit testing and integration testing became in
vogue.
• Later, the notion of testing has been expanded
significantly to include documentation, and hosts of
other related software quality assurance activities.
• We will concentrate on testing as accommodated by
running the code.
Galin, SQA from theory to implementation © Pearson Education Limited 2004
 4
Introduction
• As of 1994, some 24% of project development
budget allocated to testing; 32% of project
management budget slated for testing.
• Major emphases on increased testing has met
resistance and project managers had to reduce.
– We cannot test everything!!!
• We often ‘test’ documentation via standard SQA. But
this does not test the functionality of the code.
• Will concentrate on testing classifications and
strategies and classification of requirements types

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 5
Introduction
• Need to be able to:
– Explain testing objectives
– Discuss the differences between the various testing strategies:
advantages and disadvantages
– Describe the concepts of both black box and white box testing and their
advantages and disadvantages
– Discuss path coverage versus line coverage
– Describe the various types of black box tests.
– Testing has become a huge business and major emphasis
in software engineering!

• Testing has been described as the process of executing a

program with the intention of finding errors.
Galin, SQA from theory to implementation © Pearson Education Limited 2004
 6

Software Tests - Definition

Software testing is a formal process carried out by a

specialized testing team in which a software unit, several
integrated software units or an entire software package are
examined by running the programs on a computer. All
the associated tests are performed according to approved
test procedures on approved test cases.

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 7

• Formal: scheduled in advance; central in the

development process; not ad hoc.
• Specialized Testing Team: independent test team or
external consultancy. Unit tests by developers – okay,
but that is it.
• Running the program – a must; nothing static.
• Approved Test Procedures: use a plan approved.
• Approved Test Cases: All planned; all tests
undertaken.

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 8
Software Testing Objectives
Direct objectives
a. To identify and reveal as many errors as possible in
the tested software
b. To bring the tested software, after correction of the
identified errors and retesting, to an acceptable level
of quality.
c. To perform the required tests efficiently and
effectively, within the limits budgetary and
scheduling limitation.
Indirect objectives
a. To compile a record of software errors for use in
error prevention (by corrective and preventive
actions)  
Galin, SQA from theory to implementation © Pearson Education Limited 2004
 9
Software Testing Strategies

• Incremental testing strategies:

- Test incrementally: Unit testing;
Integration testing; System testing
• Bottom-up testing
• Top-down testing

• Big bang testing –

Test entire software at one time.

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 10
Bottom-up testing
Explain sequence

Stage 4 M11

Integration B Integration c

Stage 3 M9 M10
Integration A

Stage 2 M8

Stage 1 M1 M2 M3 M4 M5 M6 M7

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 11 Top-down testing Explain:
Integration D
Integration C
Integration B
Integration A
Stage 1 M11

Stage 2 M9 M10

Stage 3 M8

Stage 4 M6 M7

Stage 5 M1 M2

Stage 6 M3 M4 M5

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 12

• There are many variations to these approaches too.

• One can do a depth-first approach (vertically
sequenced, according to your book) or a horizontally-
sequenced approach.
• This depends on many factors

• Can also do a sandwich approach too: partially

bottom-up and then top-down.

• When to do this? Perhaps when existing modules are

reusable and already exist. Can do both!
Galin, SQA from theory to implementation © Pearson Education Limited 2004
 13

Use of Stubs and Drivers

for Incremental Testing

Top-down testing of module M8 Bottom-up testing of module M8

Module
tested in an Drive
M9 of M9
earlier
stage

M8 Module M8 Module
on test on test

Modules
Stub Stub tested in an
of M1 of M2 M1 M2 earlier
stage
Galin, SQA from theory to implementation © Pearson Education Limited 2004
Tree Class – Top Down:
Examples of Drivers and Stubs

class Tree
{
private Node root; // only data field in Tree; but key!

public void find (int key)

{
// stub: not showing details of this method here
}// end find()

public void insert (int id, double dd)

{
// stub; placeholder
}// end insert()

pubic void delete (int id)

{
// stub
}// end delete()

} // end class Tree. 14

Top Down: Drivers:

In main somewhere:

Tree theTree = new Tree();

theTree.find(); // main hunks of functionality
theTree.insert();
theTree.display();
// program runs in its entirety each time.

15
Tree Class – Example of Stubs – can do “Displays!”
class Tree
{
private Node root; // only data field in Tree; but key!

public void find (int key)

{
// stub: not showing details of this method here
System.out.println ( “Got into find()” );

}// end find()

public void insert (int id, double dd)

{
// stub; placeholder
System.out.println ( “Got into insert()” );
}// end insert()

pubic void delete (int id)

{
// stub
System.out.println (“ Got into delete routine () “ );
}// end delete()

} // end class Tree.

16
Fill in code incrementally…. Develop method…
public Node find (int key)
{ // assumes non-empty tree
Node current = root; // start at root

while (current.iData != key) // if no match

{
if (key < current.iData)
current = current.leftChild;
// recall: current = current.next??
else
current = current.rightChild;
If (current == null)
return null; // not found; boundary condition
} // end while
return current; // returns reference to node
} // end find()

17
Insert code into placeholder as developed…
 class Tree
{
 private Node root; // only data field in Tree; but key!
 public void find (int key)
 {
 Node current = root; // start at root
while (current.iData != key) // if no match
{
if (key < current.iData)
current = current.leftChild;
else
current = current.rightChild;
If (current == null)
return null; // not found; boundary condition
} // end while
return current; // returns reference to node
} // end find()

public void insert (int id, double dd)

{
// stub; placeholder
System.out.println ( “Got into insert()” );
}// end insert()
pubic void delete (int id)
{
// stub
System.out.println (“ Got into delete routine () “ );
}// end delete()
} // end class Tree.

18
 19 Tree Class – Bottom Up:

• For bottom up, you must simulate data passed to a

lower level module from above.
• Use dummy data…and parameters and returns…

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 20

Comparison: Bottom-Up versus Top Down

• Bottom Up –
– Main advantage is the relative ease of its performance.

– Main disadvantage is the lateness at which the program as a

whole can be observed.

– Sometimes the pieces may not fit too. Structure may be off.

– Sometimes awkward to pass dummy data and to accept

returned data…

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 21

Comparison: Bottom-Up versus Top Down

• Top Down: -
– Main advantage is the possibilities if offers to demonstrate
the entire program functions shortly after activation of the
upper level modules are completed.
– Can show early analysis and design flaws
– Main disadvantage – often requires complicated
programming and relative difficulty of analyzing the results
of tests.
– Supports top down programming, top down testing, and
more.
– Easy to add functionality via stubs and drivers.

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 22

Comparison: Bottom-Up versus Top Down

• Lots of debates on preferable strategy.
• Choice is usually dependent upon the developers
choice of a strategy – top down or bottom up.

• The testing strategy needs to follow the development

strategy.

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 Big Bang Approach vs Incremental
23

• Big Bang: In general, not a good approach, unless program is very small
and not terribly complicated.
– Difficult to identify errors and where they are located.
– Simply way too much code / functionality to evaluate at one time.

• Incremental testing provides a number of advantages

– Test on usually small hunks of code, like unit or integration tests.
– Easier to identify errors than with whole project
– Correction is much simpler and requires far fewer resources too.
– Find errors much earlier in process.
– Prevents migration of errors into later, more complex stages.
– But you do have overhead of developing drivers and stubs and drivers for integration testing.
– Also, you carry out many testing operations on the same program vice only a single testing
operation.

• Best: generally incremental approach is preferred despite its disadvantages.

Galin, SQA from theory to implementation © Pearson Education Limited 2004
 24 Software Test Classifications
• While quite different (each has strong proponents), there are two
accepted classification schemes:
– 1. Black Box testing
– 2. While Box testing.
• Black box Testing: (Functional Testing) – identifies bugs only
according to software malfunctioning as they are revealed in its
erroneous outputs. In cases that the outputs are found to be correct, black
box testing disregards the internal path of calculations and processing
performed.
• White Box Testing: (Structural) – Examines internal calculation paths in
order to identify bugs. Although the term ‘white’ is meant to emphasize
the contrast between this method and black box testing, the method’s
other name – ‘glass box testing’ better expresses its basic characteristic,
that of investigating the correctness of code structure.

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 25
Black Box and White
Box - IEEE Definitions

Black box testing – IEEE definition

1.Testing that ignores the internal mechanism of the
system or component and focuses solely on outputs in
response to selected inputs and execution conditions
2.Testing conducted to evaluate the compliance of a
system or component with specified functional
requirements
White box testing – IEEE definition
Testing that takes into account the internal
mechanism of a system or component

Galin, SQA from theory to implementation © Pearson Education Limited 2004

 26

Classification According to Requirements

• White box testing (to me) is more complicated than black box
testing. There are many kinds of white box testing:
– Path testing, branch testing, equivalence class testing, static analysis,
dynamic analysis, etc. Complexity testing, flow graphs, call graphs, etc.
– These can be very complicated.
– Good for verification.
• Black box testing – oversimplified – checks to see if the correct
outputs are produced for specific inputs.
– There is no internal checking.
– Simple outputs given inputs
– Great for validation testing.

Galin, SQA from theory to implementation © Pearson Education Limited 2004