SOFTWARE QUALITY

ENGINEERING
Lecture 5
Iram Hina

BSSE-VI
 HADITH OF THE DAY

Iram Hina
BSSE-VI
 TESTING
TEST DATA AND TEST CASES
• Test data Inputs which have been devised to
test the system
• Test cases Inputs to test the system and the
predicted outputs from these inputs if the
system operates according to its specification

Iram Hina
BSSE-VI
FUNCTIONAL TESTING VS STRUCTURAL
TESTING
• In the black-box testing approach, test cases are designed
using only the functional specification of the software, i.e.
without any knowledge of the internal structure of the
software. For this reason, black-box testing is known as
functional testing.
• On the other hand, in the white-box testing approach,
designing test cases requires thorough knowledge about the
internal structure of software, and therefore the white-box
testing is called structural testing.

Iram Hina
BSSE-VI
 FUNCTIONAL TESTING (BLACK BOX)
• In the black-box testing, test cases are designed from an
examination of the input/output values only and no
knowledge of design, or code is required.

• The following are the two main approaches to designing

black box test cases.
• Equivalence class portioning
• Boundary value analysis

Iram Hina
BSSE-VI
 EQUIVALENCE CLASS PORTIONING
• In this approach, the domain of input values to a program
is partitioned into a set of equivalence classes.
 The following guideline used be used for designing the
equivalence classes:

 If the input data values to a system can be specified by a range

of values, then one valid and two invalid equivalence classes
should be defined.

Iram Hina
BSSE-VI
 EQUIVALENCE CLASS PORTIONING
• For a software that computes the square root of an input
integer which can assume values in the range of 0 to 5000,
there are three equivalence classes:
• The set of negative integers, the set of integers in the range of
0 and 5000, and the integers larger than 5000.
• Therefore, the test cases must include representatives for
each of the three equivalence classes and a possible test set
can be: {-5,500,6000}.

Iram Hina
BSSE-VI
 BOUNDARY VALUE ANALYSIS
 Boundary value analysis leads to selection of test cases at the
boundaries of the different equivalence classes.
 Example: For a function that computes the square root of
integer values in the range of 0 and 5000.
 The test cases must include the following values: {-1,
0,5000,5001}.

Iram Hina
BSSE-VI
 WHITE-BOX TESTING
 Three Techniques
 Statement coverage
 Branch coverage
 Path coverage

Iram Hina
BSSE-VI
 STATEMENT COVERAGE
• The statement coverage strategy aims to design test cases so
that every statement in a program is executed at least once.
• The principal idea governing the statement coverage strategy
is that unless a statement is executed, it is very hard to
determine if an error exists in that statement.
• Disadvantage?
• However, executing some statement once and observing that
it behaves properly for that input value is no guarantee that it
will behave correctly for all input values.

10

Iram Hina
BSSE-VI
 STATEMENT COVERAGE
 Consider the Euclid’s GCD computation algorithm:

int compute_gcd(x, y)
int x, y;
{
1 while (x! = y){
2 if (x>y) then
3 x= x – y;
4 else y= y – x;
5 }
6 return x;
}

 By choosing the test set (x=4, y=3) or (x=3, y=4)}, we can exercise
the program such that all statements are executed at least once.
11

Iram Hina
BSSE-VI
 BRANCH COVERAGE
• In the branch coverage-based testing strategy, test cases are
designed to make each branch condition to assume true and
false values in turn.
• It is obvious that branch testing guarantees statement
coverage and thus is a stronger testing strategy compared to
the statement coverage-based testing.
• For Euclid’s GCD computation algorithm , the test cases for
branch coverage can be {(x=3, y=3) (x=3, y=2) (x=4, y=3)
(x=3, y=4)}.

12

Iram Hina
BSSE-VI
 PATH COVERAGE
• The path coverage-based testing strategy requires us to
design test cases such that all linearly independent paths in
the program are executed at least once.
• A linearly independent path can be defined in terms of the
control flow graph (CFG) of a program.

13

Iram Hina
BSSE-VI
 CONTROL FLOW GRAPH (CFG)
• A control flow graph describes the sequence in which the
different instructions of a program get executed.
• In other words, a control flow graph describes how the
control flows through the program.
• In order to draw the control flow graph of a program, all the
statements of a program must be numbered first.

14

Iram Hina
BSSE-VI
 CONTROL FLOW GRAPH (CFG)

15

Iram Hina
BSSE-VI
 CONTROL FLOW GRAPH (CFG)

16

Iram Hina
BSSE-VI
 CYCLOMATIC COMPLEXITY
• McCabe’s cyclomatic complexity defines an upper bound for
the number of linearly independent paths through a program.
• Thus, the McCabe’s cyclomatic complexity metric provides a
practical way of determining the maximum number of
linearly independent paths in a program.
• There are three different ways to compute the cyclomatic
complexity. The answers computed by the three methods are
guaranteed to agree.

17

Iram Hina
BSSE-VI
 CYCLOMATIC COMPLEXITY
 Method 1:
 Given a control flow graph G of a program, the cyclomatic
complexity V(G) can be computed as:
 V(G) = E – N + 2
 where N is the number of nodes of the control flow graph and E is
the number of edges in the control flow graph.
 For the CFG of example shown, E=7 and N=6. Therefore, the
cyclomatic complexity = 7-6+2 = 3.

18

Iram Hina
BSSE-VI
 CYCLOMATIC COMPLEXITY
 Method 2:

 An alternative way of computing the cyclomatic complexity of a

program from an inspection of its control flow graph is as follows:

 V(G) = Total number of bounded areas + 1

 For the CFG example, from a visual examination of the CFG

the number of bounded areas is 2.
 Therefore the cyclomatic complexity, computing with this
method is also 2+1 = 3.

19

Iram Hina
BSSE-VI
 CYCLOMATIC COMPLEXITY
 Method 3:
 The cyclomatic complexity of a program can also be easily
computed by computing the number of decision statements of the
program.
 If N is the number of decision statement of a program, then the
McCabe’s metric is equal to N+1.

20

Iram Hina
BSSE-VI
 TESTING TECHNIQUES
• Checklist-Based Testing
• Partitions and Partition Testing
• Usage-Based Testing
• FSM-Based Testing
• Markov Chains as Enhanced FSMs

21

Iram Hina
BSSE-VI
EXAMPLE: A HIGH LEVEL FUNCTION CHECKLIST
FOR SOME RELATIONAL DATABASE SIMPLE
CHECKLIST

SIMPLE CHECKLIST
• backup and restore
• communication
• co-existence
• file I/O
• gateway
• index management
• installation
• logging and recovery
• locking
• migration
• stress

22

Iram Hina
BSSE-VI
 MULTIPLE CHECKLISTS
 Hierarchical:
 Each item in the higher level checklist expandable to a
full checklist at a lower level
 Until we stop at a level of detail deemed enough for
some criteria
 Multi-dimensional:
 For example, a coding standard checklist can be combined
with a component checklist to make sure that each
component follows all the recommended practice in the
coding standards (2-D)

23

Iram Hina
BSSE-VI
 STANDARDS CHECKLIST AND A
COMPONENT CHECKLIST

24

Iram Hina
BSSE-VI
CHECKLISTS: ASSESSMENT
• Key advantage: simplicity.
• Possible drawbacks of checklists:
• Coverage: may have “holes”
• Duplication: need to improve efficiency.
• Complex interactions not modeled.
• Possible solutions:
• specialized checklists →partitions.
• alternatives to checklists: FSMs.

25

Iram Hina
BSSE-VI
 PARTITION-BASED TESTING

• Basic idea of partition-based testing:

• defining meaningful partitions/classes
• Partitions/components/sub-systems
• sampling from partitioned subsets

26

Iram Hina
BSSE-VI
 PARTITION-BASED TESTING
• Extensions to basic idea:
• Sampling from partitioned subsets
• Most common (frequent/rare)
• Coverage based: one sample from each partition
• Usage-related Testing:
• More use →failures more costly/likely
• Usage information in testing
 →operational profiles (OPs)

27

Iram Hina
BSSE-VI
 DEVELOPING OP

• OP: operations & their probabilities.

• probability: partition probabilities that sum up to 1.
• Obtaining OP information:
• identify distinct operations as disjoint alternatives.
• assign associated probabilities
• occurrences/weights → probabilities.
• OP information sources:
• actual measurement at customer installation.
• customer surveys.
• expert opinion.

28

Iram Hina
BSSE-VI
 CUSTOMER PROFILE

• Differentiate customer from users

• Customer: acquisition of software
• User: using software
• Weight assignment:
• By #customers
• By importance/marketing concerns, etc.

29

Iram Hina
BSSE-VI
 A SAMPLE CUSTOMER PROFILE

30

Iram Hina
BSSE-VI
 ESTABLISHING THE USER PROFILE

31

Iram Hina
BSSE-VI
 ESTABLISHING THE USER PROFILE

 Example:
 row: user type
 column: user profile in a customer type
 customer profile used to calculate comprehensive user
profile:
0.8 x 0.5 (com) + 0.9 x 0.4 (gov) +
0.9 x 0.05 (edu) + 0.7 x 0.05 (etc)
= 0.84

32

Iram Hina
BSSE-VI
 END OF LECTURE

THANK YOU !

QUESTIONS?

33

BSSE-VI Iram Hina