Software Testing: Dr. R. Mall
Software Testing: Dr. R. Mall
(continued)
Lecture 11b
Dr. R. Mall
1
Organization of this
Lecture:
Review of last lecture.
Data flow testing
Mutation testing
Cause effect graphing
Performance testing.
Test summary report
Summary
2
Review of last lecture
3
Review of last lecture
4
Data Flow-Based Testing
5
Data Flow-Based Testing
6
Data Flow-Based Testing
8
Definition-use chain (DU
chain)
[X,S,S1],
S and S1 are statement numbers,
X in DEF(S)
X in USES(S1), and
the definition of X in the statement S is
live at statement S1.
9
Data Flow-Based Testing
10
Data Flow-Based Testing
11
Data Flow-Based Testing
1 X(){
2 B1; /* Defines variable a */
3 While(C1) {
4 if (C2)
5 if(C4) B4; /*Uses variable a */
6 else B5;
7 else if (C3) B2;
8 else B3; }
9 B6 }
12
Data Flow-Based Testing
[a,1,5]: a DU chain.
Assume:
DEF(X) = {B1, B2, B3, B4, B5}
USED(X) = {B2, B3, B4, B5, B6}
There are 25 DU chains.
However only 5 paths are needed to cover
these chains.
13
Mutation Testing
15
Mutation Testing
A mutated program:
tested against the full test suite of the
program.
If there exists at least one test case in the
test suite for which:
a mutant gives an incorrect result,
then the mutant is said to be dead.
16
Mutation Testing
If a mutant remains alive:
even after all test cases have been exhausted,
the test suite is enhanced to kill the mutant.
The process of generation and killing of
mutants:
can be automated by predefining a set of
primitive changes that can be applied to the
program.
17
Mutation Testing
18
Mutation Testing
19
Cause and Effect Graphs
20
Cause and Effect Graphs
21
Cause and Effect Graphs
22
Steps to create cause-
effect graph
Study the functional requirements.
Mark and number all causes and
effects.
Numbered causes and effects:
become nodes of the graph.
23
Steps to create cause-
effect graph
Draw causes on the LHS
Draw effects on the RHS
Draw logical relationship between
causes and effects
as edges in the graph.
Extra nodes can be added
to simplify the graph
24
Drawing Cause-Effect
Graphs
A B
If A then B
A
B
C
If (A and B)then C
25
Drawing Cause-Effect
Graphs
A
B
C
If (A or B)then C
A
B
C
If (not(A and B))then C
26
Drawing Cause-Effect
Graphs
A
B
C
If (not (A or B))then C
A B
If (not A) then B
27
Cause effect graph-
Example
A water level monitoring
system
used by an agency involved in
flood control.
Input: level(a,b)
a is the height of water in dam in
meters
b is the rainfall in the last 24
hours in cms
28
Cause effect graph-
Example
Processing
The function calculates whether the level
is safe, too high, or too low.
Output
message on screen
level=safe
level=high
invalid syntax
29
Cause effect graph-
Example
30
Cause effect graph-
Example
Parameters A and B are real numbers:
such that the water level is calculated to be
low
or safe.
The parameters A and B are real numbers:
such that the water level is calculated to be
high.
31
Cause effect graph-
Example
32
Cause effect graph-
Example
Three effects
level = safe
level = high
invalid syntax
33
Cause effect graph-
Example
1 10 E
3
2 11
3 E
1
4
E
5 2
34
Cause effect graph-
Decision table
Test 1 Test 2 Test 3 Test 4 Test 5
Cause 1 I I I S I
Cause 2 I I I X S
Cause 3 I S S X X
Cause 4 S I S X X
Cause 5 S S I X X
Effect 1 P P A A A
Effect 2 A A P A A
Effect 3 A A A P P
35
Cause effect graph-
Example
36
Cause effect graph-
Example
37
Cause effect graph-
Example
38
Cause effect graph-
Example
39
Cause effect graph
40
Testing
Unit testing:
test the functionalities of a single module or
function.
Integration testing:
test the interfaces among the modules.
System testing:
test the fully integrated system against its
functional and non-functional requirements.
41
Integration testing
42
System Testing
System testing:
validate a fully developed
system against its
requirements.
43
Integration Testing
44
Example Structured
Design
root
Valid-numbers rms
rms
Valid-numbers
Validate-
Get-data data
45
Big bang Integration
Testing
46
Big bang Integration
Testing
Main problems with this approach:
if an error is found:
it is very difficult to localize the error
the error may potentially belong to any of
the modules being integrated.
debugging errors found during big
bang integration testing are very
expensive to fix.
47
Bottom-up Integration
Testing
Integrate and test the bottom level
modules first.
A disadvantage of bottom-up testing:
when the system is made up of a large
number of small subsystems.
This extreme case corresponds to the big
bang approach.
48
Top-down integration
testing
Top-down integration testing starts with
the main routine:
and one or two subordinate routines in the
system.
After the top-level 'skeleton’ has been
tested:
immediate subordinate modules of the
'skeleton’ are combined with it and tested.
49
Mixed integration testing
50
Integration Testing
In top-down approach:
testing waits till all top-level modules are
coded and unit tested.
In bottom-up approach:
testing can start only after bottom level
modules are ready.
51
Phased versus Incremental
Integration Testing
52
Phased versus Incremental
Integration Testing
In phased integration,
a group of related modules are
added to the partially integrated
system each time.
Big-bang testing:
a degenerate case of the phased
integration testing.
53
Phased versus Incremental
Integration Testing
Phased integration requires less
number of integration steps:
compared to the incremental integration
approach.
However, when failures are detected,
it is easier to debug if using incremental
testing
since errors are very likely to be in the
newly integrated module.
54
System Testing
55
System Testing
56
Alpha testing
57
Beta Testing
58
Acceptance Testing
59
System Testing
60
Performance Testing
Addresses non-functional
requirements.
May sometimes involve testing hardware
and software together.
There are several categories of
performance testing.
61
Stress testing
62
Stress testing
Stress tests are black box tests:
designed to impose a range of
abnormal and even illegal input
conditions
so as to stress the capabilities of the
software.
63
Stress Testing
If the requirements is to handle a
specified number of users, or
devices:
stress testing evaluates system
performance when all users or
devices are busy simultaneously.
64
Stress Testing
If an operating system is supposed to
support 15 multiprogrammed jobs,
the system is stressed by attempting to run 15
or more jobs simultaneously.
A real-time system might be tested
to determine the effect of simultaneous arrival
of several high-priority interrupts.
65
Stress Testing
67
Configuration Testing
Analyze system behavior:
in various hardware and software
configurations specified in the requirements
sometimes systems are built in various
configurations for different users
for instance, a minimal system may serve a
single user,
other configurations for additional users.
68
Compatibility Testing
69
Compatibility testing
Example
If a system is to communicate
with a large database system to
retrieve information:
a compatibility test examines speed
and accuracy of retrieval.
70
Recovery Testing
71
Maintenance Testing
72
Maintenance Testing
Verify that:
all required artifacts for
maintenance exist
they function properly
73
Documentation tests
74
Documentation tests
75
Usability tests
76
Environmental test
These tests check the system’s ability to perform
at the installation site.
Requirements might include tolerance for
heat
humidity
chemical presence
portability
electrical or magnetic fields
disruption of power, etc.
77
Test Summary Report
78
Test Summary Report
Specifies:
how many tests have been applied to a
subsystem,
how many tests have been successful,
how many have been unsuccessful, and the
degree to which they have been unsuccessful,
e.g. whether a test was an outright failure
or whether some expected results of the test were
actually observed.
79
Regression Testing
80
Regression testing
81
Regression testing
82
Summary
83
Summary
84
Summary
Cause-effect graphing:
can be used to automatically derive test
cases from the SRS document.
Decision table derived from cause-effect
graph
each column of the decision table forms
a test case
85
Summary
Integration testing:
Develop integration plan by
examining the structure chart:
big bang approach
top-down approach
bottom-up approach
mixed approach
86
Summary: System testing
Functional test
Performance test
stress
volume
configuration
compatibility
maintenance
87