0% found this document useful (0 votes)
53 views

Software Testing: Dr. R. Mall

This document discusses various white box testing techniques including data flow testing, mutation testing, and cause effect graphing. Data flow testing selects test paths according to variable definitions and uses. Mutation testing makes small deliberate changes to a program to test if existing test cases can detect the changes. Cause effect graphing converts requirements into a graph to systematically generate test cases from the requirements. The document also discusses unit, integration, and system level testing.

Uploaded by

gitu583
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
53 views

Software Testing: Dr. R. Mall

This document discusses various white box testing techniques including data flow testing, mutation testing, and cause effect graphing. Data flow testing selects test paths according to variable definitions and uses. Mutation testing makes small deliberate changes to a program to test if existing test cases can detect the changes. Cause effect graphing converts requirements into a graph to systematically generate test cases from the requirements. The document also discusses unit, integration, and system level testing.

Uploaded by

gitu583
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 87

Software Testing

(continued)
Lecture 11b

Dr. R. Mall

1
Organization of this
Lecture:
 Review of last lecture.
 Data flow testing
 Mutation testing
 Cause effect graphing
 Performance testing.
 Test summary report
 Summary

2
Review of last lecture

 White box testing:


 requires knowledge about
internals of the software.
 design and code is required.
 also called structural testing.

3
Review of last lecture

 We discussed a few white-box test


strategies.
 statement coverage
 branch coverage
 condition coverage
 path coverage

4
Data Flow-Based Testing

 Selects test paths of a


program:
 according to the locations of
 definitions and uses of different
variables in a program.

5
Data Flow-Based Testing

 For a statement numbered S,


 DEF(S) = {X/statement S contains a
definition of X}
 USES(S)= {X/statement S contains
a use of X}
 Example: 1: a=b; DEF(1)={a},
USES(1)={b}.
 Example: 2: a=a+b; DEF(1)={a},
USES(1)={a,b}.

6
Data Flow-Based Testing

 A variable X is said to be live at


statement S1, if
 X is defined at a statement S:
 there exists a path from S to S1
not containing any definition of
X.
7
DU Chain Example
1 X(){
2 a=5; /* Defines variable a */
3 While(C1) {
4 if (C2)
5 b=a*a; /*Uses variable a */
6 a=a-1; /* Defines variable a */
7 }
8 print(a); } /*Uses variable a */

8
Definition-use chain (DU
chain)

 [X,S,S1],
 S and S1 are statement numbers,
 X in DEF(S)
 X in USES(S1), and
 the definition of X in the statement S is
live at statement S1.

9
Data Flow-Based Testing

 One simple data flow testing


strategy:
 every DU chain in a program be
covered at least once.

10
Data Flow-Based Testing

 Data flow testing strategies:


 useful for selecting test paths of a
program containing nested if and
loop statements

11
Data Flow-Based Testing
 1 X(){
 2 B1; /* Defines variable a */
 3 While(C1) {
 4 if (C2)
 5 if(C4) B4; /*Uses variable a */

 6 else B5;
 7 else if (C3) B2;
 8 else B3; }
 9 B6 }

12
Data Flow-Based Testing

 [a,1,5]: a DU chain.
 Assume:
 DEF(X) = {B1, B2, B3, B4, B5}
 USED(X) = {B2, B3, B4, B5, B6}
 There are 25 DU chains.
 However only 5 paths are needed to cover
these chains.

13
Mutation Testing

 The software is first tested:


 using an initial testing method based on white-
box strategies we already discussed.
 After the initial testing is complete,
 mutation testing is taken up.
 The idea behind mutation testing:
 make a few arbitrary small changes to a
program at a time.
14
Mutation Testing

 Each time the program is


changed,
 it is called a mutated program
 the change is called a mutant.

15
Mutation Testing

 A mutated program:
 tested against the full test suite of the
program.
 If there exists at least one test case in the
test suite for which:
 a mutant gives an incorrect result,
 then the mutant is said to be dead.

16
Mutation Testing
 If a mutant remains alive:
 even after all test cases have been exhausted,
 the test suite is enhanced to kill the mutant.
 The process of generation and killing of
mutants:
 can be automated by predefining a set of
primitive changes that can be applied to the
program.

17
Mutation Testing

 The primitive changes can be:


 altering an arithmetic operator,
 changing the value of a constant,
 changing a data type, etc.

18
Mutation Testing

 A major disadvantage of mutation


testing:
 computationally very expensive,
 a large number of possible mutants
can be generated.

19
Cause and Effect Graphs

 Testing would be a lot easier:


 if we could automatically generate test cases
from requirements.
 Work done at IBM:
 Can requirements specifications be
systematically used to design functional test
cases?

20
Cause and Effect Graphs

 Examine the requirements:


 restate them as logical relation between
inputs and outputs.
 The result is a Boolean graph
representing the relationships
 called a cause-effect graph.

21
Cause and Effect Graphs

 Convert the graph to a decision


table:
 each column of the decision table
corresponds to a test case for
functional testing.

22
Steps to create cause-
effect graph
 Study the functional requirements.
 Mark and number all causes and
effects.
 Numbered causes and effects:
 become nodes of the graph.

23
Steps to create cause-
effect graph
 Draw causes on the LHS
 Draw effects on the RHS
 Draw logical relationship between
causes and effects
 as edges in the graph.
 Extra nodes can be added
 to simplify the graph

24
Drawing Cause-Effect
Graphs
A B
If A then B

A
B
C
If (A and B)then C

25
Drawing Cause-Effect
Graphs
A
B
C
If (A or B)then C
A
B
C
If (not(A and B))then C

26
Drawing Cause-Effect
Graphs
A
B
C
If (not (A or B))then C

A B
If (not A) then B

27
Cause effect graph-
Example
 A water level monitoring
system
 used by an agency involved in
flood control.
 Input: level(a,b)
 a is the height of water in dam in
meters
 b is the rainfall in the last 24
hours in cms

28
Cause effect graph-
Example
 Processing
 The function calculates whether the level
is safe, too high, or too low.
 Output
 message on screen
 level=safe
 level=high
 invalid syntax

29
Cause effect graph-
Example

 We can separate the requirements into


5 clauses:
 first five letters of the command is “level”
 command contains exactly two
parameters
 separated by comma and enclosed in
parentheses

30
Cause effect graph-
Example
 Parameters A and B are real numbers:
 such that the water level is calculated to be
low
 or safe.
 The parameters A and B are real numbers:
 such that the water level is calculated to be
high.

31
Cause effect graph-
Example

 Command is syntactically valid


 Operands are syntactically valid.

32
Cause effect graph-
Example

 Three effects
 level = safe
 level = high
 invalid syntax

33
Cause effect graph-
Example
1 10 E
3
2 11
3 E
1
4
E
5 2

34
Cause effect graph-
Decision table
Test 1 Test 2 Test 3 Test 4 Test 5
Cause 1 I I I S I
Cause 2 I I I X S
Cause 3 I S S X X
Cause 4 S I S X X
Cause 5 S S I X X
Effect 1 P P A A A
Effect 2 A A P A A
Effect 3 A A A P P

35
Cause effect graph-
Example

 Put a row in the decision table for


each cause or effect:
 in the example, there are five rows
for causes and three for effects.

36
Cause effect graph-
Example

 The columns of the decision table


correspond to test cases.
 Define the columns by examining
each effect:
 list each combination of causes that can
lead to that effect.

37
Cause effect graph-
Example

 We can determine the number of


columns of the decision table
 by examining the lines flowing into the
effect nodes of the graph.

38
Cause effect graph-
Example

 Theoretically we could have generated


25=32 test cases.
 Using cause effect graphing technique
reduces that number to 5.

39
Cause effect graph

 Not practical for systems which:


 include timing aspects
 feedback from processes is used for
some other processes.

40
Testing
 Unit testing:
 test the functionalities of a single module or
function.
 Integration testing:
 test the interfaces among the modules.
 System testing:
 test the fully integrated system against its
functional and non-functional requirements.
41
Integration testing

 After different modules of a system


have been coded and unit tested:
 modules are integrated in steps
according to an integration plan
 partially integrated system is tested at
each integration step.

42
System Testing

 System testing:
 validate a fully developed
system against its
requirements.

43
Integration Testing

 Develop the integration plan by


examining the structure chart :
 big bang approach
 top-down approach
 bottom-up approach
 mixed approach

44
Example Structured
Design
root
Valid-numbers rms
rms
Valid-numbers

Get-good-data Compute-solution Display-solution

Validate-
Get-data data

45
Big bang Integration
Testing

 Big bang approach is the simplest


integration testing approach:
 all the modules are simply put together
and tested.
 this technique is used only for very small
systems.

46
Big bang Integration
Testing
 Main problems with this approach:
 if an error is found:
 it is very difficult to localize the error
 the error may potentially belong to any of
the modules being integrated.
 debugging errors found during big
bang integration testing are very
expensive to fix.

47
Bottom-up Integration
Testing
 Integrate and test the bottom level
modules first.
 A disadvantage of bottom-up testing:
 when the system is made up of a large
number of small subsystems.
 This extreme case corresponds to the big
bang approach.

48
Top-down integration
testing
 Top-down integration testing starts with
the main routine:
 and one or two subordinate routines in the
system.
 After the top-level 'skeleton’ has been
tested:
 immediate subordinate modules of the
'skeleton’ are combined with it and tested.

49
Mixed integration testing

 Mixed (or sandwiched) integration


testing:
 uses both top-down and bottom-up
testing approaches.
 Most common approach

50
Integration Testing

 In top-down approach:
 testing waits till all top-level modules are
coded and unit tested.
 In bottom-up approach:
 testing can start only after bottom level
modules are ready.

51
Phased versus Incremental
Integration Testing

 Integration can be incremental or


phased.
 In incremental integration testing,
 only one new module is added to
the partial system each time.

52
Phased versus Incremental
Integration Testing

 In phased integration,
 a group of related modules are
added to the partially integrated
system each time.
 Big-bang testing:
 a degenerate case of the phased
integration testing.

53
Phased versus Incremental
Integration Testing
 Phased integration requires less
number of integration steps:
 compared to the incremental integration
approach.
 However, when failures are detected,
 it is easier to debug if using incremental
testing
 since errors are very likely to be in the
newly integrated module.

54
System Testing

 System tests are designed to


validate a fully developed
system:
 to assure that it meets its
requirements.

55
System Testing

 There are essentially three main


kinds of system testing:
 Alpha Testing
 Beta Testing
 Acceptance Testing

56
Alpha testing

 System testing is carried out


 by the test team within the
developing organization.

57
Beta Testing

 Beta testing is the system testing:


 performed by a select group of
friendly customers.

58
Acceptance Testing

 Acceptance testing is the system


testing performed by the customer
 to determine whether he should
accept the delivery of the system.

59
System Testing

 During system testing, in addition


to functional tests:
 performance tests are performed.

60
Performance Testing

 Addresses non-functional
requirements.
 May sometimes involve testing hardware
and software together.
 There are several categories of
performance testing.

61
Stress testing

 Evaluates system performance


 when stressed for short periods of
time.
 Stress testing
 also known as endurance testing.

62
Stress testing
 Stress tests are black box tests:
 designed to impose a range of
abnormal and even illegal input
conditions
 so as to stress the capabilities of the
software.

63
Stress Testing
 If the requirements is to handle a
specified number of users, or
devices:
 stress testing evaluates system
performance when all users or
devices are busy simultaneously.

64
Stress Testing
 If an operating system is supposed to
support 15 multiprogrammed jobs,
 the system is stressed by attempting to run 15
or more jobs simultaneously.
 A real-time system might be tested
 to determine the effect of simultaneous arrival
of several high-priority interrupts.

65
Stress Testing

 Stress testing usually involves an element


of time or size,
 such as the number of records transferred per
unit time,
 the maximum number of users active at any
time, input data size, etc.
 Therefore stress testing may not be
applicable to many types of systems.
66
Volume Testing
 Addresses handling large amounts of
data in the system:
 whether data structures (e.g. queues,
stacks, arrays, etc.) are large enough to
handle all possible situations
 Fields, records, and files are stressed to
check if their size can accommodate all
possible data volumes.

67
Configuration Testing
 Analyze system behavior:
 in various hardware and software
configurations specified in the requirements
 sometimes systems are built in various
configurations for different users
 for instance, a minimal system may serve a
single user,
 other configurations for additional users.

68
Compatibility Testing

 These tests are needed when the


system interfaces with other
systems:
 check whether the interface
functions as required.

69
Compatibility testing
Example
 If a system is to communicate
with a large database system to
retrieve information:
 a compatibility test examines speed
and accuracy of retrieval.

70
Recovery Testing

 These tests check response to:


 presence of faults or to the loss of
data, power, devices, or services
 subject system to loss of resources
 check if the system recovers properly.

71
Maintenance Testing

 Diagnostic tools and procedures:


 help find source of problems.
 It may be required to supply
 memory maps
 diagnostic programs
 traces of transactions,
 circuit diagrams, etc.

72
Maintenance Testing

 Verify that:
 all required artifacts for
maintenance exist
 they function properly

73
Documentation tests

 Check that required documents


exist and are consistent:
 user guides,
 maintenance guides,
 technical documents

74
Documentation tests

 Sometimes requirements specify:


 format and audience of specific
documents
 documents are evaluated for
compliance

75
Usability tests

 All aspects of user interfaces are


tested:
 Display screens
 messages
 report formats
 navigation and selection problems

76
Environmental test
 These tests check the system’s ability to perform
at the installation site.
 Requirements might include tolerance for
 heat
 humidity
 chemical presence
 portability
 electrical or magnetic fields
 disruption of power, etc.

77
Test Summary Report

 Generated towards the end of testing


phase.
 Covers each subsystem:
 a summary of tests which have been
applied to the subsystem.

78
Test Summary Report

 Specifies:
 how many tests have been applied to a
subsystem,
 how many tests have been successful,
 how many have been unsuccessful, and the
degree to which they have been unsuccessful,
 e.g. whether a test was an outright failure
 or whether some expected results of the test were
actually observed.

79
Regression Testing

 Does not belong to either unit


test, integration test, or system
test.
 In stead, it is a separate dimension
to these three forms of testing.

80
Regression testing

 Regression testing is the running of


test suite:
 after each change to the system or after
each bug fix
 ensures that no new bug has been
introduced due to the change or the bug
fix.

81
Regression testing

 Regression tests assure:


 the new system’s performance is at
least as good as the old system
 always used during phased system
development.

82
Summary

 We discussed two additional


white box testing
methodologies:
 data flow testing
 mutation testing

83
Summary

 Data flow testing:


 derive test cases based on definition and use
of data
 Mutation testing:
 make arbitrary small changes
 see if the existing test suite detect these
 if not, augment test suite

84
Summary

 Cause-effect graphing:
 can be used to automatically derive test
cases from the SRS document.
 Decision table derived from cause-effect
graph
 each column of the decision table forms
a test case

85
Summary

 Integration testing:
 Develop integration plan by
examining the structure chart:
 big bang approach
 top-down approach
 bottom-up approach
 mixed approach

86
Summary: System testing

 Functional test
 Performance test
 stress
 volume
 configuration
 compatibility
 maintenance
87

You might also like