17 Software Testing - introduction 2024
17 Software Testing - introduction 2024
What is Testing
“It has been said that "approximately 50% of the elapsed
time and over 50% of the total cost are expended in
testing a program or system being developed”.
Test Cases
• A best tester is not the one who finds the most bugs
errors
requirements conformance
performance
an indication
of quality
Conclusion
"Bugs lurk in corners and
congregate at boundaries…”
Boris Beizer
Error
Typographical Misunderstanding
error Misreading of
a specification of functionality of
a module
Fault
A fault is the result of an error.
It is more precise to say that a fault is the
representation of an error, where representation is the
mode of expression, such as
– narrative text
– dataflow diagrams
– hierarchy charts
– source code, and so on.
Failure
A failure occurs when a fault executes.
A particular fault may cause different failures, depending on how
it has been exercised.
Incident
When a failure occurs, it may or may not be readily apparent to
the users.
An incident is the symptom(s) associated with a failure that alerts
the user to the occurrence of a failure
Test
A test is the act of exercising software with test cases.
A Test has two distinct goals
– To find failures or
– To demonstrate correct execution
Test Case
• A test case has an identity and is associated with a
program behavior.
• It has a set of inputs and a list of expected outputs.
Inputs
Outputs
Requirement Fault
Specification Error Resolution
Fault
Fault
Design Error Isolation
Fault
Fault
Coding
Classification
Fault Incident
Testing
Testing Life Cycle
Verification
Process of determining whether or not the product
of given phase fulfill the specification from the
previous phase
Validation
Process of evaluating the software at the end of
the development to ensure the compliance with the
requirements
Classification of V&V Activities
Testing Shows Testing can show that defects are present, but cannot prove that there are no
Presence of defects. Testing reduces the probability of undiscovered defects remaining in the
Defects software but, even if no defects are found, it is not a proof of correctness.
Exhaustive Testing Testing everything (all combinations of inputs and preconditions) is not feasible
is Impossible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to
focus testing efforts.
Early Testing Testing activities should start as early as possible in the software or system
development life cycle and should be focused on defined objectives.
Testing Principles
A small number of modules contain most of the defects discovered during pre-
Defect Clustering release testing or show the most operational failures.
If the same tests are repeated over and over again, eventually the same set of test
cases will no longer find any new bugs. Tb overcome this 'pesticide paradox', the
Pesticide Paradox test cases need to be regularly reviewed and revised, and new and different tests
need to be written to exercise different parts of the software or system to potentially
find more defects.
Testing is the Testing is done differently in different contexts. For example, safety-critical
Context Dependent software is tested differently from an . e-commerce site.
Absence of Error Finding and fixing defects does not help if the system built is unusable and does not
Fallacy fulfill the users' needs and expectations
Why Testing is So Hard
• To plan and execute test, software tester must
consider
– The software and the function it computes
– The inputs and how they can be combined
– The environment in which the software will eventually
operate
– This difficult, time-consuming process requires technical
sophistication and proper planning.
Why Testing is So Hard
• As a new tester, You might believe that you can
approach a piece of software, fully test it, find all
the bugs, and assure that the software is perfect.
• Unfortunately, this isn’t possible even with the
simplest program.
• Why?
Why Testing is So Hard
• It is difficult, due to following four reasons:
1. The number of possible input is very large.
2. The number of possible output is very large
3. The number of paths through the software is very large
4. The software specification is very subjective
Why Testing is So Hard
How many test cases does it take to fully cover, or exercise,
the source code?
• There are 2n –1 ways, where ‘n’ is the number of conditions in the
statement
– If the ‘n’ is moderately large, imagine the number of test cases
required to test the program
For example, to test a clock application, How many test cases are required
to test different times in a day?
• 12 hours x 60 minutes x 60 seconds x 2 am/pm for a total of 86,400
different input values.
• That’s just the valid values
Limitations of Testing
• We want to test everything before giving the software
to the customers.
• What do we understand, when we say “everything”?
• We may expect one, two or all of the following when
we refer to “everything”:
(i) Execute every statement of the program
(ii) Execute every true and false conditions
(iii) Execute every condition of a decision node
(iv) Execute every possible path
(v) Execute the program with all valid inputs
(vi) Execute the program with all invalid inputs
Limitations of Testing
• It is impossible to achieving these six objectives due
to time and resource constraints as
• We may achieve few of them with limited coverage.
• If we do any compromise, we may miss a bug.
• Hence “Everything” is impossible and we have to
settle for “less than everything” in the real life
situations.
Functional Testing
Functional Testing/ Black Box Testing
• Black Box Testing also called as Behavioral
testing or Functional Testing
• An approach to testing where the program is
considered as a ‘black-box’
• The program test cases are based on the system
specification or the functional requirements of
the software
• It is based on the view that any program can be
considered to be a function that maps from its
input domain to values in its output range
• Test planning can begin early in the Software
process
Functional Testing / Black Box Testing
requirements
output
input events
Example: Lock and Key System
Functionality What you need to know to use
Features of a lock
Feature of a key
Action performed
State
Inputs
Expected outcome
Example: Lock and Key System
Functionality What you need to know to use
Features of a lock It is made of metal, has a hole provision to lock,
has a facility to insert
Feature of a key It is made of metal and created to fit into a
particular lock’s keyhole.
Action performed Key inserted and turned clockwise to lock
Key inserted and turned anticlockwise to unlock
State Locked
Unlocked
a x
b
Fig: Test Cases for Variables x and y
Case 1 : The Triangle Problem
• The triangle program accepts three integers a,b and
c, as input.
• These are taken to be sides of a triangle. The
integers a,b, and c must satisfy the following
condition :
C1: 1 x 100
C2: 1 y 100
C3: 1 z 100
C4: x y+z
C5: y x+z
C6: z x+y
Case 1: Triangle Problem
Boundary Value Test Cases
01
02
99
100
Case 2 : The PrevDate Function
PrevDate is a function of three variables: month, date
and year. It returns the date of the day before the
input date. The month, date, and year variables
have integer values subject to these conditions:
C1 1 month 12
C2 1 day 31
C3 1900 day 2025
Case 2: The Previous Date Function
Boundary Value Analysis Test cases
Limitations of Boundary Value Analysis
• Boundary Value Analysis works well when the program to
be tested is a function of several independent variable that
represent bounded physical quantities.
• A quick look at the boundary value analysis test cases for
PrevDate shows them to be inadequate. For example, Very
little stress occurs on February and on leap years.
• The real problem here is that interesting dependencies
exist among the month, day and year variable.
• Boundary Value Analysis test cases are rudimentary
because they are obtained with very little insight and
imagination
Adhoc Testing
• Testing carried out using no recognized test case
design technique.
• It occurs when the tester uses domain knowledge
experience and information about soft spots to derive
test cases
• Mostly intuitive and least uniform.
• Dependent on the abilities of the tester
• Also known as Special value testing/“hacking”, /“out-
of-box testing”,/ “guerilla testing”
Equivalence Class Testing
Equivalence Class Testing
•
g
•
f
•
• •
e
•
X1
a b c d
Case 1: The Triangle Problem
Equivalent Class Testing
Output domain equivalence classes are:
– O1={<x,y,z>: Equilateral triangle with sides x,y,z}
– O2={<x,y,z>: Isosceles triangle with sides x,y,z}
– O3={<x,y,z>: Scalene triangle with sides x,y,z}
– O4={<x,y,z>: Not a triangle with sides x,y,z}
Case 1: The Triangle Problem
Weak Normal Equivalent Class Testing
WN1 5 5 5 Equilateral
WN2 2 2 3 Isosceles
WN3 3 4 5 Scalene
Input Domain base classes Some inputs domain test cases can
are: be obtained using the
– I1={x: x < 1} relationship amongst x,y and z
– I2={x: x > 100} – I10={< x,y,z >: x = y = z}
– I3={x: 1 ≤ x ≤ 100} – I11={< x,y,z >: x = y, x ≠ z}
– I4={y: y < 1} – I12={< x,y,z >: x = z, x ≠ y}
– I5={y: y > 100} – I13={< x,y,z >: y = z, x ≠ y}
– I6={y: 1 ≤ y ≤ 100} – I14={< x,y,z >: x ≠ y, x ≠ z, y ≠ z}
– I7={z: z < 1} – I15={< x,y,z >: x = y + z}
– I8={z: z > 100} – I16={< x,y,z >: x > y +z}
– I9={z: 1 ≤ z ≤ 100} – I17={< x,y,z >: y = x +z}
– I18={< x,y,z >: y > x + z}
– I19={< x,y,z >: z = x + y}
– I20={< x,y,z >: z > x +y}
Equivalence Class Test
Equivalence class testing is appropriate when input
data is defined in terms of intervals and sets of
discrete values.
Equivalence class testing is strengthened by a hybrid
approach with boundary values testing.
Strong equivalence class testing makes a
presumption that the variables are independent and
the corresponding multiplication of test cases raises
issues of redundancy. If any dependencies occur,
they will often generate “error” test cases.
Structural Testing
Structural Testing
• Also called: White Box Testing
• Based on Source Code
• Can Be dynamic or static
• Necessary because:
– Part of code not fully exercised.
– Section of code may be surplus to the requirements.
– Errors may be missed by functional requirements.
The Challenge of Testing
• Definition:
– Given a program written in a programming language, its
program graph is a directed graph in which nodes are
statement fragments, and edges represent flow of control.
• If i and j are nodes in the program graph, an edge exists from
node i to node j iff the statement fragment corresponding to
node j can be executed immediately after the statement
fragment corresponding to node i.
Steps Involved in Path testing
WHILE
6
1 Int binsearch (intx,int v[], int n
Yes
2 {
3 Int low, high, mid; 7
No
4 low=0;
5 high=n-1;
8 IF
6 while (low<=high) {
No
Ye 15
7 mid=(low+high)/2
s 10 Else IF
8 if (x<v[mid] 9 No
9 high=mid‐1; Ye 12
s
10 else if(x>v[mid]) 11
11 low = mid+1;
13
12 else /* found match*/
13 return mid;
14 }
16
STOP
15 return-1; /* no match*/
16 }
procedure average
TYPE value [1..100] IS SCALAR ARRAY
TYPE average, total_input, total.valid, minimum maximum, sum IS
SCALAR
TYPE i is INTEGER
i=1;
Total _input = total _valid = 0
Sum = 0
DO WHILE value[ i] <> -999 AND total_input <100
increment total_input by 1;
IF value[ i] >= minimum AND value[ i] <= maximum
THEN increment total_valid by 1;
sum = sum+ value[ i]
ELSE skip endif
increment I by 1;
ENDDO
IF total_valid > 0;
THEN average = sum /total_valid;
ELSE average = -999
ENDIF
procedure average
TYPE value [1..100] IS SCALAR ARRAY
TYPE average, total_input, total.valid, minimum maximum, sum IS
SCALAR 1
TYPE i is INTEGER
i=1;
Total _input = total _valid = 0
Sum = 0
2 DO WHILE value[ i] <> -999 AND total_input <100 3
4 increment total_input by 1;
IF value[ i] >= minimum AND value[ i] <= maximum 6
5 THEN increment total_valid by 1;
sum = sum+ value[ i]
7
ELSE skip endif
increment I by 1; 8
ENDDO
9 > 0;
IF total_valid
THEN average =10 sum /total_valid;
ELSE average = -999 11
ENDIF 12
13
1. Flow Graph for the Procedure Average
4
10
5
12 11 6
13 7
8
9
Deriving Test Cases