Software Testing

Prepared by:Raghav
 Target Audience


Beginners

Intermediate
 What do we cover?


Basics of testing

Levels of Testing

Testing techniques
 Software testing
Relative cost
per defect

125

100

75

50

25

Req Design Code Test Production

 Software testing

Points to Ponder:

80% of all defects occur in 20% of the work

Test strategies impact test costs

Conversion of testing from craft to process can
give extensive benefits
 Software Testing

What hinders effective testing?

 Aspects that hinder Effective
testing

 Optimism
 Belief that the system works
 Negative attitude towards effective testing
 Conflict between testers and developers
 Testing is expensive
 Delivery commitments
The V Model

What is V model?
 The V Model

Customer Acceptance
Requirements
System

validation
Software
Requirements testing
Architecture (HLD) Integration
testing
Detailed Design Unit testing
(LLD)

Code
 Testing v/s Life Cycle
Stage Test Activity
Requirements Acceptance Test ,System Test

Design (HL) Integration Test

Design (LL) Unit Test
Code Unit test / Debug
Integration Integration test / debug
System test / debug
Acceptance Acceptance test / debug
 Why Test?

● Developers not fallible

● Bugs in compilers, languages, DBs, OS

● Certain bugs easier to find in testing

● Don’t want customers to find bugs

● Post release debugging is expensive

 Test team

Why separate testing team?

 Is Testing everything?

 There are other approaches possible to

create good software
 Effective methods are
•Inspection
•Design style
•Static analysis
•Language checks
•Development environment
 Can Testing be replaced?

 NO!!
 Even if we use other methods, cannot do away
with testing
 Review, inspect, read, walkthrough, better
methodologies
And then TEST
 TESTING

Definition: Testing is the process of exercising or

evaluating a system or system component by manual
or automated means to verify that it satisfies specified
requirements. (IEEE 83a)
 Testing objective


Process of executing a program with an intent of
finding an error

A good test case is one that has a high probability
of finding an as yet undiscovered error

A successful test is one that uncovers an as yet
undiscovered error
 Testing Involves

•Testplan
•Generate test conditions, cases
•Create required test environment
•Execute tests by
Initiating
application under test
Applying inputs that are specified in the already

generated test case

Compare actual results with expected results
Results: Test passed or failed!
 A ‘FAILED’ Test

● The term ‘Test Failed’ is used when the program

has failed to give the expected result
● Actually, the test has succeeded in discovering a

fault!
 Definitions

•Defect
A deviation from specification or standard
Anything that causes customer dissatisfaction

Verification
All QC activities throughout the life cycle that ensure that
interim deliverables meet their input specification
Validation
The ‘test phase’ of the life cycle which assures that the
end product meets the user’s needs
 Debugging

Debugging is the act of attempting to

determine the cause of the symptoms
of malfunctions detected by testing or
by frenzied user complaints. (Beizer)
 Testing: static and dynamic

• Static Testing: Verification performed without

executing the system’s code
•Code Inspection
•Reviews
• Dynamic Testing: Verification or validation
performed by executing the system’s code
 Testing Techniques

• Black Box
• White Box
• Basis path testing
Black box testing

What is black box testing?

 Black box testing

Also called Behavioural testing, focuses on

functional requirement of the software
 Black box testing

Detects following categories of errors


Incorrect or missing functions

Interface errors

Errors in data structures and external database
access

Behaviour or performance errors

Initialisation and termination errors
Types of Black box testing

Boundary value analysis



Equivalence partitioning


Special value

BVA(Boundary Value Analysis)

What is BVA?
 Boundary value analysis (two variables)

X2

d a<=x1 <=b
c<=x2 <=d

X1
a b
 BVA
Great number of errors tends to occur at the
boundaries of the input domain rather than
in the ‘center’
BVA leads to a selection of test cases that
exercise bounding values
Equivalence partitioning

What is equivalence partitioning?

 Equivalence partitioning testing

•EC partitions the data to partition

of a set
• Partition refers to collection of
mutually disjoint subsets whose
union is the entire set.
•Choose one data element from
each partitioned set.
• EC based testing allows
- to have a sense of complete
testing.
- helps avoid redundancy
• The KEY is the choice of
equivalence relation.
 Special value

What is special value testing?

 Special value testing

• Select test data on the basis of features of a function

to be computed
• Tester uses her/his domain knowledge, experience
with similar programs
• Ad-hoc/seat of pants/skirt testing
• No guidelines, use best engineering judgment
• Special test cases/Error guessing
• Is useful – don't discount effectiveness !
 Levels of testing
What does each level of testing uncover?

• Unit testing • Regression testing

– implementation deficiencies – fix has not broken existing
functionality
– Code / LLD
- product / code
• Integration testing
– interface deficiencies • Alpha / Beta testing
– architecture / HLD • Acceptance testing
• System testing – customer expectations
– requirement deficiencies – notion of stability (time)
– SRS • Certification testing
– external compliance
– third party
 Unit Testing

What is Unit testing?

 Unit Testing

● Unit- Smallest testable piece of software

● A unit can be compiled / assembled / linked /

loaded; and put under a test harness

● Unit testing is done to show that the unit does not

satisfy the functional specification and/or its

implemented structure does not match the
intended design structure
Integration Testing

What is integration testing?

 Integration Testing

● Integration is the process of aggregating

components to create larger components.
● Integration testing done to show that even though

components were individually satisfactory, the

combination is incorrect or inconsistent
 Integration problems

What are the general integration problems?

Types of Integration problems

● Configuration / version control

● I/O format, protocol mismatches

● Conflicting data views/ usage

● Data integrity violated

● Wrong parameters
 Integration test approach

• Top down
• Bottom up
• Sandwich M0

M11 M12 M13

M21 M22 M23

M31
 Top down integration

• Modules are integrated

by moving downward
• Depth-first or breadth-
first manner M0

• Verifies major control

or decision points early
in the test process M11 M12 M13

• Stubs replace low-level

modules M21 M22 M23

M31
 Bottom – up integration

• Construction and
testing with atomic
modules
• Stubs are NOT
needed
• Drivers are needed
 Integration testing – some thoughts
• How to choose integration
order – incremental
What to check for ….
• May be based on
criticality of • Interface integrity
interfaces/module • Functional validity
• Risk reduction principle • Information content
may be used
• Performance
• Critically based on ..
- Module used by lot of
other modules
- Module has high
cyclomatic complexity
- Has special non-
functional (e.g,
performance)
requirements
System testing

What is system testing?

 System Testing

● A system is the big component

● System testing is aimed at revealing bugs that

cannot be attributed to a component as such, to

inconsistencies between components or planned
interactions between components
● Concern: issues, behaviours that can only be

exposed by testing the entire integrated system

(e.g., performance, security, recovery)
 System testing is…
• Functionality Testing
• Compatibility testing
• Installation testing
• Stress testing
• Reliability testing
• Usability testing
• Recovery testing
• Security testing
• Supportability testing
• Performance testing
• Compliance testing
• Storage testing
• …
• Internationalization
testing
 Acceptance Testing

● Aims at uncovering ‘implied requirements’

● Aims at evaluating ‘fitness for use’

● Should not find bugs which should have been

found in earlier testing phases

 Regression Testing

 Testing after changes been made to ensure that

no unwanted changes were made
 Test Strategy

● This defines the scope and general directions for

testing for a project
● Should be developed for each project

● Should be reviewed and approved with the

project plan
 Test Plan

 Test plans specify the test conditions, features,

functions that will be tested for a specific level
of testing. Plans also contain other information
about resources, schedules, etc
 Test plans should be prepared as soon as the
corresponding document in the development
life cycle is produced
Test specification

What is test specification?

 Test Case

● A sequence of steps, sub steps, and other

actions, performed serially, in parallel, or in some
combination of consecution, that creates the
desired test conditions that the test case is
designed to evaluate
 Is complete testing possible?

● To prove that a program is free of bugs is

•Practically impossible
•Theoretically a mammoth exercise
 Stop Criteria

When to stop testing?

 STOP Criteria

● Time runs out – Poor criteria

● A certain number of errors found

● Require a certain test coverage

● Stop when testing becomes unproductive

 Software testing

Exercise
Prepare a checklist for testing
 IV & V
What is Independent Verification & Validation?