Quality Assurance

Outline
• Aim and objectives
• verification - validation: Testing Levels & Testing Strategies
• White Box - Static, Structural- functional, coverage & complexity
• Black Box - Positive –Negative, Boundary Value Analyses,
Decision Tables, Equivalence Partitioning, State Based
• Integration - top-down, bottom-up, bi-directional
• Introduction to system testing (functional and non-functional)
• Introduction to Regression & Performance Testing
Learning Outcomes
• Implement the strategies and methods of software quality assurance
Quality Assurance
• Quality Assurance involves process oriented activities.
• Quality Assurance is the process used to develop products and
focuses on the prevention of defects.
• The goal of Quality Assurance is to improve development and testing
processes to prevent defects from arising during the product
development lifecycle.
Verification-Validation
• Verification in Software Testing is a process of checking documents,
design, code, and program in order to check if the software has been
built according to the requirements or not. The main goal of verification
process is to ensure quality of software application, design,
architecture etc. The verification process involves activities like
reviews, walk-throughs and inspection.
• Validation in Software Engineering is a dynamic mechanism of
testing and validating if the software product actually meets the exact
needs of the customer or not. The process helps to ensure that the
software fulfills the desired use in an appropriate environment. The
validation process involves activities like unit testing, integration
testing, system testing and user acceptance testing.
White Box Testing
• It is a way of testing the external functionality of code by examining and testing the
program code
• Also known as Clear box / Glass box / Open box testing
• It takes into account the following:
• Program code
• Code structure
• Internal design flow
Static Testing
• Requires only the source code of the product and not
executables.
• Does not involve executing programs on computers.
• It involves select people going through the code.
• Checks if code works as per functional requirements.
• Checks for any functionality that is missed out.
• Checks if code handles errors properly.
Static Testing
• Multiple perspectives from different humans during review will
identify problems earlier on
• Human evaluation by comparing code with design is not possible
during testing
• Human evaluation can detect root causes of problems. Other testing
reveals symptoms rather than root causes
• A proactive method that minimizes the delay in identifying problems
implies less cost – defect prevention
Static Testing-Desk Checking
• Done manually by the author
• Aims to verify code for correctness
• Done by comparing code with specs and design
• When errors are found the author corrects them
• There is no formality to ensure correctness and no log is maintained
• Used for correcting obvious errors
• This is not effective in correcting errors due to incorrect
understanding of requirements and inadequate domain knowledge
Advantages of Desk Checking
• Authors knows best his own code
• Being done by individuals, fewer scheduling and logistics
overheads
• Defects are detected and corrected with minimum time delay
Code Walk through
• Less formality
• A set of people look at the program code and raise questions
to the author
• The author explains the logic of the code and answers the
questions
• If the author is unable to answer the questions he/she takes
those questions and finds the answers
• The walkthrough is complete to the extent that questions are
raised
Formal Inspection or Fagan Inspection
• Is a method with high degree of formality
• Increases the number of defects
• Its focus is to detect ALL faults in the following ways:
• Thorough preparation
• Getting multiple diverse views
• Assigning multiple roles to participants
• Going sequentially through code
Formal Inspection
• Done after basic Desk Checking and Walkthroughs
• Inspection roles
• Author
• Moderator – does formal inspection of code
• Inspectors – provide reviews and comments on code
• Scribes – take detailed notes during inspection meeting
• Review team have skills to uncover defects
How is Formal Inspection Done
• Meeting is arranged (moderator decides date, time, venue)
• 4 roles author, moderator, inspectors(review code + provide comments), scribe(for
detailing notes and circulating)
• Author/moderator selects the review team
• Inspectors get hard copies / soft copies of code and other supporting
documents(adequate time to go thru them)
• Moderator takes the team sequentially thro code, asking inspectors if there are any
defects
• Author presents his perspective
• Any defect raised by inspector classified into minor/major (immediate action)and
systemic(requires some changing of standards) / mis-execution(slip on part of author)
• Scribe documents and author fixes (needs to keep a track that all defects are fixed
from ALL meetings, inspectors can give suggestions for fixing)
Formal Inspection
Advantage
• Very effective method
Disadvantages/ challenges
• Time consuming
• Logistics and scheduling issues
• Not possible to inspect every line of code
Structural Testing
• Takes into account code, code structure and internal design
• Difference between structural and static testing
• Structural tests are done by the computer
on built product, whereas in static testing the product is tested
by humans using source code and not the executables.
Structural Testing
• Involves designing and executing test cases and finding the
percentage of code that is covered by testing
• This is done by a technique called instrumentation of code.
The instrumented code can monitor and keep an audit of what
portions of code are covered
Types of Code Coverage
• Statement Coverage
• Decision Coverage
• Condition Coverage
• Multiple Condition coverage
• Path Coverage
• Data Flow Coverage
Cyclomatic Complexity
• Cyclomatic Complexity: It is a measure of the logical
complexity of the software and is used to define the number of
independent paths. For a graph G, V(G) is its cyclomatic
complexity.
Calculating V(G):
• V(G) = P + 1, where P is the number of predicate nodes in the
flow graph
• V(G) = E – N + 2, where E is the number of edges and N is the
total number of nodes
• V(G) = Number of non-overlapping regions in the graph
Black Box Testing
• Done without knowledge of internal working of S/W
• Helps overall checking of functionality and identifies inconsistencies
with requirements
• Focuses on outputs generated in response to selected inputs
• Determines errors in external behaviour of code
• Can be applied to each and every level of software testing such as
unit, integration, system and acceptance testing.
Positive Testing
Negative Testing
 Equivalence Class Partitioning

-- Is the S/W testing technique that represents a small set of

representative values that produce as many output conditions

-- This reduces the number of permutations and combinations of input,

output values used for testing

-- The set of input values that generate one output is called partition or
equivalence class
 Advantages

Advantage
Good coverage with a small number of tests
 Example

Take a ticketing system where children under age 6 are allowed to travel for free, people under 18 as well as
senior people older than 64 pay $10 while adults need to pay $20. For testing this system, we can use
equivalence class partitioning to design our tests.
 Equivalence Class Partitions

0 to 5

6 to 17

18 to 64

<0

> 64
 Boundry Value Analysis

-- When the equivalence classes involve numbers, there are decision

boundaries between them.

-- These are places where the behavior of the system changes.

Values on the boundaries cause errors in system.
 Example-Boundary Value Analysis

Perform boundary value analysis in partition 0-6

 Decision Table

-- Decision table technique is one of the widely used case design techniques for black box testing.

-- This is a systematic approach where various input combinations and their respective system
behavior are captured in a tabular form.

-- That's why it is also known as a cause-effect table. This technique is used to pick the test cases
in a systematic manner; it saves the testing time and gives good coverage to the testing area of the
software application.

-- Decision table technique is appropriate for the functions that have a logical relationship between
two and more than two inputs.

-- This technique is related to the correct combination of inputs and determines the result of various
combinations of input. To design the test cases by decision table technique, we need to consider
conditions as input and actions as output.
 Example
-- Most of us use an email account, and when you want to use an email account, for this you need
to enter the email and its associated password.

-- If both email and password are correctly matched, the user will be directed to the email
account's homepage; otherwise, it will come back to the login page with an error message
specified with "Incorrect Email" or "Incorrect Password.“
 State Based

-- Used when application can be characterised as a set of states

-- The data values that cause state transition is well understood

-- The method of processing within each state is well understood

-- Useful in work flow modelling where depending on the current state and appropriate combinations
of input variables, specific workflows are carried out, resulting in a new output and a new state
 State Graph to Represent Workflow

-- Each state is an event

-- Arrows between between states represent data inputs

-- Start with start state and follow graph through various states until a final state is
reached
 Integration Testing

-- Integration is defined as the set of interactions among components

-- Testing the interaction between modules and interactions with other systems externally is called
integration testing

-- It starts when 2 of the product components become available

-- It ends when all component interfaces have been tested

-- The architecture and design give details of interactons within systems

Integration Testing
 Top Down Integration

-- Involves testing the topmost component interface with other components navigating from
top to bottom covering all components

-- Assumes top component ie. component 1 provides all the interface requirements of other
components even while other components are getting

-- If a component at higher level needs modification every time a module is added to the
bottom, then for each component added, integration testing needs to be repeated starting
from first step
 Bottom Up Integration

-- Opposite of top-down integration

-- Components become available in reverse order starting from the bottom

-- Navigation starts from component 1 covering all subsystems until

component 8 is reached
 Bidirectional Integration

-- Is an approach to combine top down testing with bottom up testing. Also called
Sandwich testing.

-- Individual components 1,2,3,4 and 5 are tested separately

-- Once components 6,7 and 8 become available integration testing focuses on the
new components.
Steps 1, 2 and 3 use bottom up integration
Steps 4,5 and 6 uses top down integration
 System Testing

-- ‘System testing’ testing also termed as ‘Big Bang Approach’ is done before releasing software to
the customer

-- System testing done after all components are integrated and before Acceptance testing

-- Big bang approach has main focus on black box testing of software

-- It ensure that the requirements as defined and documented in Requirement specifications and
Design specifications are met successfully –functional requirements

-- It ensures quality factors such as reliability, scalability,etc. – non-functional requirements

 System Testing

-- Testing done at the end of SDLC may show the defects pertaining to any phase of development such as
Requirements, Design, and Coding.

-- Done to find product level defects and build confidence in product before release

-- Since it is the last phase of testing, not all defects are fixed due to lack of time and effort

-- Done by independent team to get a different perspective

 System Testing

Disadvantages
When a failure is detected, it is very difficult to locate the problem i.e. to find out
in which interface the defect exists. It may require testing the interfaces again
 Functional Testing

--Involves testing a product’s functionality

-- Helps in validating what the system is doing

-- Focuses on real life customer usage

-- Simulates customer deployments

-- Requires in-depth customer and product knowledge and domain knowledge to develop test
cases
 Non Functional Testing

Performed to verify the quality factors

-- Scalability Testing

-- Reliability Testing

--Stress Testing

-- Interoperability Testing
 Regression Testing

-- Regression testing is full or partial selection of already executed test cases which
are re-executed to ensure existing functionalities work fine

-- This testing is done to make sure that new code changes should not have side
effects on the existing functionalities.

-- It ensures that old code still works once the new code changes are done.
 Purpose

-- Software undergoes changes, these can be due to

Defects that are fixed
Enhancements to existing functionality

-- When such changes are made it is important to ensure that

Changes or additions DO NOT break something that is already working
 Performance Testing

-- Performance testing is a non-functional testing technique performed to determine the system

parameters in terms of responsiveness and stability under various workload conditions.

-- Performance testing measures the quality attributes of the system, such as scalability, reliability
and resource usage.
 Factors that govern Performance Testing

-- Throughput
Capability of a product to handle multiple transactions in a given period

-- Response Time
Delay between the time of request and the first response form the product

-- Latency
In reality the response time depends on the product and environment
Latency is the delay caused by the application, OS and by the environment
 References and Credits

Jalote Pankaj, An Integrated Approach to Software Engineering, Narosa Publishing House, Third Edition
THANK YOU