Software

Testing and
Quality
Assurance
Software Testing
•The aim of program testing is to help realise identify all defects in a program. However, in
practice, even after satisfactory completion of the testing phase, it is not possible to guarantee that
a program is error free. This is because the input data domain of most programs is very large, and
it is not practical to test the program exhaustively with respect to each value that the input can
assume

•The engineer creates a series of test cases that are intended to "demolish" the software that has
been built. In fact, testing is the one step in the software process that could be viewed.
• Objectives:

1.Testing is a process of executing a program with the intent of finding an error.

2. A good test case is one that has a high probability of finding an as-yet undiscovered error.

3. A successful test is one that uncovers an as-yet-undiscovered error.

Testing Life-cycle
•Test suite design: The set of test cases using which a program is to be tested is designed possibly
using several test case design techniques.
•Running test cases and checking the results to detect failures: Each test case is run and the results
are compared with the expected results. A mismatch between the actual result and expected
results indicates a failure. The test cases for which the system fails are noted down for later
debugging.
• Locate error: In this activity, the failure symptoms are analysed to locate the errors. For each
failure observed during the previous activity, the statements that are in error are identified.
•Error correction: After the error is located during debugging, the code is appropriately changed to
correct the error.
Testing Strategies
VERIFICATION VALIDATION

•Verification is the process of determining •Validation is the process of determining

whether the output of one phase of software whether a fully developed software
development conforms to that of its conforms to its requirements specification.
previous phase.
•Validation is applied to the fully developed
•The objective of verification is to check if and integrated software to check if it
the work products produced after a phase satisfies the customer’s requirements.
conform to that which was input to the
phase
•The primary techniques used for verification •Validation techniques are primarily based
include review, simulation, formal on product testing.
verification, and testing.
•Validation requires execution of the
•Verification does not require execution of software.
the software
• Validation is carried out to check if the
•Verification is carried out during the right as required by the customer has been
development process to check if the developed.
development activities are proceeding
alright
 Defect Management :

A defect in a hardware device or component; for example, a short circuit or broken wire. (b) An
incorrect step, process, or data definition in a computer program

a quality problem that is discovered after the software has been released to end-users (or to another
activity in the software process).

The primary objective of formal technical reviews is to find errors during the process so that they
do not become defects after release of the software. The obvious benefit of formal technical
reviews is the early discovery of errors so that they do not propagate to the next step in the
software process
A defect amplification model can be used to illustrate the generation and detection of errors
during the preliminary design, detail design, and coding steps of the software engineering
process.

A box represents a software development step.

Review may fail to uncover newly generated errors and errors from previous steps, resulting in
some number of errors that are passed through.

In some cases, errors passed through from previous steps are amplified (amplification factor, x)
by current work.

The box subdivisions represent each of these characteristics and the percent of efficiency for
detecting errors, a function of the thoroughness of the review
After a failure has been detected, it is necessary to first identify the program statement(s) that are
in error and are responsible for the failure, the error can then be fixed.

Debugging is often carried out by programmers based on their ingenuity and experience.

The following are some general guidelines for effective debugging:

• Many times debugging requires a thorough understanding of the program design. Trying to
debug based on a partial understanding of the program design may require an inordinate amount
of effort to be put into debugging even for simple problems.

• Debugging may sometimes even require full redesign of the system. In such cases, a common
mistakes that novice programmers often make is attempting not to fix the error but its symptoms.
•One must be beware of the possibility that an error correction may introduce new errors.
Therefore after every round of error-fixing, regression testing (see Section 10.13) must be carried
out.
Black box and White Box Testing
White Box :

• White-box testing, sometimes called glass-box testing

•It is a test case design method that uses the control structure of the procedural design to derive
test cases.

• These control structures are described as component level design to derive test cases .
• Characteristics :

1) Guarantee that all independent paths within a module have been exercised at least once

2) Exercise all logical decisions on their true and false sides

3) Execute all loops at their boundaries and within their operational bounds, and

4) Exercise internal data structures to ensure their validity.

• Methods of white box testing :

1 ) Basis Path Testing

The basis path method enables the test case designer to derive a logical complexity measure of a
procedural design and use this measure as a guide for defining a basis set of execution paths. Test
cases derived to exercise the basis set are guaranteed to execute every statement in the program at
least one time during testing
A ) Flow Graph notation

It depicts logical control flow . To show the program control structures , flowchart are used.
Mapping of flowchart to flow graph is used .
Flow chart
Flow Graph
B) Independent program paths

An independent path is any path through the program that introduces at least one new set of
processing statements or a new condition.

When stated in terms of a flow graph, an independent path must move along at least one edge that
has not been traversed before the path is defined
Consider previous example :

path 1: 1-11

path 2: 1-2-3-4-5-10-1-11

path 3: 1-2-3-6-8-9-10-1-11

path 4: 1-2-3-6-7-9-10-1-11
Cyclomatic complexity is a software metric that provides a quantitative measure of the logical
complexity of a program.
When used in the context of the basis path testing method, the value computed for cyclomatic
complexity defines the number of independent paths in the basis set of a program and provides us
with an upper bound for the number of tests that must be conducted to ensure that all statements
have been executed at least once.
It is calculated in 3 ways :
V(G) = E- N + 2
V(G) = P + 1
How many regions are related ?
C) Graph Matrices

A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the
number of nodes on the flow graph. Each row and column corresponds to an identified node, and
matrix entries correspond to connections (an edge) between nodes
2) Control structure Testing

Although basis path testing is simple and highly effective, it is not sufficient in itself. In this
section, other variations on control structure testing are discussed. These broaden testing
coverage and improve quality of white-box testing.

A) Condition testing

Condition testing is a test case design method that exercises the logical conditions contained in a
program module. A simple condition is a Boolean variable or a relational expression,
Ex:

E1< relational operator> E2

Relational operators can be :

< - less than , ≤ - less than equal to , , ≠ (nonequality), > - Greater than, ≥ - Greater than equal to
B) Data flow testing :

The data flow testing method selects test paths of a program according to the locations of
definitions and uses of variables in the program

To illustrate the data flow testing approach, assume that each statement in a program is assigned a
unique statement number and that each function does not modify its parameters or global
variables. For a statement with S as its statement number,

DEF(S) = {X | statement S contains a definition of X}

USE(S) = {X | statement S contains a use of X}

If statement S is an if or loop statement, its DEF set is empty and its USE set is based on the
condition of statement S.
C) Loop Testing :

Loop testing is a white-box testing technique that focuses exclusively on the validity of loop
constructs. Four different classes of loops can be defined: simple loops, concatenated loops,
nested loops, and unstructured loops
Simple loops.
The following set of tests can be applied to simple loops, where n is the maximum number of
allowable passes through the loop.
1. Skip the loop entirely.
2. Only one pass through the loop.
3. Two passes through the loop.
4. m passes through the loop where m < n.
5. n 1, n, n + 1 passes through the loop.
 Nested loops

1. Start at the innermost loop. Set all other loops to minimum values.

2. Conduct simple loop tests for the innermost loop while holding the outer loops at their
minimum iteration parameter (e.g., loop counter) values. Add other tests for out-of-range or
excluded values.

3. Work outward, conducting tests for the next loop, but keeping all other outer loops at
minimum values and other nested loops to "typical" values.

4. Continue until all loops have been tested.

 Concatenated loops.

Concatenated loops can be tested using the approach defined for simple loops, if each of the loops
is independent of the other. However, if two loops are concatenated and the loop counter for loop
1 is used as the initial value for loop 2, then the loops are not independent. When the loops are not
independent, the approach applied to nested loops is recommended
 Unstructured loops

Unstructured loops. Whenever possible, this class of loops should be redesigned to reflect the use
of the structured programming constructs
D) DEF-USE Testing

•DEF-USE ( definition use ) testing is based on data flow mechanism .It performs testing on
definition and use of variables in program statement .

• The DU pair is needed in this form if testing

DEF(S)={X | statement S contains a definition of X}

USE(S) = {X | statement S contains a use of X}

• Data flow testing strategies are useful for selecting test paths of a program containing nested if
and loop statements.
EX :

DEF ( a, 3) : node 3 is defining node of variable a

USE ( a, 4) : node 4 is usage of variable a

 Black Box testing :

• Black-box testing, also called behavioral testing, focuses on the functional requirements of the
software. That is, black-box testing enables the software engineer to derive sets of input
conditions that will fully exercise all functional requirements for a program.

• Black-box testing is not an alternative to white-box techniques. Rather, it is a complementary

approach that is likely to uncover a different class of errors than white-box methods.

• Black box testing is applied in final stages of testing.

• Black-box testing attempts to find errors in the following categories:

1. incorrect or missing functions

2. interface errors

3. errors in data structures or external data base access

4. behavior or performance errors, and

5. initialization and termination errors

• Methods of black box testing :

A ) Graph based testing –

Software testing begins by creating a graph of important objects and their relationships and then
devising a series of tests that will cover the graph so that each object and relationship is exercised
and errors are uncovered.

A collection of nodes that represent objects; links that represent the relationships between objects;
node weights that describe the properties of a node (e.g., a specific data value or state behavior);
and link weights that describe some characteristic of a link.
Nodes are represented as circles connected by links that take a number of different forms.

A directed link (represented by an arrow) indicates that a relationship moves in only one
direction.

A bidirectional link, also called a symmetric link, implies that the relationship applies in both
directions.

Parallel links are used when a number of different relationships are established between graph
nodes.
EX: A)
EX :B) Word file processing system
Here , example represents :

Object #1 = new file menu select

Object #2 = document window

Object #3 = document text

B) Equivalence Partitioning –

It divides the input domain of a program into classes of data from which test cases can be derived.
An ideal test case single-handedly uncovers a class of errors (e.g., incorrect processing of all
character data) that might otherwise require many cases to be executed before the general error is
observed. Equivalence partitioning strives to define a test case that uncovers classes of errors,
thereby reducing the total number of test cases that must be developed.

Test case design for equivalence partitioning is based on an evaluation of equivalence classes for
an input condition.

An equivalence class represents a set of valid or invalid states for input conditions.
Typically, an input condition is either a specific numeric value, a range of values, a set of related
values, or a Boolean condition.

Equivalence classes may be defined according to the following guidelines:

1. If an input condition specifies a range, one valid and two invalid equivalence classes are
defined.

2. If an input condition requires a specific value, one valid and two invalid equivalence classes
are defined.

3. If an input condition specifies a member of a set, one valid and one invalid equivalence class
are defined.

4. If an input condition is Boolean, one valid and one invalid class are defined
C) Boundary Value Analysis :

•For reasons that are not completely clear, a greater number of errors tends to occur at the
boundaries of the input domain rather than in the "center." It is for this reason that boundary value
analysis (BVA) has been developed as a testing technique.

•Boundary value analysis is a test case design technique that complements equivalence
partitioning. Rather than selecting any element of an equivalence class, BVA leads to the
selection of test cases at the "edges" of the class.

•Rather than focusing solely on input conditions, BVA derives test cases from the output domain
as well.

•Guidelines for BVA are similar in many respects to those provided for equivalence partitioning:
1. If an input condition specifies a range bounded by values a and b, test cases should be designed
with values a and b and just above and just below a and b.

2. If an input condition specifies a number of values, test cases should be developed that exercise
the minimum and maximum numbers. Values just above and below minimum and maximum are
also tested.

3. Apply guidelines 1 and 2 to output conditions. For example, assume that a temperature vs.
pressure table is required as output from an engineering analysis program. Test cases should be
designed to create an output report that produces the maximum (and minimum) allowable number
of table entries.

4. If internal program data structures have prescribed boundaries (e.g., an array has a defined
limit of 100 entries), be certain to design a test case to exercise the data structure at its boundary.
D) Orthogonal Array Testing :

Orthogonal array testing can be applied to problems in which the input domain is relatively small
but too large to accommodate exhaustive testing.

The orthogonal array testing method is particularly useful in finding errors associated with region
faults—an error category associated with faulty logic within a software component.

consider a system that has three input items, X, Y, and Z. Each of these input items has three
discrete values associated with it. There are 3^3 = 27 possible test cases.
White Box and Black Box testing

WHITE BOX BLACK BOX

• Also called as glass box testing . • Also called as behavioral testing.

• It focuses on control structures by •It focuses only on functional requirements

designing test cases. of software .

• It evaluates all logic decisions and find • It evaluates performance errors or

whether they are true or false . behavioral errors .

• It begins early in testing process. • It is applied in final stages of testing .

• Mainly applicable to lower levels of testing • Mainly applicable to higher levels of testing
such as unit and integration testing . such as acceptance testing .

• Programming knowledge is required • Programming knowledge is not required

• Implementation of knowledge is required. •Implementation of knowledge is not

required.
•Done by software developers
• Done by software testers
• It can be started based on detail design
document • It can be started based on SRS document .

• It means structural test or interior test . • It means functional test or external test
Alpha and Beta Testing
ALPHA TESTING BETA TESTING

• Conducted at developer’s end by end users. • Conducted at end users place . Developer is
not present .
• The developer will keep the record of all
errors and problems encountered . • End user reports the problem and thus
developer is able to correct errors .
• Developer may modify the codes at his site
before release . • Developer may modify after getting
feedback from user .
• Test data is simulated . • Test data is live .

• Always performed in virtual environment . • Always performed in real time environment

• Always performed with the organization • Always performed outside the organization

• not known by any other name • Also known as filed testing.

• conducted for software application and • conducted for software product .

project .
Software Quality Assurance
Software Quality :

•Quality of a product is defined in terms of its fitness of purpose.

• That is, a good quality product does exactly what the users want it to do, since for almost every
product, fitness of purpose is interpreted in terms of satisfaction of the requirements laid down in
the SRS document. Although “fitness of purpose” is a satisfactory definition of quality for many
products such as a car, a table fan, a grinding machine, etc.—“fitness of purpose” is not a wholly
satisfactory definition of quality for software products.

• Quality of design refers to the characteristics that designers specify for an item. The grade of
materials, tolerances, and performance specifications all contribute to the quality of design
• Quality of conformance is the degree to which the design specifications are followed during
manufacturing. Again, the greater the degree of conformance, the higher is the level of quality of
conformance
 Quality Assurance and its factors :

• Quality assurance consists of the auditing and reporting functions of management. The goal of
quality assurance is to provide management with the data necessary to be informed about product
quality, thereby gaining insight and confidence that product quality is meeting its goals.

• Software quality assurance is composed of a variety of tasks associated with two different
constituencies—the software engineers who do technical work and an SQA group that has
responsibility for quality assurance planning, oversight, record keeping, analysis, and reporting.
 SQA Activities :

• Prepares an SQA plan for a project.

The plan is developed during project planning and is reviewed by all interested parties. Quality
assurance activities performed by the software engineering team and the SQA group are governed
by the plan.

The plan identifies

1. evaluations to be performed

2. audits and reviews to be performed

3. standards that are applicable to the project

4. procedures for error reporting and tracking

5. documents to be produced by the SQA group

6. amount of feedback provided to the software project team

• Participates in the development of the project’s software process description.

The software team selects a process for the work to be performed. The SQA group reviews the
process description for compliance with organizational policy, internal software standards,
externally imposed standards (e.g., ISO-9001), and other parts of the software project plan.

• Reviews software engineering activities to verify compliance with the defined software process.
The SQA group identifies, documents, and tracks deviations from the process and verifies that
corrections have been made.

• Audits designated software work products to verify compliance with those defined as part of the
software process.

The SQA group reviews selected work products; identifies, documents, and tracks deviations;
verifies that corrections have been made; and periodically reports the results of its work to the
project manager.
•Ensures that deviations in software work and work products are documented and handled
according to a documented procedure.
Deviations may be encountered in the project plan, process description, applicable standards, or
technical work products.
• Records any noncompliance and reports to senior management.
Noncompliance items are tracked until they are resolved. In addition to these activities, the SQA
group coordinates the control and management of change and helps to collect and analyze
software metrics.
Software Quality Factors
• Portability:

A software product is said to be portable, if it can be easily made to work in different

hardware and operating system environments, and easily interface with external hardware devices
and software products.

• Usability:

A software product has good usability, if different categories of users (i.e., both expert and
novice users) can easily invoke the functions of the product.

• Reusability:

A software product has good reusability, if different modules of the product can easily be
reused to develop new products.
• Maintainability:

A software product is maintainable, if errors can be easily corrected as and when they show
up, new functions can be easily added to the product, and the functionalities of the product can be
easily modified, etc.

• Reliability :

That evaluates software for its working according to desired precision .

• Efficiency:

Amount of various resources required to execute a given program

• Integrity:
 Efforts taken to manage access authorization .

•Flexibility :

The case with which a program can be modified and extended in future .

• Testability:

Amount of time and efforts taken to test a program.

•Correctness:
 A software product is correct, if different requirements as specified in the SRS document
have been correctly implemented.

• Interoperability :

Ability to connect one system to other system .

• McCall's Quality Factors :

Factors that effect Software Quality can be categorized as :

1. Factors that can be directly measured( defects uncovered during testing ) – They are
second-level quality attributes called as quality criteria ,

2. Factors that can be indirectly measured – They are higher level attributes.

For example, the reliability cannot be measured directly, but by measuring the number of defects
encountered over a period of time. Thus, reliability is a higher-level quality factor and number of
defects is a low-level quality factor.

Factors that affects SQ focuses on 3 aspects :

1. Its operational characteristics – Product operation

2. Its ability to undergo change – Product revision

3. Its adaptability to new environment – Product transition

Quality Evaluation Standards
• ISO 9000 certification serves as a reference for contract between independent parties. In
particular, a company awarding a development contract can form his opinion about the possible
vendor performance based on whether the vendor has obtained ISO 9000 certification or not.
•In this context, the ISO 9000 standard specifies the guidelines for maintaining a quality system.
We have already seen that the quality system of an organization applies to all its activities related
to its products or services. The ISO standard addresses both operational aspects (that is, the
process) and organizational aspects such as responsibilities, reporting, etc.
• In a nutshell, ISO 9000 specifies a set of recommendations for repeatable and high quality
product development. It is important to realize that ISO 9000 standard is a set of guidelines for
the production process and is not directly concerned about the product it self.
• ISO 9000 describes the elements of a quality assurance system in general terms. These elements
include the organizational structure, procedures, processes, and resources needed to implement
quality planning, quality control, quality assurance, and quality improvement.

•However, ISO 9000 does not describe how an organization should implement these quality
system elements. Consequently, the challenge lies in designing and implementing a quality
assurance system that meets the standard and fits the company’s products, services, and culture.
ISO 9001:

This standard applies to the organizations engaged in design, development, production, and
servicing of goods. This is the standard that is applicable to most software development
organizations.

ISO 9002:

This standard applies to those organizations which do not design products but are only involved
in production. Examples of this category of industries include steel and car manufacturing
industries who buy the product and plant designs from external sources and are involved in only
manufacturing those products. Therefore, ISO 9002 is not applicable to software development
organizations.

ISO 9003:
This standard applies to organizations involved only in installation and testing of products.