Unit-IV

Software Testing
Software Testing is a method to check whether the actual software product matches expected
requirements and to ensure that software product is Defect free. It involves execution of
software/system components using manual or automated tools to evaluate one or more properties
of interest. The purpose of software testing is to identify errors, gaps or missing requirements in
contrast to actual requirements.

Some prefer saying Software testing definition as a White Box and Black Box Testing. In simple
terms, Software Testing means the Verification of Application Under Test (AUT). This Software
Testing course introduces testing software to the audience and justifies the importance of
software testing.

Software Testing is Important because if there are any bugs or errors in the software, it can be
identified early and can be solved before delivery of the software product. Properly tested
software product ensures reliability, security and high performance which further results in time
saving, cost effectiveness and customer satisfaction.

Benefits of Software Testing

 Cost-Effective: It is one of the important advantages of software testing. Testing any IT
project on time helps you to save your money for the long term. In case if the bugs caught
in the earlier stage of software testing, it costs less to fix.
 Security: It is the most vulnerable and sensitive benefit of software testing. People are
looking for trusted products. It helps in removing risks and problems earlier.
 Product quality: It is an essential requirement of any software product. Testing ensures a
quality product is delivered to customers.
 Customer Satisfaction: The main aim of any product is to give satisfaction to their
customers. UI/UX Testing ensures the best user experience.

Following are the characteristic that process the testing templates:

 The developer should conduct the successful technical reviews to perform the testing successful.
 Testing starts with the component level and work from outside toward the integration of the
whole computer based system.
 Different testing techniques are suitable at different point in time.
 Testing is organized by the developer of the software and by an independent test group.
 Debugging and testing are different activities, then also the debugging should be accommodated
in any strategy of testing.

1
Difference between Verification and Validation
Verification Validation
Verification is the process to find whether the The validation process is checked whether the
software meets the specified requirements for software meets requirements and expectation of
particular phase. the customer.
It estimates an intermediate product. It estimates the final product.
The objectives of verification is to check whether The objectives of the validation is to check
software is constructed according to requirement and whether the specifications are correct and satisfy
design specification. the business need.
It describes whether the outputs are as per the inputs It explains whether they are accepted by the user
or not. or not.
Verification is done before the validation. It is done after the verification.
Plans, requirement, specification, code are evaluated Actual product or software is tested under
during the verifications. validation.
It manually checks the files and document. It is a computer software or developed program
based checking of files and document.

Strategy of testing
A strategy of software testing is shown in the context of spiral.
Following figure shows the testing strategy:

Unit testing
Unit testing starts at the centre and each unit is implemented in source code.

2
Integration testing
An integration testing focuses on the construction and design of the software.

Validation testing
Check all the requirements like functional, behavioral and performance requirement are validate
against the construction software.

System testing
System testing confirms all system elements and performance are tested entirely.
Testing strategy for procedural point of view
As per the procedural point of view the testing includes following steps.
1) Unit testing
2) Integration testing
3) High-order tests
4) Validation testing

These steps are shown in following figure:

Unit Testing
Unit testing is a type of software testing where individual units or components of a software are
tested. The purpose is to validate that each unit of the software code performs as expected. Unit
Testing is done during the development (coding phase) of an application by the developers. Unit
Tests isolate a section of code and verify its correctness. A unit may be an individual function,
method, procedure, module, or object.
In SDLC, STLC, V Model, Unit testing is first level of testing done before integration testing.
Unit testing is a WhiteBox testing technique that is usually performed by the developer. Though,
in a practical world due to time crunch or reluctance of developers to tests, QA engineers also do
unit testing.

Unit Testing is important because software developers sometimes try saving time doing minimal
unit testing and this is myth because inappropriate unit testing leads to high cost Defect fixing
during System Testing, Integration Testing and even Beta Testing after application is built. If
proper unit testing is done in early development, then it saves time and money in the end.

3
Unit Testing LifeCyle:

Unit Testing Techniques:

 Black Box Testing - Using which the user interface, input and output are tested.
 White Box Testing - used to test each one of those functions behaviour is tested.
 Gray Box Testing - Used to execute tests, risks and assessment methods.

Unit Testing - Advantages:

 Reduces Defects in the Newly developed features or reduces bugs when changing the
existing functionality.
 Reduces Cost of Testing as defects are captured in very early phase.
 Improves design and allows better refactoring of code.
 Unit Tests, when integrated with build gives the quality of the build as well.

Integration testing
Integration testing is the second level of the software testing process comes after unit testing. In
this testing, units or individual components of the software are tested in a group. The focus of the
integration testing level is to expose defects at the time of interaction between integrated
components or units.

Unit testing uses modules for testing purpose, and these modules are combined and tested in
integration testing. The Software is developed with a number of software modules that are coded
by different coders or programmers. The goal of integration testing is to check the correctness of
communication among all the modules.

4
Once all the components or modules are working independently, then we need to check the data
flow between the dependent modules is known as integration testing.
Let us see one sample example of a banking application, as we can see in the below image of
amount transfer.

o First, we will login as a user P to amount transfer and send Rs200 amount, the
confirmation message should be displayed on the screen as amount transfer
successfully. Now logout as P and login as user Q and go to amount balance page and
check for a balance in that account = Present balance + Received Balance. Therefore, the
integration test is successful.
o Also, we check if the amount of balance has reduced by Rs200 in P user account.
o Click on the transaction, in P and Q, the message should be displayed regarding the data
and time of the amount transfer.

Guidelines for Integration Testing

o We go for the integration testing only after the functional testing is completed on each
module of the application.

5
 o We always do integration testing by picking module by module so that a proper sequence
is followed, and also we don't miss out on any integration scenarios.
o First, determine the test case strategy through which executable test cases can be prepared
according to test data.
o Examine the structure and architecture of the application and identify the crucial modules
to test them first and also identify all possible scenarios.
o Design test cases to verify each interface in detail.
o Choose input data for test case execution. Input data plays a significant role in testing.
o If we find any bugs then communicate the bug reports to developers and fix defects and
retest.
o Perform positive and negative integration testing.

Reason Behind Integration Testing

Although all modules of software application already tested in unit testing, errors still exist due
to the following reasons:
1. Each module is designed by individual software developer whose programming logic
may differ from developers of other modules so; integration testing becomes essential to
determine the working of software modules.
2. To check the interaction of software modules with the database whether it is an erroneous
or not.
3. Requirements can be changed or enhanced at the time of module development. These
new requirements may not be tested at the level of unit testing hence integration testing
becomes mandatory.
4. Incompatibility between modules of software could create errors.
5. To test hardware's compatibility with software.
6. If exception handling is inadequate between modules, it can create bugs.

Integration Testing Techniques

Any testing technique (Blackbox, Whitebox, and Greybox) can be used for Integration Testing;
some are listed below:
Black Box Testing
o State Transition technique
o Decision Table Technique
o Boundary Value Analysis
o All-pairs Testing
o Cause and Effect Graph
o Equivalence Partitioning
o Error Guessing

6
White Box Testing
o Data flow testing
o Control Flow Testing
o Branch Coverage Testing
o Decision Coverage Testing

Types of Integration Testing

Integration testing can be classified into two parts:
o Incremental integration testing
o Non-incremental integration testing

Incremental Approach
In the Incremental Approach, modules are added in ascending order one by one or according to
need. The selected modules must be logically related. Generally, two or more than two modules
are added and tested to determine the correctness of functions. The process continues until the
successful testing of all the modules.
OR
In this type of testing, there is a strong relationship between the dependent modules. Suppose we
take two or more modules and verify that the data flow between them is working fine. If it is,
then add more modules and test again.

7
For example: Suppose we have a Flipkart application, we will perform incremental integration
testing, and the flow of the application would like this:
Flipkart→ Login→ Home → Search→ Add cart→Payment → Logout

Incremental integration testing is carried out by further methods:

o Top-Down approach
o Bottom-Up approach

Top-Down Approach
The top-down testing strategy deals with the process in which higher level modules are tested
with lower level modules until the successful completion of testing of all the modules. Major
design flaws can be detected and fixed early because critical modules tested first. In this type of
method, we will add the modules incrementally or one by one and check the data flow in the
same order.

8
In the top-down approach, we will be ensuring that the module we are adding is the child of the
previous one like Child C is a child of Child B and so on as we can see in the below image:

Advantages:
o Identification of defect is difficult.
o An early prototype is possible.
Disadvantages:
o Due to the high number of stubs, it gets quite complicated.
o Lower level modules are tested inadequately.
o Critical Modules are tested first so that fewer chances of defects.

Bottom-Up Method
The bottom to up testing strategy deals with the process in which lower level modules are tested
with higher level modules until the successful completion of testing of all the modules. Top level
critical modules are tested at last, so it may cause a defect. Or we can say that we will be adding
the modules from bottom to the top and check the data flow in the same order.

In the bottom-up method, we will ensure that the modules we are adding are the parent of the
previous one as we can see in the below image:

9
Advantages
o Identification of defect is easy.
o Do not need to wait for the development of all the modules as it saves time.
Disadvantages
o Critical modules are tested last due to which the defects can occur.
o There is no possibility of an early prototype.

In this, we have one addition approach which is known as hybrid testing.

Hybrid Testing Method

In this approach, both Top-Down and Bottom-Up approaches are combined for testing. In this
process, top-level modules are tested with lower level modules and lower level modules tested
with high-level modules simultaneously. There is less possibility of occurrence of defect because
each module interface is tested.

Advantages
o The hybrid method provides features of both Bottom Up and Top Down methods.
o It is most time reducing method.
o It provides complete testing of all modules.
Disadvantages
o This method needs a higher level of concentration as the process carried out in both
directions simultaneously.
o Complicated method.

10
Non- incremental integration testing
We will go for this method, when the data flow is very complex and when it is difficult to find
who is a parent and who is a child. And in such case, we will create the data in any module bang
on all other existing modules and check if the data is present. Hence, it is also known as the Big
bang method.

Big Bang Method

In this approach, testing is done via integration of all modules at once. It is convenient for small
software systems, if used for large software systems identification of defects is difficult.
Since this testing can be done after completion of all modules due to that testing team has less
time for execution of this process so that internally linked interfaces and high-risk critical
modules can be missed easily.

Advantages:
o It is convenient for small size software systems.
Disadvantages:
o Identification of defects is difficult because finding the error where it came from is a
problem, and we don't know the source of the bug.
o Small modules missed easily.
o Time provided for testing is very less.
o We may miss to test some of the interfaces.

11
System Testing
System Testing includes testing of a fully integrated software system. Generally, a computer
system is made with the integration of software (any software is only a single element of a
computer system). The software is developed in units and then interfaced with other software
and hardware to create a complete computer system. In other words, a computer system consists
of a group of software to perform the various tasks, but only software cannot perform the task;
for that software must be interfaced with compatible hardware. System testing is a series of
different type of tests with the purpose to exercise and examine the full working of an integrated
software computer system against requirements.

To check the end-to-end flow of an application or the software as a user is known as System
testing. In this, we navigate (go through) all the necessary modules of an application and check
if the end features or the end business works fine, and test the product as a whole system.
It is end-to-end testing where the testing environment is similar to the production environment.

There are four levels of software testing: unit testing, integration testing, system testing
and acceptance testing, all are used for the testing purpose. Unit Testing used to test a single
software; Integration Testing used to test a group of units of software, System Testing used to
test a whole system and Acceptance Testing used to test the acceptability of business
requirements. Here we are discussing system testing which is the third level of testing levels.

Hierarchy of Testing Levels

There are mainly two widely used methods for software testing, one is White box testing which
uses internal coding to design test cases and another is black box testing which uses GUI or user
perspective to develop test cases.

12
 o White box testing
o Black box testing

System testing falls under Black box testing as it includes testing of the external working of
the software. Testing follows user's perspective to identify minor defects.
System Testing includes the following steps.
o Verification of input functions of the application to test whether it is producing the
expected output or not.
o Testing of integrated software by including external peripherals to check the interaction
of various components with each other.
o Testing of the whole system for End to End testing.
o Behavior testing of the application via auser's experience

Example of System testing

Suppose we open an application, let say www.rediff.com, and there we can see that an
advertisement is displayed on the top of the homepage, and it remains there for a few seconds
before it disappears. These types of Ads are done by the Advertisement Management System
(AMS). Now, we will perform system testing for this type of field.
The below application works in the following manner:
o Let's say that Amazon wants to display a promotion ad on January 26 at precisely 10:00
AM on the Rediff's home page for the country India.
o Then, the sales manager logs into the website and creates a request for an advertisement
dated for the above day.
o He/she attaches a file that likely an image files or the video file of the AD and applies.
o The next day, the AMS manager of Rediffmail login into the application and verifies the
awaiting Ad request.
o The AMS manager will check those Amazons ad requests are pending, and then he/she
will check if the space is available for the particular date and time.
o If space is there, then he/she evaluate the cost of putting up the Ad at 15$ per second, and
the overall Ad cost for 10 seconds is approximate 150$.
o The AMS manager clicks on the payment request and sends the estimated value along
with the request for payment to the Amazon manager.
o Then the amazon manager login into the Ad status and confirms the payment request, and
he/she makes the payment as per all the details and clicks on the Submit and Pay
o As soon as Rediff's AMs manager gets the amount, he/she will set up the Advertisement
for the specific date and time on the Rediffmail's home page.

13
Types of System Testing
System testing is divided into more than 50 types, but software testing companies typically uses
some of them. These are listed below:

Differences between Black Box Testing vs White Box Testing:

Black Box Testing White Box Testing

It is a way of testing the software in which the

It is a way of software testing in which the tester has knowledge about the internal
internal structure or the program or the code structure or the code or the program of the
is hidden and nothing is known about it. software.

Implementation of code is not needed for Code implementation is necessary for white
black box testing. box testing.

It is mostly done by software testers. It is mostly done by software developers.

No knowledge of implementation is needed. Knowledge of implementation is required.

It can be referred as outer or external software

testing. It is the inner or the internal software testing.

14
Black Box Testing White Box Testing

It is functional test of the software. It is structural test of the software.

This testing can be initiated on the basis of This type of testing of software is started after
requirement specifications document. detail design document.

It is mandatory to have knowledge of

No knowledge of programming is required. programming.

It is the behavior testing of the software. It is the logic testing of the software.

It is applicable to the higher levels of testing It is generally applicable to the lower levels of
of software. software testing.

It is also called closed testing. It is also called as clear box testing.

It is least time consuming. It is most time consuming.

It is not suitable or preferred for algorithm

testing. It is suitable for algorithm testing.

Can be done by trial and error ways and Data domains along with inner or internal
methods. boundaries can be better tested.

Example: search something on google by

using keywords Example: by input to check and verify loops

Types of Black Box Testing: Types of White Box Testing:

A. Functional Testing A. Path Testing
B. Non-functional testing B. Loop Testing
C. Regression Testing C. Condition testing

Object Oriented Testing in Software Testing

Software typically undergoes many levels of testing, from unit testing to system or acceptance
testing. Typically, in-unit testing, small “units”, or modules of the software, are tested
separately with focus on testing the code of that module. In higher, order testing
(e.g, acceptance testing), the entire system (or a subsystem) is tested with the focus on testing
the functionality or external behavior of the system.

15
As information systems are becoming more complex, the object-oriented paradigm is gaining
popularity because of its benefits in analysis, design, and coding. Conventional testing methods
cannot be applied for testing classes because of problems involved in testing classes, abstract
classes, inheritance, dynamic binding, message, passing, polymorphism, concurrency, etc.
Testing classes is a fundamentally different problem than testing functions. A function (or a
procedure) has a clearly defined input-output behavior, while a class does not have an input-
output behavior specification. We can test a method of a class using approaches for testing
functions, but we cannot test the class using these approaches.

According to Davis the dependencies occurring in conventional systems are:

 Data dependencies between variables
 Calling dependencies between modules
 Functional dependencies between a module and the variable it computes
 Definitional dependencies between a variable and its types.
But in Object-Oriented systems there are following additional dependencies:
 Class to class dependencies
 Class to method dependencies
 Class to message dependencies
 Class to variable dependencies
 Method to variable dependencies
 Method to message dependencies
 Method to method dependencies

Issues in Testing Classes:

Additional testing techniques are, therefore, required to test these dependencies. Another issue
of interest is that it is not possible to test the class dynamically, only its instances i.e, objects
can be tested. Similarly, the concept of inheritance opens various issues e.g., if changes are
made to a parent class or superclass, in a larger system of a class it will be difficult to test
subclasses individually and isolate the error to one class.
In object-oriented programs, control flow is characterized by message passing among objects,
and the control flow switches from one object to another by inter-object communication.
Consequently, there is no control flow within a class like functions. This lack of sequential
control flow within a class requires different approaches for testing. Furthermore, in a function,
arguments passed to the function with global data determine the path of execution within the
procedure. But, in an object, the state associated with the object also influences the path of
execution, and methods of a class can communicate among themselves through this state
because this state is persistent across invocations of methods. Hence, for testing objects, the
state of an object has to play an important role.

Techniques of object-oriented testing are as follows:

1. Fault Based Testing:

This type of checking permits for coming up with test cases supported the consumer
specification or the code or both. It tries to identify possible faults (areas of design or code
that may lead to errors.). For all of these faults, a test case is developed to “flush” the errors
out. These tests also force each time of code to be executed.

16
 This method of testing does not find all types of errors. However, incorrect specification
and interface errors can be missed. These types of errors can be uncovered by function
testing in the traditional testing model. In the object-oriented model, interaction errors can
be uncovered by scenario-based testing. This form of Object oriented-testing can only test
against the client’s specifications, so interface errors are still missed.

2. Class Testing Based on Method Testing:

This approach is the simplest approach to test classes. Each method of the class performs a
well defined cohesive function and can, therefore, be related to unit testing of the traditional
testing techniques. Therefore all the methods of a class can be involved at least once to test
the class.

3. Random Testing:
It is supported by developing a random test sequence that tries the minimum variety of
operations typical to the behavior of the categories

4. Partition Testing:
This methodology categorizes the inputs and outputs of a category so as to check them
severely. This minimizes the number of cases that have to be designed.

5. Scenario-based Testing:
It primarily involves capturing the user actions then stimulating them to similar actions
throughout the test.
These tests tend to search out interaction form of error.

Alpha Testing
Alpha Testing is a type of acceptance testing; performed to identify all possible issues and bugs
before releasing the final product to the end users. Alpha testing is carried out by the testers who
are internal employees of the organization. The main goal is to identify the tasks that a typical
user might perform and test them.
To put it as simple as possible, this kind of testing is called alpha only because it is done early on,
near the end of the development of the software, and before beta testing. The main focus of alpha
testing is to simulate real users by using a black box and white box techniques.

Beta Testing
Beta Testing is performed by “real users” of the software application in “real environment” and
it can be considered as a form of external User Acceptance Testing. It is the final test before
shipping a product to the customers. Direct feedback from customers is a major advantage of
Beta Testing. This testing helps to test products in customer’s environment.
Beta version of the software is released to a limited number of end-users of the product to obtain
feedback on the product quality. Beta testing reduces product failure risks and provides increased
quality of the product through customer validation.

Alpha Testing Vs Beta testing:

Following are the differences between Alpha and Beta Testing:

17
Sr. Alpha Testing Beta Testing
No.

1. Alpha testing performed by a team of Beta testing performed by clients or end-

highly skilled testers who are usually users in a real-time environment, who is not
the internal employee of the an employee of the organization.
organization.

2. Alpha testing performed at the Beta testing doesn't need any lab environment
developer's site; it always needs a or the testing environment; it is performed at
testing environment or lab a client's location or end-user of the product.
environment.

3. Reliability or security testing not Reliability, security, and robustness checked

performed in-depth in alpha testing. during beta testing.

4. Alpha testing involves both white box Beta testing uses only black-box testing.
and black-box techniques.

5. Long execution cycles maybe require Only a few weeks are required for the
for alpha testing. execution of beta testing.

6. Critical issues or fixes can be Most of the issues or feedback is collecting

identified by developers immediately from the beta testing will be implemented for
in alpha testing. the future versions of the product.

7. Alpha testing performed before the At the time of software product marketing.
launch of the product into the market.

8. Alpha testing focuses on the product's Beta testing concentrates on the quality of the
quality before going to beta testing. product, but gathers users input on the
product and ensures that the product is ready
for real-time users.

9. Alpha testing performed nearly the Beta testing is a final test before shipping a
end of the software development. product to the customers.

10. Alpha testing is conducting in the Beta testing reversed of alpha testing.
presence of developers and the
absence of end-users.

Types of Beta Testing

There are different types of Beta tests in software testing, and they are as follows:
Traditional Beta testing: Product is distributed to the target market, and related data is gathered
in all aspects. This data can be used for Product improvement.
Public Beta Testing: Product is publicly released to the outside world via online channels and
data can be gathered from anyone. Based on feedback, product improvements can be done. For
18
example, Microsoft conducted the largest of all Beta Tests for its OS — Windows 8 before
officially releasing it.
Technical Beta Testing: Product is released to the internal group of an organization and gathers
feedback/data from the employees of the organization.
Focused Beta: Product is released to the market for gathering feedback on specific features of
the program. For example, important functionality of the software.
Post-release Beta: Product is released to the market and data is gathered to make improvements
for the future release of the product.

Phases of Testing
Alpha and Beta tests are typically carried for “off-the-shelf” software or product-oriented
companies. The Phases of Testing for a product company typically varies from a service-oriented
organization. Following is the testing phase adopted by product firms
Pre-Alpha:- Software is a prototype. UI is complete. But not all features are completed. At this
stage, software is not published.
Alpha: Software is near its development and is internally tested for bugs/issues
Beta: Software is stable and is released to a limited user base. The goal is to get customer
feedback on the product and make changes in software accordingly
Release Candidate (RC): Based on the feedback of Beta Test, you make changes to the
software and want to test out the bug fixes. At this stage, you do not want to make radical
changes in functionality but just check for bugs. RC is also put out to the public
Release: All works, software is released to the public.
Note: Above is a standard definition of the Testing stages but in order to garner marketing buzz,
companies combine stages like “pre-alpha beta”, “pre-beta” etc.

Entry Criteria for Alpha testing:

 Software requirements document or Business requirements specification
 Test Cases for all the requirements
 Testing Team with good knowledge about the software application
 Test Lab environment setup
 QA Build ready for execution
 Test Management tool for uploading test cases and logging defects
 Traceability Matrix to ensure that each design requirement has alteast one Test Case that
verifies it

Exit Criteria for Alpha testing

 All the test cases have been executed and passed.
 All severity issues need to be fixed and closed
 Delivery of Test summary report
 Make sure that no more additional features can be included
 Sign off on Alpha testing

Entrance Criteria for Beta Testing:

 Sign off a document on Alpha testing
 Beta version of the software should be ready
 Environment ready to release the software application to the public
 Tool to capture real time faults

19
Exit Criteria for Beta Testing:
 All major and minor issues are closed
 Feedback report should be prepared from public
 Delivery of Beta test summary report

Advantages of Alpha Testing:

 Provides better view about the reliability of the software at an early stage
 Helps simulate real time user behavior and environment.
 Detect many showstopper or serious errors
 Ability to provide early detection of errors with respect to design and functionality

Advantages of Beta Testing

 Reduces product failure risk via customer validation.
 Beta Testing allows a company to test post-launch infrastructure.
 Improves product quality via customer feedback
 Cost effective compared to similar data gathering methods
 Creates goodwill with customers and increases customer satisfaction

Disadvantages of Alpha Testing:

 In depth, functionality cannot be tested as software is still under development stage
Sometimes developers and testers are dissatisfied with the results of alpha testing

Disadvantages of Beta Testing

 Test Management is an issue. As compared to other testing types which are usually
executed inside a company in a controlled environment, beta testing is executed out in the
real world where you seldom have control.
 Finding the right beta users and maintaining their participation could be a challenge

20