SOFTWARE TESTING

UNIT I INTRODUCTION 8
Introduction to software testing – Evolution of Software Testing – Goals of Software Testing - Software
Testing Definitions - Model for Software Testing - Software Testing as a Process - software testing
terminology and methodology – Software Testing Terminology – STLC – Software Testing Methodology –
Verification and Validation
Software Testing
Software Testing is a method to check whether the actual software product matches expected requirements
and to ensure that software product is Defect free. It involves execution of software/system components
using manual or automated tools to evaluate one or more properties of interest. The purpose of software
testing is to identify errors, gaps or missing requirements in contrast to actual requirements.
Some prefer saying Software testing definition as a White Box and Black Box Testing. In simple terms,
Software Testing means the Verification of Application Under Test (AUT). This Software Testing course
introduces testing software to the audience and justifies the importance of software testing.

Why Software Testing is Important?

Software Testing is Important because if there are any bugs or errors in the software, it can be identified
early and can be solved before delivery of the software product. Properly tested software product ensures
reliability, security and high performance which further results in time saving, cost effectiveness and
customer satisfaction.

Evolution of Software Testing

Software programming has evolved over the decades and consequently, testing, which is an integral part of
software development, has also gone through a series of changes.

It’s all started with programming and debugging phase, when finding errors during debugging was
considered testing. In 1957, testing got an individual identity and was treated as a separate activity from the
debugging process. Till the late ’70s, testing was seen as an exercise to ensure that the software works as
per the specified requirements. It was then extended to find the errors, besides ensuring the proper
functioning of the software. In the ’80s, the testing activity was also considered as a measurement of
quality. With this, it gained more importance and was treated as a clearly defined and managed process of
the software development life cycle. By the mid-’90s, the testing process had its own life cycle.

Evolution Cycle
 Testing first ensured that the software works as per the specified end-user requirements (Business
Requirement or Functional Requirement).

 Then Testing was used to find the issues/bugs and also ensuring the proper functioning of the software.

 Software Testing then became a measurement of quality of Software and also got included in the software
development life cycle.

 The testing process then had its own Software Testing life cycle.
 Various Software Testing Models emerged in-line

o Waterfall Model.

o V Model.

o Agile Model.

o Spiral Model.

o Iterative Model

These different models or approaches can be used in the process where each model has its own advantages
and disadvantages. So, you must choose a particular model depending on the project deliverables and
complexity of the project.
 Agile – Growth of the software development process needed more comprehensive ways of testing. The
incremental and iterative approach employed by agile testing helped in achieving this goal.

 Automation – Testing was taken to entirely different level by Automation Testing. Automated testing is a
method in software testing that uses various automation tools to control the execution of tests, so more
time can be saved on repetition tasks and this time can be spent on higher value tasks, such as exploratory
tests. Mainly Automating time-consuming tests, such as regression tests. While you do need spend time
maintaining test scripts overall, you will increase your test coverage and scalability.
 Continuous Testing- Continuous Testing is in the trend now and has become important with the concepts
like Continuous Integrationand Continuous Deployment becoming popular. To meet the continuous
integration and delivery needs, we have turned to continuous testing backed by automation. As this is the
time of digital transformation, we need Digital Testing to meet the quality needs of future driven by AI.
 Artificial Intelligence (AI): This is the future – collaborative and smart testing. Artificial Intelligence is the
machine’s capability of imitating human behaviour by perceiving, understanding and learning. AI can be
used to help automate and reduce the amount of tedious tasks in development and testing. AI also uses
image recognition techniques to navigate through the application and verify UI objects and elements
visually. AI can also be used to sell more and spend less time on Admin tasks.

Goals of Software Testing

The main goal of software testing is to find bugs as early as possible and fix bugs and make sure that the
software is bug-free. The goals of software testing may be classified into three major categories as
follows:
1. Immediate Goals
2. Long-term Goals
3. Post-Implementation Goals

1. Immediate Goals: These objectives are the direct outcomes of testing. These objectives may be set at
any time during the SDLC process. Some of these are covered in detail below:
 Bug Discovery: This is the immediate goal of software testing to find errors at any stage of software
development. The number of bugs is discovered in the early stage of testing. The primary purpose of
software testing is to detect flaws at any step of the development process. The higher the number of
issues detected at an early stage, the higher the software testing success rate.
 Bug Prevention: This is the immediate action of bug discovery, that occurs as a result of bug discovery.
Everyone in the software development team learns how to code from the behavior and analysis of issues
detected, ensuring that bugs are not duplicated in subsequent phases or future projects.
2. Long-Term Goals: These objectives have an impact on product quality in the long run after one cycle
of the SDLC is completed. Some of these are covered in detail below:
 Quality: This goal enhances the quality of the software product. Because software is also a product, the
user’s priority is its quality. Superior quality is ensured by thorough testing. Correctness, integrity,
efficiency, and reliability are all aspects that influence quality. To attain quality, you must achieve all of
the above-mentioned quality characteristics.
 Customer Satisfaction: This goal verifies the customer’s satisfaction with a developed software product.
The primary purpose of software testing, from the user’s standpoint, is customer satisfaction. Testing
should be extensive and thorough if we want the client and customer to be happy with the software
product.
 Reliability: It is a matter of confidence that the software will not fail. In short, reliability means gaining
the confidence of the customers by providing them with a quality product.
 Risk Management: Risk is the probability of occurrence of uncertain events in the organization and the
potential loss that could result in negative consequences. Risk management must be done to reduce the
failure of the product and to manage risk in different situations.
3. Post Implemented Goals: After the product is released, these objectives become critical. Some of
these are covered in detail below:
 Reduce Maintenance Cost: Post-released errors are costlier to fix and difficult to identify. Because
effective software does not wear out, the maintenance cost of any software product is not the same as the
physical cost. The failure of a software product due to faults is the only expense of maintenance. Because
they are difficult to discover, post-release mistakes always cost more to rectify. As a result, if testing is
 done thoroughly and effectively, the risk of failure is lowered, and maintenance costs are reduced as a
result.
 Improved Software Testing Process: These goals improve the testing process for future use or software
projects. These goals are known as post-implementation goals. A project’s testing procedure may not be
completely successful, and there may be room for improvement. As a result, the bug history and post-
implementation results can be evaluated to identify stumbling blocks in the current testing process that
can be avoided in future projects.

How does software testing work?

Software testing is the process of evaluating and verifying that a software product or application does what
it is supposed to do. The benefits of testing include preventing bugs, reducing development costs and
improving performance.
Test management plan
Types of software testing

There are many different types of software tests, each with specific objectives and strategies:

 Acceptance testing: Verifying whether the whole system works as intended.

 Integration testing: Ensuring that software components or functions operate together.
 Unit testing: Validating that each software unit performs as expected. A unit is the smallest testable
component of an application.
 Functional testing: Checking functions by emulating business scenarios, based on functional
requirements. Black-box testing is a common way to verify functions.
 Performance testing: Testing how the software performs under different workloads. Load testing, for
example, is used to evaluate performance under real-life load conditions.
 Regression testing: Checking whether new features break or degrade functionality. Sanity testing can be
used to verify menus, functions and commands at the surface level, when there is no time for a full
regression test.
 Stress testing: Testing how much strain the system can take before it fails. Considered to be a type of non-
functional testing.
 Usability testing: Validating how well a customer can use a system or web application to complete a task.
Types of Software Testing Models
Process

Testing is an integral part of software development life cycle. Various models or approaches are used in the
software development process where each model has its own advantages and disadvantages. Choosing a
particular model depends on the project deliverables and complexity of the project.

Now Let us go through the various software testing models and their benefits:
1. Waterfall Model

This is the most basic software development life cycle process which is followed broadly in the industry.
Here the developers follow a sequence of processes where the processes flow progressively downwards
towards the ultimate goal. It is like a waterfall where there are a number of phases.
 These phases have their own unique functions and goals. There are, in fact, four phases – requirement
gathering and analysis phase, software design, programmed implementation and testing, maintenance. All
these four phases come one after another in the given order.

In the first phase all the possible system requirements for developing a particular software are noted and
analyzed. This in turn depends on the software requirement specifications which includes detailed
information about expectations of the end user. Based on this a Requirement Specification.

Document is created which acts an input to the next phase, i.e. software design phase. What needs to be
emphasized here is that once you move into the next phase it won’t be possible to update the requirements.
So you must be very thorough and careful about the end-user requirements.

Advantages
 Easy to implement and maintain.
 The initial phase of rigorous scrutiny of requirements and systems helps in saving time later in the
developmental phase
 The requirement of resources is minimal and testing is done after completion of each phase.
Disadvantages
 It is not possible to alter or update requirements
 You cannot make changes once you are into the next phase.
 Cannot start the next phase until the previous phase is completed
2. V Model

This model is widely recognized as superior to waterfall model. Here the development and test execution
activities are carried on side by side in the downhill and uphill shape. In this model, testing starts at the unit
level and spreads towards integration of the entire system.

So, SDLC is divided into five phases – unit testing, integration testing, regression testing, system testing
and acceptance testing.
Advantages
 Easy to use the model since testing activities like planning and test designing are done before coding
 Saves time and enhances chances of success.
 Defects are mostly found at an early stage and downward flow of defects is generally avoided
Disadvantages
 It is a rigid model
 Early prototypes of the product are not available since the software is developed during the implementation
phase
 If there are changes in the midway, then the test document needs to be updated
3. Agile model

In this SDLC model requirements and solutions evolve through collaboration between various cross
functional teams. This is known as an iterative and incremental model.

Advantages
 Ensure customer satisfaction with rapid and continuous development of deliverables.
 It is a flexible model as customers, developers and testers continuously interact with each other
 Working software can be developed quickly and product can be adapted to changing requirements regularly
Disadvantages
 In large and complex software development cases it becomes difficult to assess the effort required at the
beginning of the cycle
 Due to continuous interaction with the customer, the project can go off track if the customer is not clear
about the goals
4. Spiral model
It is more like the Agile model, but with more emphasis on risk analysis. It has four phases – planning, risk
analysis, engineering and evaluation. Here gathering of requirements and risk assessment is done at the
base level and every upper spiral builds on it.
Advantages
 Risk avoidance chance is enhanced due to the importance on risk analysis.
 Its a good model for complex and large systems.
 Depending on the changed circumstances additional functionalities can be added later on
 Software is produced early in the cycle
Disadvantages
 Its a costly model and requires highly specialized expertise in risk analysis
 It does not work well in simpler projects
Rational Unified Process

This model also consists of four phases, each of which is organized into a number of separate iterations.
The difference with other models is that each of these iterations must separately satisfy defined criteria
before the next phase is undertaken.

Advantages
 With an emphasis on accurate documentation this model is able to resolve risks associated with changing
requirements of the client
 Integration takes less time as the process goes on throughout the SDLC.
Disadvantages
 The biggest disadvantage is that the team members need to be experts in their niche.
 In big projects such continuous integration, it might give rise to confusions

Rapid application development

This is another incremental model like the Agile model. Here the components are developed parallelly to
each other. The developments are then assembled into a product.
Advantages
 The development time is reduced due to simultaneous development of components and the components can
be reused
 A lot of integration issues are resolved due to integration from the initial stage

Disadvantages
 It requires a strong team of highly capable developers with individual efficacy in identifying business
requirements
 It is a module based model, so systems that can be modularized can only be developed in this model
 As the cost is high, the model is not suitable for cheaper projects
 General Steps of Software Testing Process
These are 11 steps software testing process is an experience based practical approach for solution to test
assignment.
These are explained as following below.
 Step-1: Assess Development Plan and Status –
This initiative may be prerequisite to putting together Verification, Validation, and Testing Plan wont to
evaluate implemented software solution. During this step, testers challenge completeness and correctness of
event plan. Based on extensiveness and completeness of Project Plan testers can estimate quantity of
resources they’re going to got to test implemented software solution.
 Step-2: Develop the Test Plan –
Forming plan for testing will follow an equivalent pattern as any software planning process. The structure
of all plans should be an equivalent, but content will vary supported degree of risk testers perceive as
related to software being developed.
 Step-3: Test Software Requirements –
Incomplete, inaccurate, or inconsistent requirements cause most software failures. The inability to get
requirement right during requirements gathering phase can also increase cost of implementation
significantly. Testers, through verification, must determine that requirements are accurate, complete, and
they do not conflict with another.
 Step-4: Test Software Design –
This step tests both external and internal design primarily through verification techniques. The testers are
concerned that planning will achieve objectives of wants, also because design being effective and efficient
on designated hardware.

 Step-5: Build Phase Testing –

The method chosen to build software from internal design document will determine type and extensiveness
of testers needed. As the construction becomes more automated, less testing are going to be required during
this phase. However, if software is made using waterfall process, it’s subject to error and will be verified.
Experience has shown that it’s significantly cheaper to spot defects during development phase, than
through dynamic testing during test execution step.
 Step-6: Execute and Record Result –
This involves testing of code during dynamic state. The approach, methods, and tools laid out in test plan
are going to be wont to validate that executable code actually meets stated software requirements, and
therefore the structural specifications of design.
 Step-7: Acceptance Test –
Acceptance testing enables users to gauge applicability and usefulness of software in performing their day-
to-day job functions. This tests what user believes software should perform, as against what documented
requirements state software should perform.
 Step-8: Report Test Results –
Test reporting is continuous process. It may be both oral and written. It is important that defects and
concerns be reported to the appropriate parties as early as possible, so that corrections can be made at the
lowest possible cost.
 Step-9: The Software Installation –
Once test team has confirmed that software is prepared for production use, power to execute that software
 during production environment should be tested. This tests interface to operating software, related software,
and operating procedures.
 Step-10: Test Software Changes –
While this is often shown as Step 10, within context of performing maintenance after software is
implemented, concept is additionally applicable to changes throughout implementation process. Whenever
requirements changes, test plan must change, and impact of that change on software systems must be tested
and evaluate.
 Step-11: Evaluate Test Effectiveness –
Testing improvement can best be achieved by evaluating effectiveness of testing at top of every software
test assignment. While this assessment is primarily performed by testers, it should involve developers, users
of software, and quality assurance professionals if function exists within the IT organization.

TERMINOLOGIES

 Test Plan
 Test Scenarios
 Test Cases
 Priority
 Severity
 Verification
 Validation
 SDLC & STLC

1. Test Plan

As per ISTQB definition,

A test plan is a document outlining the scope, approach, resources, and schedule of the intended test
activities. It identifies amongst others, the test items, the features to be tested, tasks, person assigned to
each task, the test environment, test design techniques, entry and exit criteria as well as contingency
planning.
Simply put – A test plan defines the strategy to be used to test an application and elucidates the objective,
resources used and the information about the test environment. It is a blueprint which summarizes how the
testing activities will proceed for any project.

2. Test Scenarios vs Test Cases

A Test Case mentions the detailed inputs, execution conditions, steps to reproduce, test data, the actual &
expected results. Different test cases are designed and noted-down during the Test Case Development
phase of STLC which the testers later refer to, to check the application performance.
Test cases can be broadly classified into Functional & Non-functional or Positive & Negative.
People often get confused amongst – Test Scenarios and Test Cases. Here is a simple example to quickly
comprehend the difference between the two,

 Test Case 1: Check results on entering valid User Id & valid Password
 Test Case 2: Check results on entering Invalid User ID & valid Password
 Test Case 3: Check results on entering valid User ID & Invalid Password
 Test Case 4: Check results on entering Invalid User ID & Invalid Password
 Test Case 5: Check response when fields are Empty & Login Button is pressed.

Test Scenario is to Test the login functionality whereas Test Cases that are included in this Test Scenario
are,

3. Priority vs Severity

Priority specifies the level of urgency under which the bug needs to get resolved. Whereas severity
illustrates how critical the bug is, it basically describes the magnitude of impact if the bug is not resolved.
There are different levels of priority and severity, which are depicted in the image below,

4. Verification v/s Validation:

Verification is a static routine involving checking documents, structure, code, and program. It incorporates
activities involved in creating fantastic programming, including, assessment, structure investigation, and
detail examination. It is a moderate target process.
Validation is assessing the last item to check whether the product meets the clients’ desires and
prerequisites. It is a dynamic tool of approving and testing the real item.
It’s essential for a tester to possess a clear understanding of the process followed to develop the product in
their organization. Similar to SDLC, a proper process is followed to perform software testing which is
called STLC or Software Testing Life Cycle. Let’s catch some more details about these processes.
 SDLC vs STLC

 SDLC
Software Development Life Cycle (SDLC) explains the journey of Software Development. According to
Wikipedia, SDLC is a process followed for software development. It consists of a detailed strategy
outlining how to develop, maintain, replace, alter and enhance a specific software.
 STLC
STLC illustrates a systematic and well-planned testing process which includes different stages to make the
testing process quick, effective and accountable. Unlike SDLC, STLC – software testing life cycle
identifies how test cases will be implemented and how the testing would be conducted successfully.

Now Let’s a brief look at the different phases of SDLC and STLC:

Phase SDLC STLC

 Business Analyst gathers  The testing team analyzes and

SDLC requirements. reviews the Software Requirement
Requirement  Development team Document or SRD.
GatheringSTLC analyzes the requirements  Identifies the testing requirements –
Requirement & starts the brainstorming Scope, Verification, and Validation
Analysis on architecture & design of key points.

 SDLC architecture helps  In STLC, either the Test Lead or the

SDLC
you develop high-level Test Architect usually plan the test
Design
and low-level design strategy.
STLC
plans based on the
Test Planning
requirements.  Test planning, Test Analysis, and
  Business Analyst works Test Design took place here, along
on wireframe designing with Resource Allocation &
here, & after design timeline fixation.
approval, it will hand
over to stakeholders.

 The testing team writes the test

scenarios in order to examine the
 The development team
quality of the product.
starts developing the
SDLC software, Integrates it
 Detailed test cases are planned out
Development with different systems.
for all modules along with expected
STLC  Once all integration is
behavior.
Test Case achieved, a ready to test
Development software or product is  The prerequisites and the entry &
provided.
exit criteria of a test module are
mentioned here,

 The test team confirms the

SDLC
environment set-up based on the
Environment Set- Development team sets-
pre-requisites.
up up a test environment
 Performs smoke testing to make
STLC: with a developed product
sure the environment is stable for
Environment Set- to validate.
the product to be tested.
Up

 The actual testing is

carried out in this phase.
 System Integration testing starts
based on test cases.
It encompasses a wide
 Defects reported, if any, gets
SDLC: variety of testings.
retested and fixed.
Testing
STLC:  Developers fixed the bugs
 Regression testing is performed here
Test Execution & assign to tester again to
and the product is signed off once it
retest it.
meets the exit criteria.
 UAT testing is performed
here

SDLC: Once sign-off is received  Smoke and sanity testing in a

Deployment/ from the various testing production environment is
Product Release team, the application is completed here, as soon as the
STLC: deployed in a production product is deployed.
 Test Cycle- environment for real end  Test reports and matrix preparation
Closure users. are done by the testing team to
analyze the product.

It covers the post-

In this phase, the maintaining of test
deployment enhancement,
Maintenance cases, regression suits has been
supports, and updates if
done.
any.

It’s important that a tester must have knowledge of the process followed to develop any product in any IT
company. This process is named as SDLC – software development life cycle. Add on to this, a proper
process is followed to perform software testing i.e STLC – software testing life cycle. Let’s catch some
more details about these processes.
A tester has to perform numerous types of testing in his career. Let’s take a sneak peek at the different
types of software testing out there,

Functional Testing

Functional testing is a type of software testing where the system is tested against some functional
requirements.

 Unit Testing
Unit testing consists of the testing of individual modules or components. Its objective is to test each unit of
the software, validate whether the functionality implemented is stable or not and most importantly it should
work as per the design & requirement.It’s the first and foremost step to perform Software testing and is
generally carried out by software developers themselves. Unit Testing builds confidence in changing or
maintaining the codes. Reusability of code also increases.
 Integration Testing
Integration Testing is the second step of software testing, where individual modules are combined &
integrated modules are tested. It started after completion of Unit Testing. The objective is to test the defects
to discover while integrating any modules. This type of testing verifies the functionality, performance, and
reliability between the integrated modules.
 System Testing
System testing is the testing of complete and integrated software. It ensures that the developed product
meets the specific requirements.System Testing involves the examination of completely integrated
applications incorporating outer peripherals to check how modules interact with each other and with the
framework, all in all.
 Sanity Testing
Sanity testing is carried out to quickly evaluate and determine whether functionalities are working as
expected or not. Sanity Testing is also referred to as surface-level-testing.Sanity testing is a type of
Software Testing performed after receiving a software build, with minor changes in code, or functionalities,
to learn that the bugs have been fixed and no further issues are acquainted due with these progressions.The
 objective is “not” to verify the new functionality thoroughly but to determine that the developer has applied
some rationality (sanity) while developing the software.
 Smoke Testing
Smoke Testing is performed to discover that the critical functionalities of the program are working fine.
The motto of this testing is to check the “stability” of the framework and execute “before” any detailed
functional or regression tests are executed on the application.
 Interface Testing
An interface is a medium between two different software systems, whereas interface Testing is a software
testing type which verifies whether the correspondence between two distinctive programming frameworks
is done accurately.
 Regression Testing
Regression testing is the type of testing in which QA verifies whether changes made in the old functionality
still works with the newly implemented changes.
 Beta/Acceptance Testing:
Beta or User Acceptance testing is carried out by “real users” of the developed product in “real
environment.” A Beta version of the application is released for a limited number of users to gather
feedback on product quality.Beta testing decreases item disappointment hazards and provides an expanded
nature of the product through client approval.It is the final test before delivery product to the clients. Direct
criticism from clients is a noteworthy preferred standpoint of Beta Testing. This testing serves to tests the
item in the client’s environment.

Non-functional testing types,

 Performance Testing
Performance testing is defined as the type of testing where we ensure proper functionality of our
application under a heavy workload.
 Load Testing
Load testing is a sub-classification of Performance Testing. During load testing, we evaluate the behavior
of our application when N number of users simultaneously access it.
 Stress Testing
The objective of stress testing is to analyze the response of the system following a failure. In this type of
testing system should give an error message if it is operating under dire conditions. This type of testing tries
to break the system by testing with overwhelming data or resources.
 Volume Testing
Volume testing is a sort of Software Testing in which the product is subjected to an enormous volume of
information. This testing is carried out to test system performance by expanding the amount of data in the
database.
 Security Testing
Security Testing ensures that our applications are free from any vulnerabilities, dangers, risks that could
cause a major misfortune. It additionally helps in identifying all conceivable security issues in the system
and assist users in fixing these issues through coding.
 Compatibility Testing
Compatibility Testing is a sort of Software testing to check whether your product is fit for running on
multiple hardware, different OS, Applications and mobile gadgets. It’s a type of Non-functional testing.
 Recovery Testing
Recovery testing is the process of testing how well an application can recoup from accidents, hardware
failures, and other comparable issues. This testing is the constrained disappointment of the product in an
assortment of approaches to confirm that recovery is appropriately performed.
 Usability Testing
Usability testing ensures – Usefulness, Discoverability, Accessibility, Usability, and Desirability. The
objective is to see how clients interact with your product and to improve the item dependent on the
outcomes. The main focus of a Usability test is to improve a design.
 Compliance Testing :
Compliance testing also referred to as conformance testing, regulation testing, standards testing, is a sort of
testing to decide the consistency of a framework. Most importantly it is the nonfunctional testing procedure
which is done to approve, regardless of whether the framework created satisfies the association’s
recommended guidelines or not. Determining that the development and maintenance process meets the
endorsed procedure.
 Localization Testing
Localization testing is a software testing strategy, where the product is checked to guarantee that it carries
on as indicated by the nearby culture or settings.

What is Software Testing Life Cycle (STLC)?

Software Testing Life Cycle (STLC) is a sequence of specific activities conducted during the testing
process to ensure software quality goals are met. STLC involves both verification and validation activities.
Contrary to popular belief, Software Testing is not just a single/isolate activity, i.e. testing. It consists of a
series of activities carried out methodologically to help certify your software product. STLC stands for
Software Testing Life Cycle.
STLC Phases
There are following six major phases in every Software Testing Life Cycle Model (STLC Model):

1. Requirement Analysis
2. Test Planning
3. Test case development
4. Test Environment setup
5. Test Execution
6. Test Cycle closure
Differences between Verification and Validation

Verification is the process of checking that a software achieves its goal without any bugs. It is the process
to ensure whether the product that is developed is right or not. It verifies whether the developed product
fulfills the requirements that we have. Verification is static testing.
Verification means Are we building the product right?
Validation is the process of checking whether the software product is up to the mark or in other words
product has high level requirements. It is the process of checking the validation of product i.e. it checks
what we are developing is the right product. it is validation of actual and expected product. Validation is
the dynamic testing.
Validation means Are we building the right product?
The difference between Verification and Validation is as follow:

Verification Validation

It includes checking documents, design, codes It includes testing and validating the actual
and programs. product.

Verification is the static testing. Validation is the dynamic testing.

It does not include the execution of the code. It includes the execution of the code.

Methods used in verification are reviews, Methods used in validation are Black Box Testing,
walkthroughs, inspections and desk-checking. White Box Testing and non-functional testing.

It checks whether the software meets the

It checks whether the software conforms to requirements and expectations of a customer or
specifications or not. not.

It can find the bugs in the early stage of the It can only find the bugs that could not be found by
development. the verification process.

The goal of verification is application and

software architecture and specification. The goal of validation is an actual product.

Validation is executed on software code with the

Quality assurance team does verification. help of testing team.

It comes before validation. It comes after verification.

 Verification Validation

It consists of checking of documents/files and is It consists of execution of program and is

performed by human. performed by computer.

What is Software Testing Methodology?

Software Testing Methodology is defined as strategies and testing types used to certify that the Application
Under Test meets client expectations. Test Methodologies include functional and non-functional testing to
validate the AUT. Examples of Testing Methodologies are Unit Testing, Integration Testing, System
Testing, Performance Testing etc. Each testing methodology has a defined test objective, test strategy, and
deliverables.

Note: Since Software Testing is an integral part of any Development Methodology, many companies use
the term Development Methodologies & Testing Methodologies colloquially. Hence Testing
Methodologies could also refer to Waterfall, Agile and other QA models as against the above definition of
Testing Methodologies. Discussion on various testing types does not add value to the readers. Hence, we
will discuss the different development models.

In this tutorial, you will learn-

 Waterfall model
 Iterative development
 Agile methodology
 Extreme programming
 Which Software Methodology to choose?
 How to setup software testing methodologies?

Waterfall Model

What is it?

In the waterfall model, software development progress through various phases like Requirements Analysis,
Design etc – sequentially.

In this model, the next phase begins only when the earlier phase is completed.
What Is The Testing Approach?
The first phase in the waterfall model is the requirements phase in which all the project requirements are
completely defined before starting the testing. During this phase, the test team brainstorms the scope of
testing, test strategy and drafts a detailed test plan.

Only once the design of software is complete, the team will move on to execution of the test cases to ensure
that the developed software behaves as it expected.

In this methodology, the testing team proceeds to the next phase only when the previous phase is
completed.

Advantages

This software Engineering model is very simple to plan and manage. Hence, projects, where requirements
are clearly defined and stated beforehand, can be easily tested using a waterfall model.

Disadvantages

In the waterfall model, you can begin with the next phase only once the previous phase is completed.
Hence, this model cannot accommodate unplanned events and uncertainty.

This methodology is not suitable for projects where the requirements change frequently.

Iterative development

What is it?

In this model, a big project is divided into small parts, and each part is subjected to multiple iterations of
the waterfall model. At the end of an iteration, a new module is developed or an existing module is
enhanced. This module is integrated into the software architecture and the entire system is tested all
together

What is the testing Approach?

As soon as iteration is completed, the entire system is subjected to testing. Feedback from testing is
immediately available and is incorporated in the next cycle. The testing time required in successive
iteration can be reduced based on the experience gained from past iterations.

Advantages
The main advantage of iterative development is the test feedback is immediately available at the end of
each cycle.

Disadvantages

This model increases communication overheads significantly since, at the end of each cycle, feedback
about deliverables, effort etc must be given.

Agile methodology

What is it?

Traditional software development methodologies work on the premise that software requirements remain
constant throughout the project. But with an increase in complexity, the requirements undergo numerous
changes and continuously evolve. At times, the customer himself is not sure what he wants. Though the
iterative model addresses this issue, it’s still based on the waterfall model.

In Agile methodology, software is developed in incremental, rapid cycles. Interactions amongst customers,
developers and client are emphasized rather than processes and tools. The agile methodology focuses on
responding to change rather than extensive planning.

What Is The Testing Approach?

Incremental testing is used in agile development methods and hence, every release of the project is tested
thoroughly. This ensures that any bugs in the system are fixed before the next release.

Advantages

It is possible to make changes in the project at any time to comply with the requirements.
 This incremental testing minimizes risks.

Disadvantages

Constant client interaction means added time pressure on all stakeholders including the client themselves,
software development and test teams.

Extreme programming

What is it?

Extreme programming is a type of agile methodology which beliefs in short development cycles. A project
is divided into simple engineering tasks. Programmers code a simple piece of software and get back to the
customer for feedback. Review points from the customer are incorporated and the developers proceed with
the next task.

In extreme programming developers usually, work in pairs.

Extreme Programming is used in places where customer requirements are constantly changing.

What Is The Testing Approach?

Extreme programming follows a Test-driven development which is described as follows –

1. Add a Test Case to the test suite to verify the new functionality which is yet to be developed
2. Run all the tests and obviously the new test case added must fail since the functionality is not coded yet
3. Write some code to implement the feature/functionality
4. Run the test suite again. This time, the new test case should pass since the functionally has been coded

Advantages

Customers having a vague software design in mind could use extreme programming
 Continuous testing and continuous integration of small releases ensure software code is delivered is of high
quality

Disadvantages

Meetings amongst the software development team and clients add to time requirements.

UNIT II TESTING TECHNIQUES 10

Black-Box Testing Techniques – Equivalence Class Testing - State Table-Based Testing - Decision Table-
Based Testing – Cause-Effect Graphing Based Testing – Error Guessing – White- Box Testing Techniques
- Need – Basis Path Testing – Graph Matrices – Loop Testing – Data Flow Testing – Mutation Testing –
Static Testing – Progressive vs. Regressive Testing – Regression Testing Techniques
What is black box testing?
The black box is a powerful technique to check the application under test from the user’s perspective.
Black box testing is used to test the system against external factors responsible for software failures. This
testing approach focuses on the input that goes into the software, and the output that is produced. The
testing team does not cover the inside details such as code, server logic, and development method.
Black box testing is based on the requirements and checks the system to validate against predefined
requirements.

Various parameters checked in black box testing are:

 Accurate act ions performed by users
 System’s interact ion wit h the inputs
 The response time of the system
 Use of data structures Issues in the user interface
 Usabilit y issues
 Performance issues
 Abrupt applicat ion failure, unable to start or finish.
Types of Black Box Testing
There are many different types of Black Box Testing, some of them are given below:
 Functional testing – This is a t ype of black box test ing which is related to the funct ional
requirements of a system; Funct ional test ing is concerned only wit h the funct ional
requirements of a system and covers how well the system executes it s funct ions.
 Non-functional testing – This black box test ing t ype is not related to testing of specific
funct ionalit y, Non funct ional test ing is concerned wit h the non -funct ional requirements and
is designed specifically to evaluate the readiness of a system according to the various criteria
which are not covered by funct ional test ing.
 Regression testing – Regression Test ing is performed after code fixes, upgrades or any other
system maintenance to check the new changes has not affec ted any exist ing funct ionalit y.
What are Black Box testing techniques?
There are various test case design techniques applied for black-box testing:
1. Boundary Value Analysis
2. Equivalence part it ioning
3. State Transit ion Test ing
4. Decision Table Test ing
5. Graph-Based Test ing
6. Error Guessing Technique
1- Boundary Value Analysis
It is the widely used black-box testing, which is also the basis for equivalence testing. Boundary value
analysis tests the software with test cases with extreme values of test data. BVA is used to identify the
flaws or errors that arise due to the limits of input data.
For example: Taking inputs for a test case data for an age section should accept a valid data of anything
between 1-100. According to BVP analysis, the software will be tested against four test data as -1, 1, 100,
and 101 to check the system’s response using the boundary values.
2- Equivalence partitioning
This test case designing techniques checks the input and output by dividing the input into equivalent
classes. The data must be tested at least once to ensure maximum test coverage of data. It is the exhaustive
form of testing, which also reduces the redundancy of inputs.
For example: Taking inputs for a test case data for the example mentioned above will have three classes
from which one data will be tested.
Valid class: 1 to 100 (any number), Invalid class: -1 (checking the lowest of lowest), Invalid class:
101(highest of highest).
[Also Read: What is Boundary Value Analysis and Equivalence Partitioning?]
3- State Transition Testing
This testing technique uses the inputs, outputs, and the state of the system during the testing phase. It
checks the software against the sequence of transitions or events among the test data.
Based on the type of software that is tested, it checks for the behavioral changes of a system in a particular
state or another state while maintaining the same inputs.
For example, A login page will let you input username and password until three attempts. Each incorrect
password will be sent the user to the login page. After the third attempt, the user will be sent to an error
page. This state transition method considers the various states of the system and the inputs to pass only the
right sequence of the testing.
4- Decision Table Testing
This approach creates test cases based on various possibilities. It considers multiple test cases in a decision
table format where each condition is checked and fulfilled, to pass the test and provide accurate output. It is
preferred in case of various input combinations and multiple possibilities.
For example, A food delivery application will check various payment modes as input to place the order —
decision making based on the table.
Case1: If the end-user has a card, then the system will not check for cash or coupon and will take action to
place the order.
Case2: If the end-user has a coupon will not be checked for a card or cash and action will be taken.
Case3: if the end-user has cash, the action will be taken.
Case4: If the end-user doesn’t have anything, then action will not be taken.
5- Graph-Based Testing:
It is similar to a decision-based test case design approach where the relationship between links and input
cases are considered.
6- Error Guessing Technique:
This method of designing test cases is about guessing the output and input to fix any errors that might be
present in the system. It depends on the skills and judgment of the tester.
Comparison testing
This method uses the two different versions of the same software to compare and validate the results.

Equivalence Class Testing- Next date problem

Equivalence class testing (Equivalence class Partitioning) is a black-box testing technique used in
software testing as a major step in the Software development life cycle (SDLC). This testing technique is
better than many of the testing techniques like boundary value analysis, worst case testing, robust case
testing and many more in terms of time consumption and terms of precision of the test cases. Since testing
is done to identify possible risks, equivalence class testing performs better than the other techniques as the
test cases generated using it are logically identified with partitions in between to create different input and
output classes. This can be shown from the next-date problem which is stated below:
Given a day in the format of day-month-year, you need to find the next date for the given date. Perform
boundary value analysis and equivalence-class testing for this.
Conditions :
D: 1<Day<31
M: 1<Month<12
Y: 1800 <Year <2048
Boundary Value Analysis:
No. of test Cases (n = no. of variables) = 4n+1 = 4*3 +1 =13
Test Cases:
Test Case ID Day Month Year Expected Output

1 1 6 2000 2-6-2000

2 2 6 2000 3-6-2000

3 15 6 2000 16-6-2000

4 30 6 2000 1-7-2000

5 31 6 2000 Invalid Date

6 15 1 2000 16-1-2000

7 15 2 2000 16-2-2000

8 15 11 2000 16-11-2000
 9 15 12 2000 16-12-2000

10 15 6 1800 16-6-1800

11 15 6 1801 16-6-1801

12 15 6 2047 16-6-2047

13 15 6 2048 16-6-2048

Equivalence Class Testing:

Input classes:
Day:
D1: day between 1 to 28
D2: 29
D3: 30
D4: 31
Month:
M1: Month has 30 days
M2: Month has 31 days
M3: Month is February
Year:
Y1: Year is a leap year
Y2: Year is a normal year
Output Classes:
Increment Day
Reset Day and Increment Month
Increment Year
Invalid Date
Strong Normal Equivalence Class Test Cases:
Test Cases:
Test Case ID Day Month Year Expected Output

E1 15 4 2004 16-4-2004

E2 15 4 2003 16-4-2003

E3 15 1 2004 16-1-2004

E4 15 1 2003 16-1-2003

E5 15 2 2004 16-2-2004

E6 15 2 2003 16-2-2003

E7 29 4 2004 30-4-2004

E8 29 4 2003 30-4-2003
 E9 29 1 2004 30-1-2004

E10 29 1 2003 30-1-2003

E11 29 2 2004 1-3-2004

E12 29 2 2003 Invalid Date

E13 30 4 2004 1-5-2004

E14 30 4 2003 1-5-2003

E15 30 1 2004 31-1-2004

E16 30 1 2003 31-1-2003

E17 30 2 2004 Invalid Date

E18 30 2 2003 Invalid Date

E19 31 4 2004 Invalid Date

E20 31 4 2003 Invalid Date

E21 31 1 2004 1-2-2004

E22 31 1 2003 1-5-2003

E23 31 2 2004 Invalid Date

E24 31 2 2003 Invalid Date

So from this problem it is clearly seen that equivalence class testing clearly checks for many cases that
boundary value did not considered like that of February which has 28-29 days, leap year which lead to
variation in number of days in February and many more.
Hence the above example proves that equivalence partitioning generates more efficient test cases that
should be considered during risk assessment
 State Transition Testing
State Transition Testing is a type of software testing which is performed to check the change in the state
of the application under varying input. The condition of input passed is changed and the change in state is
observed.
State Transition Testing is basically a black box testing technique that is carried out to observe the behavior
of the system or application for different input conditions passed in a sequence. In this type of testing, both
positive and negative input values are provided and the behavior of the system is observed.
State Transition Testing is basically used where different system transitions are needed to be tested.

Objectives of State Transition Testing:

The objective of State Transition testing is:
 To test the behavior of the system under varying input.
 To test the dependency on the values in the past.
 To test the change in transition state of the application.
 To test the performance of the system.
Transition States:
 Change Mode:
When this mode is activated then the display mode moves from TIME to DATE.
 Reset:
When the display mode is TIME or DATE, then reset mode sets them to ALTER TIME or ALTER DATE
respectively.
 Time Set:
When this mode is activated, display mode changes from ALTER TIME to TIME.
 Date Set:
When this mode is activated, display mode changes from ALTER DATE to DATE.
State Transition Diagram:
State Transition Diagram shows how the state of the system changes on certain inputs.
It has four main components:
1. States
2. Transition
3. Events
4. Actions
Advantages of State Transition Testing:
 State transition testing helps in understanding the behavior of the system.
 State transition testing gives the proper representation of the system behavior.
 State transition testing covers all the conditions.
Disadvantages of State Transition Testing:
 State transition testing can not be performed everywhere.
 State transition testing is not always reliable.

Decision Table Based Testing in Software Testing

 Last Updated : 04 Oct, 2021
What is a Decision Table :
Decision tables are used in various engineering fields to represent complex logical relationships. This
testing is a very effective tool in testing the software and its requirements management. The output may be
dependent on many input conditions and decision tables give a tabular view of various combinations of
input conditions and these conditions are in the form of True(T) and False(F). Also, it provides a set of
conditions and its corresponding actions required in the testing.
Parts of Decision Tables :
In software testing, the decision table has 4 parts which are divided into portions and are given below :

1. Condition Stubs : The conditions are listed in this first upper left part of the decision table that is used to
determine a particular action or set of actions.
2. Action Stubs : All the possible actions are given in the first lower left portion (i.e, below condition stub) of
the decision table.
3. Condition Entries : In the condition entry, the values are inputted in the upper right portion of the decision
table. In the condition entries part of the table, there are multiple rows and columns which are known as
Rule.
4. Action Entries : In the action entry, every entry has some associated action or set of actions in the lower
right portion of the decision table and these values are called outputs.
Types of Decision Tables :
The decision tables are categorized into two types and these are given below:
1. Limited Entry : In the limited entry decision tables, the condition entries are restricted to binary values.
2. Extended Entry : In the extended entry decision table, the condition entries have more than two values.
The decision tables use multiple conditions where a condition may have many possibilities instead of only
‘true’ and ‘false’ are known as extended entry decision tables.
Applicability of Decision Tables :
 The order of rule evaluation has no effect on the resulting action.
 The decision tables can be applied easily at the unit level only.
 Once a rule is satisfied and the action selected, n another rule needs to be examined.
 The restrictions do not eliminate many applications.
Example of Decision Table Based testing :
Below is the decision table of the program for determining the largest amongst three numbers in which its
input is a triple of positive integers (x,y, and z) and values are from the interval [1, 300].
Table 1 : Decision Table of largest amongst three numbers :

Co

R10

R11

R12

R13

R14
R3

R4

R5

R6

R7

R8

R9
ndi
R1

R2

tio
ns

c1:
x>
T

T
=
F

1?

c2:
x
T

T
F

<=
300
?

c3:
y>
T

T
F

=
1?

c4:
x
T

T
F

<=
300
?
c5:
z>

T
F
=
1?

c6:
z

T
F
<=
300
?

c7:

F
x>y
?

c8: T

T
F

F
y>z
?

c9:
T

T
F

F
z>x
?

Rul
128

64

32

16

e
8

1
256

Co
unt

a1 :
Inv
X

alid
X

inp
ut

a2 :
x is
X

larg
est
 a3 :
y is

X
larg
est

a4 :
z is

X
larg
est

a5 :
Imp

X
ossi
ble

What is Cause-Effect Graph?

Cause Effect Graph is a black box testing technique that graphically illustrates the relationship between a
given outcome and all the factors that influence the outcome.
It is also known as Ishikawa diagram as it was invented by Kaoru Ishikawa or fish bone diagram because of
the way it looks.
Cause Effect - Flow Diagram

Circumstances - under which Cause-Effect Diagram used

 To Identify the possible root causes, the reasons for a specific effect, problem, or outcome.
 To Relate the interactions of the system among the factors affecting a particular process or effect.
 To Analyze the existing problems so that corrective action can be taken at the earliest.
Benefits :
 It Helps us to determine the root causes of a problem or quality using a structured approach.
 It Uses an orderly, easy-to-read format to diagram cause-and-effect relationships.
 It Indicates possible causes of variation in a process.
 It Identifies areas, where data should be collected for further study.
 It Encourages team participation and utilizes the team knowledge of the process.
 It Increases knowledge of the process by helping everyone to learn more about the factors at work and how
they relate.
Steps for drawing cause-Effect Diagram:
 Step 1 : Identify and Define the Effect
 Step 2 : Fill in the Effect Box and Draw the Spine
 Step 3: Identify the main causes contributing to the effect being studied.
 Step 4 : For each major branch, identify other specific factors which may be the causes of the EFFECT.
 Step 5 : Categorize relative causes and provide detailed levels of causes.
Cause Effect Graphing in Software Engineering
Prerequisite – Black Box Testing
Cause Effect Graphing based technique is a technique in which a graph is used to represent the situations
of combinations of input conditions. The graph is then converted to a decision table to obtain the test cases.
Cause-effect graphing technique is used because boundary value analysis and equivalence class partitioning
methods do not consider the combinations of input conditions. But since there may be some critical
behaviour to be tested when some combinations of input conditions are considered, that is why cause-effect
graphing technique is used.
Steps used in deriving test cases using this technique are:
1. Division of specification:
Since it is difficult to work with cause-effect graphs of large specifications as they are complex, the
specifications are divided into small workable pieces and then converted into cause-effect graphs
separately.
2. Identification of cause and effects:
This involves identifying the causes(distinct input conditions) and effects(output conditions) in the
specification.
3. Transforming the specifications into a cause-effect graph:
The causes and effects are linked together using Boolean expressions to obtain a cause-effect graph.
Constraints are also added between causes and effects if possible.
4. Conversion into decision table:
The cause-effect graph is then converted into a limited entry decision table. If you’re not aware of the
concept of decision tables, check out this link.
5. Deriving test cases:
Each column of the decision-table is converted into a test case.
Basic Notations used in Cause-effect graph:
Here c represents cause and e represents effect.
The following notations are always used between a cause and an effect:
1. Identity Function: if c is 1, then e is 1. Else e is 0.
2. NOT Function: if c is 1, then e is 0. Else e is 1.

3. OR Function: if c1 or c2 or c3 is 1, then e is 1. Else e is 0.

4. AND Function: if both c1 and c2 and c3 is 1, then e is 1. Else e is 0.

To represent some impossible combinations of causes or impossible combinations of effects, constraints are
used. The following constraints are used in cause-effect graphs:
1. Exclusive constraint or E-constraint: This constraint exists between causes. It states that either c1 or c2
can be 1, i.e., c1 and c2 cannot be 1 simultaneously.

2. Inclusive constraint or I-constraint: This constraint exists between causes. It states that atleast one of c1,
c2 and c3 must always be 1, i.e., c1, c2 and c3 cannot be 0 simultaneously.
3. One and Only One constraint or O-constraint: This constraint exists between causes. It states that one
and only one of c1 and c2 must be 1.

4. Requires constraint or R-constraint: This constraint exists between causes. It states that for c1 to be 1, c2
must be 1. It is impossible for c1 to be 1 and c2 to be 0.

5. Mask constraint or M-constraint: This constraint exists between effects. It states that if effect e1 is 1, the
effect e2 is forced to be 0.
 Error Guessing Technique
The test cases for finding issues in the software are written based on the prior testing experience with
similar applications. So, the scope of the test cases would generally depend upon the kind of testing Test
Analyst was involved in the past. The Error Guessing technique does not follow any specific rules.

For Example, if the Analyst guesses that the login page is error-prone, then the testers will write detailed
test cases concentrating on the login page. Testers can think of a variety of combinations of data to test the
login page.
To design test cases based on the Error Guessing technique, the Analyst can use past experiences to
identify the conditions.

This technique can be used at any level of testing and for testing the common mistakes like:
 Divide by zero
 Entering blank spaces in the text fields
 Pressing the submit button without entering values.
 Uploading files exceeding maximum limits.
 Null pointer exception.
 Invalid parameters
The achievement rate of this technique does mainly depends upon the ability of testers.

Purpose Of Error Guessing In Software Testing

 The main purpose of this technique is to guess possible bugs in the areas where formal testing would not
work.
 It should obtain an all-inclusive set of testing without any skipped areas, and without creating redundant
tests.
 This technique compensates for the characteristic incompleteness of Boundary Value Analysis and
Equivalence Partitioning techniques.

White Box Testing

The box testing approach of software testing consists of black box testing and white box testing. We are
discussing here white box testing which also known as glass box is testing, structural testing, clear box
testing, open box testing and transparent box testing. It tests internal coding and infrastructure of a
software focus on checking of predefined inputs against expected and desired outputs. It is based on inner
workings of an application and revolves around internal structure testing. In this type of testing
programming skills are required to design test cases. The primary goal of white box testing is to focus on
the flow of inputs and outputs through the software and strengthening the security of the software.

The term 'white box' is used because of the internal perspective of the system. The clear box or white box
or transparent box name denote the ability to see through the software's outer shell into its inner workings.

Developers do white box testing. In this, the developer will test every line of the code of the program. The
developers perform the White-box testing and then send the application or the software to the testing team,
where they will perform the black box testing and verify the application along with the requirements and
identify the bugs and sends it to the developer.
 The white box testing contains various tests, which are as follows:

o Path testing
o Loop testing
o Condition testing
o Testing based on the memory perspective
o Test performance of the program

Reasons for white box testing

o It identifies internal security holes.
o To check the way of input inside the code.
o Check the functionality of conditional loops.
o To test function, object, and statement at an individual level.

Advantages of White box testing

o White box testing optimizes code so hidden errors can be identified.
o Test cases of white box testing can be easily automated.
o This testing is more thorough than other testing approaches as it covers all code paths.
o It can be started in the SDLC phase even without GUI.

Disadvantages of White box testing

o White box testing is too much time consuming when it comes to large-scale programming applications.
o White box testing is much expensive and complex.
o It can lead to production error because it is not detailed by the developers.
o White box testing needs professional programmers who have a detailed knowledge and understanding of
programming language and implementation.

Basis Path Testing in Software Testing

Prerequisite – Path Testing
Basis Path Testing is a white-box testing technique based on the control structure of a program or a
module. Using this structure, a control flow graph is prepared and the various possible paths present in the
graph are executed as a part of testing. Therefore, by definition,
Basis path testing is a technique of selecting the paths in the control flow graph, that provide a basis set of
execution paths through the program or module.
Since this testing is based on the control structure of the program, it requires complete knowledge of the
program’s structure. To design test cases using this technique, four steps are followed :
1. Construct the Control Flow Graph
2. Compute the Cyclomatic Complexity of the Graph
3. Identify the Independent Paths
4. Design Test cases from Independent Paths
Let’s understand each step one by one.
 1. Control Flow Graph –
A control flow graph (or simply, flow graph) is a directed graph which represents the control structure of a
program or module. A control flow graph (V, E) has V number of nodes/vertices and E number of edges in
it. A control graph can also have :
 Junction Node – a node with more than one arrow entering it.
 Decision Node – a node with more then one arrow leaving it.
 Region – area bounded by edges and nodes (area outside the graph is also counted as a region.).

Below are the notations used while constructing a flow graph :

 Sequential Statements –
 If – Then – Else –

 Do – While –

 While – Do –
 Switch – Case –

Cyclomatic Complexity –
The cyclomatic complexity V(G) is said to be a measure of the logical complexity of a program. It can be
calculated using three different formulae :
1. Formula based on edges and nodes :
V(G) = e - n + 2*P
Where,
e is number of edges,
n is number of vertices,
P is number of connected components.
For example, consider first graph given above,
where, e = 4, n = 4 and p = 1

So,
 Cyclomatic complexity V(G)
=4-4+2*1
=2
2. Formula based on Decision Nodes :
V(G) = d + P
where,
d is number of decision nodes,
P is number of connected nodes.
For example, consider first graph given above,
where, d = 1 and p = 1

So,
Cyclomatic Complexity V(G)
=1+1
=2
3. Formula based on Regions :
V(G) = number of regions in the graph
For example, consider first graph given above,
Cyclomatic complexity V(G)
= 1 (for Region 1) + 1 (for Region 2)
=2
Hence, using all the three above formulae, the cyclomatic complexity obtained remains same. All these
three formulae can be used to compute and verify the cyclomatic complexity of the flow graph.
Note –
1. For one function [e.g. Main( ) or Factorial( ) ], only one flow graph is constructed. If in a program, there
are multiple functions, then a separate flow graph is constructed for each one of them. Also, in the
cyclomatic complexity formula, the value of ‘p’ is set depending of the number of graphs present in total.
2. If a decision node has exactly two arrows leaving it, then it is counted as one decision node. However, if
there are more than 2 arrows leaving a decision node, it is computed using this formula :
d=k-1
Here, k is number of arrows leaving the decision node.
Independent Paths :
An independent path in the control flow graph is the one which introduces at least one new edge that has
not been traversed before the path is defined. The cyclomatic complexity gives the number of independent
paths present in a flow graph. This is because the cyclomatic complexity is used as an upper-bound for the
 number of tests that should be executed in order to make sure that all the statements in the program have
been executed at least once.
Consider first graph given above here the independent paths would be 2 because number of independent
paths is equal to the cyclomatic complexity.
So, the independent paths in above first given graph :
 Path 1:
A -> B
 Path 2:
C -> D
Note –
Independent paths are not unique. In other words, if for a graph the cyclomatic complexity comes out be N,
then there is a possibility of obtaining two different sets of paths which are independent in nature.
Design Test Cases :
Finally, after obtaining the independent paths, test cases can be designed where each test case represents
one or more independent paths.
Advantages :
Basis Path Testing can be applicable in the following cases:
1. More Coverage –
Basis path testing provides the best code coverage as it aims to achieve maximum logic coverage instead of
maximum path coverage. This results in an overall thorough testing of the code.
2. Maintenance Testing –
When a software is modified, it is still necessary to test the changes made in the software which as a result,
requires path testing.
3. Unit Testing –
When a developer writes the code, he or she tests the structure of the program or module themselves first.
This is why basis path testing requires enough knowledge about the structure of the code.
4. Integration Testing –
When one module calls other modules, there are high chances of Interface errors. In order to avoid the case
of such errors, path testing is performed to test all the paths on the interfaces of the modules.
5. Testing Effort –
Since the basis path testing technique takes into account the complexity of the software (i.e., program or
module) while computing the cyclomatic complexity, therefore it is intuitive to note that testing effort in
case of basis path testing is directly proportional to the complexity of the software or program.

Graph Matrices in Software Testing

A graph matrix is a data structure that can assist in developing a tool for automation of path testing.
Properties of graph matrices are fundamental for developing a test tool and hence graph matrices are very
useful in understanding software testing concepts and theory.
What is a Graph Matrix ?
A graph matrix is a square matrix whose size represents the number of nodes in the control flow graph. If
you do not know what control flow graphs are, then read this article. Each row and column in the matrix
identifies a node and the entries in the matrix represent the edges or links between these nodes.
Conventionally, nodes are denoted by digits and edges are denoted by letters.
 Let’s take an example.

Let’s convert this control flow graph into a graph matrix. Since the graph has 4 nodes, so the graph matrix
would have a dimension of 4 X 4. Matrix entries will be filled as follows :
 (1, 1) will be filled with ‘a’ as an edge exists from node 1 to node 1
 (1, 2) will be filled with ‘b’ as an edge exists from node 1 to node 2. It is important to note that (2, 1) will
not be filled as the edge is unidirectional and not bidirectional
 (1, 3) will be filled with ‘c’ as edge c exists from node 1 to node 3
 (2, 4) will be filled with ‘d’ as edge exists from node 2 to node 4
 (3, 4) will be filled with ‘e’ as an edge exists from node 3 to node 4
The graph matrix formed is shown below :

Connection Matrix :
A connection matrix is a matrix defined with edges weight. In simple form, when a connection exists
between two nodes of control flow graph, then the edge weight is 1, otherwise, it is 0. However, 0 is not
usually entered in the matrix cells to reduce the complexity.
For example, if we represent the above control flow graph as a connection matrix, then the result would be :
 As we can see, the weight of the edges are simply replaced by 1 and the cells which were empty before are
left as it is, i.e., representing 0.
A connection matrix is used to find the cyclomatic complexity of the control graph.
Although there are three other methods to find the cyclomatic complexity but this method works well too.
Following are the steps to compute the cyclomatic complexity :
1. Count the number of 1s in each row and write it in the end of the row
2. Subtract 1 from this count for each row (Ignore the row if its count is 0)
3. Add the count of each row calculated previously
4. Add 1 to this total count
5. The final sum in Step 4 is the cyclomatic complexity of the control flow graph
Let’s apply these steps to the graph above to compute the cyclomatic complexity.

We can verify this value for cyclomatic complexity using other methods :
Method-1 :
Cyclomatic complexity
=e-n+2*P
Since here,
e=5
n=4
and, P = 1
Therefore, cyclomatic complexity,
=5-4+2*1
=3
Method-2 :
Cyclomatic complexity
=d+P
Here,
d=2
 and, P = 1
Therefore, cyclomatic complexity,
=2+1
=3
Method-3:
Cyclomatic complexity
= number of regions in the graph
li
 Region 1: bounded by edges b, c, d, and e
 Region 2: bounded by edge a (in loop)
 Region 3: outside the graph
Therefore, cyclomatic complexity,
=1+1+1
=3
It can be seen that all other methods give the same result. Method1, 2, and 3 have been discussed here in
detail.

Loop Software Testing

Loop Testing is a type of software testing type that is performed to validate the loops. It is one of the type
of Control Structure Testing. Loop testing is a white box testing technique and is used to test loops in the
program.
Objectives of Loop Testing:
The objective of Loop Testing is:
 To fix the infinite loop repetition problem.
 To know the performance.
 To identify the loop initialization problems.
 To determine the uninitialized variables.
Types of Loop Testing:
Loop testing is classified on the basis of the types of the loops:
1. Simple Loop Testing:
Testing performed in a simple loop is known as Simple loop testing. Simple loop is basically a normal
“for”, “while” or “do-while” in which a condition is given and loop runs and terminates according to true
and false occurrence of the condition respectively. This type of testing is performed basically to test the
condition of the loop whether the condition is sufficient to terminate loop after some point of time.
Example:
while(condition)
{
statement(s);
 }

2. Nested Loop Testing:

Testing performed in a nested loop in known as Nested loop testing. Nested loop is basically one loop
inside the another loop. In nested loop there can be finite number of loops inside a loop and there a nest is
made. It may be either of any of three loops i.e., for, while or do-while.
Example:
while(condition 1)
{
while(condition 2)
{
statement(s);
}
}
3. Concatenated Loop Testing:
Testing performed in a concatenated loop is known as Concatenated loop testing. It is performed on the
concatenated loops. Concatenated loops are loops after the loop. It is a series of loops. Difference between
nested and concatenated is that in nested loop is inside the loop but here loop is after the loop.
Example:
while(condition 1)
{
statement(s);
}
while(condition 2)
{
statement(s);
}

4. Unstructured Loop Testing:

Testing performed in an unstructured loop is known as Unstructured loop testing. Unstructured loop is the
combination of nested and concatenated loops. It is basically a group of loops that are in no order.
Example:
while()
{
for()
{}
while()
{}
}
 Advantages of Loop Testing:
The advantages of Loop testing are:
 Loop testing limits the number of iterations of loop.
 Loop testing ensures that program doesn’t go into infinite loop process.
 Loop testing endures initialization of every used variable inside the loop.
 Loop testing helps in identification of different problems inside the loop.
 Loop testing helps in determination of capacity.
Disadvantages of Loop Testing:
The disadvantages of Loop testing are:
 Loop testing is mostly effective in bug detection in low-level software.
 Loop testing is not useful in bug detection.

Data Flow Testing

Data Flow Testing is a type of structural testing. It is a method that is used to find the test paths of a
program according to the locations of definitions and uses of variables in the program. It has nothing to do
with data flow diagrams.
It is concerned with:
 Statements where variables receive values,
 Statements where these values are used or referenced.
To illustrate the approach of data flow testing, assume that each statement in the program assigned a unique
statement number. For a statement number S-
DEF(S) = {X | statement S contains the definition of X}
USE(S) = {X | statement S contains the use of X}
 If a statement is a loop or if condition then its DEF set is empty and USE set is based on the condition of
statement s.
Data Flow Testing uses the control flow graph to find the situations that can interrupt the flow of the
program.
Reference or define anomalies in the flow of the data are detected at the time of associations between
values and variables. These anomalies are:
 A variable is defined but not used or referenced,
 A variable is used but never defined,
 A variable is defined twice before it is used
Advantages of Data Flow Testing:
Data Flow Testing is used to find the following issues-
 To find a variable that is used but never defined,
 To find a variable that is defined but never used,
 To find a variable that is defined multiple times before it is use,
 Deallocating a variable before it is used.
Disadvantages of Data Flow Testing
 Time consuming and costly process
 Requires knowledge of programming languages
Example:
1. read x, y;
2. if(x>y)
3. a = x+1
else
4. a = y-1
5. print a;
Control flow graph of above example:

Define/use of variables of above example:

Variable Defined at node Used at node
 Variable Defined at node Used at node

X 1 2, 3

Y 1 2, 4

A 3, 4 5

Software Testing | Mutation Testing

Testing is a type of Software Testing that is performed to design new software tests and also evaluate the
quality of already existing software tests. Mutation testing is related to modification a program in small
ways. It focuses to help the tester develop effective tests or locate weaknesses in the test data used for the
program.
History of Mutation Testing:
Richard Lipton proposed the mutation testing in 1971 for the first time. Although high cost reduced the
use of mutation testing but now it is widely used for languages such as Java and XML.

Mutation Testing is a White Box Testing.

Mutation testing can be applied to design models, specifications, databases, tests, and XML. It is a
structural testing technique, which uses the structure of the code to guide the testing process. It can be
described as the process of rewriting the source code in small ways in order to remove the redundancies
in the source code.
Objective of Mutation Testing:
The objective of mutation testing is:
 To identify pieces of code that are not tested properly.
 To identify hidden defects that can’t be detected using other testing methods.
 To discover new kinds of errors or bugs.
 To calculate the mutation score.
 To study error propagation and state infection in the program.
 To assess the quality of the test cases.
 Types of Mutation Testing:
Mutation testing is basically of 3 types:
1. Value Mutations:
In this type of testing the values are changed to detect errors in the program. Basically a small value is
changed to a larger value or a larger value is changed to a smaller value. In this testing basically
constants are changed.
Example:
Initial Code:

int mod = 1000000007;

int a = 12345678;
int b = 98765432;
int c = (a + b) % mod;

Changed Code:

int mod = 1007;

int a = 12345678;
int b = 98765432;
int c = (a + b) % mod;
2. Decision Mutations:
In decisions mutations are logical or arithmetic operators are changed to detect errors in the program.
Example:
Initial Code:

if(a < b)
c = 10;
else
c = 20;

Changed Code:

if(a > b)
c = 10;
else
c = 20;
3. Statement Mutations:
In statement mutations a statement is deleted or it is replaces by some other statement.
Example:
Initial Code:

if(a < b)
c = 10;
else
c = 20;
 Changed Code:

if(a < b)
d = 10;
else
d = 20;
Advantages of Mutation Testing:
 It brings a good level of error detection in the program.
 It discovers ambiguities in the source code.
Disadvantages of Mutation Testing:
 It is highly costly and time-consuming.
 It is not able for Black Box Testing.
Software Testing | Static Testing
Static Testing is a type of a Software Testing method which is performed to check the defects in software
without actually executing the code of the software application. Whereas in Dynamic Testing checks the
code is executed to detect the defects.
Static testing is performed in early stage of development to avoid errors as it is easier to find sources of
failures and it can be fixed easily. The errors that can’t not be found using Dynamic Testing, can be easily
found by Static Testing.
Static Testing Techniques:
There are mainly two type techniques used in Static Testing:

1. Review:
In static testing review is a process or technique that is performed to find the potential defects in the design
of the software. It is process to detect and remove errors and defects in the different supporting documents
like software requirements specifications. People examine the documents and sorted out errors,
redundancies and ambiguities.
Review is of four types:
 Informal:
In informal review the creator of the documents put the contents in front of audience and everyone gives
their opinion and thus defects are identified in the early stage.
 Walkthrough:
It is basically performed by experienced person or expert to check the defects so that there might not be
problem further in the development or testing phase.
 Peer review:
Peer review means checking documents of one-another to detect and fix the defects. It is basically done in a
team of colleagues.
 Inspection:
Inspection is basically the verification of document the higher authority like the verification of software
requirement specifications (SRS).
2. Static Analysis:
Static Analysis includes the evaluation of the code quality that is written by developers. Different tools are
used to do the analysis of the code and comparison of the same with the standard.
It also helps in following identification of following defects:
(a) Unused variables
(b) Dead code
(c) Infinite loops
(d) Variable with undefined value
(e) Wrong syntax
Static Analysis is of three types:
 Data Flow:
Data flow is related to the stream processing.
 Control Flow:
Control flow is basically how the statements or instructions are executed.
 Cyclomatic Complexity:
Cyclomatic complexity is the measurement of the complexity of the program that is basically related to the
number of independent paths in the control flow graph of the program

What is regression testing?

Regression testing is a black box testing techniques. It is used to authenticate a code change in the software
does not impact the existing functionality of the product. Regression testing is making sure that the product
works fine with new functionality, bug fixes, or any change in the existing feature.

Regression testing is a type of software testing. Test cases are re-executed to check the previous
functionality of the application is working fine, and the new changes have not produced any bugs.

There are three types of Regression Testing:

Regression Testing Techniques

Mainly there are 5 types of Regression testing techniques. Each of the techniques is applicable for different
purposes. Some technique requires fewer resources while some require a lot of resources but that technique
will provide the result with greater details.

The type of technique used in regression testing varies from project to project and from organization to
organization. Some organization prefers to use a particular regression testing technique in all their
projects. The 5 main categories of regression testing methods are:

Corrective Regression testing

It is used when no changes are introduced in the current application or software or product specifications.
In this case, we can use the test cases which have already been created to perform the testing.

Progressive Regression testing

This technique of regression testing is used in situations when new changes are introduced in the software’s
specification and new test cases need to be designed to perform the desired tests. The progressive
regression testing works very effectively when certain changes are introduced in the software’s
specification and when new test cases are designed.

In this technique, a part of the test cases is taken from the prevailing test cases and used which reduces the
cost and effort involved in designing all the new test cases.

Selective Regression Testing

This technique of regression testing is used to analyze the effects of the newly added code which is
introduced in the software to the already existing code in the software. In order to reduce the retesting
efforts and cost involved in selective regression testing, we use a part of the current experiments.
In this technique, a test case is re-executed if any new change is introduced in any module or part of the
software or application like factors, capacities and so on. In selective regression testing, the condition
between the experiment and the elements covered by the program are very difficult to determine.

Retest-all regression testing

As the name suggests this technique of regression testing involves reusing all the test cases and also testing
all the possible aspects of the product or software even when a new change is introduced in the software or
not.

This technique is time-consuming and very tedious. This is very expensive as it requires a huge amount of
resources because we test all the aspects of the software and all the tests in the existing test bucked have to
be re-executed.

The retest-all regression testing is not advised to use when only a small change is introduced in the
software or the product.

Complete Regression Testing

We have already discussed this in the types of regression testing. The complete regression testing is used
more often when multiple changes are introduced in the software and it is also used in the case when the
changes made in the software affect the root or the core of the software.

This is beneficial as it helps in identifying the unseen problems. After the testing is complete the final
software or the product is provided to the end-users.
 UNIT III MANAGING THE TESTING PROCESS 9
Test Management – Software Metrics – Testing Metrics for Monitoring and Controlling the Testing
Process – Efficient Test Suit Management

Test Management :
Test Management is a process where testing activities are managed to ensure high-quality and high-end
testing of software applications. This method consists of tracking, organization, controlling process,
checks the visibility of the testing process in order to deliver a high-quality software application. It
makes sure the software testing process run as expected.
Test Management Process :
It is a software process that manages the start to the end of all software testing act ivities. This
management process provides planning, controlling, tracking, and monitoring facilities throughout the
whole group cycle, these process includes several activities like test case design and test execution, test
planning, etc. It also gives initial plan and discipline specifications to the software testing process.
Responsibilities:
 Works in collaboration with test analyst and technical test analyst to select and customizes the
appropriate templates and also establish standards.
 Provides all facilities to keep track and control the testing throughout the project.
 Gives the clear concept of understanding of the testing activity of the prior upcoming project and also
post ones
Test management process has two main parts of test Management Process:
Planning :
 Risk analysis
 Test Estimation
 Test planning
Execution :
 Testing Activity
 Issue Management
 Test report and evolution
The activity of the test process :
1. Test plan –
Rough sketches are served in order to test plans to convey the process of testing. Gives a clear vision of
the complete testing process.
2. Test design –
Test design affords the implementation process of testing.
3. Test execution –
It shows the actual system result against the expected result during test execution.
4. Exit criteria –
It gives the signal when to stop the test execution process.
5. Test reporting –
Test reporting picturizes the test process and result for the particular testing cycle.
 Tools :
Some commonly used tools are listed below-
1. qTest
2. Zephyr
3. Test Collab
4. XQual
5. TestRail
6. Testpad
Advantages :
 Reuses current test and compares the results with last trials.
 Prevents duplicate issues
 Enables conceptual graphical visualization regarding reports
 Reports errors via email
 Combines easily with automation tools and C
 Deals with tests accordingly to the requirements effortlessly
 Integrated quickly with slack
 Deals test based on cycle and sprints
Disadvantages :
 It is not cost-effective
 Doesn’t support cloud-based application
 No mobile app support
Software Testing Metrics: Definition, Types & Example
Software TestingSoftware & CodingTesting Tools

Software Testing metrics are quantitative steps taken to evaluate the software testing process's quality,
performance, and progress. This helps us to accumulate reliable data about the software testing process and
enhance its efficiency. This will allow developers to make proactive and precise decisions for upcoming
testing procedures.
What is a metric in software testing metrics?
A Metric is a degree to which a system or its components retains a given attribute. Testers don't define a
metric just for the sake of documentation. It serves greater purposes in software testing. For example,
developers can apply a metric to assume the time it takes to develop software. It can also be assigned to
determine the numbers of new features and modifications, etc., added to the software.
Importance of Software Testing Metrics
As mentioned, test metrics are crucial to measuring the quality and performance of the software. With
proper software testing metrics, developers can−
 Determine what types of improvements are required to deliver a defect-free, quality software
 Make sound decisions about the subsequent testing phases, such as scheduling upcoming projects as well as
estimating the overall cost of those projects
 Evaluate the current technology or process and check whether it needs further modifications
Types of software testing metrics
There are three types of software testing metrics−
 Process Metrics: Process metrics define the characteristics and execution of a project. These
characteristics are essential to the improvement and maintenance of the process in the SDLC (Software
Development Life Cycle).
 Product Metrics: Product metrics define the size, design, performance, quality, and complexity of a
product. By using these characteristics, developers can enhance their software development quality.
 Project Metrics: Project Metrics determine the overall quality of a project. It is used to calculate costs,
productivity, defects and estimate the resource and deliverables of a project.
It is incredibly vital to identify the correct testing metrics for the process. Few factors to consider−
 Choose your target audiences wisely before preparing the metrics
 Define the goal behind designing the metrics
 Prepare metrics by considering the specific requirements of the project
 Evaluate the financial gain behind each metrics
 Pair the metrics with the project lifestyle phase that delivers optimum output
Software testing can be further divided into manual and automated testing.
In manual testing, the test is performed by QA analysts in a step-by-step process. Meanwhile, in automated
testing, tests are executed with the help of test automation frameworks, tools, and software.
Both manual and automated testing has its strength and weakness.
Manual testing is a slow process, but it allows testers to handles complex scenarios.
The most significant advantage of automated testing is that it enables testers to run more testing in less
time, covering a substantial level of permutations, which is nearly impossible to calculate manually.
Types of Manual Test Metrics
Manual Test Metrics are of two types−
Base Metrics
Base metrics are data collected by analysts during test case development and execution. These metrics are
submitted to test leads and project managers by preparing a project status report. It is quantified by using
calculated metrics −
 Number of test cases
 Number of test cases executed
Calculated Metrics
Calculated metrics are derived using data from base metrics. The test lead gathers these data and converts
them to more meaningful information for tracking the progress of projects at the module level, tester level,
etc.
It comprises a significant part of SDLC and empowers developers to make vital improvements in software.
Most used Metrics
Below are the types of metrics, popularly used by developers and testers
 Defect metrics: This metric allows developers to understand the various quality aspects of software,
including functionality, performance, installation stability, usability, compatibility, etc.
 Defects finding rate: It is used to identify the pattern of defects during a specific timeframe
 Defect severity: It enables the developer to understand how the defect is going to impact the quality of the
software.
 Defect cause: It is used to understand the root cause of the defect.
 Test Coverage: It defines how many test cases are assigned to the program. This metric ensures the testing
is conducted to its full completion. It further aids in checking the code flow and test functionalities.
 Defect fixing time: It determines the amount of time it takes to resolve a defect
 Test case efficiency: It tells the efficiency rate of test cases in finding defects
 Schedule adherence: Its primary motive is to figure out the time difference between the planned schedule
and the actual time of executing a schedule.
Test Metrics Life Cycle
The life cycle of test metrics consists of four stages−

 Analysis: In this stage, developers identify the required metrics and define them.
 Communicate: Once metrics are identified, developers have to explain their importance to stakeholders
and the testing team.
 Evaluation: This stage includes quantifying and verifying the data. Then testers have to use the data to
calculate the value of the metric.
 Report: Once the evaluation process is finished, the development team needs to create a report including a
detailed summary of the conclusion. Then the report is distributed among stakeholders and relevant
representatives. The stakeholders then give their feedback after reading the information carefully.

What Is Test Monitoring And Test Control?

Test Monitoring and Test Control is basically a management activity. Test Monitoring is a process of
evaluating and providing feedback on the “currently in progress” testing phase. Test Control is an activity
of guiding and taking corrective action based on some metrics or information to improve efficiency and
quality.
What is Test Monitoring?
Test Monitoring in test execution is a process in which the testing activities and testing efforts are
evaluated in order to track current progress of testing activity, finding and tracking test metrics, estimating
 the future actions based on the test metrics and providing feedback to the concerned team as well as
stakeholders about current testing process.
What is Test Control?
Test Control in test execution is a process of taking actions based on results of the test monitoring process.
In the test control phase, test activities are prioritized, test schedule is revised, test environment is
reorganized and other changes related to testing activities are made in order to improve the quality and
efficiency of future testing process.
What do we monitor?
Monitoring will allow you to make comparisons between your original plan and your progress so far. You
will be able to implement changes, where necessary, to complete the project successfully.In your project, as
the Test Manager, you should monitor the key parameters as below

How to monitor?
As your project comes to life, keep these questions in mind:

 Are you on schedule? If not, how far behind are you, and how can you catch up?
 Are you over budget?
 Are you still working toward the same project goal?
 Are you running low on resources?
 Are there warning signs of impending problems?
 Is there pressure from management to complete the project sooner?

These are just a few of the questions you should ask yourself as you monitor the progress of your project.

It is important to monitor the progress of the project so you will know if adjustments need to be made to get
it moving back in the right direction. To monitor project progress effectively, you should follow the
following steps
 best Practices in Test Monitoring and Control
 Follow the standards: One important consideration of project planning is to ensure standardization. It
means that all the project activities must follow the standard process guideline. Standardized processes,
tools, templates, and measurement values make analysis easy, facilitate easy communications, and help the
project team members understand the situation better.
 Documentation: What will happen if you do not write down any discussion or decision in a document?
You may forget them and lose many things. You should write down discussions and decisions at the
appropriate place, and establishing a formal documentation procedure for meetings. Such documentation
helps you to resolve issues of miscommunication or misunderstandings among the project team.
 Proactivity: Issues occur in all projects. The important thing is that you have to adopt a proactive approach
to solve issues and problems that arise during project execution. Such issues could be budget, scope, time,
quality, and human resources

Formula for Test Metrics

To get the percentage execution status of the test cases, the following formula can be used: Percentage
test cases executed = (No of test cases executed / Total no of test cases written) x 100
Percentage test cases executed = (No of test cases of test cases written) x 100
Similarly, it is possible to calculate for other parameters also such as test cases that were not executed,
test cases that were passed, test cases that were failed, test cases that were blocked, and so on. Below are
some of the formulas:
1. Test Case Effectiveness: Test Case Effectiveness = (Number of defects detected / Number of test
cases run) x 100
Test Case Effectiveness = (Number of defects detected / Number of test cases run) x 100
2. Passed Test Cases Percentage: Test Cases that Passed Coverage is a metric that indicates the
percentage of test cases that pass. Passed Test Cases Percentage = (Total number of tests ran / Total
number of tests executed) x 100
Passed Test Cases Percentage = (Total number of tests ran / Total executed) x 100
3. Failed Test Cases Percentage: This metric measures the proportion of all failed test cases. Failed
Test Cases Percentage = (Total number of failed test cases / Total number of tests executed) x 100
Failed Test Cases Percentage = (Total number of failed test cases / Total number) x 100
4. Blocked Test Cases Percentage: During the software testing process, this parameter determines the
percentage of test cases that are blocked. Blocked Test Cases Percentage = (Total number of blocked
tests / Total number of tests executed) x 100
Blocked Test Cases Percentage = (Total number of blocked tests / Total number) x 100
5. Fixed Defects Percentage: Using this measure, the team may determine the percentage of defects that
have been fixed. Fixed Defects Percentage = (Total number of flaws fixed / Number of defects
reported) x 100
Fixed Defects Percentage = (Total number of flaws fixed / Number of defects reported) x 100
6. Rework Effort Ratio: This measure helps to determine the rework effort ratio. Rework Effort Ratio =
(Actual rework efforts spent in that phase/ Total actual efforts spent in that phase) x 100
Rework Effort Ratio = (Actual rework efforts spent in that phase/ Total actual efforts spent in that
phase) x 100
7. Accepted Defects Percentage: This measures the percentage of defects that are accepted out of the
total accepted defects. Accepted Defects Percentage = (Defects Accepted as Valid by Dev Team / Total
Defects Reported) x 100
Accepted Defects Percentage = (Defects Accepted as Valid by Dev Team / Total Defects Reported) x
100
8. Defects Deferred Percentage: This measures the percentage of the defects that are deferred for future
release. Defects Deferred Percentage = (Defects deferred for future releases / Total Defects Reported)
x 100
Defects Deferred Percentage = (Defects deferred for future releases / Total Defects Reported) x 100

Example of Software Test Metrics Calculation

Let’s take an example to calculate test metrics:

S Data retrieved during test case

No. Testing Metric development

1 No. of requirements 5

The average number of test cases written per

2 requirement 40

Total no. of Test cases written for all

3 requirements 200

4 Total no. of Test cases executed 164

5 No. of Test cases passed 100

6 No. of Test cases failed 60

S Data retrieved during test case
No. Testing Metric development

7 No. of Test cases blocked 4

8 No. of Test cases unexecuted 36

9 Total no. of defects identified 20

10 Defects accepted as valid by the dev team 15

11 Defects deferred for future releases 5

12 Defects fixed 12

1. Percentage test cases executed = (No of test cases executed / Total

What is a Test Suite?

Test suite is a container that has a set of tests which helps testers in executing and reporting the test
execution status. It can take any of the three states namely Active, Inprogress and completed.
A Test case can be added to multiple test suites and test plans. After creating a test plan, test suites are
created which in turn can have any number of tests.
Test suites are created based on the cycle or based on the scope. It can contain any type of tests, viz -
functional or Non-Functional.
 Test Suite - Diagram:

Software testing is a continuous process that takes place throughout lifecycle of project. Test cases in an
existing test suite can often be used to test a modified program. However, if the test suite is inadequate for
retesting, new test cases may be developed and added to test suite. Thus, size of test suite groups as
software evolves. Test suite can also be reused later as software evolves. Such test suite, reuse in the form
of regression testing. Running all the test cases, however can require lot of efforts. Thus due to obsolete and
redundant data, size of test suite continues to grow unnecessarily.
Following are the reasons why use need test suite minimization
 Release date of product is near.
 Limited staff to execute all the test cases.
 Limited test equipment’s or unavailability of testing tools.
Advantages of minimization:
 Reluctant test cases will be eliminated.
 Lowers the cost by reducing test suite to minimal subset.
 Reducing size of test suite decreases the overhead of maintenance of test suite and reduces the efforts and
cost required in regression testing.
The tester’s day-to-day activities include:
1. Creating and maintaining release/project cycle/component information.
2. Creating and maintaining the test artifacts specific to each release/cycle for which we have- requirements,
test cases, etc.
3. Establishing traceability and coverage of the test assets.
4. Test execution support – test suite creation, test execution status capture, etc.
5. Metric collection/report-graph generation for analysis.
6. Bug tracking/defect management.
List of Best Test Management Tools in 2022
 #1) Zephyr Scale
 #2) PractiTest
 #3) Zephyr Squad
 #4) TestCollab
 #5) Requirements and Test Management for Jira (RTM)
 #6) XQual
 #7) Xray – Cutting Edge Test Management
 #8) TestRail
 #9) Kualitee
 #10) TestFLO for JIRA

UNIT IV BUILDING AGILITY & TOOL SUPPORT 9

Building Agility into the Testing Process- Using Agile Methods to Improve Software Testing –– Tool
Support For Testing
What is Agile Testing? Process, Methodology and Strategies
Process

The agile Testing methodology has gained a lot of importance and popularity nowadays when it comes
to software testing modern-day software. It is an intelligent way of testing complex software that bears
more efficient end results than traditional testing methods.
This is a complete guide to Agile Testing, explaining all the important aspects related to it.
 What is Agile Testing?
 Agile Testing Methodology
 Agile Testing Life-Cycle
 Agile Test Plan
 Agile Testing Strategies
 Agile Testing Quadrants
 Difference Between Agile Testing Vs Waterfall Testing
 Advantages of Agile Testing
What is Agile Testing?
Agile testing is an iterative and incremental testing process in which software requirements are gathered
gradually throughout the testing process.
Agile Testing is a part of the Agile Software development process which is carried out with the help of
customer’s involvement. The agile testing process starts at the beginning of the software development
process and there is no integration required between the development and testing processes.

As the development and testing are integrated into this process, it allows early error detection and removal
which results in less development time as well as costs. This also ensures high product quality as errors get
removed in the initial stage itself.
The basic principles on which Agile Testing is conducted are:
 Agile testing is a continuous process and it ensures smooth project progress.
 Product and business requirements are realized through continuous feedback. Feedback are provided on an
on-going basis.
 In Agile Testing, the development team, testing team, and the customers, all are involved in the testing
process.
 There is very little documentation required in Agile Testing as all the testers us a reusable checklist instead
of lengthy documents. The main focus is on the test in this process.
 The errors are resolved by the testing team as and when they occur, in the same iteration. This results in
simpler and clean code.
 Traditional testing is performed after the software is developed. However, agile testing is performed along
with the development process which helps reducing development time
 Agile Testing Methodology

TDD (Test Driven Development)

As the name suggests, the test-driven development approach starts with the test itself. It depends upon the
repetition of the development cycle.
The first step in this development cycle is to create a unit test case. The next step involves creating the code
which fits the test case so that the test case can be run. Thus, the entire code is created until the unit test
passes.
This test is usually performed with the help of automated testing tools and run on units and components of
the code.
ATDD (Acceptance Test Driven Development)
In this testing methodology, the developers, testers and the customers together build an acceptance test
from the point of view of the customers in order to understand the requirements of the customers.
The code is developed according to the acceptance test case created.
This testing method is very customer-centered, and the main aim is to get to develop a program according
to the user’s perception.
BDD (Behaviour Driven Development)
BDD is built on the same principles as TDD and ATDD. Thus, the code is developed according to the test
case created in this testing methodology too.
The main aim of this development is to focus on the identification of business needs and outcomes. The
 development should be related to a business outcome.
The steps that are followed in BDD are:
 First, describe the behavior
 Creating the test case
 Writing code according to the test case defined
 Continuing the process until the code passes the test case.
Exploratory Testing
In this software testing methodologies, the testers have complete freedom to explore with the code and
develop the most efficient software.
The testers create multiple test cases in order to explore each aspect of the software functionality. Testers
perform different tests and record the process to study it and figure out the exact flow of the software.
This type of testing is a very important part of the agile test as it helps uncover hidden risks from the
software which could not have been detected by simple testing methods.
Session-Based Testing
Session-based testing is built on the principles of exploratory testing. However, unlike exploratory
testing which is conducted spontaneously without any planning, there is some structure in session-based
testing.
This structure is provided by conducting tests during uninterrupted sessions where testers are required to
report the tests that took place during the process.
The session-based testing methodology also helps in finding hidden bugs and defects in the software.
Agile Testing Life-cycle

There are 5 phases in the Agile Testing Life-cycle:

 Agile Test Planning

The first phase is creating Agile Test Plans. In this phase, all the stakeholders: developers, testers,
customers and users, come together and plan testing schedules.

 Daily Scrums

The goals and targets of the day are set at the start of each day. It helps testers realize the status of the
testing.

 Test Agility Review

 Agility reviews are performed frequently in the development process. These are done to analyze the
progress of the development.

 Release Readiness

In this phase, testers decide which assess which features are completely developed and ready to go live and
which ones need to go back to the previous development phase.

 Impact Assessment

This phase is the feedback phase of the life cycle which helps testers set goals for the next life-cycle. The
inputs and feedback are collected from users and stakeholders to perform an impact assessment.

Advantages of Agile Testing

 saves time and money. Because the testing is carried along with the development. The errors and bugs are
fixed at the same time and no additional time s required to test the entire software, find errors and solve
them.
 There is little to no documentation required in Agile testing as testers use reusable checklists.
 Feedback is also a part of the Agile testing life-cycle and users give feedback after each process iteration.
Possible disadvantages of Agile methodology
However, no system is perfect. Improperly implemented, Agile testing can weaken team structure and
product development, preventing a viable product from ever being released. Even when properly
used, all Agile methodologies have their weaknesses. For example, exploratory testing can lack the
structure necessary to ensure a product is comprehensively tested; ATDD accounts for customer feedback,
but not for business outcomes.

Agile Testing Strategy

With dedication, each of these pitfalls can be overcome and the three powerful benefits experienced. The
first step towards successful Agile testing is determining when Agile testing should not be used. Blind
adoption of Agile testing can result in a weak, crash-prone product.

Here are a few guidelines for cases in which Agile may not be the best way to test:

1. When the project scope is crystal clear and very unlikely to change
2. When the project is governed by a single product owner or stakeholder with minimal requirements
3. When the people on your team lack the deep, broad skills necessary to perform Agile testing
4. When the customer insists on using a traditional waterfall approach to testing

Tool Support for Testing Summary

Tool Support for Testing

 Tools are used for test execution, Test data generations, and requirements management.
 Tools that help in managing the tests.

Purpose of tool
 To automate repetitive tasks and to improve efficiency
 Automate activities that require significant resources when done manually
 Automate tasks that can’t be executed manually
 Increase reliability of testing

Tool Support for Management of Testing and Test

Test Management tool- Provides interfaces for test execution, defect logging and managing requirements etc

Requirements Management Tool- Provide interfaces for storing requirements, uniquely identify requirements,

tracing requirements

Incident Management Tool- Store and Manage incident reports

Configuration Management Tool-For storage and version management of test ware and related software

Tool Support for Static Testing

Review Tools-Helps in review process, check lists, guidelines and are used to store and communicate review

comments.

Static Analysis Tools (D)-Helps both developers and testers find defects prior to dynamic testing

(including secure coding and analysis of structures and dependencies

Modeling tools-Helps to validate the models like PDM (Physical Data Model)

Tool Support for Test Specification

Test Design Tool-Used to generate test inputs or executable tests and / or test oracles from requirements,

design models or code

Test Data Preparation Tool-Manipulate DB, files or data transmissions to set up test data to be used during the

execution of tests

Tool Support for Test Execution and Logging

Test Execution Tools-To execute tests automatically, uses scripting languages and parameterization techniques

Test Harness/Unit Frame Work Tools(D)- It facilitates the testing of components or parts of a system by

simulating the environment in which that object will run, provision of mock objects such as stubs, drivers.
 Test Comparators-Compares the diff between files, DBs or test results.

Coverage Measurement Tools (D) - Measuring the % of specific types of code structures that have been exercised

Security Testing Tools- To evaluate the security characteristics of a SW like data confidentiality, integrity,

authentication, authorization, availability.

Tool Support for Performance and Monitoring

Dynamic Analysis Tool (D)-Find defects which are evident only when the SW is running such as time

dependencies and memory leaks. Use in Integration.

Stress Testing/Load Testing/Performance Testing tools- Monitor and report on how a system behaves under a

variety of simulated conditions like load, concurrent users, and relative percentage of transactions

Monitoring Tools- Continuously analyze, verify and report on usage of specific system resources and give

warnings of possible service problems

Tool Support for Specific Testing Needs

Data Quality Assessment Tool- Data is at the center of some projects such as data conversion/migration projects

and applications like data ware houses.

Benefits of Tools

 Repetitive work is reduced

 Greater consistency and repeatability
 Objective assessment
 Ease of access to information about tests or testing

Risks of Tools

 Unrealistic expectations on the tool

 Underestimating the time, cost and effort required for introduction of tool
 Underestimating the time and effort required to achieve the significant change using the tool
 Poor support from the vendor
 Risk of suspension for open source / free total project
 Over reliance on the tool
 Unforeseen such as the inability to support new platform
 Introducing a tool into an Organization

 Assessment of organizational maturity, strengths and weaknesses for an improved test process

supported by tools

 Evaluation against clear requirements and objective criteria

 A proof of concept by using a test tool during the evaluation phase to establish whether it performs effectively with
infrastructure or to identify changes needed to that infrastructure to use the tool
 Evaluation of the vendor
 Evaluation of the training needs considering the current test team's test automation skills
 Estimation of a cost benefit ratio
 Introducing a tool into an Organization Objectives
 Learn more about the tool
 Evaluate how the tool fits with the existing processes and practices and determine what need to be changed
 Decide on standard ways of using, managing, storing and maintaining the tool and the test assets
 Assess whether the benefits will be achieved at reasonable cost

Success Factors of a Tool

 Rolling out to the rest of the organization

 Adapting and improving processes to fit the use of the tool
 Providing training
 Defining usage guidelines
 Monitoring the tool usage and benefits
 Providing support for the test team for a tool
 Gathering lessons learned from all teams

UNIT V TEST AUTOMATION &TESTING THE APPLICATIONS 9

Test Automation – Software Testability – Components of a test case – A test Automation framework -
Testing Object-Oriented Software – Testing Web-Based Systems – Testing Web Applications – Functional
Web testing with Twill – Selenium – Testing a simple Web Application – Testing Mobile Smartphone
Applications
What is Test Automation?
Software Test automation makes use of specialized tools to control the execution of tests and compares the
actual results against the expected result. Usually, regression tests, which are repetitive actions, are
automated.
Testing Tools not only helps us to perform regression tests but also helps us to automate data set up
generation, product installation, GUI interaction, defect logging, etc. Automation tools are used for both
Functional and Non-Functional testing.
Criteria for Tool Selection:
For automating any application, the following parameters should be considered:
 Data driven capabilities
 Debugging and logging capabilities
 Platform independence
 Extensibility & Customizability
 E-mail Notifications
 Version control friendly
 Support unattended test runs
Types of Frameworks:
Typically, there are 4 test automation frameworks that are adopted while automating the applications:
 Data Driven Automation Framework
 Keyword Driven Automation Framework
 Modular Automation Framework
 Hybrid Automation Framework
What is Software Testability?
It is the state of software artifact, which decides the difficulty level for carrying out testing activities on that
artifact. These software artifacts may include software modules, UML models, requirements & design
documents and software application itself.

It is, basically the fundamental attribute associated with each and every software artifact that enables to
compute amount of effort required in performing testing. Lesser the testability, larger will be the efforts
whereas greater testability ensures minimal efforts.

How to measure Software's Testability?

Below given are some of the heuristics through which software testability can be determined

 Controllability: It defines the control over software and hardware behaviour and comp. Testers should be
able to control each module or layers of the software, independently. The better our control, the more
effective will be testing.
 Observability: You can't test unless & until you perceive something. You can only test, what is visible to
you. It is related to observation of states and factors affecting the output of the software.

 Availability: It defines the availability of the objects or entities, to carry out the testing. This may include
software product evolution at various stages of development, bugs in the software, access to the source
code, etc.

 Simplicity : The simplicity makes everything easier to use. Lesser efforts are required in testing the simple
software product. The simplicity of software may depend upon its functional, structural and code
simplicity.

 Stability: Lesser the changes, better will be the testing. The software product should be stable enough and
does not require frequent modification in it. It also examines that changes (if any needed), should be
controlled and communicated.

Key Benefits of Software Testability

 Provides ease to test engineers, to estimate the difficulty, in exploring the defects in the software product.

 Decides the scope of automated testing on the software product, based on its controllability.

 Increased in testability ensures the easy and early detection of bugs, thereby saves both time and cost.

 Calculates, minimizes and control the efforts needed by the testers, to perform testing.

Typical Test Case Parameters:

 Test Case ID
 Test Scenario
 Test Case Description
 Test Steps
 Prerequisite
 Test Data
 Expected Result
 Test Parameters
 Actual Result
 Environment Information
 Comments
Test Automation Frameworks
In this article
What is a Test Framework?
Types of Automated Testing Frameworks
Conclusion
Testing frameworks are an essential part of any successful automated testing process. They can reduce
maintenance costs and testing efforts and will provide a higher return on investment (ROI) for QA teams
looking to optimize their agile processes.

The goal of this article is to walk through the most common types of frameworks used today and the
benefits and disadvantages of each. For QA professionals new to automated testing, or those who need a
quick refresher, this article will provide a high-level overview of each type of framework and how they can
contribute to the success of any automated testing process.

What is a Test Framework?

Before diving into the most common types of frameworks and their benefits, let’s clarify what a test
automation framework actually is. A testing framework is a set of guidelines or rules used for creating and
designing test cases. A framework is comprised of a combination of practices and tools that are designed to
help QA professionals test more efficiently.

These guidelines could include coding standards, test-data handling methods, object repositories, processes
for storing test results, or information on how to access external resources.

While these are not mandatory rules and testers can still script or record tests without following them, using
an organized framework typically provides additional benefits that would otherwise be missed out on.

Benefits of a Test Automation Framework

Utilizing a framework for automated testing will increase a team’s test speed and efficiency, improve test
accuracy, and will reduce test maintenance costs as well as lower risks. They are essential to an efficient
automated testing process for a few key reasons:

 Improved test efficiency

 Lower maintenance costs
 Minimal manual intervention
 Maximum test coverage
 Reusability of code
 A common trend to minimize risk is to test earlier in the Test Automation Framework. By using tools such
as TestLeft, you can test in your own IDE. Download a free 30-day trial to Automate in your IDE today!

Types of Automated Testing Frameworks

There are six common types of test automation frameworks, each with their own architecture and differing
benefits and disadvantages. When building out a test plan, it’s important to choose the framework that is
right for you.

1. Linear Automation Framework

2. Modular Based Testing Framework
3. Library Architecture Testing Framework
4. Data-Driven Framework
5. Keyword-Driven Framework
6. Hybrid Testing Framework

Linear Automation Framework

With a linear test automation framework, also referred to as a record-and-playback framework, testers don’t
need to write code to create functions and the steps are written in a sequential order. In this process, the
tester records each step such as navigation, user input, or checkpoints, and then plays the script back
automatically to conduct the test.

Advantages of a linear framework:

 There is no need to write custom code, so expertise in test automation is not necessary.
 This is one of the fastest ways to generate test scripts since they can be easily recorded in a minimal
amount of time.
 The test workflow is easier to understand for any party involved in testing since the scripts are laid out in a
sequential manner.
 This is also the easiest way to get up and running with automated testing, especially with a new tool. Most
automated testing tools today will provide record-and-playback features, so you also won’t need to plan
extensively with this framework.
Disadvantages:

 The scripts developed using this framework aren’t reusable. The data is hardcoded into the test script,
meaning the test cases cannot be re-run with multiple sets and will need to be modified if the data is altered.
 Maintenance is considered a hassle because any changes to the application will require a lot of rework. This
model is not particularly scalable as the scope of testing expands.
Modular Based Testing Framework
Implementing a modular framework will require testers to divide the application under test into separate
units, functions, or sections, each of which will be tested in isolation. After breaking down the application
 into individual modules, a test script is created for each part and then combined to build larger tests in a
hierarchical fashion. These larger sets of tests will begin to represent various test cases.

A key strategy in using the modular framework is to build an abstraction layer, so that any changes made in
individual sections won’t affect the overarching module.

Advantages of a Modular Framework:

 If any changes are made to the application, only the module and it’s associated individual test script will
needs to be fixed, meaning you won’t have to tinker with the rest of the application and can leave it
untouched.
 Creating test cases takes less effort because test scripts for different modules can be reused.
Disadvantages of a Modular Framework:

 Data is still hard-coded into the test script since the tests are executed separately, so you can’t use multiple
data sets.
 Programming knowledge is required to set up the framework.
Library Architecture Testing Framework
The library architecture framework for automated testing is based on the modular framework, but has some
additional benefits. Instead of dividing the application under test into the various scripts that need to be run,
similar tasks within the scripts are identified and later grouped by function, so the application is ultimately
broken down by common objectives. These functions are kept in a library which can be called upon by the
test scripts whenever needed.

Advantages of a LIbrary Architecture Testing Framework:

 Similar to the modular framework, utilizing this architecture will lead to a high level of modularization,
which makes test maintenance and scalability easier and more cost effective.
 This framework has a higher degree of reusability because there is a library of common functions that can
be used by multiple test scripts.
Disadvantages:

 Test data is still hard coded into the script. Therefore, any changes to the data will require changes to the
scripts.
 Technical expertise is needed to write and analyze the common functions within the test scripts.
 Test scripts take more time to develop.
Data-Driven Framework
Using a data-driven framework separates the test data from script logic, meaning testers can store data
externally. Very frequently, testers find themselves in a situation where they need to test the same feature
or function of an application multiple times with different sets of data. In these instances, it’s critical that
the test data not be hard-coded in the script itself, which is what happens with a Linear or Modular-based
testing framework.
 Setting up a data-driven test framework will allow the tester to store and pass the input/ output parameters
to test scripts from an external data source, such as Excel Spreadsheets, Text Files, CSV files, SQL Tables,
or ODBC repositories.

The test scripts are connected to the external data source and told to read and populate the necessary data
when needed.

Advantages of a Data-Driven Framework:

 Tests can be executed with multiple data sets.

 Multiple scenarios can be tested quickly by varying the data, thereby reducing the number of scripts
needed.
 Hard-coding data can be avoided so any changes to the test scripts do not affect the data being used and
vice versa.
 You’ll save time by executing more tests faster.
Disadvantages

 You’ll need a highly-experienced tester who is proficient in various programming languages to properly
utilize this framework design. They will need to identify and format the external data sources and to write
code (create functions) that connect the tests to those external data sources seamlessly.
 Setting up a data-driven framework takes a significant amount of time.
Keyword-Driven Framework
In a keyword-driven framework, each function of the application under test is laid out in a table with a
series of instructions in consecutive order for each test that needs to be run. In a similar fashion to the data-
driven framework, the test data and script logic are separated in a keyword-driven framework, but this
approach takes it a step further.

With this approach, keywords are also stored in an external data table (hence the name), making them
independent from the automated testing tool being used to execute the tests. Keywords are the part of a
script representing the various actions being performed to test the GUI of an application. These can be
labeled as simply as ‘click,’ or ‘login,’ or with complex labels like ‘clicklink,’ or ‘verifylink.’

In the table, keywords are stored in a step-by-step fashion with an associated object, or the part of the UI
that the action is being performed on. For this approach to work properly, a shared object repository is
needed to map the objects to their associated actions.

Keyword Table:
Step Number Description Keyword Object Action
Step 1 Click user portal link on homepage clicklink Login Button
Step 2 Enter user name Inputdata Login Username
Step 3 Enter password Inputdata Login password
Step 4 Verify user log in information verifylogin
 Step 5 Log in to the application login Submit Button
Once the table has been set up, all the testers have to do is write the code that will prompt the necessary
action based on the keywords. When the test is run, the test data is read and pointed towards the
corresponding keyword which then executes the relevant script.

Advantages of Keyword-Driven Frameworks:

 Minimal scripting knowledge is needed.

 A single keyword can be used across multiple test scripts, so the code is reusable.
 Test scripts can be built independent of the application under test.

Disadvantages:

 The initial cost of setting up the framework is high. It is time-consuming and complex. The keywords need
to be defined and the object repositories / libraries need to be set up.
 You need an employee with good test automation skills.
 Keywords can be a hassle to maintain when scaling a test operation. You will need to continue building out
the repositories and keyword tables.
Hybrid Test Automation Framework
As with most testing processes today, automated testing frameworks have started to become integrated and
overlap with one another. As the name suggests, a hybrid framework is a combination of any of the
previously mentioned frameworks set up to leverage the advantages of some and mitigate the weaknesses
of others.

Every application is different, and so should the processes used to test them. As more teams move to an
agile model, setting up a flexible framework for automated testing is crucial. A hybrid framework can be
more easily adapted to get the best test results.

Object Oriented Testing in Software Testing

Software typically undergoes many levels of testing, from unit testing to system or acceptance testing.
Typically, in-unit testing, small “units”, or modules of the software, are tested separately with focus on
testing the code of that module. In higher, order testing (e.g, acceptance testing), the entire system (or a
subsystem) is tested with the focus on testing the functionality or external behavior of the system.
As information systems are becoming more complex, the object-oriented paradigm is gaining popularity
because of its benefits in analysis, design, and coding. Conventional testing methods cannot be applied for
testing classes because of problems involved in testing classes, abstract classes, inheritance, dynamic
binding, message, passing, polymorphism, concurrency, etc.
Testing classes is a fundamentally different problem than testing functions. A function (or a procedure) has
 a clearly defined input-output behavior, while a class does not have an input-output behavior specification.
We can test a method of a class using approaches for testing functions, but we cannot test the class using
these
approaches.
According to Davis the dependencies occurring in conventional systems are:
 Data dependencies between variables
 Calling dependencies between modules
 Functional dependencies between a module and the variable it computes
 Definitional dependencies between a variable and its types.
But in Object-Oriented systems there are following additional dependencies:
 Class to class dependencies
 Class to method dependencies
 Class to message dependencies
 Class to variable dependencies
 Method to variable dependencies
 Method to message dependencies
 Method to method dependencies
Issues in Testing Classes:
Additional testing techniques are, therefore, required to test these dependencies. Another issue of interest is
that it is not possible to test the class dynamically, only its instances i.e, objects can be tested. Similarly, the
concept of inheritance opens various issues e.g., if changes are made to a parent class or superclass, in a
larger system of a class it will be difficult to test subclasses individually and isolate the error to one class.
In object-oriented programs, control flow is characterized by message passing among objects, and the
control flow switches from one object to another by inter-object communication. Consequently, there is no
control flow within a class like functions. This lack of sequential control flow within a class requires
different approaches for testing. Furthermore, in a function, arguments passed to the function with global
data determine the path of execution within the procedure. But, in an object, the state associated with the
object also influences the path of execution, and methods of a class can communicate among themselves
through this state because this state is persistent across invocations of methods. Hence, for testing objects,
the state of an object has to play an important role.
Techniques of object-oriented testing are as follows:
1. Fault Based Testing:
This type of checking permits for coming up with test cases supported the consumer specification or the
code or both. It tries to identify possible faults (areas of design or code that may lead to errors.). For all of
these faults, a test case is developed to “flush” the errors out. These tests also force each time of code to be
executed.
This method of testing does not find all types of errors. However, incorrect specification and interface
errors can be missed. These types of errors can be uncovered by function testing in the traditional testing
model. In the object-oriented model, interaction errors can be uncovered by scenario-based testing. This
form of Object oriented-testing can only test against the client’s specifications, so interface errors are still
missed.
2. Class Testing Based on Method Testing:
This approach is the simplest approach to test classes. Each method of the class performs a well defined
cohesive function and can, therefore, be related to unit testing of the traditional testing techniques.
Therefore all the methods of a class can be involved at least once to test the class.
3. Random Testing:
It is supported by developing a random test sequence that tries the minimum variety of operations typical to
the behavior of the categories
4. Partition Testing:
This methodology categorizes the inputs and outputs of a category so as to check them severely. This
minimizes the number of cases that have to be designed.
5. Scenario-based Testing:
It primarily involves capturing the user actions then stimulating them to similar actions throughout the test.
These tests tend to search out interaction form of error.

Software Testing – Web Based Testing

Web testing is a software testing technique to test web applications or websites for finding errors and bugs.
A web application must be tested properly before it goes to the end-users. Also, testing a web application
does not only means finding common bugs or errors but also testing the quality-related risks associated
with the application. Software Testing should be done with proper tools and resources and should be done
effectively. We should know the architecture and key areas of a web application to effectively plan and
execute the testing.
Testing a web application is very common while testing any other application like testing functionality,
configuration, or compatibility, etc. Testing a web application includes the analysis of the web fault
compared to the general software faults. Web applications are required to be tested on different browsers
and platforms so that we can identify the areas that need special focus while testing a web application.
Types of Web Testing:
Basically, there are 4 types of web-based testing that are available and all four of them are discussed below:
 Static Website Testing: A static website is a type of website in which the content shown or displayed is
exactly the same as it is stored in the server. This type of website has great UI but does not have any
dynamic feature that a user or visitor can use. In static testing, we generally focus on testing things like UI
as it is the most important part of a static website. We check things font size, color, spacing, etc. testing
also includes checking the contact us form, verifying URLs or links that are used in the website, etc.
 Dynamic Website Testing: A dynamic website is a type of website that consists of both frontend i.e, UI,
and the backend of the website like a database, etc. This type of website gets updated or change regularly as
per the user’s requirements. In this website, there are a lot of functionalities involved like what a button
will do if it is pressed, are error messages are shown properly at their defined time, etc. We check if the
backend is working properly or not, like does entering the data or information in the GUI or frontend gets
updated in the databases or not.
 E-Commerce Website Testing: An e-commerce website is very difficult in maintaining as it consists of
different pages and functionalities, etc. In this testing, the tester or developer has to check various things
like checking if the shopping cart is working as per the requirements or not, are user registration or login
functionality is also working properly or not, etc. The most important thing in this testing is that does a user
 can successfully do payment or not and if the website is secured. And there are a lot of things that a tester
needs to test apart from the given things.
 Mobile-Based Web Testing: In this testing, the developer or tester basically checks the website
compatibility on different devices and generally on mobile devices because many of the users open the
website on their mobile devices. So, keeping that thing in mind, we must check that the site is responsive
on all devices or platforms.
Steps in Software Testing
There is a total of 11 steps in software testing. You can read all of them from the article “General Steps
of Software Testing Process”. In web-based testing, various areas have to be tested for finding the
potential errors and bugs, and steps for testing a web app are given below:
 App Functionality: In web-based testing, we have to check the specified functionality, features, and
operational behavior of a web application to ensure they correspond to its specifications. For example,
Testing all the mandatory fields, Testing the asterisk sign should display for all the mandatory fields,
Testing the system should not display the error message for optional fields, and also links like external
linking, internal linking, anchor links, and mailing links should be checked properly and checked if
there’s any damaged link, so that should be removed. We can do testing with the help of Functional
Testing in which we test the app’s functional requirements and specifications.

 Usability: While testing usability, the developers face issues with scalability and interactivity. As
different numbers of users will be using the website, it is the responsibility of developers to make a group
for testing the application across different browsers by using different hardware. For example, Whenever
the user browses an online shopping website, several questions may come to his/her mind like, checking
the credibility of the website, testing whether the shipping charges are applicable, etc.

 Browser Compatibility: For checking the compatibility of the website to work the same in different
browsers we test the web application to check whether the content that is on the website is being
displayed correctly across all the browsers or not.

 Security: Security plays an important role in every website that is available on the internet. As a part of
security, the testers check things like testing the unauthorized access to secure pages should not be
permitted, files that are confined to the users should not be downloadable without the proper access.

 Load Issues: We perform this testing to check the behavior of the system under a specific load so that
we can measure some important transactions and the load on the database, the application server, etc. are
also monitored.

 Storage and Database: Testing the storage or the database of any web application is also an important
component and we must sure that the database is properly tested. We test things like finding errors while
executing any DB queries, checking the response time of a\the query, testing whether the data retrieved
from the database is correctly shown on the website or not.

What is Web Application Testing?

Web application testing, a software testing technique exclusively adopted to test the applications that are
hosted on web in which the application interfaces and other functionalities are tested.
 Web Application Testing - Techniques:
1. Functionality Testing - The below are some of the checks that are performed but not limited to the
below list:
 Verify there is no dead page or invalid redirects.
 First check all the validations on each field.
 Wrong inputs to perform negative testing.
 Verify the workflow of the system.
 Verify the data integrity.
2. Usability testing - To verify how the application is easy to use with.
 Test the navigation and controls.
 Content checking.
 Check for user intuition.
3. Interface testing - Performed to verify the interface and the dataflow from one system to other.
4. Compatibility testing- Compatibility testing is performed based on the context of the application.
 Browser compatibility
 Operating system compatibility
 Compatible to various devices like notebook, mobile, etc.
5. Performance testing - Performed to verify the server response time and throughput under various load
conditions.
 Load testing - It is the simplest form of testing conducted to understand the behaviour of the system under
a specific load. Load testing will result in measuring important business critical transactions and load on the
database, application server, etc. are also monitored.
 Stress testing - It is performed to find the upper limit capacity of the system and also to determine how the
system performs if the current load goes well above the expected maximum.
 Soak testing - Soak Testing also known as endurance testing, is performed to determine the system
parameters under continuous expected load. During soak tests the parameters such as memory utilization is
monitored to detect memory leaks or other performance issues. The main aim is to discover the system's
performance under sustained use.
 Spike testing - Spike testing is performed by increasing the number of users suddenly by a very large
amount and measuring the performance of the system. The main aim is to determine whether the system
will be able to sustain the work load.
6. Security testing - Performed to verify if the application is secured on web as data theft and unauthorized
access are more common issues and below are some of the techniques to verify the security level of the
system.
 Injection
 Broken Authentication and Session Management
 Cross-Site Scripting (XSS)
 Insecure Direct Object References
 Security Misconfiguration
 Sensitive Data Exposure
 Missing Function Level Access Control
 Cross-Site Request Forgery (CSRF)
 Using Components with Known Vulnerabilities
 Unvalidated Redirects and Forwards
What is Selenium?
Selenium is a free (open-source) automated testing framework used to validate web applications across
different browsers and platforms. You can use multiple programming languages like Java, C#, Python etc
to create Selenium Test Scripts. Testing done using the Selenium testing tool is usually referred to as
Selenium Testing.
Selenium Software is not just a single tool but a suite of software, each piece catering to different Selenium
QA testing needs of an organization. Here is the list of tools

 Selenium Integrated Development Environment (IDE)

 Selenium Remote Control (RC)
 WebDriver
 Selenium Grid

Introduction to Selenium
Brief Introduction Selenium IDE
Selenium Integrated Development Environment (IDE) is the simplest framework in the Selenium suite
and is the easiest one to learn. It is a Firefox plugin that you can install as easily as you can with other
plugins. However, because of its simplicity, Selenium IDE should only be used as a prototyping tool. If
you want to create more advanced test cases, you will need to use either Selenium RC or WebDriver.
 Brief Introduction Selenium Remote Control (Selenium RC)
Selenium RC was the flagship testing framework of the whole Selenium project for a long time. This is
the first automated web testing tool that allowed users to use a programming language they prefer. As
of version 2.25.0, RC can support the following programming languages:

 Java
 C#
 PHP
 Python
 Perl
 Ruby

Brief Introduction WebDriver

The WebDriver proves itself to be better than both Selenium IDE and Selenium RC in many aspects. It
implements a more modern and stable approach in automating the browser’s actions. WebDriver, unlike
 Selenium RC, does not rely on JavaScript for Selenium Automation Testing. It controls the browser by
directly communicating with it.

The supported languages are the same as those in Selenium RC.

 Java
 C#
 PHP
 Python
 Perl
 Ruby

Selenium Grid
Selenium Grid is a tool used together with Selenium RC to run parallel tests across different machines
and different browsers all at the same time. Parallel execution means running multiple tests at once.

Features:

 Enables simultaneous running of tests in multiple browsers and environments.

 Saves time enormously.
 Utilizes the hub-and-nodes concept. The hub acts as a central source of Selenium commands to each node
connected to it.

How to test web applications

Creating a top-notch web application requires a lot of testing which, if performed manually, can be tedious
and time-consuming. For this reason, many QA teams rely on automation to create fast, efficient, and
reliable test cases for their web applications.

Test automation offloads these routine and repetitive testing tasks from humans to machines. The tests
compare actual outcomes with predicted outcomes. This approach can help find bugs in specific operations
and simple-use cases (e.g. logging in, creating a new account, doing password resets).
By automating web application tests, testers are able to save time and effort on monotonous tasks.
Automated tests can be run continuously or scheduled at intervals. This offloads tester from time-
consuming tasks, and they can focus on exploratory testing or other tests that require a human perspective.

Test automation is the use of software (separate from the software under test) to control the execution of
tests. It lets software robots perform repetitive tasks and emulate end-user interaction with the system
under test, in order to increase the range, depth, and reliability of one’s quality assurance efforts.

Still, automation in not a plug-and-play system requiring no human intervention. Effective automation
requires testers to have a thorough knowledge of the software under test, as well as an “automation first”
mindset.

What can you automate in web apps?

There are different test cases that you can automate depending on your business objectives.

More components of the life cycle are being automated, but you should choose those that best suit the
capabilities of your testing team.

These are the types of tests that you can automate for web applications:

Functional Testing

Functional testing is used to ensure that functionality of the software work as intended for an end-user.

A single end-user can make the whole system crash in minutes, even after unit, integration, and
performance tests have passed. This usually happens because the user does something the developers did
not expect.
Automating tests ensures that a web application is working as intended. However, functional test
automation cannot fully replace the human perspective. That’s why you should always supplement your
automated test cases with exploratory testing to provide a positive user experience.
If you want to dig deeper into automated functional UI testing, you can get immediate access to this
whitepaper on Functional UI testing: An introduction to codeless test automation.

Regression testing

Regression testing can be described as “repeated functional testing”. It is used to make sure that a
software’s functionality continues to work after parts of it have been modified with new code or
configurations. For instance, when new features are built, regression testing ensures that old features of the
software continue to work as intended.

Automated regression testing uses computer-based tools and techniques to test software that has been
changed or updated.

When regression testing is automated, you can quickly and reliably run through simple scenarios and check
a variety of changes to get fast feedback. This, in turn, frees up time for testers to focus on product
improvements or conduct manual exploration into more unusual cases that require special attention.

To become an expert in automated regression testing, download our whitepaper: How to do regression
testing (faster) in agile teams

Cross-browser testing

Cross-browser testing ensures that your web application is performing as expected across different
browsers, both on desktop and mobile.

Browsers tend to be updated fairly frequently, meaning that by the time you are ready to deploy your
application, it might not work as intended in your target browsers.

Users might have the latest browser version while using an old operating system. By automating cross-
browser testing, you make sure that incompatibilities like the one just mentioned are found even before
they reach an end-user.
 Performance testing

Performance testing, such as stress and load testing, ensures that a web application can endure extended
periods of activity or peak user loads. Reaching the necessary stress conditions or load level wouldn’t be
feasible if done manually, therefore automation is key in proving that your application can perform in any
situation.

Web app test automation best practices

Before you start automating your web application tests, make sure you draft a test automation strategy to
keep you on track. Things to keep in mind before you start automation are:

1. What are the specific requirements of your web application?

2. What do you need to automate?
3. Which test automation tool best suits your requirements and goals?
4. How much maintenance will automation require?

Mobile Web application, in fact, is the website opened in the gadget (smartphone or tablet) with the help
of the mobile browser.

Some merits of the Mobile Web Apps:

 Easy development.
 Easy access.
 Easy update.
 Mobile Web App requires no installation.

Some demerits of the Mobile Web Apps:

 No offline capabilities support.

 Limited functionality in the comparison with Hybrid and Native Apps. (no access to the file system and
local resources).
 Problems with redistribution: Google Play and App Store don’t support redistribution of the Mobile
Web Apps.

Native App is the application, which has been developed specifically for one platform (Android, iOS,
Tizen, Windows 10 M0bile, BlackBerry).

Some merits of the Native Apps:

 Native app works offline.

 It can use all features of its device.
 Advanced user experience.
 Push notifications can be used for users alert.

Some demerits of the Native Apps:

 Native Apps creation is expensive in comparison to the Mobile Web apps.
 It requires high costs for the maintenance.

Hybrid App is the mix of the Native App and Mobile Web App. It can be defined like mobile website
content exposition in the application format.

Some merits of the Hybrid Apps:

 More cost effective in comparison to the Native App.

 Easy distribution.
 Embedded browser.
 Device features.

Some demerits of the Hybrid Apps:

 It works not so fast as Native App.

 Graphics are less accustomed to the OS in comparison to Native App.

Mobile site testing strategy key points

Now, we can think about our testing strategy. Let’s consider the main points and challenges we should face
to.

Devices selection

There is no doubt, that the real device is the best decision if you want to test mobile application. Testing on
a real device always gives you the highest accuracy of results.

In fact, this is really not easy to choose the most appropriate device. Anyway, here are some actions you
should do while selecting device for the mobile testing:

 Make the analysis to define the most popular and used gadgets in the market.
 Choose devices with different OS.
 Choose devices with different screen resolutions.
 Pay attention to the next factors: compatibility, memory size, connectivity etc.

As it was mentioned before you have lots of advantages for testing mobile apps on the real devices:

 High accuracy of the testing result.

 Simple bug replication.
 The points like battery drainage, geolocation, push notifications, devices built-in sensors are easy for
testing.
 Ability to test incoming interrupts (calls, SMS).
 Ability to test mobile application in the real environment and conditions.
 No false positives.

And also some disadvantages:

 A huge number of the often used devices.

 Additional expenses for the maintenance of the devices.
 Limited access to the devices often used in the foreign countries.

As you can see testing on the real devices is the good decision, but also it has some limitations. You should
overcome them to make mobile apps testing process real effective.

Emulators or simulators?

There is no difficult to guess, that they are special tools which emulate/simulate functionality and behavior
of the mobile devices.

“Emulator” and “simulator meanings are often confused. Despite theirs almost similars pronunciation, they
have no equal meaning.

In fact, an emulator is the original device replacement. Though you can run soft and apps on your gadget,
you have no ability to modify them.

The simulator doesn’t replicate device’s hardware, but you have an ability to set up the similar environment
as the original device’s OS.

So, it is better to use mobile simulators to test mobile application. Emulators are more appropriate for the
mobile site testing.

Here, you can read more about emulators and simulators.

Some advantages of using the simulators to test mobile application:

 Easy setup.
 Fast working.
 Helps to verify and explore the behavior of your mobile app.
 Cost effective.

Some disadvantages of using the simulators to test mobile application:

 Device hardware is not taken into the consideration.

 False positives are possible.
 Incomplete data of the simulation results, which makes some difficulties for the complete analysis of the
test results.

Cloud-based testing of the mobile application

 Testing mobile applications with cloud-based tools seems to be the optimal choice. It can help you to
overcome disadvantages of the real devices and simulators.

The main advantages of this approach:

 Easy availability.
 An ability to run mobile devices on multiple systems and networks.
 An ability not only to test, but also update and manage apps in the cloud.
 Cost effective.
 High scalability.
 The same script can be run on several devices in parallel.

Some weak points of the cloud mobile testing:

 Less of the control.

 No so high level of the security.
 Dependence of the Internet connection.

Some useful cloud-based tools, which can help you to test mobile application: Xamarin Test
Cloud, Perfecto Mobile Continuous Quality Lab, Keynote Mobile Testing. Here you can read more
about mobile testing tools.

Mobile manual and automated testing

Nowadays many specialists support the opinion that manual testing is going to die. Sure, it is not true. Of
course, we can not do without test automation, but there also situations when manual testing is preferable.

Some merits of the manual mobile application testing:

 It is more cost effective in the short-termed period.

 Manual testing is more flexible.
 Better simulation of user actions.

Some demerits of the manual mobile application testing:

 Manual test cases are hard to be reused.

 Less effective of execution certain and constant task.
 Test running process is slow.
 Some kinds of test cases couldn’t be executed manually ( load testing).

Some advantages of the app automation testing:

 Test running process is quite faster.

 Cost effective in the long-termed period.
 Automated test cases are easy to be reused.
 The only decision for some kind of testing (perfomance testing).
 Test results are easy to be shared.
 Some disadvantages of the app automation testing:

 Automated mobile testing tools have some limitations.

 Time-consuming process.
 Automated testing has less efficiency in the defining user-friendliness or positive customer experience.

As you can see you should make different decisions creating your strategy for the mobile testing. Of
course, there are no univocal answers on them.

The combination of different approaches seems to be the optimal way. For example, you can use simulators
in the earliest stages of your testing process. But is better to use real devices (physical or cloud-based) in
the final stages. Automated testing is preferable for the load and regression testing. But manual mobile
testing tools are better to be used for usability and exploratory testing.

Mobile application testing stages

So, let’s start to consider the main stages of the mobile app testing process. They more mostly similar to the
website testing stages. Mostly, but not quite similar. As you have read before, there are some basic
differences between mobile and desktop applications. Therefore, we need to pass some additional stages
and make some additional verifications.

main differences between mobile and desktop apps:

 The mobile device is a system, that has not powerful stuffing. So, it can not work as a personal computer.
 The mobile application testing in provided on handsets ( Apple, Samsung, Nokia, etc.), while the desktop
app is tested on a central processor.
 Mobile devices screens variety, their extensions, and colors. Mobile phone screen size is smaller than
desktop ones.
 Making and receiving calls is the main task of the phone, that is why the application should not interfere
with this major function.
 A wide variety of specific operating systems and component configurations: Android, iOS, BlackBerry
etc.
 Mobile phone OS quickly becomes obsolete. In addition, there is a limit to updating their OS.
 Mobile devices use network connections ( 3G, 4G, Wi-Fi), desktop use broadband connection or Wi-Fi.
 Mobile devices constantly search the network. That is why you should test the application at different data
rates.
 Tools, which are good for the desktop apps testing, are not fully suitable for the mobile application testing.
 Mobile applications must support multiple input channels (keyboard, voice, gestures, etc.), multimedia
technologies and other features that increase their usability.