Lecture 1: Software Testing

What is Testing
Testing is the process of evaluating a system or its component(s) with the intent to find
whether it satisfies the specified requirements or not.
Testing is executing a system in order to identify any gaps, errors, or missing requirements
in contrary to the actual requirements.
Testing is the process of evaluating a system or its component(s) with the intent to find
whether it satisfies the specified requirements or not. In simple words, testing is executing a
system in order to identify any gaps, errors, or missing requirements in contrary to the actual
requirements.

According to ANSI/IEEE 1059 standard, Testing can be defined as - A process of analyzing a

software item to detect the differences between existing and required conditions (that is
defects/errors/bugs) and to evaluate the features of the software item

Testing, Quality Assurance and Quality Control

Most people get confused when it comes to pin down the differences among Quality
Assurance, Quality Control, and Testing. Although they are interrelated and to some extent,
they can be considered as same activities, but there exist distinguishing points that set them
apart. The following table lists the points that differentiate QA, QC, and Testing.

Quality Assurance Quality Control Testing

QA includes activities that ensure the It includes activities that ensure It includes activities that
implementation of processes, the verification of a developed ensure the identification
procedures and standards in context to software with respect to of bugs/error/defects in a
verification of developed software and documented (or not in some cases) software.
intended requirements. requirements.

Focuses on processes and procedures Focuses on actual testing by Focuses on actual

rather than conducting actual testing on executing the software with an testing.
the system. aim to identify bug/defect through
implementation of procedures and
process.

Process-oriented activities. Product-oriented activities. Product-oriented

activities.

Preventive activities. It is a corrective process. It is a preventive

process.

It is a subset of Software Test Life QC can be considered as the Testing is the subset of
Cycle (STLC). subset of Quality Assurance. Quality Control.

Why to Learn Software Testing?

In the IT industry, large companies have a team with responsibilities to evaluate the
developed software in context of the given requirements. Moreover, developers also conduct
testing which is called Unit Testing. In most cases, the following professionals are involved
in testing a system within their respective capacities −

Software Tester
 Software Developer
Project Lead/Manager
End User
Different companies have different designations for people who test the software on the
basis of their experience and knowledge such as Software Tester, Software Quality
Assurance Engineer, QA Analyst, etc.

Applications of Software Testing

Cost Effective Development - Early testing saves both time and cost in many
aspects, however reducing the cost without testing may result in improper design of a
software application rendering the product useless.
Product Improvement - During the SDLC phases, testing is never a time-consuming
process. However diagnosing and fixing the errors identified during proper testing is a time-
consuming but productive activity.
Test Automation - Test Automation reduces the testing time, but it is not possible to
start test automation at any time during software development. Test automaton should be
started when the software has been manually tested and is stable to some extent. Moreover,
test automation can never be used if requirements keep changing.
Quality Check - Software testing helps in determining following set of properties of
any software such as
-
Functionality o
- Reliability
o
- Usability
- Efficiency
-
Maintainability
- Portability
Who does Testing?

It depends on the process and the associated stakeholders of the project(s). In the IT industry,
large companies have a team with responsibilities to evaluate the developed software in
context of the given requirements. Moreover, developers also conduct testing which is called
Unit Testing. In most cases, the following professionals are involved in testing a system
within their respective capacities −

Software Tester
Software Developer
Project Lead/Manager
End User
Different companies have different designations for people who test the software on the
basis of their experience and knowledge such as Software Tester, Software Quality
Assurance Engineer, QA Analyst, etc.
It is not possible to test the software at any time during its cycle. The next two sections state
when testing should be started and when to end it during the SDLC.

When to Start Testing?

An early start to testing reduces the cost and time to rework and produce error-free software
that is delivered to the client. However in Software Development Life Cycle (SDLC), testing
can be started from the Requirements Gathering phase and continued till the deployment of
the software.
It also depends on the development model that is being used. For example, in the Waterfall
model, formal testing is conducted in the testing phase; but in the incremental model, testing
is performed at the end of every increment/iteration and the whole application is tested at the
end.
Testing is done in different forms at every phase of SDLC −
During the requirement gathering phase, the analysis and verification of requirements
are also considered as testing.
Reviewing the design in the design phase with the intent to improve the design is also
considered as testing.
Testing performed by a developer on completion of the code is also categorized as
testing.
When to stop Testing:
It is difficult to determine when to stop testing, as testing is a never-ending process and no
one can claim that software is 100% tested. The following aspects are to be considered for
stopping the testing process −
Testing Deadlines
Completion of test case execution
Completion of functional and code coverage to a certain point
Bug rate falls below a certain level and no high-priority bugs are identified
Management decision
Roles and Responsibilities of a Software Tester
Test lead/manager: A test lead is responsible for:

Defining the testing activities for subordinates – testers or test engineers.

All responsibilities of test planning.
To check if the team has all the necessary resources to execute the testing activities.
To check if testing is going hand in hand with the software development in all
phases.
Prepare the status report of testing activities.
Required Interactions with customers.
Updating project manager regularly about the progress of testing activities.

Test engineers/QA testers/QC testers are responsible for:

To read all the documents and understand what needs to be tested.

Based on the information procured in the above step decide how it is to be tested.
Inform the test lead about what all resources will be required for software testing.
Develop test cases and prioritize testing activities.
Execute all the test case and report defects, define severity and priority for each
defect.
Carry out regression testing every time when changes are made to the code to fix
defects.

Overview of Software Engineering Team

How a software application shapes up during the development process entirely depends on
the how the software engineering team organizes work and implements various
methodologies. For an application to develop properly, it is important that all processes
incorporated during the software development are stable and sustainable. Many times
developers come under pressure as the delivery date approaches closer this often affects the
quality of the software. Rushing through the processes to finish the project on time will only
produce a software application which has no or minimal use for the customers. Hence, work
organization and planning is important and sticking to the plan is very important. The
project manager should ensure that there are no obstacles in the development process and if
at all there is an issue it must be resolved with immediate attention.

Overview of Software Testing Team

How soon and how well you can achieve your testing goals depends solely on the
capabilities of the testing team. Within the testing team itself it is important to have the
correct blend of testers who can efficiently work together to achieve the common testing
goals. While forming a team for testing, it is important to ensure that the members of the
team jointly have a combination of all the relevant domain knowledge that is required to
test the software under development.

It is very important to ensure that the software testing team has a proper structure. The
hierarchy and roles should be clearly defined and responsibilities too should be well defined
and properly distributed amongst the team members. When the team is well organized the
work can be handled well. If every team member knows what duties he or she has to
perform then they will be able to finish their duties as required well within the time limit. It
is important to keep track of the testers’ performance. It is very important to check what
kind of defects the tester is able to uncover and what kind of detects he tends to miss. This
will give you a fair idea about how serious your team is about the work.

All the team members should work together to prepare a document that clearly defines the
roles and responsibilities of all the team members. Once the document is prepared the role of
each member should be communicated clearly to everyone. Once the team members are
clear about who is going to handle which area of the project, then in case of any issue it will
be easy to determine who needs to be contacted.

Each member of the team should be provided with the necessary documents that provide
information on how the task would be organized, what approach will be followed, how
things are scheduled, how many hours have been allocated to each member and all details
related to applicable standards and quality processes.
Software Tester Role

A Software tester (software test engineer) should be capable of designing test suites and
should have the ability to understand usability issues. Such a tester is expected to have sound
knowledge of software test design and test execution methodologies. It is very important for
a software tester to have great communication skills so that he can interact with the
development team efficiently. The roles and responsibilities for a usability software tester are
as follows:

A Software Tester is responsible for designing testing scenarios for usability testing.
He is responsible for conducting the testing, thereafter analyze the results and then
submit his observations to the development team.
He may have to interact with the clients to better understand the product
requirements or in case the design requires any kind of modifications.
Software Testers are often responsible for creating test-product documentation and
also has to participate in testing related walk through.

A software tester has different sets of roles and responsibilities. He should have in depth
knowledge about software testing. He should have a good understanding about the system
which means technical (GUI or non-GUI human interactions) as well as functional product
aspects. In order to create test cases it is important that the software tester is aware of
various testing techniques and which approach is best for a particular system. He should
know what are various phases of software testing and how testing should be carried out in
each phase. The responsibilities of the software tester include:

Creation of test designs, test processes, test cases and test data.
Carry out testing as per the defined procedures.
Participate in walkthroughs of testing procedures.
Prepare all reports related to software testing carried out.
Ensure that all tested related work is carried out as per the defined standards and
procedures.
Software Test Manager Role
Managing or leading a test team is not an easy job. The company expects the test manager to
know testing methodologies in detail. A test manager has to take very important decisions
regarding the testing environment that is required, how information flow would be managed
and how testing procedure would go hand in hand with development. He should have sound
knowledge about both manual as well as automated testing so that he can decide how both
the methodologies can be put together to test the software. A test manager should have sound
knowledge about the business area and the client’s requirement, based on that he should be
able to design a test strategy, test goal and objectives. He should be good at project planning,
task and people coordination, and he should be familiar with various types of testing tools.
Many people get confused between the roles and responsibilities of a test manager and test
lead.For a clarification, a test lead is supposed to have a rich technical experience which
includes, programming, handling database technologies and various operating systems,
whereas he may not be as strong as Software Test Manager regarding test project
management and coordination. The responsibilities of the test manager are as follows:

Since the test manager represents the team he is responsible for all
interdepartmental meetings.
Interaction with the customers whenever required.
A test manager is responsible for recruiting software testing staff. He has to
supervise all testing activities carried out by the team and identify team members who
require more training.
Schedule testing activities, create budget for testing and prepare test effort
estimations.
Selection of right test tools after interacting with the vendors. Integration of testing
and development activities.
Carry out continuous test process improvement with the help of metrics.
Check the quality of requirements, how well they are defined.
Trace test procedures with the help of test traceability matrix.

Software Test Automator Role

Software test automator or an automated test engineer should have very good
understanding of what he needs to test- GUI designs, load or stress testing. He should be
proficient in automation of software testing, and he should be able to design test suites
accordingly. A software test automator should be comfortable using various kinds of
automation tools and should be capable of upgrading their skills with changing trends. He
should also have programming skills so that he is able to write test scripts without any
issues. The responsibilities of a tester at this position are as follows:

He should be able to understand the requirement and design test procedures and
test cases for automated software testing.
Design automated test scripts that are reusable.
Ensure that all automated testing related activities are carried out as per the
standards defined by the company.

Interactions between Software Test Team and Business Teams

If at all a customer has any issues related to testing activities and operational matters of the
project then it is the software testing manager who is responsible for communicating the
details to the client regarding how things are being managed. The software testing manager
not only answers the queries of the customers but also ensures that the project is
completed on time as per the requirement of the customer.
Interactions between Software Test Team and Development Teams
In order to produce good software applications, it is important that software testing and
software development teams work together with good understanding. For this it is
important that the testers and developers are comfortable with each other’s role and
understand well that they have a common goal and it is wise to listen each other. A good
communication skill is very important both for testers and developers.

Before getting started with testing work it is important to discuss the basic guidelines and
expectations so that there is no confusion in later stages. Criticism should be taken in a
positive sense. It is important to understand that developers and testers have a common goal
of producing high quality software. A tester is not discovering bugs to show someone
down, the idea is to learn from mistakes and avoid repeating them in future. A culture of
constructive criticism can be of great help.
Interactions between Software Test Team And Release Management Teams
The release management teams are responsible for moving the software from development
into production. This team is responsible for planning the releases for hardware, software and
testing. It is also responsible for development of software development procedures and for
coordinating interactions and training of releases. Software testing is considered to be a very
important aspect of software engineering life cycle but it does not get over with development.
Testing and verification is a very important part of release management exercise.
Interactions between Software Test Manager And Software Project Manager
The job of a software test manager is not an easy one. He has to recruit testing team and take
responsibility for getting them trained. A software manager has to perform ongoing analysis of
various testing processes and ensure that the testing team is carrying out all the processes
correctly. This job is of great responsibility as the software testing manager is the one who
selects, introduces and implement various tools for testing. A software test manager is
responsible for finalizing templates for testing documents, test reports and other procedures.

Since a software tester manager has to deal with all the details of various testing activities, it is
very important for him to be in constant touch with the project manager and provide necessary
support in project planning and scheduling so that the project can be successfully completed in
time within the specified financial budget limits.

Verification & Validation

These two terms are very confusing for most people, who use them interchangeably. The
following table highlights the differences between verification and validation.
Sr.No
. Verification Validation

1 Verification addresses the concern: "Are you Validation addresses the concern: "Are
building it right?" you building the right thing?"

2 Ensures that the software system meets all the Ensures that the functionalities meet the
functionality. intended behavior.
 3 Verification takes place first and includes the Validation occurs after verification and
checking for documentation, code, etc. mainly involves the checking of the overall
product.

4 Done by developers. Done by testers.

5 It has static activities, as it includes collecting It has dynamic activities, as it includes

reviews, walkthroughs, and inspections to verify executing the software against the
a software. requirements.

6 It is an objective process and no subjective It is a subjective process and involves

decision should be needed to verify a software. subjective decisions on how well a
software works.

Testing Principles
Software testing is a procedure of implementing software or the application to identify the
defects or bugs. For testing an application or software, we need to follow some principles to
make our product defects free, and that also helps the test engineers to test the software with
their effort and time. Here, in this section, we are going to learn about the seven essential
principles of software testing.

Let us see the seven different testing principles, one by one:

Testing shows the presence of defects

Exhaustive Testing is not possible
Early Testing
Defect Clustering
Pesticide Paradox
Testing is context-dependent
Absence of errors fallacy
The test engineer will test the application to make sure that the application is bug or defects
free. While doing testing, we can only identify that the application or software has any errors.
The primary purpose of doing testing is to identify the numbers of unknown bugs with the
help of various methods and testing techniques because the entire test should be traceable to
the customer requirement, which means that to find any defects that might cause the product
failure to meet the client's needs.

By doing testing on any application, we can decrease the number of bugs, which does not
mean that the application is defect-free because sometimes the software seems to be bug-free
while performing multiple types of testing on it. But at the time of deployment in the
production server, if the end-user encounters those bugs which are not found in the testing
process.

Exhaustive Testing is not possible

Sometimes it seems to be very hard to test all the modules and their features with effective
and non- effective combinations of the inputs data throughout the actual testing process.

Hence, instead of performing the exhaustive testing as it takes boundless determinations

and most of the hard work is unsuccessful. So we can complete this type of variations
according to the importance of the modules because the product timelines will not permit
us to perform such type of testing scenarios.

Early Testing

Here early testing means that all the testing activities should start in the early stages of the
software development life cycle's requirement analysis stage to identify the defects
because if we find the bugs at an early stage, it will be fixed in the initial stage itself, which
may cost us very less as compared to those which are identified in the future phase of the
testing process.

To perform testing, we will require the requirement specification documents; therefore, if

the requirements are defined incorrectly, then it can be fixed directly rather than fixing
them in another stage, which could be the development phase.

Defect clustering

The defect clustering defined that throughout the testing process, we can detect the
numbers of bugs which are correlated to a small number of modules. We have various
reasons for this, such as the modules could be complicated; the coding part may be
complex, and so on.

These types of software or the application will follow the Pareto Principle, which states that
we can identify that approx. Eighty percent of the complication is present in 20 percent of the
modules. With the help of this, we can find the uncertain modules, but this method has its
difficulties if the same tests are performing regularly, hence the same test will not able to
identify the new defects.
Pesticide paradox

This principle defined that if we are executing the same set of test cases again and again over a particular time, then these kinds of
the test will not be able to find the new bugs in the software or the application. To get over these pesticide paradoxes, it is very
significant to review all the test cases frequently. And the new and different tests are necessary to be written for the implementation
of multiple parts of the application or the software, which helps us to find more bugs.

Testing is context-dependent

Testing is a context-dependent principle states that we have multiple fields such as e-commerce websites, commercial websites, and
so on are available in the market. There is a definite way to test the commercial site as well as the e-commerce websites because
every application has its own needs, features, and functionality. To check this type of application, we will take the help of various
kinds of testing, different technique, approaches, and multiple methods. Therefore, the testing depends on the context of the
application.

Absence of errors fallacy

Once the application is completely tested and there are no bugs identified before the release, so we can say that the application is
99 percent bug-free. But there is the chance when the application is tested beside the incorrect requirements, identified the flaws,
and fixed them on a given period would not help as testing is done on the wrong specification, which does not apply to the client's
requirements. The absence of error fallacy means identifying and fixing the bugs would not help if the application is impractical
and not able to accomplish the client's requirements and needs.