Chapter No 6

Software Testing

Software Engineering I
By: Saifullah shakir
For: Nisar Ahamad saqib
Department of Computer Science
Software Testing
 Testing is the process of evaluating a system or its

component(s) with the intent to find whether it satisfies the

specified requirements or not.

 In simple words, testing is executing a system in order to

identify any gaps, errors, or missing requirements in contrary

to the actual requirements.

Software Testing

 According to ANSI/IEEE 1059 standard, Testing can be defined

as - A process of analyzing a software item to detect the

differences between existing and required conditions (that is

defects/errors/bugs) and to evaluate the features of the

software item.
Who does Testing?
 It depends on the process and the associated stakeholders of

the project(s). In the IT industry, large companies have a team

with responsibilities to evaluate the developed software in

context of the given requirements.

 Moreover, developers also conduct testing which is called Unit

Testing.
Who does Testing?
 In most cases, the following professionals are involved in

testing a system within their respective capacities:

 Software Tester
 Software Developer
 Project Lead/Manager
 End User
Who does Testing?
 Different companies have different designations for people who

test the software on the basis of their experience and knowledge

such as Software Tester, Software Quality Assurance Engineer, QA

Analyst, etc.

 It is not possible to test the software at any time during its cycle.

The next two sections state when testing should be started and

when to end it during the SDLC.

Types of Testing
 Mainly types of testing is divided into two part.

 Manual Testing

 Automation Testing

 Which each of the above may contain more types for testing to

test a software from different perspective.

Manual Testing
 Manual testing includes testing a software manually, i.e., without

using any automated tool or any script.

 In this type, the tester takes over the role of an end-user and tests

the software to identify any unexpected behavior or bug.

 There are different stages for manual testing such as unit testing,

integration testing, system testing, and user acceptance testing.

Manual Testing

 Testers use test plans, test cases, or test scenarios to

test a software to ensure the completeness of testing.

Manual testing also includes exploratory testing, as

testers explore the software to identify errors in it.

Automation Testing
 Automation testing, which is also known as Test Automation, is

when the tester writes scripts and uses another software to

test the product.

 This process involves automation of a manual process.

Automation Testing is used to re-run the test scenarios that

were performed manually, quickly, and repeatedly.

Automation Testing
 Apart from regression testing, automation testing is

also used to test the application from load,

performance, and stress point of view. It increases the

test coverage, improves accuracy, and saves time

and money in comparison to manual testing.

What can be Automated?
 It is not possible to automate everything in a software. The areas at

which a user can make transactions such as the login form or registration

forms, any area where large number of users can access the software

simultaneously should be automated.

 Furthermore, all GUI items, connections with databases, field validations,

etc. can be efficiently tested by automating the manual process.

Testing
Techniques
Black-Box Testing
 The technique of testing without having any

knowledge of the interior workings of the application

is called black-box testing.

 The tester is oblivious to the system architecture and

does not have access to the source code

Black-Box Testing
 Typically, while performing a black-box test, a tester

will interact with the system's user interface by

providing inputs and examining outputs without

knowing how and where the inputs are worked upon.

Black-Box Testing
White-Box Testing
 White-box testing is the detailed investigation of

internal logic and structure of the code. White-box

testing is also called glass testing or open-box

testing.

 In order to perform white-box testing on an

application, a tester needs to know the internal

White-Box Testing
 The tester needs to have a look inside the source

code and find out which unit/chunk of the code is

behaving inappropriately.
White-Box Testing
Software Testing - Levels
 Levels of testing include different methodologies that can be

used while conducting software testing. The main levels of

software testing are:

 Functional Testing
 Non-functional Testing
Functional Testing
 This is a type of black-box testing that is based on the

specifications of the software that is to be tested.

 The application is tested by providing input and then

the results are examined that need to conform to the

functionality it was intended for.

Functional Testing

 Functional testing of a software is conducted on a

complete, integrated system to evaluate the system's

compliance with its specified requirements.

Unit Testing

 This type of testing is performed by developers before

the setup is handed over to the testing team to

formally execute the test cases. Unit testing is

performed by the respective developers on the

individual units of source code assigned areas.

Unit Testing
 The goal of unit testing is to isolate each part of the program

and show that individual parts are correct in terms of

requirements and functionality.

Integration Testing
 Integration testing is defined as the testing of

combined parts of an application to determine if they

function correctly. Integration testing can be done in

two ways: Bottom-up integration testing and Top-

down integration testing.

Integration Testing Method
 Bottom-up integration

 This testing begins with unit testing, followed by tests of

progressively higher-level combinations of units called

modules or builds.

 Top-down integration

 In this testing, the highest-level modules are tested first and

progressively, lower-level modules are tested thereafter.

Integration Testing Method
 In a comprehensive software development environment,

bottom-up testing is usually done first, followed by top-down

testing.
System Testing
 System testing tests the system as a whole. Once all

the components are integrated, the application as a

whole is tested rigorously to see that it meets the
specified Quality Standards.

 This type of testing is performed by a specialized

testing team.
Characteristics of System Testing
 System testing is the first step in the Software Development

Life Cycle, where the application is tested as a whole.

 The application is tested thoroughly to verify that it meets the

functional and technical specifications.

 The application is tested in an environment that is very close

to the production environment where the application will be

deployed.

 System testing enables us to test, verify, and validate both the

business requirements as well as the application architecture.

Regression Testing
 Whenever a change in a software application is made,

it is quite possible that other areas within the

application have been affected by this change.

 Regression testing is performed to verify that a fixed

bug hasn't resulted in another functionality or

business rule violation.

Regression Testing
 The intent of regression testing is to ensure that a change,

such as a bug fix should not result in another fault being

uncovered in the application.

Characteristics of Regression Testing

 Minimize the gaps in testing when an application with changes

made has to be tested.

 Testing the new changes to verify that the changes made did

not affect any other area of the application.

 Mitigates risks when regression testing is performed on the

application.
Acceptance Testing
 This is arguably the most important type of testing, as it is

conducted by the Quality Assurance Team who will gauge

whether the application meets the intended specifications and

satisfies the client’s requirement.

 The QA team will have a set of pre-written scenarios and test

cases that will be used to test the application.

Acceptance Testing
 Acceptance tests are not only intended to point out

simple spelling mistakes, cosmetic errors, or interface

gaps, but also to point out any bugs in the application

that will result in system crashes or major errors in

the application.
Acceptance Testing
 By performing acceptance tests on an application, the

testing team will deduce how the application will

perform in production. There are also legal and

contractual requirements for acceptance of the

system.
Alpha Testing
 This test is the first stage of testing and will be performed

amongst the teams (developer and QA teams). Unit testing,

integration testing and system testing when combined

together is known as alpha testing. During this phase, the

following aspects will be tested in the application:

Alpha Testing
 Spelling Mistakes

 Broken Links

 Cloudy Directions

 The Application will be tested on machines with the lowest

specification to test loading times and any latency problems.

Beta Testing
 This test is performed after alpha testing has been
successfully performed.

 In beta testing, a sample of the intended audience tests the

application. Beta testing is also known as pre-release

testing.

 Beta test versions of software are ideally distributed to a wide

audience on the Web, partly to give the program a "real-world"

test and partly to provide a preview of the next release.
Beta Testing
 In this phase, the audience will be testing the following:

 Users will install, run the application and send their feedback

to the project team.

 Typographical errors, confusing application flow, and even

crashes.

 Getting the feedback, the project team can fix the problems

before releasing the software to the actual users.

Beta Testing
 The more issues you fix that solve real user problems, the

higher the quality of your application will be.

 Having a higher-quality application when you release it to the

general public will increase customer satisfaction.

Non-Functional Testing
 This section is based upon testing an application from

its non-functional attributes. Non-functional testing

involves testing a software from the requirements

which are nonfunctional in nature but important such

as performance, security, user interface, etc.

Performance Testing
 It is mostly used to identify any bottlenecks or performance

issues rather than finding bugs in a software. There are

different causes that contribute in lowering the performance of

a software:
 Network delay
 Client-side processing
 Database transaction processing
 Load balancing between servers
 Data rendering
Performance Testing
 Performance testing is considered as one of the important and
mandatory testing type in terms of the following aspects:

 Speed (i.e. Response Time, data rendering and accessing)

 Capacity (Over all capacity)

 Stability (Show errors while load or change)

 Scalability (How Many Concurrent Tasks)

 Performance testing can be either qualitative or quantitative
and can be divided into different sub-types such as Load
testing and Stress testing.
Load Testing
 It is a process of testing the behavior of a software by applying

maximum load in terms of software accessing and

manipulating large input data. It can be done at both normal

and peak load conditions. This type of testing identifies the

maximum capacity of software and its behavior at peak time.

Load Testing
 Virtual users (VUsers) are defined in the automated testing

tool and the script is executed to verify the load testing for the

software. The number of users can be increased or decreased

concurrently or incrementally based upon the requirements.

Stress Testing
 Stress testing includes testing the behavior of a software

under abnormal conditions. For example, it may include taking

away some resources or applying a load beyond the actual

load limit.

 The aim of stress testing is to test the software by applying the

load to the system and taking over the resources used by the

software to identify the breaking point.

Stress Testing
 This testing can be performed by testing different scenarios

such as:

 Shutdown or restart of network ports randomly

 Turning the database on or off

 Running different processes that consume resources such as

CPU, memory, server, etc.

Usability Testing
 Usability testing is a black-box technique and is used to

identify any error(s) and improvements in the software by

observing the users through their usage and operation.

UI vs Usability Testing
 UI testing involves testing the Graphical User Interface of the

Software. UI testing ensures that the GUI functions according

to the requirements and tested in terms of color, alignment,
size, and other properties.

 On the other hand, usability testing ensures a good and user-

friendly GUI that can be easily handled. UI testing can be

considered as a sub-part of usability testing.
Security Testing
 Security testing involves testing a software in order to identify

any flaws and gaps from security and vulnerability point of

view. Listed below are the main aspects that security testing

should ensure:
 Confidentiality
 Integrity
 Authentication
 Availability
 Authorization
Security Testing
 Software is secure against known and unknown vulnerabilities

 Software data is secure

 Software is according to all security regulations

 Input checking and validation

 SQL insertion attacks

Portability Testing
 Portability testing includes testing a software with the aim to

ensure its reusability and that it can be moved from another

software as well. Following are the strategies that can be used

for portability testing:

 Transferring an installed software from one computer to

another.

 Building executable (.exe) to run the software on different

Portability Testing
 Portability testing can be considered as one of the sub-parts of

system testing, as this testing type includes overall testing of

a software with respect to its usage over different

environments.

 Computer hardware, operating systems, and browsers are the

major focus of portability testing. Some of the pre-conditions

for portability testing are as follows:

Portability Testing
 Software should be designed and coded, keeping in mind the

portability requirements.

 Unit testing has been performed on the associated

components.

 Integration testing has been performed.

 Test environment has been established.