CCS366 - Software Testing and Automation UNIT IV
CCS366 - Software Testing and Automation UNIT IV
Performance Testing
1. Test environment setup : Set up a special test environment just for carrying out
the load testing. It guarantees that testing will be carried out correctly.
2.Load test scenario : Load test scenarios are produced in the second stage. Data
is then prepared for each transaction and load testing transactions are chosen for
the application.
3.Execution of test scenarios : This step involves running the load tests that were
prepared in the previous phase: To acquire the data, many metrics and measures
are collected.
4.Test result analysis : The testing results are examined and numerous suggestions
are given
5.Retest : If the test is unsuccessful, it is repeated to ensure that the results are
accurate.
Parameters for Load Testing
Metrics are used to understand how load testing performs in various scenarios. It
provides information on how correctly load testing is performing under various test
situations. Typically, it is completed after the creation of load test scripts and
scenarios. The load testing may be evaluated using a variety of measures.
Following is a list of some of them.
• Average response time : This number indicates how long it typically takes to
react to a request made by a client, •customer or user. Additionally, it displays the
application's performance based on how quickly all of the requests are processed.
• Error rate : The error rate, which is expressed as a percentage, indicates thé
proportion of requests for which mistakes were made to the total number of
requests. These errors often appear when the programme is unable to process the
request at the time specified or when there are other technical issues. When the
mistake rate continues rising, the application becomes less effective.
•Throughput : This metric is utilized to determine the bandwidth range required
during load routines or tests as well as the volume of data being used to measure
the request flow between the user sewer and application main server. Kilobytes per
second is the unit of measurement.
• Requests per second : This information indicates the number of requests being
sent to the application server per second. Any kind of request, including those for
photographs, documents, web pages, articles or other materials, may be made.
• Concurrent people : This measure is used to count the number of people who
are online at any one moment or at a certain period. Without making any requests
inside the programme, it simply counts the number of users who have visited the L
application at any given moment. This allows us to quickly determine when a large
number of people are accessing the application or website.
Peak response time is a measure of how long it takes to respond to a request.
Additionally, it aids in determining the length of the longest period of time during
which the request and response cycle is handled as well as which resource is taking
the longest to react to requests.
LoadNinja
Apache JMeter
NeoLoad
HP Performance Tester
WebLoad
LoadView
Advantages of load testing :
● Load testing improves the system or software application's resilience.
● It enhances the system's or software application's capacity to scale.
● It assists in reducing the risks associated with system outages.
● It lowers the expenses associated with system failure.
● It raises client satisfaction.
● Load testing is a crucial component of software testing since it offers a
number of benefits.
● Finding bottlenecks : Load testing identifies system bottlenecks including
sluggish -database queries, limited memory or clogged networks. This aids
system optimization and guarantees that it can support the anticipated
volume of users or transactions.
● Increased scalability: Load testing helps guarantee that the system can
manage a rising number of users or transactions over time by determining
the maximum capacity of the system. This is especially crucial for
web-based apps and systems that must manage a lot of traffic.
● Increased reliability : Load testing aids in the detection of any possible
problems that can arise under situations of high load, such as elevated error
rates or sluggish reaction times. This makes it easier to make sure the
system is dependable and stable when it is put into use in production.
● Lower risk : By spotting possible problems before deployment, load testing
helps lower the possibility of system failure or subpar performance in actual
use.
● Economical : Load testing is more economical than correcting issues that
arise during production. The cost of finding and fixing problems before
deployment is substantially lower than it is during testing
● Better user experience : Load testing assists in ensuring that users have a
pleasant system use experience by identifying and removing bottlenecks. As
a result, consumer loyalty and satisfaction may increase.
Disadvantages of load testing :
There is a requirement for programming skills in order to execute load testing.
Tools for load testing might be expensive. Additionally, there are several
drawbacks to load testing, such as :
● Resource-intensive : To simulate a high number of users or transactions,
load testing may need a lot of hardware and software resources. Due to this,
load testing may be; costly and time-consuming.
● Complexity: To set up and conduct load testing successfully, it might be
complicated and need specialized knowledge and skills. This might make
load testing challenging for teams with minimal resources or expertise.
● Limited testing scope : Because load testing focuses on how the system
performs under stress, it may not be able to find every kind of problem or
flaw. It's crucial to include load testing with other testing techniques, such
as functional; regression and acceptance testing.
● Inaccurate results : The test's outcomes may not be accurate if the load
testing environment is not a good representation Of the production
environment or if the load test scenarios don't correctly reflect real-world
use.
● Difficulty reproducing real-world use : Predicting how users will interact
with the system and simulating real-world usage are both challenging tasks.
As a result, it is difficult to predict whether the system can support the
anticipated demand.
● Difficulty in analyzing the results : Because load testing creates a lot of
data, it. may be challenging to analyze the findings and the underlying
causes of performance problems.
To ensure that the system is completely tested and that any flaws are found
and fixed before deployment, it is crucial to remember that load testing is
one sort of software testing and that it should be supplemented with other
types of testing.
Stress Testing
The performance of the stress is evaluated using metrics and this is often
done after the conclusion of-the stress scripts or tests. Below are some of the
measures.
Pages per second : The number of pages that are requested and loaded per second.
Pages retrievable : The average time required to obtain all the data from a certain
page.
Byte retrieved : The initial byte of data is usually retrieved from the page in a
reasonable amount of time.
Transaction response time : The amount of time it typically takes for transactions
between programmes to load or complete.
Transactions per second: It keeps track of how many transactions are loaded
successfully each Second as well as how many failures are experienced.
Connection failure: It keeps track of how often a client's system
experienced-connection fàilure.
System attempt failure : The system keeps track of how many times an attempt
has failed.
Rounds : It maintains track of how many test or script conditions were
successfully run by clients, as well as how many rounds were unsuccessful.
Requires solid scripting expertise : To develop the script test-cases for the
specific tool, solid scripting skills are needed.
Demand for external resources : In order to execute stress testing, extemal
resources are required- It results in an increase in time and resources
Volume Testing
A form of software testing called volume testing involves exposing the
programme to a significant amount of data. It is also known as flood testing. By
increasing the amount of data in the database, volume testing is done to
evaluate the system's performance.
The influence on reaction time and system behavior when exposed to a big
amount of data may be evaluated with the aid of volume testing.
As an example, examine the music site's behavior while millions of users are
downloading a song.
● Maintaining a database.
Due to the fact that we are working with relational databases, which have a
strong structure and a number of nearby tables, it is the difficulty that is
encountered most often during the volume testing.
In order to ensure that the test data is of the highest quality, the tester must
collect a variety of fields, including both the mandatory elective ones that
include big binary files.
● Review the data types and connections between them.
Learning the relationships between them, the distinctions between the kinds
and how the software responds is challenging for a testing novice.
During the volume testing, the test specialists must deal with a wide variety of
data that may be legitimate, invalid, missing, boundary-related or incorrect.
1. HammerDB
It is a database benchmarking tool that is open-source for the whole world's
database industry. 'It is multi-threaded, utilised for automation and supports
run-time scripting.
Features of HammerDB
Following are the commonly used features of HammerDb :
● It is a completely open-source corporate rating tool with no limits on virtual
users.
● It can be utilized by all of the leading IT firms.
● It offers complete support for a number of databases, including Oracle, MY
SQL, SQL Server, PostgreSQL etc.
● It enables help at the expert level.
● It offers thorough and detailed documentation.
● It is supported and completely functioning on Linux and Windows systems.
Dbf-it
A free tool that facilitates test-driven development is called DbFit. The DbFit
tests may be used as a working example of our system behaviour documenta,tion.
Benefits of DbFit
These are some of the DbFit features that are often used :
• It supports agile development techniques including test-driven
development and refactoring.
• It supports a variety of database types, including Oracle, SQL Server etc.
• It aids in improving the product's quality, maintainability and design.
• It provides Comprehensible syntax, enhancing communication with
non-technical persons.
• It offers examples-filled online documentation.
JdbcSlim
JdbcSlim is a different volume testing tool that makes it simple to include
database statements and queries into Slim FitNesse testing: It focuses mostly on
maintaining the setup, test data and SQL instructions clearly.
JdbcSlim's Characteristics
The following are some of JdbcSlim's frequently used features :
● SQL-literate developers, testers and business users may utilize the JdbcSlim
framework.
● It supports any database that a JDBC driver is available for.
● It also involves ensuring that the requirements are clear for users to
comprehend and write independently of execution.
NoSQLMap
● An open-source Python programme called ÑoSQLMap is designed to
automatically inject outbreaks and interfere with Database setups in order to
assess the hazard.
Advantage of Volume Testing
The following are a few advantages of volume testing :
It helps reduce maintenance expenses related to application development.
It only works with the MongoDB database.
Early bottleneck discovery is aided by this.
It facilitates séalability plans' speedier starts.
The volume testing ensures that the technology can be used in the actual
world.
Disadvantages of Volume Testing
The following are drawbacks of volume testing :
The exact partition of memory utilized in the actual world cannot be achieved
using the volume tests.
To acquire the volume testing that would result in an additional cost to the
project, a knowledgeable database performance testing team is necessary.
The personalized representation of the actual world is challenging and
complex.
Executing complete volume testing, which includes all test scenarios, writing
scripts and running those scripts, takes a lot of time. This might push back the
application's release date.
Fail-Over Testing
4. List the outcomes in order of the degree of damage that each failure would cause
and then note which failures are most likely to occur.
Important considerations for failover testing :
1. Maintain a list of actions to be done after a test.
2. Concentrate On carrying out the test plan.
3. Establish a baseline to ensure that performance standards can be met.
4. Write a report on thé problem requirements and/or asset requirements.
Working of Failover testing :
1. Before doing failover testing, take into account variables like money, time, team,
technology, etc.
2.Analyze the causes of failover and provide fixes.
3.Create test cases to examine possibilities for failover.
4.Execute the test plan based on the results.
5.Write a thorough failover report.
6.Based on the report, take the appropriate action.
The following are some advantages of failover testing :
1. Users may adjust anything, including user access, network settings and other
things.
2. Verifies that the setup is functioning correctly.
3.The system's server can quickly fix all of the issues in advance.
4. Offers improved services to ensure seamless operation of users' servers.
5. Assures that no loss occurs during downtime.
Recovery Testing
System testing techniques called "recovery testing" are used to determine if a
system can recover from errŒs. The method entails failing the system and then
confirming that the system recovery was successfully completed.
Recovery testing must be carried out to make sure that a system is
fault-tolerant and capable of quickly recovering from errors.A system is
anticipated to repair errors and go on with its task within a certain time frame. Any
mission-critical system, such as a defense system, medical equipment etc.,.
requires recovery testing. In such systems, a tight protocol is placed on how the
system should recover from failure, how long it should take to do so and how it
should act while failing.
It is important to test a system or piece of software for failures like :
● Power supply failure
● The external server is unreachable
● Wireless network signal loss
● Physical conditions
● The external device not responding
● The external device is not responding as expected etc.
l . Recovery Analysis: It's critical to assess the system's capacity to assign more
resources, such as servers or additional CPUs. This would make it easier to
cornþrehend how modifications linked to recovery could affect how the system
functions. Additionally, each potential failure, its potential effect, its severity and
how to handle it should be researched.
2.Creating the test scenarios while considering the environment and the outcomes
of the recovery analysis.
3.Designing the test environment in accordance with the findings of the recovery
analysis.
4.Keeping a backup - It's important to keep a backup of any information pertaining
to the programme, including its different states and database. Additionally, if the
data is crucial, it is crucial to back it up in many different places.
5.Allocation of recovery workers It's crucial to assign recovery employees who are
knowledgeable and skilled enough for the recovery tests being done.
6.Documentation - This stage focuses on recording all the actions taken before and
during the recovery testing so that the system's performance in the event of a
failure may be evaluated.
Example of recovery testing :
• By cutting off the power to a system that is receiving data through a network for
processing, we may cause software failure. After some time has passed, we may
plug the system back in to see whether it can resume receiving data where it left
off.
• Another example would be the ability to simulate software failure by
restarting a browser that is running several sessions. We can check to see
whether the system recovers from the failure and IQads all of the sessions it
was previously working on after restarting it.
Why If we relocate to an area without a. network when a movie is downloading
via a Wifi network, the download will be paused. We now return to a location with
a Wifi network. to see whether the procedure is able to resume functioning after
the interruption: The programme has an excellent recovery rate if the downloading
may be restarted.
Advantages of recovery testing
● Enhances the system's quality by removing any possible defects and
ensuring that it performs as intended
● Disaster Recovery Testing is another name for recovery testing. Many
businesses have disaster recovery facilities to ensure that there is a backup
system in case one of the systems is destroyed or fails for any other reason.
● Since the potential weaknesses are found and fixed, risks may be completely
eliminated. Better performance as faults are fixed and the system becomes
more dependable and operates more effectively in the event of a breakdown.
Configuration testing may also be divided into the two categories below:
1. Testing at the client level : Testing at the client level includes usability and
functionality testing. The consumers immediate interest is taken into
consideration while doing this testing.
Usability Testing
Today, there are several applications in the app store that may assist individuals
in their job. And when they can, they write a'bad review or a low rating, which
directs a certain product in their direction before a select group of end consumers
download or install it.
In a nutshell, we may say that one negative review can ruin all of the hard work,
talent and strategy that went into creating the product.
This is why usability testing, which plays a significant role and is carried out by
the test engineers throughout the STLC (Software Testing Life Cycle), enters the
picture to address these kinds of problems.
• The end-users must be able to see all the features of the software or apps.
Easy to Access
• A user-friendly programme needs to be available to all users.
Easy to Access
• The application's appearance and feel must be superb and alluring in order
to capture the user's attention.
• The user interface (GUI) of the programme should be decent since a poor
GUI might lead the user to lose interest in the application or software.
• The product meets the client's expectations for quality.
Faster to Access
● The programme should operate more quickly when accessed, indicating that
the application responds quickly.
● If an answer takes a long time, the user could get agitated. [We need to make
sure that the response time for our application is between 3 and 6 seconds.
Effective Navigation
● The most important feature of the programme is effective navigation.
Among the elements for successful navigation are :
o Effective Internal Linking
o A helpful header and footer
o A useful search function
Good Error Handling
• Handling errors at the code level ensures that the programme or application
is resilient and bug-free.
• By displaying the appropriate error message, the application's usability and
user experience will be improved.
Step 1 : Planning
• Planning is the first stage in the usability testing process. During this
phase, the team creates a test strategy and some document samples to aid
in the completion of the usability testing tasks. It is a critical and one of
the most important steps in the usability testing process.
• The planning stage determines the usability test's goal. Here, the goal is
not to have the volunteers use our programme. while documenting their
actions; rather, we need to address the system's most important functions
and components.
• Our test engineers who are acquainted with these essential elements
need to be given the jobs. Additionally, during the planning stage, the
usability testing methodology, the quantity and demographics of usability
test engineers and test report formats are determined
Step 6 : Reporting
● We will eventually get at the last phase of the usability testing pröcess,
which is referTed to as Reporting, after successfully completing all the
aforementiòned processes.
● With the development team, designer and other project participants, we may
report the results and recommend changes, as well as exchange all the
relevant materials, including audio, databases, screen recordings etc.
There are various types of testing documents in software testing. Some of them are
as follows :
● Test scenarios : The methods used to test an application: programme are
called test scenarios. It excludes the inputs and outputs needed for
application navigation. However, it is often used to comprehend how the
programme works from point to point. whereas test scenarios and test cases
both include similar kinds of principles, test scenarios involve many stages
for evaluating the programme whereas test cases just include a test case.
● Test case : Test scenarios and test cases are both used to test the application,
as we recently talked with test scenarios. However, a test case is a single
input and output scenario used to evaluate an application. Onlÿ one test case
is insufficient to test an applicätion, thus numerous test cases are created to
test the programme and these test cases are recorded in a test document.
● Test plan : As the name suggests, we will create a strategy in this document
for the software testing that will be done either during the programme or
thereafter. For efficient software application testing, this strategy is quite
helpful. Although the test plan document is crucial for effective testing,
whatever we perform with' a plan becomes successful.
• Test data : Test data is the data that is present before the testing processes are
carried out. The test data may be used to compare test case findings, whether
they are accurate or not.The test data document is created in an Excel sheet
format.
Benefits of using Documentation
● Clear documentation makes the effectiveness of approaches and goals clear.
● When a consumer employs a software programme, it guarantees internal
co-operation.
● It guarantees transparency about the consistency of duties and output.
● It offers feedback on tasks involving prevention.
● It offers feedback for your cycle of planning.
● It provides unbiased proof of the effectiveness of the quality management
system.
● The values that we entered in the first step cannot be forgotten while writing
the test document.
● We can quickly consult the text document, which saves time in the
procedure
The fact that we will test on the same number makes it consistent as well.
Security testing
Software testing that identifies system flaws and establishes whether the
system's data and resources are secure from potential hackers is known as
security testing. It makes sure that the software system and application remain
secure and unaffected by any dangers or threats that might result in harm. Any
system's security testing aims to identify any potential flaws and
vulnerabilities that might lead to the loss of data or the organization's
reputation. Software testing that focuses on assessing the security of a system
or applicåtion is known as security testing. In order to guarantee that the
system is secure against unauthorized access, data breaches and other
security-related problems, security testing aims to discover vulnerabilities and
possible threats.
Availability
• This ensures that the data and statement services will be available whenever
we need them by requiring that the data be kept on file by an authorised
individual.
Integrity
● In doing so, we shall protect the data that an unauthorised individual
modified. Integrity's main goal is to provide the receiver control over the
data that the system provides.
● The secrecy structures and integrity systems often use some of the same
underlying strategies. However, instead of encrypting every communication,
they often incorporate the communication data to establish the basis of an
algorithmic check. Additionally, make sure that accurate data is sent from
one programme to the next.
Authorization
● It involves determining whether a customer is. allowed to both execute an
activity and receive services. Access control is a good illustration of
authori$ation.
Confidentiality
● Because it is the only method we can ensure the security of our data, it is a
security procedure that delays the leak of data from outsiders.
Authentication
● In order to provide access to the system or private information,- it is
important to trace the
source of a product and validate a person's uniqueness.
Non- repudiation
● It is a term used to refer to digital security, and it provides assurance that
neither the sender nor the receiver of a communication may dispute having
transmitted or received the message.
● The non-repudiation principle is used to confirm that a communication has
been sent and received by the individual claiming to have done so.
Involvement of the entire team : As far as we are aware, the testing team is the
sole team in the software development life cycle that is in charge of the testing
process. However, Business Analysts (BA) and developers may also test the
product or the application in agile testing.
Test-driven : Agile testing requires us to cany out the testing procedure during
implementation, which reduces the amount of time needed for development.
However, testing is done either after implementation or during the conventional
process of developing software-
Quick response : The business team participates in each agile testing iteration. As
a result,we may get constant input, which helps us shorten the time it takes to
respond to comments on development activity.
Agile Test Plan
The agile test plan is produced and modified for every release, in contrast to the
waterfall style.
The agile test plan also includes the testing kinds that were carried out during a
particular iteration, such as test environments, test data specifications, test
outcomes and infrastructure. The-agile test plans emphasize the following :
● Testing Scope : The testing scope outlines the test coverage, test scope and
sprint targets that will be used to implement the test.
● Performance and load testing : This section describes the various testing
techniques.
● Testing kinds or levels according to the complexity of the feature : It
specifies the testing types or levels that will be applied. Additionally, it
describes the test's data and specifications as well as the environment in
which the test will be run.
● Mitigation or risks plan : It describes the fallback strategy created to deal
with risks or problems. Additionally, it lists any difficulties that could arise
when testing the programme for the most recent version.
● Deliverables and Milestones : It establishes test deliverables and
milestones from the customer's point of view.
● Infrastructure consideration : It controls the equipment needed to do the
tests.
● Resourcing : It enumerates the test-related jobs and the frequency of tests,
which specifies how often the tests will be run.
● Establish the newly tested functionality.
Agile testing has four different approaches, which help us to enhance our
product quality.
• Iteration 0
• Construction iteration
• Release End Game or Transition Phase
• Production
Let us discuss them one by one in detail .
1. Iteration 0
● Iteration 0 is the initial technique or strategy in agile testing. Here, we carry
out the preparatory setup duties including recruiting test subjects, setting up
test instruments, arranging resources or a usability testing lab etc.
● In Iteration 0, the below steps are accomplished .
○ Validating a business case for the project's boundaries and situational
circumstances.
○ List the key specifications and use cases that will influence the
strategic trade-offs.
○ Schedule the first project and estimate costs
○ Spotting the danger.
○ Describe one or more potential designs.
2.Construction Iteration
● Construction Iteration is the following agile testing technique. The bulk of
the testing is done using this method.
● In order to produce an increment of the answer, the building iteration is
carried out as a series of iterations.
● Simply said, the agile team executes the specified requirements throughout
each iteration, acquiring the most important business needs or requirements
that have been left behind from the work item stack.
● The construction iteration process divided into the following two types of
testing :
l. Confirmatory testing
2. Investigative testing
1. Confirmatory Testing : We will catTY out the confirmatory testing to
make sure the product satisfies the needs of all the stakeholders.
Confirmatory testing may be further divided into the following two
categories of testing :
o Agile acceptance testing
o Developer testing
Agile acceptance testing : It combines acceptability testing with
functional testing. The development team and stakeholders may jointly
conduct the agile acceptance testing.
Developer testing : It combines integration testing with unit testing.
Additionally, it verifies the database structure as well as the application
code.
2. Investigative Testing : We will do investigative testing in order to test
thoroughly and find any problem that confirmatory testing missed.
3.Release End Game or Transition Phase
● Release is the third strategy used in agile testing . This particular strategy's
goal is to successfully put our technology into production.
● In the end, the test engineer will be working on its defect tales. In the release
end game or transition stage, we have the following activities :
○ Support Individuals
○ Training of end-users
○ Operational People
● Similarly, it involves some additional activities as well .
○ Back-up and Restoration
○ Marketing of the product release
○ User documentation
○ Completion of system
Whole system testing and acceptability testing are included in the final testing
phase of the agile approach- We should have to test the product more
extensively throughout building iterations if we want to successfully finish
our final testing pháse.
4. Production
When the release stage is over, the product will proceed to the manufacturing
step.
Agile Testing life cycle completed into five different phases, as we can see in the
following image :
In order to evaluate the development process's progress, we may state that agility
reviews are implemented on a regular basis.
● The biggest drawback of agile testing is that if two or more team members
quit, the project would fail.
● In order to test the programme or the application, the testing team must be
determined, which leads to uneven results.
● The lack of documentation that translates into precise criteria and
requirements makes it difficult for us to forecast expected outcomes.
● Agile testing often involves bug patches, revisions and releases, which
might sometimes cause us to introduce new problems into the system.