UNIT 4

Performance Testing | Software Testing

Performance Testing is a type of software testing that ensures software applications to perform
properly under their expected workload. It is a testing technique carried out to determine system
performance in terms of sensitivity, reactivity and stability under a particular workload.
Performance testing is a type of software testing that focuses on evaluating the performance and
scalability of a system or application. The goal of performance testing is to identify bottlenecks,
measure system performance under various loads and conditions, and ensure that the system can
handle the expected number of users or transactions.

There are several types of performance testing, including:

 Load testing: Load testing simulates a real-world load on the system to see how it performs
under stress. It helps identify bottlenecks and determine the maximum number of users or
transactions the system can handle.
 Stress testing: Stress testing is a type of load testing that tests the system’s ability to handle a
high load above normal usage levels. It helps identify the breaking point of the system and any
potential issues that may occur under heavy load conditions.
 Spike testing: Spike testing is a type of load testing that tests the system’s ability to handle
sudden spikes in traffic. It helps identify any issues that may occur when the system is
suddenly hit with a high number of requests.
 Soak testing: Soak testing is a type of load testing that tests the system’s ability to handle a
sustained load over a prolonged period of time. It helps identify any issues that may occur after
prolonged usage of the system.
 Endurance testing: This type of testing is similar to soak testing, but it focuses on the long-
term behavior of the system under a constant load.
 Performance Testing is the process of analyzing the quality and capability of a product. It is a
testing method performed to determine the system performance in terms of speed, reliability
and stability under varying workload. Performance testing is also known as Perf Testing.
Performance Testing Attributes:
 Speed:
It determines whether the software product responds rapidly.
 Scalability:
It determines amount of load the software product can handle at a time.
 Stability:
It determines whether the software product is stable in case of varying workloads.
 Reliability:
It determines whether the software product is secure or not.
Objective of Performance Testing:
1. The objective of performance testing is to eliminate performance congestion.
2. It uncovers what is needed to be improved before the product is launched in market.
3. The objective of performance testing is to make software rapid.
4. The objective of performance testing is to make software stable and reliable.
5. The objective of performance testing is to evaluate the performance and scalability of a system
or application under various loads and conditions. It helps identify bottlenecks, measure system
performance, and ensure that the system can handle the expected number of users or
transactions. It also helps to ensure that the system is reliable, stable and can handle the
expected load in a production environment.
Types of Performance Testing:
1. Load testing:
It checks the product’s ability to perform under anticipated user loads. The objective is to
identify performance congestion before the software product is launched in market.
2. Stress testing:
It involves testing a product under extreme workloads to see whether it handles high traffic or
not. The objective is to identify the breaking point of a software product.
3. Endurance testing:
It is performed to ensure the software can handle the expected load over a long period of time.
4. Spike testing:
It tests the product’s reaction to sudden large spikes in the load generated by users.
5. Volume testing:
In volume testing large number of data is saved in a database and the overall software
system’s behavior is observed. The objective is to check product’s performance under varying
database volumes.
6. Scalability testing:
In scalability testing, software application’s effectiveness is determined in scaling up to support
an increase in user load. It helps in planning capacity addition to your software system.
Performance Testing Process:

Performance Testing Tools:

1. Jmeter
2. Open STA
3. Load Runner
4. Web Load
Advantages of Performance Testing :
 Performance testing ensures the speed, load capability, accuracy and other performances of
the system.
 It identifies, monitors and resolves the issues if anything occurs.
 It ensures the great optimization of the software and also allows large number of users to use it
on same time.
 It ensures the client as well as end-customers satisfaction.Performance testing has several
advantages that make it an important aspect of software testing:
 Identifying bottlenecks: Performance testing helps identify bottlenecks in the system such as
slow database queries, insufficient memory, or network congestion. This helps developers
optimize the system and ensure that it can handle the expected number of users or
transactions.
 Improved scalability: By identifying the system’s maximum capacity, performance testing
helps ensure that the system can handle an increasing number of users or transactions over
time. This is particularly important for web-based systems and applications that are expected to
handle a high volume of traffic.
 Improved reliability: Performance testing helps identify any potential issues that may occur
under heavy load conditions, such as increased error rates or slow response times. This helps
ensure that the system is reliable and stable when it is deployed to production.
 Reduced risk: By identifying potential issues before deployment, performance testing helps
reduce the risk of system failure or poor performance in production.
 Cost-effective: Performance testing is more cost-effective than fixing problems that occur in
production. It is much cheaper to identify and fix issues during the testing phase than after
deployment.
 Improved user experience: By identifying and addressing bottlenecks, performance testing
helps ensure that users have a positive experience when using the system. This can help
improve customer satisfaction and loyalty.
 Better Preparation: Performance testing can also help organizations prepare for unexpected
traffic patterns or changes in usage that might occur in the future.
 Compliance: Performance testing can help organizations meet regulatory and industry
standards.
 Better understanding of the system: Performance testing provides a better understanding of
how the system behaves under different conditions, which can help in identifying potential
problem areas and improving the overall design of the system.
Disadvantages of Performance Testing :
 Sometimes, users may find performance issues in the real time environment.
 Team members who are writing test scripts or test cases in the automation tool should have
high-level of knowledge.
 Team members should have high proficiency to debug the test cases or test scripts.
 Low performances in the real environment may lead to lose large number of users
 Performance testing also has some disadvantages, which include:
 Resource-intensive: Performance testing can be resource-intensive, requiring significant
hardware and software resources to simulate a large number of users or transactions. This can
make performance testing expensive and time-consuming.
 Complexity: Performance testing can be complex, requiring specialized knowledge and
expertise to set up and execute effectively. This can make it difficult for teams with limited
resources or experience to perform performance testing.
 Limited testing scope: Performance testing is focused on the performance of the system under
stress, and it may not be able to identify all types of issues or bugs. It’s important to combine
performance testing with other types of testing such as functional testing, regression testing,
and acceptance testing.
 Inaccurate results: If the performance testing environment is not representative of the
production environment or the performance test scenarios do not accurately simulate real-world
usage, the results of the test may not be accurate.
 Difficulty in simulating real-world usage: It’s difficult to simulate real-world usage, and it’s hard
to predict how users will interact with the system. This makes it difficult to know if the system
will handle the expected load.
 Complexity in analyzing the results: Performance testing generates a large amount of data, and
it can be difficult to analyze the results and determine the root cause of performance issues.
Agile Software Testing
Agile Testing is a type of software testing that follows the principles of agile software
development to test the software application. All members of the project team along with the
special experts and testers are involved in agile testing. Agile testing is not a separate phase and
it is carried out with all the development phases i.e. requirements, design and coding, and test
case generation. Agile testing takes place simultaneously throughout the Development Life Cycle.
Agile testers participate in the entire development life cycle along with development team
members and the testers help in building the software according to the customer requirements and
with better design and thus code becomes possible. The agile testing team works as a single team
towards the single objective of achieving quality. Agile Testing has shorter time frames called
iterations or loops. This methodology is also called the delivery-driven approach because it
provides a better prediction on the workable products in less duration time.
 Agile testing is an informal process that is specified as a dynamic type of testing.
 It is performed regularly throughout every iteration of the Software Development Lifecycle
(SDLC).
 Customer satisfaction is the primary concern for agile test engineers at some stage in the agile
testing process.
Features of Agile Testing
Some of the key features of agile software testing are:
 Simplistic approach: In agile testing, testers perform only the necessary tests but at the same
time do not leave behind any essential tests. This approach delivers a product that is simple
and provides value.
 Continuous improvement: In agile testing, agile testers depend mainly on feedback and self-
learning for improvement and they perform their activities efficiently continuously.
 Self-organized: Agile testers are highly efficient and tend to solve problems by bringing teams
together to resolve them.
 Testers enjoy work: In agile testing, testers enjoy their work and thus will be able to deliver a
product with the greatest value to the consumer.
 Encourage Constant communication: In agile testing, efficient communication channels are
set up with all the stakeholders of the project to reduce errors and miscommunications.
 Constant feedback: Agile testers need to constantly provide feedback to the developers if
necessary.
Agile Testing Principles
 Shortening feedback iteration: In Agile Testing, the testing team gets to know the product
development and its quality for each and every iteration. Thus continuous feedback minimizes
the feedback response time and the fixing cost is also reduced.
 Testing is performed alongside Agile testing is not a different phase. It is performed
alongside the development phase. It ensures that the features implemented during that
iteration are actually done. Testing is not kept pending for a later phase.
 Involvement of all members: Agile testing involves each and every member of the
development team and the testing team. It includes various developers and experts.
 Documentation is weightless: In place of global test documentation, agile testers use
reusable checklists to suggest tests and focus on the essence of the test rather than the
incidental details. Lightweight documentation tools are used.
 Clean code: The defects that are detected are fixed within the same iteration. This ensures
clean code at any stage of development.
 Constant response: Agile testing helps to deliver responses or feedback on an ongoing basis.
Thus, the product can meet the business needs.
 Customer satisfaction: In agile testing, customers are exposed to the product throughout the
development process. Throughout the development process, the customer can modify the
requirements, and update the requirements and the tests can also be changed as per the
changed requirements.
 Test-driven: In agile testing, the testing needs to be conducted alongside the development
process to shorten the development time. But testing is implemented after the implementation
or when the software is developed in the traditional process.
Agile Testing Methodologies
Some of the agile testing methodologies are:
1. Test-Driven Development (TDD): TDD is the software development process relying on
creating unit test cases before developing the actual code of the software. It is an iterative
approach that combines 3 operations, programming, creation of unit tests, and refactoring.
2. Behavior Driven Development (BDD): BDD is agile software testing that aims to document
and develop the application around the user behavior a user expects to experience when
interacting with the application. It encourages collaboration among the developer, quality
experts, and customer representatives.
3. Exploratory Testing: In exploratory testing, the tester has the freedom to explore the code
and create effective and efficient software. It helps to discover the unknown risks and explore
each aspect of the software functionality.
4. Acceptance Test-Driven Development (ATDD): ATDD is a collaborative process where
customer representatives, developers, and testers come together to discuss the requirements,
and potential pitfalls and thus reduce the chance of errors before coding begins.
5. Extreme Programming (XP): Extreme programming is a customer-oriented methodology that
helps to deliver a good quality product that meets customer expectations and requirements.
6. Session-Based Testing: It is a structured and time-based approach that involves the progress
of exploratory testing in multiple sessions. This involves uninterrupted testing sessions that are
time-boxed with a duration varying from 45 to 90 minutes. During the session, the tester
creates a document called a charter document that includes various information about their
testing.
7. Dynamic Software Development Method (DSDM): DSDM is an agile project delivery
framework that provides a framework for building and maintaining systems. It can be used by
users, developers, and testers.
8. Crystal Methodologies: This methodology focuses on people and their interactions when
working on the project instead of processes and tools. The suitability of the crystal method
depends on three dimensions, team size, criticality, and priority of the project.
Agile Testing Strategies

1. Iteration 0

It is the first stage of the testing process and the initial setup is performed in this stage. The testing
environment is set in this iteration.
 This stage involves executing the preliminary setup tasks such as finding people for testing,
preparing the usability testing lab, preparing resources, etc.
 The business case for the project, boundary situations, and project scope are verified.
 Important requirements and use cases are summarized.
 Initial project and cost valuation are planned.
 Risks are identified.
 Outline one or more candidate designs for the project.

2. Construction Iteration

It is the second phase of the testing process. It is the major phase of the testing and most of the
work is performed in this phase. It is a set of iterations to build an increment of the solution. This
process is divided into two types of testing:
 Confirmatory testing: This type of testing concentrates on verifying that the system meets the
stakeholder’s requirements as described to the team to date and is performed by the team. It is
further divided into 2 types of testing:
 Agile acceptance testing: It is the combination of acceptance testing and functional
testing. It can be executed by the development team and the stakeholders.
 Developer testing: It is the combination of unit testing and integration testing and
verifies both the application code and database schema.
 Investigative testing: Investigative testing detects the problems that are skipped or ignored
during confirmatory testing. In this type of testing, the tester determines the potential problems
in the form of defect stories. It focuses on issues like integration testing, load testing, security
testing, and stress testing.

3. Release End Game

This phase is also known as the transition phase. This phase includes the full system testing and
the acceptance testing. To finish the testing stage, the product is tested more relentlessly while it
is in construction iterations. In this phase, testers work on the defect stories. This phase involves
activities like:
 Training end-users.
 Support people and operational people.
 Marketing of the product release.
 Back-up and restoration.
 Finalization of the system and user documentation.
4. Production
It is the last phase of agile testing. The product is finalized in this stage after the removal of all
defects and issues raised.
Agile Testing Quadrants
The whole agile testing process is divided into four quadrants:
1. Quadrant 1 (Automated)
The first agile quadrat focuses on the internal quality of code which contains the test cases and
test components that are executed by the test engineers. All test cases are technology-driven and
used for automation testing. All through the agile first quadrant of testing, the following testing can
be executed:
 Unit testing.
 Component testing.
2. Quadrant 2 (Manual and Automated)
The second agile quadrant focuses on the customer requirements that are provided to the testing
team before and throughout the testing process. The test cases in this quadrant are business-
driven and are used for manual and automated functional testing. The following testing will be
executed in this quadrant:
 Pair testing.
 Testing scenarios and workflow.
 Testing user stories and experiences like prototypes.

3. Quadrant 3 (Manual)

The third agile quadrant provides feedback to the first and the second quadrant. This quadrant
involves executing many iterations of testing, these reviews and responses are then used to
strengthen the code. The test cases in this quadrant are developed to implement automation
testing. The testing that can be carried out in this quadrant are:
 Usability testing.
 Collaborative testing.
 User acceptance testing.
 Collaborative testing.
 Pair testing with customers.

4. Quadrant 4 (Tools)

The fourth agile quadrant focuses on the non-functional requirements of the product like
performance, security, stability, etc. Various types of testing are performed in this quadrant to
deliver non-functional qualities and the expected value. The testing activities that can be
performed in this quadrant are:
 Non-functional testing such as stress testing, load testing, performance testing, etc.
 Security testing.
 Scalability testing.
 Infrastructure testing.
 Data migration testing.

Agile Testing Life Cycle

The agile testing life cycle has 5 different phases:
1. Impact Assessment: This is the first phase of the agile testing life cycle also known as the
feedback phase where the inputs and responses are collected from the users and
stakeholders. This phase supports the test engineers to set the objective for the next phase in
the cycle.
2. Agile Testing Planning: In this phase, the developers, customers, test engineers, and
stakeholders team up to plan the testing process schedules, regular meetings, and
deliverables.
3. Release Readiness: This is the third phase in the agile testing lifecycle where the test
engineers review the features which have been created entirely and test if the features are
ready to go live or not and the features that need to be sent again to the previous development
phase.
4. Daily Scrums: This phase involves the daily morning meetings to check on testing and
determine the objectives for the day. The goals are set daily to enable test engineers to
understand the status of testing.
5. Test Agility Review: This is the last phase of the agile testing lifecycle that includes weekly
meetings with the stakeholders to evaluate and assess the progress against the goals.

Agile Test Plan

An agile test plan includes types of testing done in that iteration like test data requirements, test
environments, and test results. In agile testing, a test plan is written and updated for every release.
The test plan includes the following:
 Test Scope.
 Testing instruments.
 Data and settings are to be used for the test.
 Approaches and strategies used to test.
 Skills required to test.
 New functionalities are being tested.
 Levels or Types of testing based on the complexity of the features.
 Resourcing.
 Deliverables and Milestones.
 Infrastructure Consideration.
 Load or Performance Testing.
 Mitigation or Risks Plan.
Benefits of Agile Testing
Below are some of the benefits of agile testing:
 Saves time: Implementing agile testing helps to make cost estimates more transparent and
thus helps to save time and money.
 Reduces documentation: It requires less documentation to execute agile testing.
 Enhances software productivity: Agile testing helps to reduce errors, improve product
quality, and enhance software productivity.
 Higher efficiency: In agile software testing the work is divided into small parts thus developer
can focus more easily and complete one part first and then move on to the next part. This
approach helps to identify minor inconsistencies and higher efficiency.
 Improve product quality: In agile testing, regular feedback is obtained from the user and
other stakeholders, which helps to enhance the software product quality.
Limitations of Agile Testing
Below are some of the limitations of agile software testing:
 Project failure: In agile testing, if one or more members leave the job then there are chances
for the project failure.
 Limited documentation: In agile testing, there is no or less documentation which makes it
difficult to predict the expected results as there are explicit conditions and requirements.
 Introduce new bugs: In agile software testing, bug fixes, modifications, and releases happen
repeatedly which may sometimes result in the introduction of new bugs in the system.
 Poor planning: In agile testing, the team is not exactly aware of the end result from day one,
so it becomes challenging to predict factors like cost, time, and resources required at the
beginning of the project.
 No finite end: Agile testing requires minimal planning at the beginning so it becomes easy to
get sidetracked while delivering the new product. There is no finite end and there is no clear
vision of what the final product will look like.
Challenges During Agile Testing
Below are some of the challenges that are faced during agile testing:
 Changing requirements: Sometimes during product development changes in the
requirements or the specifications occur but when they occur near the end of the sprint, the
changes are moved to the next sprint and thus become the overhead for developers and
testers.
 Inadequate test coverage: In agile testing, testers sometimes miss critical test cases because
of the continuously changing requirements and continuous integration. This problem can be
solved by keeping track of test coverage by analyzing the agile test metrics.
 Tester’s availability: Sometimes the testers don’t have adequate skills to perform API and
Integration testing, which results in missing important test cases. One solution to this problem
is to provide training for the testers so that they can carry out essential tests effectively.
 Less Documentation: In agile testing, there is less or no documentation which makes the task
of the QA team more tedious.
 Performance Bottlenecks: Sometimes developer builds products without understanding the
end-user requirements and following only the specification requirements, resulting in
performance issues in the product. Using load testing tools performance bottlenecks can be
identified and fixed.
 Early detection of defects: In agile testing, defects are detected at the production stage or at
the testing stage, which makes it very difficult to fix them.
 Skipping essential tests: In agile testing, sometimes agile testers due to time constraints and
the complexity of the test cases put some of the non-functional tests on hold. This may cause
some bugs later that may be difficult to fix.
Risks During Agile Testing
 Automated UI slow to execute: Automated UI gives confidence in the testing but they are
slow to execute and expensive to build.
 Use a mix of testing types: To achieve the expected quality of the product, a mixture of
testing types and levels must be used.
 Poor Automation test plan: Sometimes automation tests plan is poorly organized and
unplanned to save time which results in a test failure.
 Lack of expertise: Automated testing sometimes is not the only solution that should be used, it
can sometimes lack the expertise to deliver effective solutions.
 Unreliable tests: Fixing failing tests and resolving issues of brittle tests should be the top
priority to avoid false positives.
Difference between Load Testing and Stress Testing
  Load Testing: Load Testing is a type of performance testing that determines the
performance of a system, software product, or software application under real life based
load conditions.
 Stress Testing: Stress testing is a type of software testing that verifies the stability and
reliability of the system. This test particularly determines the system on its robustness and
error handling under extremely heavy load conditions.
 Difference between Load Testing and Stress Testing:

S.
No. Load Testing Stress Testing

Load Testing is performed to test the Stress Testing is performed to test the
1. performance of the system or software robustness of the system or software
application under extreme load. application under extreme load.

In load testing load limit is the In stress testing load limit is above the
2.
threshold of a break. threshold of a break.

In load testing, the performance of the

In stress testing, the performance is tested
3. software is tested under multiple
under varying data amounts.
number of users.

4. Huge number of users. Too much users and too much data.

Load testing is performed to find out

Stress testing is performed to find the
5. the upper limit of the system or
behavior of the system under pressure.
application.

The factor tested during load testing The factor tested during stress testing
6.
is performance. is robustness and stability.

Load testing determines the operating

7. Stress testing ensures system security.
capacity of a system or application.

The purpose of load testing is to The goal of stress testing is to prevent

8. generate more traffic for a web server crashes under sudden, high loads for
application. an extended period.

9. Load testing is useful in finding bugs Stress testing is useful as it aids the testing
like memory overflows, etc., finding unit by testing the system in failure
the adequacy of current infrastructure situations, checks for data saving by the
in running the applications, system before it crashed, to see if any
determining the number of concurrent unanticipated failures potentially harm the
users that an application can handle, security of the system, and more.
 checks for application’s scalability for
accommodating more users and more.
