VIEW POINT

CONTINUOUS TESTING: TESTING

EARLY AND TESTING OFTEN

In this fast-changing world, continuous delivery has become a critical

concept for many organizations. However, despite embracing continuous
integration and continuous deployment, organizations are facing issues
in deploying quality software without compromising time and effort.
Continuous testing helps address these gaps by automating testing and
ensuring immediate identification of issues. This report sheds light on
continuous testing, its implementation process and future.
 Continuous testing:
Testing early and testing often
As per a MarketWatch research, the global DevOps platform
market is expected to reach 5360 million USD by the end of
2024, growing at a CAGR of 15.6% between 2019 and 2024.
In a world where change is the only constant, industries are
adopting DevOps in their attempt to efficiently navigate
complex business scenarios. DevOps aims at rapid software
delivery through continuous integration and delivery i.e
continuous development.
However, in order to enjoy the full benefits of DevOps,
companies must introduce continuous testing throughout
their software development and delivery processes.

External Document © 2019 Infosys Limited

Continuous development process
Continuous development aims at in web service. Any failures in this possible feedback about the quality
automating and streamlining the step will prevent new changes of their code, and a manager timely
process of building, testing and from being deployed. information about the risks associated
deploying new code into a live 5. After validating the new changes with deploying that code. The primary
environment. on staging, the application can goal of CT is to speed up the process
be deployed to production, after of providing feedback.
Following are the steps involved in
a continuous development process. which tests are run again on the Continuous testing helps validate
live API. and get immediate feedback on
1. Develop locally and push your
changes to a branch For a continuous development process to be new changes or new versions of the
successful, there is a need for early feedback software made; it is a culmination of
2. Continuous integration server
and correction at each step, thereby making many modern development methods
picks up your changes, builds
continuous testing an integral part of the such as Test Driven Development
your code, and runs unit tests
continuous development process. (TDD), extreme programming and
3. Application is deployed to a other collaborative approaches.
staging or review environment
at the ‘delivery step’
Continuous testing These tests hooked into continuous
integration servers, build pipelines and
4. Once deployed, automated API Continuous Testing (CT) is the run through an infinite loop of steps.
tests are run against the new process of executing automated tests
version and the user is alerted throughout the delivery pipeline
of any test failures or degradation giving a developer the earliest

Push
Code

Post Deploy Continuous Build/Unit CT

CT Testing Development Tests

Deploy
New Code

External Document © 2019 Infosys Limited

 Implementing a continuous testing program
A successful continuous testing • Democratisation of performance Readiness of the testing environment,
program requires good preparation testing reduction of manual efforts and a
of the environment it operates in. • Open-source testing tool change in organizational culture are
Following is an exhaustive list of integration prerequisites in creating a successful
requirements for continuous testing. continuous testing program. The
• Comprehensive cloud-based API
• Automated generation of below diagram shows elements in
testing
test automation script from respect to environment, automation
• Built-in automated application efforts and processes to ensure a fool
requirements
security testing proof CT program.
• Simulated test environment
• Orchestration and automation
• Accessibility to test data on demand of the pipeline to promote artifacts
and ensuring PII compliance when a test passes
• Multi-layer/backend ‘request and • Harness application insight
response testing’ before UI testing across SDLC

Continuous testing is the process of executing series of automated tests, as part of the software delivery pipeline. The QA
team acts as a glue between development and operations and as a key enabler for DevOps

DEV QA OPS

Environment Automation Process

• Hosting of CT platform either on • Test data management • Collaborative team, continuous

cloud or local server • Service virtualization automation
• Micro services-based architecture • Automated acceptance test • Lean test process and test
• Test environment management governance
• Continuous monitoring,
• Integration with Integrated interactive dashboard and • Flawless continuity in delivery
Developer Environments (IDEs) incident management tools cycle
and Continuous Integration (CI) by Ops • Defect prevention rather than
tools • Compatibility with complex defect detection
• Customization algorithms that support DevOps • Rapid feedback on the impact of
• Ease of setup scalability • End-to-end testing changes

A well-set continuous testing process • Generate test data and test cases • Virtualize testing resources to
ensures that the application is tested automatically eliminate bottlenecks
early and often. In order to ensure • Automate API functional and • Test and optimize performance
the readiness of continuous testing, non‑functional testing early in development
organizations resort to multiple tools
• Integrate risk assessment and
and technologies to:
security testing

External Document © 2019 Infosys Limited

Future of continuous testing
Today, developers are leveraging better test coverage, manual testing DevOps are AI and RPA, both of which
multiple technologies to improve elimination and so on. Today, the ensure reliability and efficiency of
the continuous testing process in most frequently used technologies the continuous testing process and
terms of shorter throughput times, to improve continuous testing in contribute to successful DevOps.

Validate code
Sprint planning Commit and build Deploy Automated
and build
Impact and Test to code Predicts potential Targeted testing,
ambiguity P1 errors in higher optimize and test for UI
coverage
assessment Ve r s i o n co n t r o l environment and content and RPA

Production deploy Performance tests

Commit and build
Predict potential Prediction models
P1 errors in higher
environment

1. Leveraging AI in DevOps • Anomaly detection in response 2. Leveraging RPA to automate

continuous testing times and errors. test data management
• Impact analysis to identify the
AI increases reliability in software With DevOps focusing on continuous
impacted automation scripts for
applications. Incorporating AI in integration and delivery, it becomes
any code change.
testing tools enables users to build and essential that applications be tested
execute personalized tests. Here is a • Application health analysis to every time they are delivered hence,
thematic representation of how AI will identify and track application emphasizing the need for Test Data
transform continuous testing. quality, responsiveness and Management (TDM). TDM helps in
automation script issues creating targeted test data of the
Following are the benefits of leveraging across builds. right size without cloning the entire
AI in DevOps continuous testing:
• UI analysis to highlight user interface production environment. In order to
• Finding correlation in errors changes between test runs. ensure faster throughput and optimal
and defect. coverage, it is highly advised to
• Automation script optimization
• Continuously monitoring test runs to eliminate redundancies across automate TDM.
to generate actionable insights test scripts. Following diagram gives a glimpse as
for QA.
• Test coverage analysis to identify to how RPA can transform TDM.
missing test paths.

• No manual intervention
Robot will execute the data • Monitoring through
Robot will check TDM portal request based on priority Benefits
with RPA Dashboard
• Reduced work time
• Auto-emails of report
Robot keeps polling every Robot executes data request
30 min interval to check if masking and sends the mai
any data request received to requestor

External Document © 2019 Infosys Limited

 Following are the benefits of • Cloud-based automated test captures the results with network
automating test data management environment that is fully scalable performance comparison graph and
with RPA: and can be seamlessly integrated tests the application performance
• Helps create stateful test data on with any of the CI/CD tools that can for distributed users under
demand for end-to-end testing share data through APIs. distributed networks.
• Provides stateful test data • Service virtualization supports • Process automation using RPA, QA
management seamlessly integrated codeless UI recording and testing, process automation automates
into test case design and execution synchronized user simulation, repetitive tasks performed by QA
virtual service environment. team. Bots can automate testing of
• Extraction and masking of data
• Test data management, sensitive each step and its expected action
• Self-service test data provisioning /output and testing of multiple
data discovery, data subset,
and management systems is performed by QA team.
data masking, data request and
• Increases testing efficiency data extraction, synthetic data • Leveraging AI capabilities
• Reduces skilled resource generation, data copy, gold copy in DevOps, improved testing
requirement creation, custom relationships efficiency through focused QA
• Eliminates manual testing builder, reporting etc. of impacted areas, reduced testing
• Reduces automation effort • Automated functional testing cycle time.
includes shift left approach, • Customer experience solutions

Infosys capabilities automated test script generation

using AI technique on manual
help evaluate all parameters
affecting end customer
Today, most of the organizations test cases, support of TDD/BDD experience and provides
deal with tight deadlines and approaches, components, UI detailed recommendations to
ever‑changing demands from testing, and mobile testing. improve customer experience
informed customer segments. • Supports API testing, easy-to-use numeric scoring. This helps
Organizations that are early adopters GUI for rapid development of benchmark against past releases
of agile methodologies and DevOps automated test cases in a codeless or competition, provides better
can help in automating testing needs, manner and execute functional visibility into end-user issues and
accomplishing delivery schedules, and regression tests, shift left with application quality, and improves
and bolstering agile development functional automation, automation customer experience.
processes. scripting even in the absence
Infosys provides different automated of APIs.
solutions to support DevOps • Load network performance testing
continuous testing. Here are a few creates user load, distributed real
Infosys offerings that enable DevOps networks, multiple test execution
continuous testing: and executes in sequence,

External Document © 2019 Infosys Limited

External Document © 2019 Infosys Limited
Conclusion
Organizations opting for DevOps welcome continuous testing at each stage of product development. This practice helps companies test early and
test often, bringing in a cultural shift that fosters communication, collaboration and innovation. Continuous testing, when implemented diligently,
allows flawless continuity in the delivery cycle. Integrating a fully automated continuous testing process into the SDLC is the most effective solution
for a successful continuous delivery process.

References
Perrow, 2016, November, 10. A real world guide to continuous testing.
https://techbeacon.com/real-world-guide-continuous-testing
Infosys internal artifacts

Author

Shashikala J
Senior Associate Consultant – IVS-FS
Shashikala J is a Senior Associate Consultant with 11 years of experience and
is currently working with IVS FS team in Infosys. In her ten years of experience
in financial services domain, Shashikala has worked on trade finance, equity
finance, Forex payments, and United States mortgage projects across six major
banking clients.

To know more about the topic please contact [email protected]

For more information, contact [email protected]

© 2019 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys
acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this
documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the
prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.

Infosys.com | NYSE : INFY Stay Connected