A Comprehensive Test Automation Guide For IT Teams
A Comprehensive Test Automation Guide For IT Teams
A Comprehensive Test
Automation Guide for
IT Teams
E-guide
In this e-guide
In this e-guide:
A comprehensive test automation
guide for IT teams p. 2 This one-stop test automation guide walks through the benefits and
Why is automated testing important? challenges, how to craft an automated testing strategy, and how to
. p. 2 compare tools.
What are the advantages of test
automation? p. 4
Further reading p. 23
Page 1 of 23
E-guide
In this e-guide
Page 2 of 23
E-guide
such as Selenium, Appium, Cucumber, Silk Test and dozens more allow for the creation
In this e-guide of customized tests that can accommodate the specific needs of the software. Later in
A comprehensive test automation this test automation guide, we'll walk through the most popular and effective test
guide for IT teams p. 2 automation tools.
Further reading p. 23 Most software projects will still benefit from the attention of skilled QA testers who
can perform tests that are difficult to simulate with automated tools or that are
infrequent enough to justify the investment necessary to automate them. Automated
Page 3 of 23
E-guide
and manual testing are often performed together in varying degrees throughout the
In this e-guide development cycle.
A comprehensive test automation
guide for IT teams p. 2
Page 4 of 23
E-guide
especially when results can be compared with previous results to gauge resolution
In this e-guide effectiveness and efficiency.
A comprehensive test automation
guide for IT teams p. 2
Page 5 of 23
E-guide
testing brings to the development process. One way that manual testing adds value is
In this e-guide through the addition of technical notes and documentation that QA personnel create.
A comprehensive test automation Such documentation can prove invaluable to supplement test cases, create training
guide for IT teams p. 2 materials or build user documentation. For example, deep QA expertise in using
software can aid help desk operations.
Why is automated testing important?
. p. 2
When deployed together, manual QA and automated testing allow each to focus on its
What are the advantages of test strengths. Automated testing, for example, is often best suited to smoke and
automation? p. 4 regression testing tasks, while new development tasks can benefit from the agility of
manual testing. The biggest challenge in such shared responsibility is keeping both
What are the challenges of automated
testing? p. 5 automated and manual efforts organized and effective in the face of constantly
changing priorities.
What are the types of automated
testing? p. 6
How to perform automated testing What are the types of automated testing?
. p. 11
Automated software testing can apply a wide range of test types to check integrations,
Testing automation frameworks p. 18
interfaces, performance, the operation of specific modules and security. Testing is not
How to choose an automated testing restricted to a single test type, and multiple test types can be applied (layered) at the
tool p. 19 same time or performed in rapid succession to test a multitude of issues.
Page 6 of 23
E-guide
In this e-guide
Further reading p. 23
Page 7 of 23
E-guide
Testing automation frameworks p. 18 Smoke tests. Smoke tests are generally simple go/no-go tests designed to ensure that
an application still functions properly when a new build is completed. The tests often
How to choose an automated testing
tool p. 19 are used to determine whether the most important features or functions of the
application operate as expected and if the application is suitable for further, more
The future of test automation p. 21
detailed testing. For example, a smoke test might determine whether the application
Further reading p. 23 launches, an interface opens, buttons work or dialogs open and so on. If a smoke test
fails, the application may be too broken to justify further testing. At that point, the
Page 8 of 23
E-guide
application returns to the developers for retooling. Smoke tests are often referred to
In this e-guide as build-verification tests or build-acceptance tests.
A comprehensive test automation
guide for IT teams p. 2
API and integration tests. Communication and integration are vital aspects of modern
software. API testing is used to validate the interaction of requests and responses for
Why is automated testing important?
the application's APIs. These can involve various endpoints, including databases,
. p. 2
mainframes, UIs, enterprise service busses, web services and enterprise resource
What are the advantages of test planning applications. Not only does API testing look for reasonable requests and
automation? p. 4 responses, it also checks unusual or edge cases and evaluates potential problems with
latency, security and graceful error handling.
What are the challenges of automated
testing? p. 5
API testing is often included in integration testing. This provides more comprehensive
What are the types of automated testing of the application's modules and components to ensure that everything
testing? p. 6 operates as expected. For example, an integration test might simulate a complete
How to perform automated testing
order entry process that takes a test order from entry to processing to billing to
. p. 11 shipping and beyond so that every part of the application is involved from start to
finish.
Testing automation frameworks p. 18
User interface and input/output tests. The user interface (UI) represents the front end
How to choose an automated testing
tool p. 19 of any application, allowing the user to interact with the app. The UI itself may be as
simple as a command line interface or a well-designed graphical user interface (GUI).
The future of test automation p. 21
UI testing can be an intricate and highly detailed effort; the number of possible button-
Further reading p. 23 press sequences or command-line variations can be staggering.
Page 9 of 23
E-guide
Input/output (I/O) testing basically transforms input data into output data. For
In this e-guide example, an application intended to perform calculations and derive an output might
A comprehensive test automation use a sample data set and check the output to ensure that the underlying processing
guide for IT teams p. 2 functions correctly. I/O testing is often connected to UI testing because the data set is
frequently selected through the UI, and results may be graphed or otherwise displayed
Why is automated testing important?
. p. 2
in the UI.
What are the advantages of test Security and vulnerability tests. Security testing helps to ensure that the application --
automation? p. 4 and its constituent data -- remains secure in the wake of application faults and
deliberate attempts at unauthorized access. Security tests can check authorization
What are the challenges of automated
testing? p. 5 behaviors as well as common attack vectors such as SQL injection and cross-site
scripting.
What are the types of automated
testing? p. 6 Vulnerability testing is often performed on the code base before a build is executed.
How to perform automated testing
This checks the code for known flaws such as missing error handling in a subroutine or
. p. 11 an insecure configuration setting. Vulnerability testing is usually associated with
penetration, or pen, testing as a means of checking the security readiness of an
Testing automation frameworks p. 18
application or data center environment.
How to choose an automated testing
tool p. 19 Performance tests. An application may pass functional tests properly but still fail under
stress. Performance tests are intended to measure an application's key performance
The future of test automation p. 21
indicators, which can include computing load, traffic levels and scalability. In effect,
Further reading p. 23 performance tests are meant to simulate real-world conditions, often pushing the
application beyond its requirements until it fails. Such evaluation provides a baseline
Page 10 of 23
E-guide
for further development and a benchmark for adding limits or warnings to prevent
In this e-guide unexpected problems.
A comprehensive test automation
guide for IT teams p. 2
Acceptance tests. Software is developed using a software requirements specification
(SRS). The SRS contains acceptance criteria that outline the features and functionality
Why is automated testing important?
expected of the application. Acceptance tests are typically used to validate that the
. p. 2
criteria are in accordance with the SRS or other client documentation. In other words,
What are the advantages of test acceptance tests determine when the project is done. Because acceptance tests can be
automation? p. 4 extremely difficult to automate, they are generally reserved for late in a project's
development cycle.
What are the challenges of automated
testing? p. 5
Page 11 of 23
E-guide
In this e-guide
Page 12 of 23
E-guide
Developers and software QA/testing professionals are typically tasked to prepare tests,
In this e-guide and test code is often indistinguishable from other segments of code. In most cases,
A comprehensive test automation test code takes the form of scripts intended to execute certain behaviors in a
guide for IT teams p. 2 prescribed order. Some tests can also be generated automatically. One popular
example of this is record and playback testing tools, which create tests based on user
Why is automated testing important?
. p. 2
actions or behaviors.
What are the advantages of test Generally, record and playback tools match the user activities to a library of objects
automation? p. 4 that classify the behavior and then translate the object into code. That code forms the
basis of a script or other test data set. Once the tool generates a test, the test can be
What are the challenges of automated
testing? p. 5 reused, edited or merged with other tests.
What are the types of automated Record and playback testing helps QA teams develop tests that simulate user activities.
testing? p. 6 These could be UI tests, regression test or integration tests that implement and repeat
How to perform automated testing
complex sequences of actions. Such tools can also be used to check for performance
. p. 11 issues, such as to ensure that a feature or function responds within an adequate
timeframe.
Testing automation frameworks p. 18
Numerous automated software testing tools have record and playback capabilities,
How to choose an automated testing
tool p. 19 including Ranorex, Selenium and Microsoft Visual Studio. Selection of a tool will
depend on an organization's existing development needs and framework.
The future of test automation p. 21
An organization that chooses to adopt automated software testing usually grapples
Further reading p. 23
with scope -- defining exactly what automated testing is and what types of testing
need automation.
Page 13 of 23
E-guide
Further reading p. 23 Also, pay close attention to versioning and version control. New builds often need new
tests, and it is critical to maintain control over test versions. Otherwise, new test cycles
can fail or produce unusable results because the tool is implementing older -- and now
Page 14 of 23
E-guide
invalid -- test scripts of data. In practice, tests should observe the same version control
In this e-guide as any other codebase.
A comprehensive test automation
guide for IT teams p. 2
Even with the best tools and strategies, don't expect to fully automate testing for every
test requirement within a software project. Automation does many things well, and it
Why is automated testing important?
brings measurable value to the business, but people are also a crucial part of the
. p. 2
software QA process. When implementing automated testing, adopt a tool or platform
What are the advantages of test that supports the widest range of capabilities and scenarios possible. This puts
automation? p. 4 maximum value in the tool and minimizes the number of tools that developers and QA
testers must master. Look for logging capabilities that allow a comprehensive analysis
What are the challenges of automated
testing? p. 5 of test results.
What are the types of automated To minimize the learning curve, avoid the use of proprietary languages in test scripts
testing? p. 6 and other coding. Plus, use of those languages makes it difficult, if not impossible, to
How to perform automated testing
migrate to an alternate tool later without the need to rebuild all the scripts and other
. p. 11 test data from scratch. Test tools should ideally support common programming
languages, such as Ruby, Python or Java.
Testing automation frameworks p. 18
Page 15 of 23
E-guide
these are false errors, but it underscores the need to include test case development
In this e-guide and version control as a regular and recurring part of the software QA process.
A comprehensive test automation
guide for IT teams p. 2
There is no one way to implement such maintenance, but emerging tools seek to
democratize test automation knowledge by simplifying test creation. Tools such as
Why is automated testing important?
Eggplant Functional employ visual models, so testers need only minimal knowledge of
. p. 2
modeling and scripting to use them. They are also versatile for editing and easy
What are the advantages of test reporting. The goal is to make the tool and its tests far easier for newer users to
automation? p. 4 understand and create with -- and effectively allow more developers and testers to
participate in the software testing process. Tooling is usually augmented with
What are the challenges of automated
testing? p. 5 processes and policies that accommodate various team members in test creation and
implementation.
What are the types of automated
testing? p. 6
Further reading p. 23
Page 16 of 23
E-guide
In this e-guide
Further reading p. 23
Page 17 of 23
E-guide
In this e-guide
Test automation frameworks
A comprehensive test automation
guide for IT teams p. 2 Automated testing does not occur in a vacuum. The selection and implementation of
an automated testing tool won't be enough to deliver results. Successful test
Why is automated testing important?
. p. 2
automation demands careful attention to the guidelines, coding standards, reporting,
processes and workflows, and other groundwork involved in the test execution
What are the advantages of test environment. This mixture of concepts is referred to as the test automation
automation? p. 4
framework.
What are the challenges of automated
testing? p. 5
When implemented properly, a test automation framework can help developers and
testers create, execute and report on test automation events efficiently and uniformly
What are the types of automated
across projects and business units. Other benefits of a well-considered framework
testing? p. 6
include better code reusability, opportunities to automate testing across more code
How to perform automated testing (modules, components and even entire projects), easier maintenance and support of
. p. 11 the test automation tools, and less human intervention for manual QA testing.
Testing automation frameworks p. 18
Despite the potential benefits, automated software testing can pose serious challenges
How to choose an automated testing for an organization. It is critical to craft a test automation framework that defines and
tool p. 19 optimizes tests so that they can run with minimal human intervention.
Page 18 of 23
E-guide
deciding who codes and executes tests, who maintains the tools, the time and
In this e-guide circumstances when specific test types are executed, where test media will be stored
A comprehensive test automation and how test versions are managed.
guide for IT teams p. 2
The test automation framework requires strong policies for reporting, logging and
Why is automated testing important?
maintenance. This helps developers know where to quickly find test result reports or
. p. 2
execution logs, plus it speeds the fixing and remediation work to be done for the next
What are the advantages of test build. The tool often produces error logs in response to script problems -- not
automation? p. 4 necessarily application problems -- so accessing those related logs can over time help
script and test maintenance. Ultimately, a strong framework needs regular testing to
What are the challenges of automated
testing? p. 5 ensure that the tests are adequate and appropriate for the applications being
developed.
What are the types of automated
testing? p. 6
Page 19 of 23
E-guide
though there are serious tradeoffs in price, maintenance and support, and feature-
In this e-guide integration roadmap.
A comprehensive test automation
guide for IT teams p. 2
First, understand what problem the testing tool is intended to solve, and narrow the
field to the particular tools that can perform the required tests. Every tool is built to
Why is automated testing important?
address specific problems or perform certain tests, be it UI, documentation, API or
. p. 2
regression testing. It is possible that a tool can solve more than one problem. In fact,
What are the advantages of test some are promoted as all-in-one testing tools, while others are classified as stand-
automation? p. 4 alone products.
What are the challenges of automated
Next, determine whether the prospective tool fits the current development framework
testing? p. 5
of OSes and application languages. Also, be sure it integrates with other tools, such as
What are the types of automated version-control systems, along the software development workflow. For example, one
testing? p. 6 tool might not support a Java application in a web browser, while another tool might
How to perform automated testing
claim to support any platform -- as long as it's a Windows platform. Still other tools
. p. 11 might demand the addition of a web server, database or other supporting application.
Don't just consider what is being developed today; also think ahead to decide if the
Testing automation frameworks p. 18
tool will support future platforms.
How to choose an automated testing
tool p. 19 Third, examine less-desirable factors. As one example, a tool can be much easier to use
when it supports a developer-native language, such as Ruby, Python, .NET or Java. If
The future of test automation p. 21
the tool requires a unique or proprietary language, then users must learn yet another
Further reading p. 23 language for testing. That requires more time, and it will likely introduce errors into the
test process. Similarly, consider how the tool is licensed, paid for, installed and
secured.
Page 20 of 23
E-guide
Finally, take the automated tool for a test drive. Chances are the tool vendor offers a
In this e-guide free trial. Although the functionality may be limited, developers and testers should be
A comprehensive test automation able to determine whether the prospective tool meets the team's testing needs. It's a
guide for IT teams p. 2 good idea to use the tool under consideration in parallel with existing tools, enabling a
team to compare test results.
Why is automated testing important?
. p. 2
Ultimately, an organization will adopt more than one automated testing tool. This
What are the advantages of test allows developers and testers to create a toolbox capable of handling a multitude of
automation? p. 4 test types across a wide variety of software project requirements, from websites to
mobile apps to traditional desktop and client-server applications.
What are the challenges of automated
testing? p. 5
Page 21 of 23
E-guide
implement manually with existing tools. In addition, AI can help to locate gaps in
In this e-guide testing -- and even suggest ways to fill them.
A comprehensive test automation
guide for IT teams p. 2
Another avenue of evolution for automated software testing is robotic process
automation, or RPA, which is designed to perform repetitive operations as needed. For
Why is automated testing important?
software development, RPA technologies can mimic user actions and test the
. p. 2
interactions between disparate systems. The goal is to translate complex, multistep,
What are the advantages of test multisystem actions into repeatable and scripted processes. RPA, in effect, can bolster
automation? p. 4 end-to-end testing that evaluates customer activities and expectations for software
behaviors. One emerging use of RPA is in low-code software development platforms, in
What are the challenges of automated
testing? p. 5 which the tool stitches together existing templates to generate and then perform more
complex business functions.
What are the types of automated
testing? p. 6 Test automation, AI, RPA and low-code technologies are still in their infancy. The
How to perform automated testing
future for these technologies will ultimately rest in their business value and the
. p. 11 creativity with which they solve business problems. Ironically, success will depend on
deep human understanding of the business and its inner workings in order to find
Testing automation frameworks p. 18
inefficient or tedious tasks that will benefit from tomorrow's testing platforms.
How to choose an automated testing
tool p. 19
Further reading
The future of test automation p. 21
Further reading p. 23
Page 22 of 23
E-guide
In this e-guide
Further reading p. 23
Page 23 of 23