Contents

State of automation in modern enterprises p4 / How does a virtual

environment work? p6 / Existing methods of automation in a virtual
environment p10 / Automation capability analysis of Automation Anywhere
Enterprise v.11 in virtual environments p12

Robotic process automation

in a virtual environment

www.pwc.in
Robotic process automation in a virtual environment 3
State of automation
in modern enterprises

Enterprises in the service industry are now widely cognisant

of the applicability of robotic process automation (RPA)
techniques to achieve significant productivity gains while
being cost conscious. RPA’s ability to reduce processing
time and errors, enhance scalability and compliance, and
enable staff to focus on value-adding activities rather
than repetitive tasks has been vindicated across industries
and is now widely accepted. Firms which embraced these
automation techniques early in the game have been able to
gain significant benefits:

Cost reduction Quality and compliance

Software robotics brings immediate The automated nature of software
reduction in operational cost beyond robotics reduces errors and leaves a
labor arbitrage, and realises rapid digital audit trail that increases accuracy
return on investment. and regulatory compliance, enabling
programmable controls.

Revenue enhancement
Value focused talent Software robotics increases revenue
The priorities of the workforce shift growth through a shorter cycle time to
to innovation, strategy and other service customers.
business development activities. There is a one-to-one relationship between
a bot and the tasks it automates for a
single worker (full-time equivalent or FTE).
In others, a single bot automates tasks
performed by three to five workers. Savings
are easy to calculate.
Scalability
A virtual workforce can respond to growth
events (e.g. organic, acquisitive) with
speed, agility and resiliency. Robots 24/7
are managed from the control room and Robots never sleep and many of today’s
require little IT expertise. digitally enabled processes can be
orchestrated to operate autonomously 24x7,
driving real-time transactions.

Employee and customer satisfaction

Increased employee satisfaction through Speed to value and low risk
focus on higher value activities will, Software robotics has a quick time to
together with fewer errors, result in more RPA benefits delivery and avoids invasive traditional
satisfied customers. system integration. Weeks or months
instead of years.

Source: PwC analysis

4 PwC
In this era of technology disruption, enterprises are The RPA market is expanding rapidly, but to be sustained,
under immense pressure to digitise operations, and RPA must offer broader support for application types, and
they see a future where human work can be augmented for both virtual and cloud-based systems. As RPA continues
through the use of software robotics. These enterprises to progress both in capability and scope, business use cases
are beginning to view a digital workforce as part of their are expanding beyond mainframe, legacy, desktop, and
digital transformation strategy by combining elements of web applications into the realm of remote and virtualised
RPA, artificial intelligence (AI) and analytics to automate environments. We are already seeing a healthy mix of
business processes. While implementing organisations reap automation environments such as physical machines,
the above benefits of successful RPA implementation, the Windows servers, terminal servers, virtual machines (VMs),
RPA product vendors and service providers also undergo a virtual desktop infrastructure (VDI) and remote desktop
constant evolution cycle to tackle technical obstacles that protocol (RDP).
adopting organisations face in automating processes and
RPA tool integration/interaction with all applications/ Having said that, every RPA architect or developer who
systems within their environment. has tried to automate a process on a ‘virtual environment’
knows the pain that comes with building automations in
that environment. For the purpose of this paper, we will
focus on how the RPA tools access the remote applications
on virtual environments, insights into pain points, and
existing workarounds for automating such processes.
While there are multiple RPA solutions in the market, this
paper provides presents a case study on how Automation
Anywhere (a leading RPA service provider) has further
enhanced its current virtual environment automation
capabilities in their upcoming release of Automation
‘It was not long ago that we were talking Anywhere v.11.
to a potential client about the relative
merits of automation, especially robotic
process automation. Although the client
acknowledged the above merits to a great
extent, they put forth a list of challenges that
they had heard about RPA implementations
pertaining to the RPA tool capabilities on
virtualized environments (e.g. Citrix/remote
desktops/VDIs).’

– Sumit Srivastav, Intelligent Process

Automation Leader, PwC India

Robotic process automation in a virtual environment 5

How does a virtual
environment work?

There are many service providers in the market today

who provide server/cloud-based application and desktop
virtualisation for organisations (e.g. Citrix, VMware
Horizon, Microsoft RDP, Oracle’s Secure Global Desktop).
This helps organisations to remotely display an interface,
typically from a VDI instance, enabling individuals of an
enterprise to work and collaborate remotely regardless
of device or network. This allows Windows desktops
and applications that are located behind a firewall to be
accessed from any remote device with any operating system.

Server/desktop virtualisation renders the display of a

business application in the form of images from a desktop
instance running behind the firewall. Let’s call the device
behind the firewall the native device, and the device outside
the firewall the remote device. For clarity, the business
applications run only on the native device and the remote
device only runs the virtual display software.

Indicative application deployment over a virtual environment

User

Receiving virtual server Firewall

User

Remote device

Cloud-based

Virtual Virtual
desktops desktops

Business applications hosted on native device

6 PwC
It may be noted that Citrix is one of the most extensively
used applications and desktop virtualisations products
within financial services organisations. Having said that,
the challenges with using RPA tools to automate processes
on such environments apply to all remote display products.

Why do enterprises opt for virtual

environments?

Enhanced security and compliance

One important benefit that drives the use of virtual

environments is that they keep data secure behind
the firewall. They remotely render the user interface,
providing no access to the underlying application, data,
or user-interface objects. By rendering the display only,
the transactional data remains behind the firewall and
within the data centre boundaries. This enforces complete
isolation of business data from the user’s operating system
and other applications installed on it, thus ensuring
business applications and data are secure from any threats
or attacks. RPA vendors provide additional RPA-specific
security controls to ensure secure and compliant virtual
desktop execution. (See the section below on ‘managing
Citrix infrastructure’.)

Low cost

Host applications can be run on a few servers running the

native devices as VDI instances rather than on hundreds
of laptops or desktops. Any change in configuration or
application update needs to be done only on these few Flexibility
servers where the native devices reside. This also helps
organisations implement strategies like bring your own
device (BYOD) on the remote device, subsequently Users can access the provisioned application executing on
helping them save millions in management, device and the native device from anywhere with any remote device
licensing costs. like a smartphone or laptop.

High availability Maintenance

Improved failover strategies provide better 24x7 application Data backup and restoration can be easily done within the
availability. data centre where the application is hosted.
Robotic process automation in a virtual environment 7
So, what is so hard about automating a process on a virtual
environment? Some of the common challenges faced by
RPA developers are discussed below:

1 Managing the virtual

infrastructure

• Many enterprises with virtual installations will not

allow RPA to execute on the native device for security
and confidentiality reasons, which forces the RPA
tool to execute the automation on the remote device.
The preferred method to run RPA would be for the
automation to execute on a run-time installed on the
native device, enabling the RPA tool to read and use the
underlying UI elements (objects). Native devices typically
run on VDI instances from where the display will be
rendered to the remote devices. Building automations
on the native device makes development easier and
the resulting automation is more accurate and easier
to maintain.
• Remote devices typically reside outside a firewall,
travelling over the Internet using remote display
protocol. In this scenario, the automation executes on
the remote device running only the rendered images—
for example, a pixel display with no UI software
drawing the display. This was previously described as a
surface automation.

8 PwC
2 Developing automation on
virtual environments: Surface
automations
In a virtual environment, the bot is looking at a picture.
Therefore, developers have to rely on the image the bot sees
to automate the respective process step and eventually the
entire process, which is not foolproof. The term ‘surface
automation’ is frequently used to refer to automations
that rely on the picture/image. Although this may sound
simplistic to a human, many RPA platforms are unable to
accommodate this type of functionality to 100% efficiency
and surface automations are generally regarded difficult
and challenging for the reasons outlined below:
• RPA tools use the UI via Object IDs for automation and
this is not available in any virtualised environment.
• Coordinated use of pixel-by-pixel scanning of image,
anchors, X/Y coordinates and optical character
recognition (OCR) are required in order to correctly
locate and interact with the image. This can fail when the
screen resolution changes.
• For security reasons, sometimes the clipboard is not
enabled between the virtual server and client, making it
difficult to move data between the server and client. If
the clipboard is not enabled, one needs to find a way (e.g.
sending keystrokes) of working with the data. The task
would become even more difficult if the volume of data
is huge.
• Even with safeguards in place, additional control criteria
and dynamic delays need to be implemented to take care
of system latency.
• Automation becomes increasingly difficult on virtual
applications with screen objects like date pickers, which
cannot be controlled using the keyboard (wherein the
date needs to be selected using the mouse only).
As a result of the above challenges, developers can use only
image recognition, co-ordinate based (mouse clicks/
keystrokes) and OCR-based functions to automate in a
virtual environment. The next section provides insights on
how this can be achieved.

Robotic process automation in a virtual environment 9

Existing methods of automation in a
virtual environment

There are a few workarounds available to developers

automating processes on Citrix environments:

1 Coordinate-based automation

This is the most static automation approach where the

RPA tool searches the target UI element at predefined X/Y
coordinates. Once the image is found, the tool can perform
mouse actions on that image such as left click, right click
and double click. After that, data entry at that location
can be performed using keystrokes or data extraction can
be performed using OCR. Coordinate-based automation
provides fastest automation execution.
Failure modes: This method would fail if the screen
resolution or element/object position changes.
Workarounds:
• Explore if image-based automation or OCR-based
automation can be performed.
• Identifying elements on the screen and using them as an
anchor to then do a relative click on the actual element is
a temporary fix that can be put in place to avoid failures
due to screen resolution changes.
• Building utilities that check screen resolution changes
before triggering the process bot.

10 PwC
2 Image-based automation 3 OCR-based automation

In this approach, the RPA tool scans the business application In this approach, the business application image is
pixel by pixel, until the targeted UI element’s (object) image converted to text using OCR technology and the RPA tool
is found. Once the image is found, the tool can perform searches for the target application object by locating a
mouse actions on that image such as left click, right click specific text occurrence. Once the text is found, the tool
and double click. After that, data entry at that location can can perform mouse actions on that object (text) such as
be performed using keystrokes and data extraction can left click, right click and double click. After that, data entry
be performed using OCR. The RPA tool can be trained to at that object can be performed using keystrokes and data
search for a target image by configuring parameters such extraction can be performed using OCR.
as match percentage, comparison Mode (e.g. grayscale, Failure modes:
monochrome), tolerance and image occurrence.
Accuracy percentage (like to like match) of the data
Failure modes: extracted via the OCR functions is not 100% and, in some
Change in system display and UI properties. cases, may require manual human intervention to validate
Changes in the default font/style and colour properties of the data before this can be further processed by the bots.
the Windows apps and buttons. Workarounds:
Workarounds: • Try the automation using a different OCR engine since
• Configure RPA tool parameters such as match some business applications give best results with a
percentage, comparison mode (e.g. monochrome), particular OCR engine.
tolerance and image occurrence. • Explore if image-based or coordinate-based automation
• Explore if coordinate-based automation or OCR-based can be used.
automation can be used.

Robotic process automation in a virtual environment 11

Automation capability analysis of Automation
Anywhere Enterprise v.11 in virtual environments

The currently available mitigations to the potential Anywhere v.11. The table below compares and assess
automation roadblocks may not be perfect and a foolproof the abilities of the two versions and scores them on the
solution may not exist. Automation Anywhere has been parameters listed below:
working behind the scenes to tackle such roadblocks • Read and extract data;
to Citrix automation and is gearing up to bring about a
• Write/update data;
revelation in how Citrix-based automation will be done in
the near future. PwC was given early access to the latest • Launch applications on Citrix;
version of Automation Anywhere Enterprise v.11. We • Perform advanced functions such as match percentage
performed a thorough study and comparison of some of and tolerance (image comparison mode);
the key functions/commands used in the previous version • Search options and on-screen automations.
(Automation Anywhere v10.5) and the latest Automation

The table below provides a view on how Automation Anywhere Enterprise v.11 scores on the same assessment parameters
compared to the older version:

Segment Assessment dimension Automation Anywhere Automation Anywhere

V10.5 V11

Read/extract Scrape data

Scrape relative

Select and copy (editable fields)

Image occurrence

Bulk data extraction (out of the box)

Write/update Image and text occurrence

Click on a text in an image (OCR click)

Click relative to text/image

Set text

Bulk data entry (out of the box)

Launch apps Keyboard actions

Click image

Advanced Intelligent automation using SMART screen

automation technology

Comparison modes with match% and tolerance

Search a locally saved (PC) image

Resolution independent image recognition

Auto mode to search UI element

Overall capability

Best in class High Medium Low NA

12 PwC
Automation Anywhere Enterprise v.11 – a • Auto mode to search UI element: AI-powered smart
screen automation automatically switches between
preview
OCR mode and image mode to increase accuracy of the
Here are some of the key highlights and a quick insight into software robot.
Automation Anywhere Enterprise v.11. • Image and text occurrence: If the system searches
Next-generation virtual environment automation for a particular UI element through image or text
identification (OCR) and the given image/text is present
• AISense: Automation Anywhere has embedded AI, more than once on the screen, Automation Anywhere’s
machine learning (ML), natural language processing smart screen automation enables the user to pick the
(NLP) and computer vision (CV) to develop learning desired image or text. This is important because many
algorithms for their RPA suite of products. AISense IR/OCR methods pick the first occurrence of the image,
simplifies Citrix automation capabilities and becomes which may not be accurate.
more accurate as it learns from the user.
• Data burst: Automation Anywhere’s ‘data burst’
AISense uses computer vision to intelligently create technology inputs data from all UI elements after just
dynamic linking between objects by determining their a single scan of the application screen or image. This
composition. This intelligent linking delivers change- saves many hours of execution time compared to other
resilient automation. AISense excels in accurately techniques that require multiple scans of the same screen
automating applications even where labels and text before each data input.
change their relative position.
• Bulk data extraction: Automation Anywhere’s bulk data
AISense is enabled in cases where standard object-based extraction technology extracts data from all UI elements
automation is not available or unreliable, for example, in after just a single scan of the application screen or image.
applications exposed over Citrix, applications accessed Like data burst, this saves many hours of execution time
over remote desktop, or legacy applications like Delphi, compared to other techniques that require multiple scans
QT, or Flex. of the same screen before each data extraction.
• Support dynamic objects: AISense uses advanced Automation Anywhere has made significant changes to
Automation Anywhere has made significant changes to enhance
computer vision to understand the composition of a page. enhance the automation capabilities of its Enterprise
the automation capabilities of its Enterprise v.11 offering, whilst
This automation is resilient to changes in the location of v.11 offering, whilst it continues to maintain the ease at
it continues to maintain the ease at which both business and
UI objects, changes in format and color of the objects, which both business and technical users can automate
technical users can automate processes. On the basis of our
and changes in the structure of the objects. processes. On the basis of our study, we find that this
study, we find that this version is an interactive and easy-to-use
version is an interactive and easy-to-use RPA tool with
RPA tool with an equally impressive enhanced Citrix/virtual
• Resolution-independent image recognition: an equally impressive ‘enhanced Citrix/virtual desktop
desktop automation capability.
Automation Anywhere’s advanced image recognition automation capability.
engine can accurately locate application images even
when the machine resolution, scale or zoom level has Enterprises have seen and accepted that automation
changed. This ensures automation accuracy in cases on virtual environments/remote desktops using RPA
where development and production machines have tools is not as easy as standard UI-based automation. As
different configurations. other RPA vendors in the market are working on future
versions of their tools and similar enhancements, we will
bring about a series of thought papers to cover other key
players and product enhancements in the RPA world.

Irrespective of movement of text boxes or labels, the system is able to automate seamlessly.

Indicative example:
Irrespective of movements
to the UI elements
(username,password),
AISense ensures breakfree
automation.

Robotic process automation in a virtual environment 13

14 PwC
About PwC
At PwC, our purpose is to build trust in society and solve important
problems. We’re a network of firms in 158 countries with more than
2,36,000 people who are committed to delivering quality in assurance,
advisory and tax services. Find out more and tell us what matters to you
by visiting us at www.pwc.com

In India, PwC has offices in these cities: Ahmedabad, Bengaluru,

Chennai, Delhi NCR, Hyderabad, Kolkata, Mumbai and Pune. For more
information about PwC India’s service offerings, visit www.pwc.com/in

PwC refers to the PwC International network and/or one or more of its
member firms, each of which is a separate, independent and distinct
legal entity. Please see www.pwc.com/structure for further details.

© 2018 PwC. All rights reserved

Contact us
Sumit Srivastav
Partner and Intelligent Process Automation Leader
PwC India
[email protected]

Authors
Hariprasad Gajapathy
Ashwin Guptha
Shrinath Gadiyar
Sumit Srivastav
pwc.in
Data Classification: DC0
This document does not constitute professional advice. The information in this document has been obtained or derived from sources believed by
PricewaterhouseCoopers Private Limited (PwCPL) to be reliable but PwCPL does not represent that this information is accurate or complete. Any opinions
or estimates contained in this document represent the judgment of PwCPL at this time and are subject to change without notice. Readers of this publication
are advised to seek their own professional advice before taking any course of action or decision, for which they are entirely responsible, based on the
contents of this publication. PwCPL neither accepts or assumes any responsibility or liability to any reader of this publication in respect of the information
contained within it or for any decisions readers may take or decide not to or fail to take.
© 2018 PricewaterhouseCoopers Private Limited. All rights reserved. In this document, “PwC” refers to PricewaterhouseCoopers Private Limited (a limited
liability company in India having Corporate Identity Number or CIN : U74140WB1983PTC036093), which is a member firm of PricewaterhouseCoopers
International Limited (PwCIL), each member firm of which is a separate legal entity.
GM/May2018-13098