ISSN : 2347 - 8446 (Online) International Journal of Advanced Research in

ISSN : 2347 - 9817 (Print) Vol. 2 Issue Special 1 Jan-March 2014 Computer Science & Technology (IJARCST 2014)

An Efficient Framework for Unified Automation Testing: A Case

Study on Software Industry
I
A.Divya, IIMrs.S.Devi Mahalakshmi
I,II
Mepco Schlenk Engineering College, Sivakasi, Tamil Nadu, India

Abstract
Manual software testing becomes difficult, time consuming and costly as software systems evolve since it is performed by a human
sitting in front of a computer carefully going through application screens, trying various usage and input combinations, comparing
the results to the expected behaviour and recording their observations. Manual tests are repeated often during the development
cycle for source code changes and othersituations like multiple operating environments and hardware configurations. Therefore
tests should be automated to reduce testing efforts. Automated software testing is like writing the code to test the source code, which
performs predefined actions, compares the results to the expected behaviour and report the success or failure of these tests. In this
paper, automated tests are developed and extended to perform tasks impossible with manual testing in very less amount of time. In
order to make automated testing effective and efficient, so that it can reduce overall costs, new techniques and methodologies were
applied during the project like Modular Keyword Framework and hybrid approach. It is observed that automated software testing
is an essential component of successful development project.

Keywords
Modular Keyword Framework, Manual Tests, Automated software testing, Hybrid approach, Automated Tests

I. Introduction II. The Proposed System

Every software product needs to be tested adequately, but as
quickly and thoroughly as possible with minimal resources. To A. Design Goals
accomplish this goal automated testing is used. Automation Testing has found its place in the software industry,
Faced with this reality and realising that a big number of test cases with the crucial role that it plays in quality software production.
cannot be executed manually (For e.g. Simulating 1,000 virtual As business requirements grow, so does the pressure to deliver
users for volume testing), automated testing is introduced [1], the product with fewer resources, in reduced time and with high
[2]. In the environment of continual changes and additions to quality.
the software through each software build in rapid application
development, where requirements are encouraged to evolve,
software testing takes on an iterative nature itself. Each build is
accompanied by a considerable number of new tests as well as QA Team
rework to existing test scripts, just as there is rework on previously
released software modules. Given the continual changes and Review Fail
additions to software applications, automated software testing
becomes an important control mechanism to ensure accuracy and Manual
stability of the software through each build. Test Cases
Review
A test development effort may be as timeconsuming as effort
required developing the software application. Much of the
test effort required on a project now needs to be supported by Review
automated test tools. Manual testing is labour intensive and error Pass
prone. And it does not support the same kind of quality checks Test
that are possible with automation testing. Automation testing can Automation
replace manual test activities with a more efficient and repeatable Unit Test
automated test environment.
Automated test capabilities for software products include testing Maintenanc
Fail
Unit
of the graphical user interface, requirements compliance, load e
Test
performance, database validations, etc. New capabilities continue
to be added to keep pace with the growing demand of test support. Unit Test Pass
In regression testing which is a risk reduction exercise, it is needed Batch Test Fail
Batch
to find the bugs in the code as fast as possible with minimum effort. Batch
This can be achieved using automation testing [3]. Test Pass
Test
Automated Software Testing is the best way to increase the
effectiveness, efficiency and coverage of the software testing. Fig. 1: General Flow of test automation
Once automated tests are created they can easily be repeated and
they can be extended to perform tasks impossible with manual B. General Flow of test automation
testing. Because of this, it is found that automated software testing Based on the product/application/module requirement, types
is an essential component of successful development projects. of testing that need to be performed are identified. Testing
requirements and their nature are studied for the application.

www.ijarcst.com 15 © All Rights Reserved, IJARCST 2014

International Journal of Advanced Research in ISSN : 2347 - 8446 (Online)
Computer Science & Technology (IJARCST 2014) Vol. 2 Issue Special 1 Jan-March 2014 ISSN : 2347 - 9817 (Print)

Each requirement has its own actions, validations for testing. D. General flow of data creation
For example, contribute A test case may or may not need the pre-requisite test data. Pre-
• Scenario 1: For an application, form validation functionalities, requisite test data is nothing but the data that should already have
database validation and accessibility functionalities needs to been created or available before executing the automated test. The
be validated. test data is also created by the application through automation.
• Scenario 2: For an application, all the web-service methods The general flow of data is as follows:
need to be validated. This would also include the delay time
for the request’s response from third party systems.
Appropriate testing tool is selected which is best suited for the AUT. Prerequisites of Data Details Populate the
Test Case Gathering Controller Tables
Every tool has its own limitations. A feasibility study needs to be
conducted for the requirements against the tools. This study would
result in deciding the testing tool [4]. Test Automation framework
is developed which is to be used for developing and running
automated tests.QA team which is responsible for management
of all Quality Assurance functions writes test cases and does
Controller
implementation of test process. HP-QC is used as a repository
tables
for documenting these test cases. These well documented test management
Database
cases are then manually reviewed by test automation team step (Controller Tables+
by step. If the test case is manually passed, it is automated. If Application Tables)
manual review is failed (i.e. the application is not behaving the
way it is supposed to according to steps in the test case in HP-
QC), it is reported back to QA team since it can be a bug in the
Batch for Data Scripts to
application. Automating test case is nothing but writing a code Generate
Application
creation
to test the code. Automated tests are then unit tested. If unit test the data
passed, the test case goes for batch testing. If unit test failed, it is
again modified so that the execution of the automated test matches
the manually reviewed steps of the test case. All automated tests Fig. 2: Data Gathering and Test Data Creation Flow
of the same functionality are batched and integration testing is
done. The automated test cases which are failed in batch testing E. Steps for Data Creation and Maintenance
goes under maintenance and are corrected. Theses test cases again While manual review of a test case from HP-QC the pre-requisites
are unit tested and batch tested. Once the batch is passed, all these are analysed given in the details tab of the test case. If any pre-
automated test cases are delivered to QA team, so that these can be requisite data is needed, the details of the data to be created are
used for regression testing reducing the manual testing effort. gathered in a shared sheet by the test case developer. The Controller
tables are the tables that are used for storing automation data and
C. Design of test automation framework information. These tables are populated with the information that
Generally, testing is started with creating test scripts based on the the application will use while creating the particular data. There
scenarios. This includes multiple actions to be performed against are some indicators in the tables that will be used by data creating
each object. This approach leads to an ad-hoc test script creation scripts to identify which data is to be created and which is not.
and duplicate testing effort, i.e. testers, would create test scripts A batch is prepared for creating the data. This batch is nothing
for a single action in different scenarios. but a VB script which calls other multiple scripts according to
There is an approach that takes a different path as explained below. data creation needs. It is a VB script because we are using HP-
For designing a framework, various elements need to be taken QTP as automation tool for data creation of the AUT. Scripts
into consideration. Utilities/Components (re-usable) would be in the batch runs one by one and make the connection with the
designed for the following elements that include: database then fetch the information about the what kind of data
Actions to be performed - Identification of actions to be automated is to be created, in what amount from the controller tables[5].
for each object of the application and thus identify and abstract Then the data is created by the application through automation.
common functions used across multiple test scripts. When the data is created, application tables are populated with
• Database Communication - Database validation and check the information about the data that is created. This information
point validations is used during execution of the test cases for which the test data
• Data retrieval - Retrieval of data from multiple input data has been created. Once the test data is ready, the test developer
stores can run the automated test so that it will not fail because of data
• Error Handlers - Error handlers to handle known and unknown validation issue or data unavailability issue [6].
errors and log the information
• Custom Messages - Display of relevant defined messages III. The Proposed System
• Result Presentation - Customized and presentable reports on
completion of test execution A. Framework Design
• Easy to expand, maintain, and perpetuate. To start the testing with QTP, there is an important concept to
• Decouple test data from the test scripts. understand i.e. framework. Frameworks basically define the way
• Structure scripts with minimal dependencies - Ensuring to handle the different operations in different methods. The test can
scripts executing unattended even on failures be created in two ways in QTP i.e. by recording and by scripting.
[6] We should also know that how we can improve the quality

© 2014, IJARCST All Rights Reserved www.ijarcst.com

16
ISSN : 2347 - 8446 (Online) International Journal of Advanced Research in
ISSN : 2347 - 9817 (Print) Vol. 2 Issue Special 1 Jan-March 2014 Computer Science & Technology (IJARCST 2014)

of test by using Framework concept. QTP supports five kinds of TestCase flow according to Keywords
framework i.e. Linear, Modular, Keyword driven and Data driven
and Hybrid Framework.
Function fnApp_Login()
>>App_Login …........
1. Linear Framework
End Function
Linear Framework is very easy and catchy. Basically Linear >> Count_Unread_Mails Function
Framework deals with individual script which is recorded under fnCount_Unread_Mails()
one Action and running individual. So this is very simple and easy >>App_Logout ……..
way to create the test by navigating through the application. End Function
Function fnApp_Logout()
…........
2. Modular Framework End Function
Sometime when we test some business processes in one
application, we may feel that some operations we repeat in all
Linear Framework Scripts. To handle this situation we go with
Modularity which solves this kind of problem by dividing the Fig. 3: Mapping between keywords and functions
single test in multiple parts or modules.
For implementing modularity, we basically divide the test in Each keyword is associated with a separate function as shown
different parts so that we can form the functions for reusing. That in the figure.
solves many problems of repetition of some actions and gives the From the above image we can see that basically a Keyword
facility to make the script reusable. So in one sentence, Division Driven Framework is nothing but a collection of keywords &
of Linear Framework script in different parts called Modular functions (or actions or operations) and the association between
Framework. these two entities.

3. Keyword driven Framework 4. Data driven Framework

Tester should know the basic of programming because Keyword Data Driven Testing Framework is a framework which is driven by
driven Framework deals with functions. In programming, function test data, which means that test data is the important factor here.
is an important part of programming as they allow creating chunks The basic expectation with this kind of test automation framework
of code that performs a specific task. Basically we create the is scripts should be built in such a way that it should work for
functions forming the function library and calling these functions different sets of data without any changes to the test script.
as keywords in the tests therefore this becomes keyword driven Test cases that are executed multiples times—once for each
framework with modularity. This is very important and useful input of a given set of data use data driven approach. In
framework to deal and handle many critical tasks in testing through Linear Framework because the data is hard-coded within
QTP. script, we cannot use the same code with multiple data
For example: Consider that we have to automate a flow where we values. We overcome this shortcoming by using Data Driven
would need to do the following things in a Mailing Application. approach.
• Login to Application. Consider a scenario where we have a large number of user
• Count the number of unread emails in Inbox. credentials for an application. We need to login to the application
• Logout from Application. with each of these credentials to find out which of the user
If we look at the above test case from pure Modular Framework credentials are working properly and which are not. In order to
point of view, we will be able to easily identify that we would complete this task, we can create a script in QTP which would
need to write 3 different functions for the above test case. These read each of the user credentials as variables and try to login to
3 functions will be used to Login to Application, count unread the application and at the end would report the results.
emails and logout.
Once we have identified the functions, the next step is to identify 5. Hybrid Framework
some keywords and then associate the functions with these In this approach, we mix data driven approach with modular
keywords. Below figure shows a pictorial representation of this approach or with keyword driven approach. Scope of Hybrid
concept. framework is very high as we mix different approaches. This
approach is flexible for performing different tasks.
Test Case Flow
B. MoKey Framework
>> Login to Application MoKey is a modular keyword-driven automation framework.
>>Count unread mails The current version of MoKey is implemented using QTP and
>>Logout from Application VBScript.MoKey combines industry best practice concepts like
modular programming and keyword-driven testing into a single
Identify keywords for each flow hybrid framework MoKey reduces and simplifies automation test
Keywords can be anything with a meaningful name as follows and code maintenance. These tests are readable and easy to debug
the tests. It reduces the training requirements for new users.
>>App_Login
>>Count_Unread_Mails C. MoKey Structure
>>App_Logout The user interface to MoKey exercises the keyword-driven testing
concept. The code supporting the keywords is implemented using

www.ijarcst.com 17 © All Rights Reserved, IJARCST 2014

International Journal of Advanced Research in ISSN : 2347 - 8446 (Online)
Computer Science & Technology (IJARCST 2014) Vol. 2 Issue Special 1 Jan-March 2014 ISSN : 2347 - 9817 (Print)

the modular programming concept. Table 1: Test case datasheet written to be executed using MoKey
framework
Input/
Input Data Object
MOKEY Keyword Validation Comments
in Excel Name
Data
Start Web
StartTest
Application
Generic QTP MoKey MoKey
Enters value
Test Script Engine Functions Action Username John into user name
field
Enters value into
Password Mypwd
password field
Clicks on Sign-
QC AUT Keyword AUT Sign-In
Functions In
Validates
whether correct
Fig. 4: MoKey Structure Find a
Validate BrowserTitle screen
Flight
Appears
QTP Automated test: EndTest Ends test

Browser(“Welcome:Mercury Tours”).
Page(“Welcome:MercuryTours”).WebEdit(“userName”).Set D. Components of Framework
“keith”
Browser(“Welcome: Mercury Tours”).Page(“Welcome: 1. Function Library
M e r c u r y To u r s ” ) . We b E d i t ( “ p a s s w o r d ” ) . S e t S e c u r e Function libraries usually play a very important role. All the
“4678374e3e3faccb50be844f56637acb93d4fa07” necessary intelligence is built in the function library so that it
Browser(“Welcome: Mercury Tours”).Page(“Welcome: Mercury can read the excel sheets and call the different functions from the
Tours”).Image(“Sign-In”).Click 38,9 function libraries itself based on the Keywords. And so provides
MOKEY Test Script: modularity.
Call startTestScript (“D:\Automation\Demo\MercuryTours\Data\
data1.xls”) 2. Object Repository
Based on the design of Framework, we can use an object
MOKEY Data File: repository. An object repository is a very essential entity in any
UI automation tool. A repository allows a tester to store all the
All MoKey tests consist of a QTP test script and a data file as shown objects that will be used in the scripts in one or more centralized
above. The 1-line QTP script references the data file name.If we locations rather than letting them be scattered all over the test
automate that manual test case with MOKEY, it will result in an scripts. An Object repository is a file that maintains a logical
automated test script and a supporting input/validation data file. representation of each application object that is referenced in an
The test script will be one simple line that points to the supporting automated script. The purpose of the Object repository is to allow
data file. The data file comprises both the control and test data for the separation of application object property data known as the
and is stored in an Excel file. The data is represented as a triplet Physical Description, from the automated script via a parameter
of keyword, object name, input/validation data. The latter two are known as the Logical Name.
considered supporting parameters for the keyword.
E. Key Benefits of Framework
Keyword - represents a typical action or validation that a tester Standard process in Production: Due to use of Test automation
performs during a test step and can also represent business framework, a single standard is established across the organization.
processes or tasks. Keywords are used instead of the automation This helps the organization as the standard processes are followed
tool’s functions/methods. as compared to pre-empted ad-hoc processes, which yield no
Object - represents the object name that the keyword operates on. results.
Value - represents the input or validation data for the object. Action
- represents the default action performed on an object validate - 1. Free from Dependencies
represents the default validation of an object’s attribute Complete coding and component usage standards are defined in
Control Data: The data or part of the automated test that performs production. Independency from the individual coding standards
actions on AUT is called as Control data. and the utilities/components created. Complete documentation
Test Data: The data that is provided as input to the automated test helps the organization in inducting the new members with minimal
or perquisite is called as Test Data. effort.

2. Complete Coverage
Requirements are collected from an overall application’s
perspective. This overall coverage minimizes the testing effort

© 2014, IJARCST All Rights Reserved www.ijarcst.com

18
ISSN : 2347 - 8446 (Online) International Journal of Advanced Research in
ISSN : 2347 - 9817 (Print) Vol. 2 Issue Special 1 Jan-March 2014 Computer Science & Technology (IJARCST 2014)

during the later stages of the releases, for the entire product suite
across the organization. A.Divya received the B.E degree in
Future Enhancements Support: Automation team need not worry computer science engineering from
about testing future enhancements. Only minimal changes and Velammal College of Engineering and
the validations related to the enhancements need to be added to Technology, Madurai, India in 2012.She
the existing base framework and that too with minimal effort. did her Internship as a part of her Master’s
degree in Oracle India Private Limited,
Cost Estimation: The cost includes, Bangalore in 2013.She is currently a
• Acquisition cost - In the procurement process of the tool, student pursuing Master of engineering
following cost needs to be considered: Tool Cost, Cost based in Mepco Schlenk engineering college,
on number of licenses, based on our requirements and Version Sivakasi, India. Her research interests
Upgrade Cost. include Network Security, Wireless
• Training - Training cost incurred for training test engineers, Security and Networking.
business users, developers and creating supportive training
documentations must be taken into consideration.
• Environment - Cost involved in setting up the system
environment (Hardware and Software) must be taken into
consideration.
• Development - Development cost can be calculated based on
the components designed in the framework development.
• Maintenance - Each tool has its own maintenance
requirements.

IV. Conclusion
Test automation implemented during this project accelerated
the regression testing cycle and promoted software quality.
Automating tests and other repetitive tasks reduced time required
in regression testing and expanded the quality of testing. Test
coverage is increased by extending automation to parts of the
application that may not have been thoroughly tested in prior
releases. The use of Modular Keyword Framework and hybrid
automation approach further increased testing efficiency.
The Automation approach used for pre-requisite test data creation
has reduced time consumed in manual creation of test data. Pre-
batch developed for checking availability of test data before
running the test avoided tiresome work in manually checking test
data availability and thus drastically reduced the time consumed.
Test Automation required less manpower to perform tedious
processes and therefore cut man hours and costs.

V. Acknowledgement
This research was supported by Oracle India Private
Limited,Bangalore during my internship program (August 2013-
December 2013) in Unified Automation testing team- RGBU.

References
[1] http://www.automatedtestinginstitute.
[2] http://www.automationrepository.com.
[3] http://www.qtpselenium.com/
[4] http://www.softwaretestingmentor.com/automation/
[5] http://www.oracle.com/technetwork/tutorials/index.html
[6] Mark Fewster& Dorothy Graham (1999). Software Test
Automation. ACM Press/Addison-Wesley.

www.ijarcst.com 19 © All Rights Reserved, IJARCST 2014