DESIGN AND IMPLEMENTATION OF AN HYBRID SOFTWARE TESTING

MODEL.

CHAPTER 1

INTRODUCTION

1.0 INTRODUCTION

Software testing is an essential process in software development that helps ensure that
software products are reliable, effective, and efficient. A Hybrid software testing model
combines elements from different testing models to create a unique testing approach that best
suits the software development project. However, a combination of an Automated and
Manual software testing model will be considered in this article. Manual testing involves
human testers who manually execute test cases and record the results. Automated testing uses
software tools to execute test cases and compare the actual results to the expected results.
Both methods have their advantages and disadvantages. Manual testing is more flexible and
can identify certain types of defects that automated testing cannot, while automated testing is
more efficient and can test large amounts of code quickly. The combination of automated and
manual software testing can provide several benefits to software development teams,
including increased testing coverage and more efficient use of resources. However, to
effectively implement a combination of automated and manual testing, it is important to
consider the design and implementation of the testing model. This module will discuss the
key design and implementation considerations for a combination of automated and manual
testing system.

1.1 STATEMENT OF PROBLEM

The following problems were identified:

 Automated testing can be fast, repeatable, and reliable, making it ideal for testing
large amounts of code and catching regressions quickly. However, automated tests
may not catch all the edge cases and complex scenarios that manual testing can detect.

 Manual testing, on the other hand, can provide a more nuanced and exploratory
approach to testing, enabling testers to uncover unexpected bugs and usability issues.
However, manual testing can be time-consuming, labor-intensive, and prone to human
error.

The challenge, therefore, is to find the optimal combination of both testing models that can
effectively detect defects and improve the quality of the software, while minimizing costs and
time to market. This requires a well-planned and coordinated testing strategy, as well as the
use of appropriate testing tools and techniques.

1.2 OBJECTIVES

1. To design a hybrid software testing model that combines the strengths of automated
and manual testing system testing models to improve software quality assurance.
2. To implement the hybrid software testing model in a real-world software development
project to evaluate its effectiveness.
 3. To discuss the challenges and limitations of implementing a hybrid testing approach
in software development projects.
4. To provide recommendations for software development teams looking to adopt a
hybrid testing approach.

1.3 SIGNIFICANCE OF THE STUDY

The significance of the study are:
 It will help institutions and organizations evaluate the effectiveness of a software
product.
 The study will also serve as a useful reference material to other researchers seeking
for information concerning the subject.

1.4 SCOPE OF THE STUDY

This study covers design and implementation a hybrid software testing model using
automation and manual testing system.
 CHAPTER 2

LITERATURE REVIEW

2.1 BACKGROUND OF STUDY

Testing refers to the process of determining whether or not a given system meets its intended
requirements. It primarily consists of a validation and verification process to determine
whether the developed system meets the user's requirements. Therefore, this activity results
in a difference between actual and expected result. Software Testing is a method to check
whether the actual software product matches expected requirements and to ensure that
software product is defect free. It involves execution of software/system components using
manual or automated tools to evaluate one or more properties of interest. The purpose of
software testing is to identify errors, gaps or missing requirements in contrast to actual
requirements.

Software Testing can also be considered as a risk-based activity. The important thing during
testing process, the software testers must understand how to minimise a large number of tests
into manageable tests set, and make wise decisions about the risks that are important to test or
what are not [1]. Figure 1 shows the testing cost and errors found a relationship.

The Figure 1 clearly shows that cost goes up dramatically in testing both types i.e. functional
and nonfunctional. The decision making for what to test or reduce tests then it can cause to
miss many bugs. The effective testing goal is to do that optimal number of tests so that extra
testing effort can be minimised [1].

Fig. 1 Graph showing optimal amount of testing [1]

Previous research has shown that professional programmers makes average six defects per
1000 lines of code [2]. Software tests allow us to compare the design and architecture with
the functionality of software to learn the precision captured in the development process.
Intensive tests are conducted to analyse the behavior of various input spaces.
A study conducted by Microsoft indicated that it takes about 12 programming hours to
manually locate and correct software defects. At this rate, it could take 11.4 years to debug a
program of 350000 lines of code with costs in millions [2]. The results demonstrate the
requirement for methods to test software effectively and efficiently. Testing is made less
cumbersome, effective, and more efficient by implementing automated and manual testing
methods. Apart from speeding up the testing process, it also increases the amount of test
coverage that can be done in a limited time resulting in higher quality software.

A. TESTING
The testing process involves planning, test case creation, execution of the test, validation, and
debugging. A strong test strategy that focuses on testing new functionality in a dedicated
testing environment is required for a successful test [3]. After cycles of revision are made on
the software application, regression testing is done to ensure that the new functionalities and
modifications have not adversely affected the previously existing code. The software is tested
with both good and bad test cases to observe the reaction and stability of the software.

Generating test cases and expected outcomes for every possibility of execution is labor-
intensive and time-consuming. Testing should continue for as long as the costs of finding and
correcting defects are lower than the potential cost of a software failure after release to the
end-user.

Testing is considered to be a phase that can be initiated after fundamental development

initiation. But it is recommended to perform testing throughout the development process as
the costs to correct defects increases rapidly throughout the development cycle. After every
level of development, unit system, integration, and acceptance testing are initiated to execute
debugging with minimal risk.
 Unit testing validates various modules individually.
 System testing is verifying each subsystem's interfaces and interdependencies
separately, then merging them to test the system as a whole to check the fulfillment of
functionalities specified in SRS.
 Integration testing incorporates all system components, including hardware, software,
and human interaction, to guarantee that system requirements are met in actual or
simulated scenarios.
 Acceptance testing is usually carried out by users and is performed using a BETA
release of the software.
The expense of repairing software problems after they have been released can be 100 times
higher than the cost of preventing the same issue from developing in the first place [2]. As
software has become more complex, new testing techniques have been developed to improve
the effectiveness and efficiency of the testing process. These include:
 Automated testing: Using tools and scripts to automate the testing process and reduce
the time and effort required for testing.
 Exploratory testing: Testing the software by exploring its functionality and features
without preconceived test plans or scripts.
 Agile testing: Testing the software in an iterative and collaborative manner, in line
with agile software development methodologies.
B. TYPES OF SOFTWARE TESTING
Software testing is divided into 2 types
 Manual software testing: Testing application without any tools or program code.
 Automation software testing: Testing application using program codes and tools

I. MANUAL SOFTWARE TESTING

Manual software testing is a process of testing where a tester tests the software/application
manually. This kind of testing does not involve any coding or programs instead testing is
done based on the test cases written. End-to-end testing can be achieved by this method. All
the scenarios that cannot be automated have to be tested manually. Although manual testing
is time consuming it is still practiced in almost every software companies for its reliability.
Manual testing is the most primitive of all testing types and helps find defects in the software
system. A person doing manual testing should be patient, have good observing skills,
knowledge about the product/software they are testing. Each bug is tested manually in the
software by a manual software tester to assure the quality of the software. The process
involved in manual software testing is explained in next section.

How to perform manual testing

Here’s how to perform manual testing step by step:
 Analyze requirements from the software requirement specification document
 Create a clear test plan
 Write test cases that cover all the requirements defined in the document
 Get test cases reviewed by the QA lead
 Execute test cases and detect any bugs
 Report bugs, if any, and once fixed, run the failed tests again to re-verify the fixes

II. AUTOMATION SOFTWARE TESTING

Automation software testing is type of testing which involves scripts and codes to run a
particular test. An automation tester writes the code for a test case or scenario to reduce the
amount of time taken for manually testing the same scenario again and again. Even though to
run the automation script and testing a scenario takes very less time, the development of the
script is time consuming. When there is an urgent requirement to test and deliver the software
it is better to choose manual testing procedure. Automation testing is also used to test the
application from load, performance, and stress point of view. It increases the test coverage,
improves accuracy, and saves time and money in comparison to manual testing. Automation
tools like selenium, eclipse, maven, git etc. are used in companies according to their needs.

Tests that can be automated:

1. Background processes, file logging, and database entry are examples of hard-to-reach
areas of the system.
2. Frequently utilised features with a high risk of error: payment systems, registrations,
and so on. Because crucial functioning tests take several minutes on average,
automation assures quick errors.
 3. Load tests, which examine a system's functioning in the face of a huge number of
requests.
4. Template operations, such as data searches, the entry of multi-field forms, and the
verification of their preservation.
5. Validation messages: erroneous data should be filled in the fields, and the validation
should be checked.
6. End-to-end situations that take a long time.
7. Data verification including precise mathematical computations, such as accounting or
analytical processing.
8. Verifying the accuracy of the data search. [4]

2.2. ADVANTAGES OF COMBINING MANUAL AND AUTOMATION SOFTWARE

TESTING:
1. Greater test coverage: Automated testing can cover a wide range of test scenarios,
while manual testing can provide human expertise and judgment to identify issues that
automated testing may not detect.
2. Enhanced testing effectiveness: While manual testing can provide useful feedback on
user experience and usability, automated testing can save time and money by
performing tests that take a long time to complete.
3. Early detection of defects: During the early stages of the software development life
cycle, automated testing can aid in the identification of flaws, thereby reducing the
time and money required to address them later.
4. Consistent testing: Automated testing can ensure consistent test execution, reducing
the risk of human error in manual testing [5].
5. Improved test accuracy: Automated testing can provide more accurate and reliable
test results compared to manual testing, reducing the risk of false positives or false
negatives [6].

2.3. DISADVANTAGES OF COMBINING MANUAL AND AUTOMATION SOFTWARE

TESTING:

1. High setup costs: Automated testing requires significant upfront investment in tools,
infrastructure, and resources, making it expensive to implement.
2. Limited test coverage: Automated testing may not cover all possible test scenarios,
requiring manual testing to supplement test coverage.
3. Maintenance costs: Automated testing requires ongoing maintenance and updates,
adding to the cost and time required to maintain the test suite [7].
4. False sense of security: Automated testing may create a false sense of security,
leading to the belief that all critical tests have been covered when there may be gaps
in test coverage.
5. Lack of human judgment: Automated testing cannot replace human judgment, which
is required to identify issues that may be missed by automated tests.

2.4. THE BEST PRACTICES FOR COMBINING AUTOMATED AND MANUAL

TESTING

Figure 2 includes the main parts of the systems development processes and the sequence
mechanics of these parts or phases in a logical arrangement, which insures the suitability of
this model to different phases of development systems, either small, medium or large. The
primary shape of the Hybrid model includes the following processes:

1. Planning: Contains the planning of necessary tasks to define the resources and
timelines, and making plans for the system development process and other
information that is related to the project. [8]

2. Requirements: Three types of requirements can be determined in this phase:

 Abstract functional requirements: determine the system's functions in
idealistic methods.
 System properties: define the non-functional requirements for the system.
 Undesirable characteristics: determine which system behavior is
unacceptable.
These requirements must define and determine the overall organizational objectives of
the system. [9]
 System objectives
 Functional objectives.
 Organizational objectives.
In this phase, the systems engineer should understand the information range for the
software. In addition to this, the functions, behavior, and performance should be
considered. At the end of this phase, software and system requirements should be
documented and revised with the customer. [10]

3. Design: A Process that has many steps. It contains four important parts:
 Data structuring.
 Software building.
 Data representation.
 Processes details (algorithms).
The requirements for the design process are transformed into a representation for the
software which can be evaluated for good quality, before starting the following phase, i.e,
"Implementation". Through this phase, the design is documented to become a part of the
software collection. [11]
4. Implementation: This transforms the design into formula which computers can read.
Also, it structures the system components by using one of the programming languages
according to its planning. [12]

5. Integration development: This connects the different components of a system into

one community and subsequently forms one formal system. In this phase, the different
parts should complete each other without any negative effect on the rest of the system
components. [13]

6. Deployment: This means sending the system after complete to the customer for use
and work in order to show the problems based on its use for the first time. [1]

7. Testing: The testing process starts with the first phase of any system, namely
"planning'.' It includes the planning test, requirements collection test, design,
representation and integration. [14]
 8. Maintenance: Software undergoes some modifications after handing it over to the
customer. These modifications occur because of the difficulties faced. Other reasons
may be the necessity of adjusting to the changes in the external environment or to the
customer's requests to make improvements in performance or function. [9]

9. Risk Analysis: This phase includes all development phases starting with planning
processes and finishing at maintenance processes. It lists all expected risks and
suggests all the necessary activities to reduce such risks. [15]

It is known that the process of developing medium and large systems starts with the
planning phase. In this phase, system requirements, like human cadre, time, materials and
costs are determined. Small projects, on the other hand, usually start with the designing or
programming phase, in which the plan is tested to find out whether it is appropriate to apply it
to the project. Moreover, this phase is used to assess project risks that are related to
programming, human and material factors. This phase is followed by the requirements of
collection and analysis. Through this phase, the requirements are gathered by contacting
customers, then analyzed based on customers' decisions whether such requirements fulfill the
system objectives or not (taking into account the existing risks). The designing phase comes
after the requirement phase. Here, the system is primarily designed in papers with algorithms
and diagrams showing system progress. In this phase, the design is also tested to make sure of
its appropriateness to the requirements. The system representation phase follows all the
previous phases. This phase is an implementation for the design of computer devices
according to certain specifications shown in the plan. In addition, the implemented
programming parts are tested for the purpose of assuring their appropriate application. The
integration phase immediately follows the system representation phase. This phase is
important, as all implemented programming parts are integrated and tested for two purposes.
The first one is to ensure that they are free of any faults and the second is to decide whether
they fulfill all system requirements. This phase should be completely implemented before
loading the system for customers. The last phase is spreading and loading the system on
customer's devices. This phase is one of the most critical and important phases and should be
considered according to plan and risks analysis. [12][9]
The hybrid model is characterized by its appropriateness to all small, medium and large
systems and its flexibility in this model, one can start with any phase, whether planning,
requirements collection and analysis, designing, or programming representation in
accordance with the system type and complexity.
 Fig. 2 Best practices for combining automated and manual testing

2.5. EFFECTIVENESS AND EFFICIENCY OF HYBRID SOFTWARE TESTING

There are several factors that can influence the effectiveness of combining automated and
manual testing. Here are some of the most commonly cited factors:
1. Test case selection: The effectiveness of combining automated and manual testing
depends on selecting the right test cases for automation and manual testing based on
their complexity, importance, and business criticality. [16]

2. Test environment: The effectiveness of combining automated and manual testing

depends on ensuring that the testing environment is stable and consistent across
different testing phases. [17]

3. Test data management: The effectiveness of combining automated and manual

testing depends on ensuring that the test data used for automation and manual testing
is relevant, accurate, and consistent. [18]

4. Test automation framework: The effectiveness of combining automated and manual

testing depends on using a well-designed test automation framework that supports
both automation and manual testing. [6]
5. Test team collaboration: The effectiveness of combining automated and manual
testing depends on fostering collaboration and communication between the
automation and manual testing teams. [5]
 6. Test result analysis: The effectiveness of combining automated and manual testing
depends on analyzing the test results from both automation and manual testing to
identify and prioritize defects. [7]

7. Test execution scheduling: The effectiveness of combining automated and manual

testing depends on scheduling the test executions for both automation and manual
testing in a way that optimizes the use of resources. [19]

8. Test feedback loop: The effectiveness of combining automated and manual testing
depends on establishing a feedback loop between automation and manual testing to
ensure that the testing results are shared and acted upon. [20]

Testing is defined as a process of evaluation that either the

specific system meets its originally specified requirements or
not. It is mainly a process encompassing validation and
verification process that whether the developed system meets the
requirements defined by user. Therefore, this activity results in a
difference between actual and expected result. Software Testing
refers to finding bugs, errors or missing requirements in the
developed system or software. So, this is an investigation that
provides the stakeholders with the exact knowledge about the
quality of the product.
Software Testing can also be considered as a risk-based
activity. The important thing during testing process the software
testers must understand that how to minimise a large number of
tests into manageable tests set, and make wise decisions about
the risks that are important to test or what are not [1].
Figure 1 shows the testing cost and errors found a
relationship. The Figure 1 clearly shows that cost goes up
dramatically in testing both types i.e. functional and non-
functional. The decision making for what to test or reduce tests
then it can cause to miss many bugs. The effective testing goal is
to do that optimal amount of tests so that extra testing effort can
be minimised [1].
According to Figure 1, Software testing is an important
component of software quality assurance. The importance of
testing can be considered from life-critical software (e.g., flight
control) testing which can be highly expensive because of risk
regarding schedule delays, cost overruns, or outright
cancellation [2], and more about this [3][4].
Testing is defined as a process of evaluation that either the
specific system meets its originally specified requirements or
not. It is mainly a process encompassing validation and
verification process that whether the developed system meets the
requirements defined by user. Therefore, this activity results in a
difference between actual and expected result. Software Testing
refers to finding bugs, errors or missing requirements in the
developed system or software. So, this is an investigation that
provides the stakeholders with the exact knowledge about the
quality of the product.
Software Testing can also be considered as a risk-based
activity. The important thing during testing process the software
testers must understand that how to minimise a large number of
tests into manageable tests set, and make wise decisions about
the risks that are important to test or what are not [1].
Figure 1 shows the testing cost and errors found a
relationship. The Figure 1 clearly shows that cost goes up
dramatically in testing both types i.e. functional and non-
functional. The decision making for what to test or reduce tests
then it can cause to miss many bugs. The effective testing goal is
to do that optimal amount of tests so that extra testing effort can
be minimised [1].
According to Figure 1, Software testing is an important
component of software quality assurance. The importance of
testing can be considered from life-critical software (e.g., flight
control) testing which can be highly expensive because of risk
regarding schedule delays, cost overruns, or outright
cancellation [2], and more about this [3][4].
Testing is defined as a process of evaluation that either the
specific system meets its originally specified requirements or
not. It is mainly a process encompassing validation and
verification process that whether the developed system meets the
requirements defined by user. Therefore, this activity results in a
difference between actual and expected result. Software Testing
refers to finding bugs, errors or missing requirements in the
developed system or software. So, this is an investigation that
provides the stakeholders with the exact knowledge about the
quality of the product.
Software Testing can also be considered as a risk-based
activity. The important thing during testing process the software
testers must understand that how to minimise a large number of
tests into manageable tests set, and make wise decisions about
the risks that are important to test or what are not [1].
Figure 1 shows the testing cost and errors found a
relationship. The Figure 1 clearly shows that cost goes up
dramatically in testing both types i.e. functional and non-
functional. The decision making for what to test or reduce tests
then it can cause to miss many bugs. The effective testing goal is
to do that optimal amount of tests so that extra testing effort can
be minimised [1].
According to Figure 1, Software testing is an important
component of software quality assurance. The importance of
testing can be considered from life-critical software (e.g., flight
control) testing which can be highly expensive because of risk
regarding schedule delays, cost overruns, or outright
cancellation [2], and more about this [3][4].
Testing is defined as a process of evaluation that either the
specific system meets its originally specified requirements or
not. It is mainly a process encompassing validation and
verification process that whether the developed system meets the
requirements defined by user. Therefore, this activity results in a
difference between actual and expected result. Software Testing
refers to finding bugs, errors or missing requirements in the
developed system or software. So, this is an investigation that
provides the stakeholders with the exact knowledge about the
quality of the product.
Software Testing can also be considered as a risk-based
activity. The important thing during testing process the software
testers must understand that how to minimise a large number of
tests into manageable tests set, and make wise decisions about
the risks that are important to test or what are not [1].
Figure 1 shows the testing cost and errors found a
relationship. The Figure 1 clearly shows that cost goes up
dramatically in testing both types i.e. functional and non-
functional. The decision making for what to test or reduce tests
then it can cause to miss many bugs. The effective testing goal is
to do that optimal amount of tests so that extra testing effort can
be minimised [1].
According to Figure 1, Software testing is an important
component of software quality assurance. The importance of
testing can be considered from life-critical software (e.g., flight
control) testing which can be highly expensive because of risk
regarding schedule delays, cost overruns, or outright
cancellation [2], and more about this [3][4]

CHAPTER 3

METHODOLOGY
The methodology outlines the procedures and methods that was used to design and
implement a hybrid software testing model. The research design considered for this study is a
mixed-method approach that incorporates both quantitative and qualitative research methods.
Fifteen (15) respondents were interviewed using Google Hangout, WhatsApp, Stack and
Trello via chat while ten (10) were engaged in a face-to-face interview using an open-ended
questionnaire.

REFERENCES:

[1]. P. Ron. Software testing. Vol. 2. Indianapolis: Sam’s, 2001.

[2]. Candea, George, Stefan Bucur, and Cristian Zamfir. "Automated software testing as a
service." Proceedings of the 1st ACM symposium on Cloud computing. 2010
[3]. Huang, Zhenyu, and Lisa Carter. "Automated solutions: Improving the efficiency of
software testing." Issues in Information Systems 4 (2003): 171-177.
[4]. Rafi, Dudekula Mohammad, et al. "Benefits and limitations of automated software
testing: Systematic literature review and practitioner survey." 2012 7th International
Workshop on Automation of Software Test (AST). IEEE, 2012
[5]. R. Shelke, "How to Combine Manual and Automated Testing to Achieve Quality
Assurance," DZone, 2018.
[6]. D. Patel, "Manual Vs Automated Testing: Which One to Choose and When?"
Software Testing Help, 2019.
[7]. A. van Veenendaal, "How to Integrate Manual and Automated Testing," Agile
Testing Days, 2019.
[8]. Barry Boehm edited by Wilfred J. Hansen, "Spiral Development: Experience,
Principles, and Refinements", 2000.
[9]. Roger S Pressman, "Software Engineering: A Practitioner's Approach", 7th edition,
McGrawHill, 2009.
[10]. Steve Easterbrook, "Software Lifecycles", University of Toronto Department of
Computer Science, 2001.
[11]. Rlewallen, "Software Development Life Cycle Models" , 2005
[12]. Lan Sommerville, "Software Engineering", Addison Wesley, 7th edition, 2004.
[13]. Karlm, "Software Lifecycle Models", KTH,2006 .
[14]. Lifecycle Models , National Instruments Corporation , 2006.
[15]. B. W. Boehm, " Classics in Software Engineering", Yourdon Press -Upper Saddle
River, NJ, USA, Pages: 323 – 361, ISBN:0-917072-14-6, 1979.
[16]. S. Guggenheim, "The Benefits of Automated Testing," IBM, 2018.
[17] J. Bataller, "Why Automated Testing is Essential for Software Development,"
Techopedia, 2017.
[18]. K. Smalley, "Manual Testing vs. Automated Testing," Techwell, 2016.
[19]. J. Walsh, "How to Use Exploratory Testing in Your QA Process," SmartBear, 2018.
[20]. B. Hamill, "The Pros and Cons of Automated Testing," Rainforest QA, 2019.