Course Name- Software Testing and Automation

Course Code-COM-702(B)
Lecture No-

Topic – Testing frameworks

Date- 14-10-2024

Model Institute of Engineering & Technology

Course Outcomes

By the end of the course, students shall be able to:

CO1 Understand the concepts of testing automation applications.

CO2 Analyze the variations in the framework and its suitability to various projects

CO3 Understand various testing tools

CO4 Understand what kind of testing carried out in different applications

CO5 Gain understanding of applicability of popular testing framework

 Selecting Appropriate Frameworks
3

1. Project Requirements
2. Team Skills / Learning Curve
3. Budget
4. Ease of Test Case Creation and Maintenance
5. Reporting
6. Support for Collaboration
7. Community Support and Documentation
8. Flexibility and Extensibility:

Test Automation Fundamentals. Unit 3

 1.Project Requirements
4

list down project requirements and the problems to solve.

• Type of application that needs to be tested: It could be web,

mobile, API or a desktop application.

• Platforms that need to be tested: If yours is a desktop

application, list down the operating systems that should be

tested. If yours is a mobile application, then list down the
supported mobile operating systems. If yours is a web
application, then list down the supported browsers.
• Language your application is built in: This can help if you are

planning to use a programming language for automation.

• Need for cross-browser testing/cross-device testing: If

yours is a web application or a mobile application then you

will, most probably, need this.
Test Automation Fundamentals. Unit 3
 2. Team Skills / Learning Curve
5

When selecting a tool for automation there could be 2 types of tools:

• A codeless test automation tool

• An automation tool that requires coding

 If the team already has people that are skilled in some programming
language then using an automation tool in that programming language is
suitable.
 if the plan is to hire skilled people for automation then this option not
suitable at that time.
 But, if the planning is to have an automation tool that will not need to look
for people with the required skillset, then codeless automation tools
will be a better option. These tools allow the automation of test cases
without the need for knowing a programming language.
Test Automation Fundamentals. Unit 3
Coding vs. Non-Coding Testing Frameworks
6

 Merits of Using a Coding-Required Testing Framework

1. Flexibility and Customization: Frameworks that require coding (e.g.,
Selenium, JUnit, PyTest) provide greater flexibility in defining complex test
scenarios that might not be feasible with codeless frameworks.
2. Integration with Development Environments: These frameworks can be
seamlessly integrated into IDEs and CI/CD pipelines, enabling automated
builds and testing.
3. Control and Debugging: Code-based frameworks offer deeper control over
test execution and easier debugging of test scripts.
4. Version Control: Test scripts written in code can be easily versioned and
stored in source control management systems alongside the application code.
5. Community and Resources: Coding-based frameworks often have large
communities and extensive documentation.

Test Automation Fundamentals. Unit 3

Coding vs. Non-Coding Testing Frameworks
7

 Demerits
 Time-Consuming: Writing, maintaining, and updating test scripts can be time-
consuming, especially as the complexity of the application grows. This might lead
to longer development cycles.
 Resource Intensive: Effective use of these frameworks often requires skilled
resources, which may not be readily available or could be costly for some
organizations.
 High learning curve: Intense learning and skillset may be required

 Examples of Testing Frameworks

• Coding-Required Frameworks: Selenium (for web applications, requires
scripting in languages like Java, Python, C#), JUnit (primarily for unit testing in
Java), PyTest (a powerful tool for testing Python applications).
• Codeless Testing Frameworks: Tools like TestComplete, Leapwork, or Tricentis
Tosca offer a graphical user interface to create tests without writing code, focusing
on a more visual approach where you can drag and drop test components.
 3. Budget
8

 This is a very important criteria for selection of testing tool. The following points
can be considered while selecting framework:

• Cost of human resources being used for automation: If there is a tool that does not
need to hire new resources, especially for automation, consider it a saving.
• Time spent on learning the tool: If there is a tool that has a low learning curve, that
is an indirect saving in the cost one might have spent in terms of the time the
resources spend on learning the tool. Or hiring resources that are skilled in that
particular tool.
• Time being spent on automation: If there is a tool that makes it easy to create and
maintain test cases, thereby saving time, consider it a saving in cost.

Test Automation Fundamentals. Unit 3

 3. Budget
9

 Budget-Friendly Testing Frameworks

1. Selenium: This is an open-source framework that supports multiple
programming languages and browsers. It's free to use, which makes
it a budget-friendly option for many organizations. However,
because it requires coding skills, the cost could increase based on
the need for skilled personnel or training.
2. JUnit (for Java) and PyTest (for Python): These are also open-
source and free to use. They are mainly used for unit testing and are
popular due to their extensive plugin ecosystems which extend their
functionality at no extra cost.
3. TestNG: Similar to JUnit but with more advanced features like
annotations, grouping, and sequencing of tests. It's free and open-
source, suitable for both small and large projects.
Test Automation Fundamentals. Unit 3
 3. Budget
10

 More Expensive Testing Frameworks

1. QTP/UFT (Unified Functional Testing): This is a commercial tool
provided by Micro Focus, commonly used for functional and
regression testing. It is considered expensive due to its licensing
fees. However, it offers a robust set of features including a powerful
scripting language, good integration with other enterprise tools, and
extensive support which can justify its cost.
2. Tricentis Tosca: Known for its model-based testing approach,
Tricentis Tosca minimizes the maintenance required for test scripts,
but it comes at a high cost due to licensing fees. The platform also
provides extensive automation capabilities across mobile, API, and
GUI testing which can be a significant advantage for enterprises
requiring comprehensive testing solutions.
Test Automation Fundamentals. Unit 3
4. Ease of Test Case Creation and
Maintenance
 Not every tool is made to handle all kinds of scenarios. So, to make
sure that the chosen tool meets needs, try automating a few test cases
of your application to know if the tool suits the needs. That could be
done with the trial version of a tool if your search has narrowed down
to premium tools.

 Also, to avoid spending more time in test case maintenance as

compared to test case creation, make sure to choose a tool that fits the
budget including the maintenance costs.
 5. Reporting

 Good reporting capabilities can help in quickly identifying issues and

assessing the health of the software. The framework should provide
clear, informative, and customizable reports.

 Few of them would be:

• Screenshots for failed steps
• Video for test execution
• Stack trace for the error
• A clear indication of failed test cases/steps
• Time taken for execution of test steps and test cases is reported
 6. Support for collaboration

 If you are doing automation of a project for a client, the

client will want to review the quality of automated test
cases.
 It will also be beneficial if other non-technical members
of the team are able to automate/review the test cases.
So, in such scenarios, look for tools that make
collaboration with the management and clients easy.
 6. Community Support and
Documentation
 A strong community and comprehensive documentation can
greatly help in troubleshooting and learning the framework.
Well-supported frameworks are more likely to receive updates
and security patches.

7. Flexibility and Extensibility:

 The framework should be flexible enough to handle various
test cases and be extensible to include custom functionalities
as needed. This could involve the ability to write plugins or
modify its behavior.
Some popular frameworks

1. Selenium

• Primary Use: Web application testing across various browsers and

platforms.
• Language Support: Supports multiple programming languages including
Java, C#, Python, Ruby, and JavaScript.
• Key Features: Open-source and widely used in the industry.
• Supports all major browsers and operating systems.
• Integrates well with frameworks for parallel testing like TestNG and JUnit.
• Extensive community support and resources.
 Typical Use Cases: Automating web applications for testing purposes,
creating robust, browser-based regression automation suites and tests, and
scaling and distributing scripts across many environments.
Some popular frameworks

1. Appium

• Primary Use: Mobile application testing.

• Language Support: Supports languages that work with Selenium like
Java, Ruby, Python, PHP, JavaScript, and C#.
• Key Features: Open-source and cross-platform.
• Allows testing of both native and hybrid mobile applications.
• Utilizes WebDriver protocol for testing.
• Typical Use Cases: Suitable for organizations looking to automate testing
for iOS and Android mobile applications without modifying the apps for
automation purposes.
Some popular frameworks

 3. QTP/UFT (Quick Test Professional/Unified Functional Testing)

• Primary Use: Functional and regression testing.
• Language Support: Uses VBScript for scripting.
• Key Features:
• Commercial tool offered by Micro Focus.
• Supports a wide range of applications including web, desktop, SAP,
Java, Oracle, and mobile apps.
• Integrated with an IDE and offers record and playback features.
• Extensive reporting and logging capabilities.
• Typical Use Cases: Suitable for enterprises looking for an all-in-one
solution for functional testing across different platforms and
technologies with minimal coding.
Some popular frameworks

4. TestComplete
• Primary Use: Automated UI testing.

• Language Support: Supports scripting in JavaScript, Python,

VBScript, DelphiScript, C++Script, and C#Script.

• Key Features:

• Commercial tool developed by SmartBear Software.

• Provides a powerful and robust record and playback feature.
• Offers object recognition capabilities that can automatically detect and
update UI objects.
• Supports data-driven testing and keyword-driven testing.
• Typical Use Cases: Ideal for teams that require a tool with strong
GUI testing capabilities, including complex desktop applications and
rich client applications.
 Summary
19

 Selecting framework  Popular framework

1. Project Requirements 1. Selenium
2. Team Skills / Learning 2. Appium
Curve 3. QTP/UFT
3. Budget 4. TestComplete
4. Ease of Test Case Creation
and Maintenance
5. Reporting
6. Support for Collaboration
7. Community Support and
Documentation
8. Flexibility and
Extensibility:
Test Automation Fundamentals. Unit 3