ST Unit IV Notes
ST Unit IV Notes
UNIT – IV
Test Planning, Cost–Benefit Analysis of Testing, Monitoring and Control, Test Reporting, Test Control,
Specialized Testing, Object Oriented Testing, Automated Tools for Testing, Tool Selection and
Implementation, Challenges in Test Automation, GUI Testing.
COURSE OBJECTIVES:
• To understand the concepts of Object Oriented Testing..
COURSE OUTCOMES:
Upon completion of this unit, student will be able to:
Use Object Oriented Testing and understand the essence of automated testing tolls.
TEST PLANNING
A test plan is a detailed document which describes software testing areas and activities. It outlines the
test strategy, objectives, test schedule, required resources (human resources, software, and hardware),
test estimation and test deliverables.
Testing usually starts with test plan and ends with acceptance testing. Test plan identifies what
levels of testing will be done, what units will be tested, etc., in the project. Test plan is a template for
conducting software testing activities that are fully monitored and controlled by the testing manager.
Types of Test Plan: There are three types of the test plan
1. Master Test Plan: Master Test Plan is a type of test plan that has multiple levels of testing. It
includes a complete test strategy.
2. Phase Test Plan: A phase test plan is a type of test plan that addresses any one phase of the testing
strategy. For example, a list of tools, a list of test cases, etc.
3. Specific Test Plans: Specific test plan designed for major types of testing like security testing, load
testing, performance testing, etc. In other words, a specific test plan designed for non-functional
testing.
How to write a Test Plan: Making a test plan is the most crucial task of the test management process.
According to IEEE the following seven steps are used to prepare a test plan.
• Analyze product structure and architecture.
• Design the test strategy.
• Define all the test objectives.
• Define the testing area.
• Define all the useable resources.
• Schedule all activities in an appropriate manner.
• Determine all the Test Deliverables.
1
Lecture Notes for Software Testing
Test plan components or attributes: The test plan consists of various parts, which help us to
derive the entire testing activity.
1. Objectives: It consists of information about modules, features, test data etc., which indicate the aim of
the application means the application behavior, goal, etc.
2. Scope: It contains information that needs to be tested with respective of an application. The Scope can
be further divided into two parts:
In scope: These are the modules that need to be tested rigorously (in-detail).
Out scope: These are the modules, which need not be tested rigorously.
For example, in Gmail application, features to be tested are Compose mail, Sent Items, Inbox,
Drafts and the features which not be tested are Help, and so on. So in planning stage, we will decide
that which functionality has to be checked or not based on the time limit given in the product.
3. Test methodology: It contains information about performing a different kind of testing like
Functional testing, Integration testing, and System testing, etc. on the application. In this, we will
decide what type of testing; we will perform on the various features based on the application
requirement.
For example to test https://www.cricbuz.com/ application, we will perform following types of testing:
2
Lecture Notes for Software Testing
4. Approach: This attribute is used to describe the flow of the application while performing testing and
for the future reference. We can understand the flow of the application with the help of below aspects:
• By writing the high-level scenarios
• By writing the flow graph
5. Assumption: It contains information about a problem or issue which maybe occurred during the
testing process and when we are writing the test plans, the assured assumptions would be made like
resources and technologies, etc.
6. Risk: These are the challenges which we need to face to test the application in the current release and
if the assumptions will fail then the risks are involved.
For example, the effect for an application, release date becomes postponed.
7. Mitigation Plan or Contingency Plan: It is a back-up plan prepared to overcome the risks or issues.
The assumptions, risk, and mitigation or contingency plan are always precise on the product itself. The
various types of risks are as follows:
• Customer perspective
• Resource approach
• Technical opinion
8. Role & Responsibility: It defines the complete task which needs to be performed by the entire testing
team. For a large project, the Test Manager is a person who writes the test plan. If there are 3-4 small
projects, then the test manager will assign each project to each Test Lead. And then, the test lead
writes the test plan for the project, which he/she is assigned.
9. Schedule: It is used to explain the timing to work, which needs to be done or this attribute covers
when exactly each testing activity should start and end? And the exact data is also mentioned for
every testing activity for the particular date.
10. Defect tracking: It is generally done with the help of tools because we cannot track the status of each
bug manually. And we also comment about how we communicate the bugs which are identified
during the testing process and send it back to the development team and how the development team
will reply. Here we also mention the priority of the bugs such as high, medium, and low.
3
Lecture Notes for Software Testing
11. Test Environments: These are the environments where we will test the application, and here we have
two types of environments, which are of software and hardware configuration.
• The software configuration means the details about different Operating Systems and
various Browsers.
• The hardware configuration means the information about different sizes of RAM, ROM,
and the Processors.
12. Entry and Exit criteria: It is a necessary condition, which needs to be satisfied before starting and
stopping the testing process.
Entry Criteria: The entry criteria contain the following conditions:
• White box testing should be finished.
• Understand and analyze the requirement and prepare the test documents.
• Test data should be ready.
• Build or the application must be prepared
• Modules or features need to be assigned to the different test engineers.
• The necessary resource must be ready.
Exit Criteria: The exit criteria contain the following conditions:
• When all the test cases are executed.
• Most of the test cases must be passed.
• Depends on severity of the bugs i.e., there must not be any blocker or major bug.
13. Test Automation: In this, the following issues will be determined:
• Which feature has to be automated and not to be automated?
• Which test automation tool we are going to use on which automation framework?
• We automate the test case only after the first release.
14. Effort estimation: In this, we will plan the effort need to be applied by every team member.
15. Test Deliverable: These are the documents which are the output from the testing team, which we
handed over to the customer along with the product. It includes Test plan, Test Cases, Test Scripts,
RTM, Defect Report, Test Execution Report, Graphs and metrics, Release Notes.
Release Note: It is a document that is prepared during the release of the product and signed by the
Test Manager. Ex: Beta Releases
16. Template: This part contains all the templates for the documents that will be used in the product, and
all the test engineers will use only these templates in the project to maintain the consistency of the
product. Different types of the templates are:
• Test case template
• Test case review template
• RTM Template
• Bug Report Template
• Test execution Report
4
Lecture Notes for Software Testing
Test Plan Guidelines:
• Avoid overlapping and redundancy.
• Be specific. For example, when you specify a software system as the part of the test
environment, then mention the software version instead of only name.
• Avoid lengthy paragraphs.
• Use lists and tables wherever possible.
• Update plan when needed.
• Do not use an outdated and unused document.
Cost-benefit analysis facilitates a structured cost management process, helping project managers and
company executives prioritize projects and allocate resources effectively to achieve the organization’s
main goals.
5
Lecture Notes for Software Testing
Benefits may include:
1. Tangible benefits: These are measurable outcomes that can be easily quantified in monetary terms,
e.g., increased revenue or reduced costs.
2. Intangible benefits: These benefits are difficult to measure in monetary terms. They are indirect or
qualitative outcomes, such as improved customer satisfaction or increased employee morale.
• Although intangible benefits may be difficult to quantify in financial terms, it is necessary to
factor them in when conducting a CBA, as they still have a significant impact on the overall
value of a project.
6
Lecture Notes for Software Testing
• Include direct, indirect, opportunity, and future costs.
• After identifying the individual costs, assess the potential benefits of the project.
• Include all tangible and intangible benefits, even those that are difficult to quantify.
3. Add up all potential costs and benefits: Once you have your lists of costs and benefits, assign
current, realistic monetary values to each one and then sum up both sides. Ensure to use credible data
sources. Also, perform a project cost analysis and estimate the timing of the costs and benefits.
By accurately calculating the timing of the project costs and benefits, you can determine their
present value and evaluate the project’s financial feasibility.
4. Evaluate predicted outcomes: After adding both sides of the analysis, calculate the total cost and
benefit for the proposed project. If the benefits exceed the costs, the project may be worth pursuing.
Conversely, if the costs exceed the benefits, it’s advisable to reconsider investing in it.
At this stage, revisiting the project goals and objectives set out in the first step of the CBA is
helpful to check if the analysis shows you can achieve the set goals.
• Ask questions like:
• Can your organization pool the needed funds to cover all costs?
• Will the benefits come in time to keep business operations running smoothly?
• Next, conduct a sensitivity analysis to test the robustness of the CBA results.
• Finally, interpret the results of the CBA and communicate them to stakeholders to make informed
decisions about the project.
• A general rule of thumb to remember when evaluating project cost-benefit analysis is that the
costs should be less than 50% of the benefits, and the payback period should not exceed a year.
The Monitoring & Controlling process group includes eleven processes, which are:
1. Monitor and control project work: The generic step under which all other monitoring and
controlling activities fall under.
2. Perform Integrated Change Control: When changes to the schedule, cost, or any other area of the
project management plan are necessary, the program is changed and re-approved by the project
sponsor.
7
Lecture Notes for Software Testing
3. Validate Scope: Activities involved with gaining approval of the project's deliverables.
4. Control Scope: Ensuring that the scope of the project does not change and that unauthorized activities
are not performed as part of the plan (scope creep).
5. Control Schedule: The functions involved with ensuring the project work is performed according to
the schedule, and that project deadlines are met.
6. Control Costs: Tasks involved with ensuring project costs stay within approved budget.
7. Control Quality: Ensuring that the quality of the projects deliverables is to the standard defined in
the project management plan.
8. Control Communications: Providing communication needs of each project stakeholder.
9. Control Risks: Safeguarding the project from unexpected events that negatively impact the project's
budget, schedule, stakeholder needs, or any other project success criteria.
10. Control Procurements: Ensuring project subcontractors and vendors meet project goals.
11. Control Stakeholder Engagement: The tasks involved with ensuring that all of the project's
stakeholders are left satisfied with the project work.
TEST CONTROL
Test Control occurs based on the results of Test Monitoring. It refers to taking corrective action
based on test monitoring reports to improve quality and efficiency. Test control is about guiding and
corrective actions to try to achieve the best possible outcome for the project.
Some examples of test control activities would be:
• Prioritize testing efforts in a different way
• Reorganize test schedules and deadlines
• Restructure the test environment
• Reprioritize the test case and conditions
8
Lecture Notes for Software Testing
Test Control goes hand-in-hand with Test Monitoring. Obviously, once Monitoring identifies any
bottlenecks that may prevent a test cycle from meeting its goals, Control activities will have to come into
play to ensure otherwise.
Test Control is essentially modifying the testing process so that it becomes better suited for
meeting the defined objectives. This may require adding extra resources, reducing the scope of release or
splitting the release into multiple releases, etc.
Obviously, what specific test control activities will be implemented depends on a variety of factors –
stakeholders’ opinions, budget, project complexity, availability of testers, and the like.
TEST REPORTING
A software testing report is a synopsis of all the test activities of the team, test objectives, and the
results of a testing project. Test Reporting is the process of generating such reports.
• It helps in appraising how well testing was performed and identifying the reasons behind a failed
test report.
• It provides insights into how well the team tackled or resolved the defects.
• Test reporting analysis helps the stakeholders understand the quality of overall testing activities. It
helps them figure out the origin of the issue. It also helps to identify the grounds for negative test
results.
• It helps to analyze whether the problem occurred because of defective automation scripts,
mismanaged backend, unstable infrastructure, or weak implementation.
• Additionally, a software testing report should mention testing strategies, goals, and testing efforts.
• A well-organized testing report contributes not only towards the quality improvement of the
product, but also allows monitoring testing activities, and facilitates early product releases.
9
Lecture Notes for Software Testing
SPECIALIZED TESTING
Specialized testing is a type of testing that can be customized for an application from the
perspectives of enterprise users. The criteria can be changed in the test in different scenarios for the same
application. The source of criteria is a team of developers, users and business analysts.
Types of specialized testing include Web based application testing, Mobile application testing, Modular
testing, Agile testing, Cloud testing etc.
10
Lecture Notes for Software Testing
2. Scenario-based Testing: It primarily involves capturing the user actions then stimulating them to
similar actions throughout the test. These tests tend to search out interaction form of error.
3. Class Testing Based on Method Testing: This approach is the simplest approach to test classes.
Each method of the class performs a well defined cohesive function and can, therefore, be related to
unit testing of the traditional testing techniques. Therefore all the methods of a class can be involved
at least once to test the class.
4. Random Testing: It is supported by developing a random test sequence that tries the minimum
variety of operations typical to the behaviour of the categories
5. Partition Testing: This methodology categorizes the inputs and outputs of a category so as to check
them severely. This minimizes the number of cases that have to be designed.
11
Lecture Notes for Software Testing
1) Selenium: It is an open-source and most commonly used tool in automation testing. This tool is
used to test web-based applications with the help of test scripts, and these scripts can be written in any
programming language such as java, python, C#, Php, etc.
Features of Selenium:
• Selenium supports only web-based application, which means that the application can be opened by
the browser or the URL like Gmail, Amazon, etc.
• Selenium does not support a stand-alone application, which means that the application is not
opened in the browser or URL like Notepad, MS-Word, Calculator, etc.
• Selenium web driver is the latest tool in the selenium community that removes all the drawbacks of
the previous tool (selenium-IDE).
• Selenium web-driver is powerful as it supports multiple programming languages, various
browsers, different operating systems and also supports mobile applications.
2) Watir: Watir stands for web application testing in ruby, written in the Ruby programming language.
It is a web application testing tool, which is open-source and supports cross-browser testing. Humans
can validate the text, click on the links and fill out the forms.
Features of Watir:
• It supports various browsers like Google Chrome, Opera, Firefox, Internet Explorer, and Safari.
• Watir is a lightweight and powerful tool.
• We can easily download the test file for the UI.
• This tool has some inbuilt libraries to check the alerts, browser windows, page-performance, etc.
3) QTP: QTP tool is used to test functional regression test cases of the web-based application. QTP
stands for Quick Test Professional. it is also known as Micro Focus UFT [Unified Functional
Testing]. This is very helpful for the new test engineer because they can understand this tool in a few
minutes. QTP is designed on the scripting language like VB script to automate the application.
Features of QTP:
• QTP can test both desktop and web-based applications.
• QTP supports record and playback feature.
• QTP uses scripting language to deploy objects, and for analysis purposes, it provides test
reporting.
• Both technical and non-technical tester can use QTP.
• QTP supports multiple software development environments like Oracle, SAP, JAVA, etc.
4) Telerik Studio: It is modern web application which supports functional test automation. With this
tool, we can test the load, performance, and functionality of the web and mobile applications and also
identify the cross-browser issues.
12
Lecture Notes for Software Testing
Features of Telerik test studio:
• Telerik test studio allows us to deliver quality products on time.
• It supports all types of applications, such as desktop, web, and mobile.
• It supports Asp.Net, AJAX, HTML, JavaScript, WPF, and Silverlight application testing.
• It supports multiple browsers like Firefox, Safari, Google Chrome, and Internet Explorer for the
test execution process.
5) Testim: It can execute the test case in very little time and run them in various web and mobile
applications. This tool will help us to enhance the extensibility and stability of our test suites. It
provides the flexibility to cover the functionalities of the platform with the help of JavaScript and
HTML.
Features of Testim:
• The Test stability is very high in testim tool.
• This tool will support parallel execution.
• In this tool, we can capture the screenshots.
• With the help of this tool, we can perform requirements-based testing.
6) Applitools: This tool is used to check the look and feel and user's feedback on the application.
Applitools is a monitoring software, which provides visual application management and AI-powered
visual UI testing. It is an open-source tool that helps us to deliver a quality product.
Features of Applitools:
• This tool has active user access management.
• For various devices, it allows cross-browser testing.
• It will deliver the visual test reports to the users.
• It is available on the public and dedicated cloud.
13
Lecture Notes for Software Testing
• Consider the vendor’s reputation, after sale support, tool update frequency, etc. while taking
your decision.
• Evaluate the quality of the tool by taking the trial usage & launching a pilot. Many vendors
often make trial versions of their software available for download
3) Estimate Cost and Benefit: To ensure the test tool is beneficial for business, the Test Manager
should perform cost-benefit analysis before acquiring a tool. The balance between cost & benefit of
the tool may affect the final decision.
4) Make the Final Decision: To make the final decision, the Test Manager must:
• Have a strong awareness of the tool. He must understand strong and weak points of it.
• Balance cost and benefit.
• Conduct a meeting with the project team, consultants to get the deeper knowledge of the tool.
1. Team Collab Issues: With the growing technology, firms are now moving towards automation and
that has become a challenge for QA teams. The reason being they do lack a skill set and are not much
well aware of the backend codes. So, this becomes very challenging for them to work with the script.
This issue can be solved by keeping the whole team in a loop and making sure that they all are on
the same path where they can exchange information and communicate as per their needs.
2. Network Issues: During the automation testing, it is being observed that network stability &
disconnection has been one of the most challenging issues for QAs. The dedicated team needs to have
a good and stable connection, and if the connection is poor, it becomes hard to access databases, third-
party services, etc.
Any delay in testing, specifically due to network issues can cause delays in the whole
development cycle.
3. Cost Factor: Automation comes at a costly price especially during its initial phase. Going for an
open-source solution will lead you to train and maintenance costs, which makes it even from all
aspects so test manager need to make sure the management plans for it.
14
Lecture Notes for Software Testing
4. Right Expectations: It is a must to understand that expectations should always be right either it’s for
selecting testing tools or the methods you’re going to implement or maybe the outcome you’re
expecting. No matter how much automation we’re going to make but human interaction and
involvement are necessary and will play a very crucial role in fixing bugs and running tests.
So, make sure that you’re getting the right expectation with stakeholders and management so that
they can plan accordingly.
5. Testing Accuracy: If you try to run processes on outdated data, then it will show you the false
outputs which are the most common mistakes in automation. In Automation, DevOps needs a quick
response from testers in order to get more accuracy over the data. Thus, data relevancy and accuracy
are the only factors to achieve in testing accuracy.
6. Selection of Tool: The selection of the right tool is the most crucial part of the testing phase. Lack of
expertise can hamper the whole development cycle. Make sure to dig into these considerations before
selecting any tool for your project:
• Gather information about your project and its requirement.
• Technical support & assistance.
• Check for cross-browsing testing compatibility.
• Ease in maintenance
• Cost
7. Fixing Code smell: It is very important to address and fix up a bad code smell from any frame. A
Code smell is a part of a code that is not “technically wrong” but its implementation is chaotic and
becomes a pain during the execution. So, it’s a responsibility for both programmers and testers to take
care of the quality of code so that it doesn’t affect the development cycle.
However, it will also lead to maintaining a pace in project development and makes automation
more robust.
GUI TESTING
Graphical User Interface Testing (GUI) Testing is the process for ensuring proper functionality of
the graphical user interface (GUI) for a specific application. GUI testing generally evaluates a design of
elements such as layout, colors and also fonts, font sizes, labels, text boxes, text formatting, captions,
buttons, lists, icons, links, and content.
GUI testing processes may be either manual or automatic and are often performed by third-party
companies, rather than developers or end users.
15
Lecture Notes for Software Testing
• Sometimes the internal functions of the system work correctly but the user interface doesn’t then
GUI testing is good to have in addition to the other types.
1. Analog Recording: In analog recording, the testing tool basically captures specific mouse clicks,
keyboard presses, and other user actions and then stores them in a file for playback. For example, it
might record that a user left-clicked at position X = 500 pixels, Y = 400 pixels or typed the word
“Search” in a box and pressed the [ENTER] key on their keyboard.
2. Object based Recording: In object based recording, the testing tool is able to connect
programmatically to the application being tested and “see” each of the individual UI components (a
button, a text box, a hyperlink) as separate entities. It is able to perform operations (click, enter text)
and read the state (is it enabled, what is the label text, what is the current value) reliably regardless of
where that object is on the screen.
Manual testing is often not productive as sometimes the execution of manual testing is slow,
monotonous, and error-prone. If we want to release high-quality software appropriately, we should try
to automate our testing strategy insistently.
But in a current testing strategy, manual testing still plays an important role. Precisely talking about
GUI testing, the manual test engineer could have more subjective facets of the interface, like its look
and feel and usability.
16
Lecture Notes for Software Testing
2) Model-Based Testing: A model is a visual narrative of System performance which helps us to
understand and predict the system performance. Model-based Testing is testing such models.
The models are beneficial in order to develop a practical test case with the help of the system
requirements.
Some of the essential requirements need to be considered while executing the model-based testing
approach:
Create the model
Verify the inputs for the model
For the particular model analyze the expected result
Implement the tests
Balance the actual result with the expected result
An evaluation on added action on the model
Compared to the other GUI testing approaches, model-based testing provides the benefits of fixing the
adverse states that GUI can accomplish.
We can obtain the test cases with the help of some the other model-based approaches:
Decision Tables: The decision tables are use to control the outputs for each related input.
Charts: The charts technique represents the state of a system and tests the state after some input.
3) Record and Replay: We can perform the GUI testing with the help of Automation tools, which can
be completed in two types. Throughout the record part, the test steps are encapsulated by the
automation tool. And in the playback, these recorded test steps are implemented on the application
under test. Ex: QTP.
The automation approach is the most common technique in which the GUI automated testing
demonstrates itself is beyond record-and-playback methods.
17
Lecture Notes for Software Testing
As its name recommends, the record and replay approach depends on having a test engineer use a
specific tool to record a testing session.
The significant benefit of the Record and replay approach is that it doesn't need any coding skills to
use it. The major drawback of record-and-replay tests is their weakness.
4) Hybrid Tests: This type of testing is a beneficial technique for non-technical background users to
develop a test case by recording their sessions. After that, the user who is familiar with coding can
further control these recorded tests technically. The person who has the coding knowledge can further
manipulate these recorded tests to modify them for more complex situations.
5) Code-based Testing: To develop test cases by using the code the GUI testing provides some GUI
testing tools. To discover more difficult test scenarios, we can use the code-based testing approach.
The perceptible disadvantage of the code-based methods that they either involve us to take developers
away from coding and have them write test cases or teach our test engineer to code or programming.
GUI Testing Tools: In order to identify the bugs or defects, which occurred during the design phase, we
will use the Graphical User Interface (GUI) testing tools that help us improve the quality of the
software. By using these tools, we can easily detect the loopholes rather than implementing the GUI
testing manually.
Based on the application behavior, we will test the application that is involved the mouse and keyboard
actions and some of the additional GUI items such as Dialog boxes, buttons, Menu bars, toolbars, and
the edit fields.
18
Lecture Notes for Software Testing
Some of the most commonly used GUI testing tools are as follows:
Ranorex Studio
Eggplant
Squish
AutoIT
RIATest
19