MSL

CHAPTER 4: QUALITY ASSURANCE, CONTROL, AND TESTING

The Concept of Software Quality: Quality Assurance (QA), Quality Control (QC), and
Testing
Software Quality
- It refers to different processes that work together to ensure the overall quality of the software
product.
Two Basic Levels of Software Quality
- Functional
o the product’s compliance with functional (explicit) requirements and design
specifications.
- Non-functional
o system’s inner characteristics and architecture, i.e. structural (implicit) requirement.

Quality Management Activities

- Refers to the actions and processes implemented throughout the lifecycle of a product or service
to ensure that it meets or exceeds the desired quality standards and requirements.
Quality Assurance
- It focuses more on organizational aspects of quality management, monitoring the consistency of
the production process.
Quality Control
- A part of quality management that verifies the product’s compliance with standards set by QA.
Testing
- The primary activity of detecting and solving technical issues in the software source code and
assessing the overall product usability, performance, security, and compatibility.

The Main Principles of Software Testing

1. Testing shows presence of mistakes – Testing aims to detect the defects within a piece of software.
2. Exhaustive testing is impossible – There is no way to test all combinations of data inputs, scenarios,
and preconditions within an application.
3. Early testing – As mentioned in the previous slides, the cost of an error grows exponentially
throughout the stages of the software development life cycle (SDLC).
4. Defect clustering – This principle is often referred to as an application of the Pareto Principle to
software testing.
5. Pesticide paradox - Running the same set of tests again and again won’t help you find more issues.
6. Testing is context dependent – Depending on their purpose or industry, different applications should
be tested differently.
7. Absence-of-errors fallacy - The complete absence of errors in your product does not necessarily
means its success.

The Role of Testing in Software Development Life Cycle

1. Waterfall Model-SDLC - A traditional software development life cycle, it has 6 consecutive phases
2. Agile Testing - Main purpose of such process is to deliver new software features fast and with the best
quality
3. DevOps Testing - This new software development methodology is a common practice. Often referred
to as an extension of Agile testing.
MSL

The Process of Software Testing in Practice

1. Test Planning: Artifacts & Strategy - Like any formal process, testing activities are typically preceded
by thorough preparations and planning.
- Seven types of Strategies:
o Analytical
o Model-based
o Methodical
o Process or Standard-compliant
o Dynamic
o Consultative or Directed
o Regression-averse
2. Design & Execution - As a starting point for the test execution, we need to identify what is subject to
testing.
- Software testing process:
o Static testing - examines the software project documents to prevent defects early in the
software testing cycle.
o Dynamic testing - the software is tested during execution.
▪ Process Segments:
• Software Testing Methods
• Software Testing Levels
• Software Testing Types
3. Documentation & Reporting - Agreed conditions to officially complete testing, ensuring all critical
tests are executed, defects are addressed, documentation is updated, and stakeholders are informed for
decision-making.

The Levels of Software Testing

Software Testing Levels - vary in the scope of what is checked, ranging from a single component to
an entire program as a whole.
Commonly, before release, the code goes through four test layers:
1. Unit/Component Testing - is the smallest testable part of the software system.
2. Integration Testing - The objective of this level is to verify that units work well together as a group
and also smoothly interact with other system elements — databases, external APIs, etc.
3. System Testing - At this level, a complete software system is tested as a whole. The stage verifies the
product’s compliance with the functional and non-functional requirements.
4. User Acceptance Testing - This is where the product is validated against the end-user requirements
and for accuracy.
- Three common stages of UAT:
o Alpha testing - performed by internal testers in a staging environment;
o Beta testing - handled by a group of real customers in the production environment to
verify that the app or website satisfies their expectations; and
o Gamma testing - when a limited number of customers check particular specifications
(primarily related to security and usability). At this stage, the product is 99 percent ready
for release, so developers won’t make any critical changes. Feedback is to be considered
in upcoming versions.

Software Testing Methods - are the ways tests at different levels are conducted.
1. Black box testing
- The main purpose is to check the functionality of the software, making sure that it works
correctly and meets user demands.
- Black box testing applies to any level but is mostly used for system and user acceptance testing.
2. White box testing
- White box testing aims to enhance security, reveal hidden defects, and address them.
MSL

- This method is used at the unit and integration levels.

3. Gray box testing
- The method combines the two previous ones, testing both functional and structural parts of the
application.
4. Smoke Testing
- is a popular white box technique to check whether a build added to the software is bug-free.
- It contains a short series of test runs, evaluating a new feature against critical functionality.
5. Ad hoc Testing
- This informal testing technique is performed without documentation, predefined design, and
test cases.
6. Exploratory Testing
- Similar to the ad hoc method, exploratory testing does not rely on predefined and documented
test cases.
- Instead, it is a creative and freestyle process of learning the system while interacting with it.

Types of Software Testing

1. Functional Testing
- Functional testing typically involves the following steps:
1. Outline the functions for the software to perform
2. Compose the input data depending on function specifications
3. Determine the output depending on function specifications
4. Execute the test case
5. Juxtapose the received and expected outputs
2. Performance Testing - System performance under varying workloads is assessed through different
types of performance testing.
- Load Testing — at continuously increasing workload
- Stress Testing — at or beyond the limits of the anticipated workload
- Endurance Testing — at continuous and significant workload
- Spike Testing — at suddenly and substantially increased workload
3. Use Case Testing
- It’s the most widely used testing technique, followed by exploratory testing
- Describes how a system will respond to a given scenario created by the user
4. Exploratory Testing
MSL

- It is an interactive and free-form process, with the main focus on validating user experience, not
code
- It has much in common with the ad hoc or intuitive testing but is more systematic
5. Usability Testing
- Performed from the end user’s perspective to see if the system is easy to use

Test Automation
- is critical in terms of continuous testing as it eases the burden of managing all of the testing
needs, allowing more time and effort to be spent on creating effective test cases
The Process of Test Automation
1. Preliminary Project Analysis - This is where you figure out what needs to be automated.
2. Framework Engineering - This involves building the foundation for your automated tests.
3. Test Cases Development - This stage translates manual test cases into automated scripts
4. Test Cases Implementation - This is the stage where planned tests become executable.
5. Iterative Framework Support - This step ensures the automation framework and the tests themselves
are maintained and updated.

Benefits of Test Automation

- Automation can be applied to almost every testing type, at every level.
- As a result, the automation minimizes the human effort required to efficiently run tests, reduces
time-to-market and the cost of errors because the tests are performed up to 10 times faster when
compared to manual testing process.
- Moreover, such a testing approach is more efficient as the framework covers over 90 percent of
the code, unveiling the issues that might not be visible in manual testing and can be scaled as
the product grows.

Test Automation in Numbers

According to the ISTQB® survey,
64.4 percent of their respondents
vote for test automation activities
as the main improvement area in
software testing. At the same time,
43,4 percent of the respondents
name test automation the top
challenge in Agile projects.
Here are the most striking
problems faced in applying test
automation based on the survey by
Katalon Studio.
Automated test activities
include test execution,
functional test case design,
test data generation, and
testing of end-to-end
business scenarios
However, the most effective
testing approaches combine
manual and automated
testing activities in order to
achieve the best results.
MSL

Regression Testing
- is the practice of verifying software behavior after updates to ensure that the changes haven’t
impacted existing system functions, stability, and overall integrity.
- Regression testing techniques:
o Retesting all test cases
o Selecting specific test cases
o Prioritizing test cases to verify the most critical ones first and then test the rest
o Hybrid techniques

The Future of Testing – As a part of technological progress, testing is continually evolving to meet
ever-changing business needs as it adopts new tools that allow the tester to push the boundaries of quality
assurance.
1. Security
- The World Quality Report survey shows that security is one of the most important elements of an
IT strategy. Input from security is vital to protecting the business.
- Security testing intends to uncover system vulnerabilities and determine how well it can protect
itself from unauthorized access, hacking, any code damage, etc.
- Four main focus areas in security testing:
o Network security
o System software security
o Client-side application security
o Server-side application security
2. Artificial intelligence
- The challenges of testing are increasing and their solutions have unlimited number of situations
requiring artificial intelligence to test them thoroughly.
- Different implementations of AI using machine learning-based algorithms will soon become
embedded in applications to perform tasks once reserved for humans.
3. Big Data
- Big data testing is aimed at checking the quality of data and verifying data processing.
- Big data testing demands a high level of testing skills as the processing is very fast.