WHAT IS SOFTWARE QUALITY?

Quality Glossary Definition: Software quality assurance (SQA)

Software quality is defined as a field of study and practice that describes the desirable attributes
of software products. There are two main approaches to software quality: defect management
and quality attributes.

SOFTWARE QUALITY DEFECT MANAGEMENT APPROACH

A software defect can be regarded as any failure to address end-user requirements. Common
defects include missed or misunderstood requirements and errors in design, functional logic,
data relationships, process timing, validity checking, and coding errors.

The software defect management approach is based on counting and managing defects. Defects
are commonly categorized by severity, and the numbers in each category are used for planning.
More mature software development organizations use tools, such as defect leakage matrices
(for counting the numbers of defects that pass through development phases prior to detection)
and control charts, to measure and improve development process capability.

SOFTWARE QUALITY ATTRIBUTES APPROACH

This approach to software quality is best exemplified by fixed quality models, such as ISO/IEC
25010:2011. This standard describes a hierarchy of eight quality characteristics, each
composed of sub-characteristics:

1. Functional suitability
2. Reliability
3. Operability
4. Performance efficiency
5. Security
6. Compatibility
7. Maintainability
8. Transferability
What is Quality Control?

Quality control is a set of methods used by organizations to achieve quality parameters or

quality goals and continually improve the organization's ability to ensure that a software
product will meet quality goals.

Quality Control Process:

The three class parameters that control software quality are:

• Products
• Processes
• Resources
The total quality control process consists of:
• Plan - It is the stage where the Quality control processes are planned
• Do - Use a defined parameter to develop the quality
• Check - Stage to verify if the quality of the parameters are met
• Act - Take corrective action if needed and repeat the work

Quality Control characteristics:

• Process adopted to deliver a quality product to the clients at best cost.

• Goal is to learn from other organizations so that quality would be better each time.
• To avoid making errors by proper planning and execution with correct review process
McCall’s Factor Model

This model classifies all software requirements into 11 software quality factors. The 11 factors
are grouped into three categories – product operation, product revision, and product transition
factors.
• Product operation factors − Correctness, Reliability, Efficiency, Integrity, Usability.
• Product revision factors − Maintainability, Flexibility, Testability.
• Product transition factors − Portability, Reusability, Interoperability.

Product Operation Software Quality Factors

According to McCall’s model, product operation category includes five software quality
factors, which deal with the requirements that directly affect the daily operation of the
software. They are as follows −
Correctness
These requirements deal with the correctness of the output of the software system. They
include −
• Output mission
• The required accuracy of output that can be negatively affected by inaccurate data or
inaccurate calculations.
• The completeness of the output information, which can be affected by incomplete data.
• The up-to-datedness of the information defined as the time between the event and the
response by the software system.
• The availability of the information.
• The standards for coding and documenting the software system.
Reliability
Reliability requirements deal with service failure. They determine the maximum allowed
failure rate of the software system, and can refer to the entire system or to one or more of its
separate functions.
Efficiency
It deals with the hardware resources needed to perform the different functions of the software
system. It includes processing capabilities (given in MHz), its storage capacity (given in MB
or GB) and the data communication capability (given in MBPS or GBPS).
It also deals with the time between recharging of the system’s portable units, such as,
information system units located in portable computers, or meteorological units placed
outdoors.
Integrity
This factor deals with the software system security, that is, to prevent access to unauthorized
persons, also to distinguish between the group of people to be given read as well as write
permit.
Usability
Usability requirements deal with the staff resources needed to train a new employee and to
operate the software system.

Product Revision Quality Factors

According to McCall’s model, three software quality factors are included in the product
revision category. These factors are as follows −
Maintainability
This factor considers the efforts that will be needed by users and maintenance personnel to
identify the reasons for software failures, to correct the failures, and to verify the success of
the corrections.
Flexibility
This factor deals with the capabilities and efforts required to support adaptive maintenance
activities of the software. These include adapting the current software to additional
circumstances and customers without changing the software. This factor’s requirements also
support perfective maintenance activities, such as changes and additions to the software in
order to improve its service and to adapt it to changes in the firm’s technical or commercial
environment.
Testability
Testability requirements deal with the testing of the software system as well as with its
operation. It includes predefined intermediate results, log files, and also the automatic
diagnostics performed by the software system prior to starting the system, to find out whether
all components of the system are in working order and to obtain a report about the detected
faults. Another type of these requirements deals with automatic diagnostic checks applied by
the maintenance technicians to detect the causes of software failures.

Product Transition Software Quality Factor

According to McCall’s model, three software quality factors are included in the product
transition category that deals with the adaptation of software to other environments and its
interaction with other software systems. These factors are as follows −
Portability
Portability requirements tend to the adaptation of a software system to other environments
consisting of different hardware, different operating systems, and so forth. The software
should be possible to continue using the same basic software in diverse situations.
Reusability
This factor deals with the use of software modules originally designed for one project in a
new software project currently being developed. They may also enable future projects to make
use of a given module or a group of modules of the currently developed software. The reuse
of software is expected to save development resources, shorten the development period, and
provide higher quality modules.
Interoperability
Interoperability requirements focus on creating interfaces with other software systems or with
other equipment firmware. For example, the firmware of the production machinery and testing
equipment interfaces with the production control software.

Testing Object-Oriented Systems

Once a program code is written, it must be tested to detect and subsequently handle all errors
in it. A number of schemes are used for testing purposes.
Another important aspect is the fitness of purpose of a program that ascertains whether the
program serves the purpose which it aims for. The fitness defines the software quality.

Testing Object-Oriented Systems

Testing is a continuous activity during software development. In object-oriented systems,

testing encompasses three levels, namely, unit testing, subsystem testing, and system testing.
Unit Testing
In unit testing, the individual classes are tested. It is seen whether the class attributes are
implemented as per design and whether the methods and the interfaces are error-free. Unit
testing is the responsibility of the application engineer who implements the structure.
Subsystem Testing
This involves testing a particular module or a subsystem and is the responsibility of the
subsystem lead. It involves testing the associations within the subsystem as well as the
interaction of the subsystem with the outside. Subsystem tests can be used as regression tests
for each newly released version of the subsystem.
System Testing
System testing involves testing the system as a whole and is the responsibility of the quality-
assurance team. The team often uses system tests as regression tests when assembling new
releases.

Object-Oriented Testing Techniques

Grey Box Testing

The different types of test cases that can be designed for testing object-oriented programs are
called grey box test cases. Some of the important types of grey box testing are −
• State model based testing − This encompasses state coverage, state transition
coverage, and state transition path coverage.
• Use case based testing − Each scenario in each use case is tested.
• Class diagram based testing − Each class, derived class, associations, and
aggregations are tested.
• Sequence diagram based testing − The methods in the messages in the sequence
diagrams are tested.
Techniques for Subsystem Testing
The two main approaches of subsystem testing are −
• Thread based testing − All classes that are needed to realize a single use case in a
subsystem are integrated and tested.
• Use based testing − The interfaces and services of the modules at each level of
hierarchy are tested. Testing starts from the individual classes to the small modules
comprising of classes, gradually to larger modules, and finally all the major
subsystems.
Categories of System Testing
• Alpha testing − This is carried out by the testing team within the organization that
develops software.
• Beta testing − This is carried out by select group of co-operating customers.
• Acceptance testing − This is carried out by the customer before accepting the
deliverables.

What is Web Application Testing?

Web application testing, a software testing technique exclusively adopted to test the
applications that are hosted on web in which the application interfaces and other
functionalities are tested.

Web Application Testing - Techniques:

1. Functionality Testing - The below are some of the checks that are performed but not
limited to the below list:
• Verify there is no dead page or invalid redirects.
• First check all the validations on each field.
• Wrong inputs to perform negative testing.
• Verify the workflow of the system.
• Verify the data integrity.
2. Usability testing - To verify how the application is easy to use with.
• Test the navigation and controls.
• Content checking.
• Check for user intuition.
3. Interface testing - Performed to verify the interface and the dataflow from one system to
other.
4. Compatibility testing- Compatibility testing is performed based on the context of the
application.
• Browser compatibility
• Operating system compatibility
 • Compatible to various devices like notebook, mobile, etc.
5. Performance testing - Performed to verify the server response time and throughput under
various load conditions.
• Load testing - It is the simplest form of testing conducted to understand the behaviour
of the system under a specific load. Load testing will result in measuring important
business critical transactions and load on the database, application server, etc. are also
monitored.
• Stress testing - It is performed to find the upper limit capacity of the system and also
to determine how the system performs if the current load goes well above the expected
maximum.
• Soak testing - Soak Testing also known as endurance testing, is performed to
determine the system parameters under continuous expected load. During soak tests
the parameters such as memory utilization is monitored to detect memory leaks or
other performance issues. The main aim is to discover the system's performance under
sustained use.
• Spike testing - Spike testing is performed by increasing the number of users suddenly
by a very large amount and measuring the performance of the system. The main aim
is to determine whether the system will be able to sustain the work load.
6. Security testing - Performed to verify if the application is secured on web as data theft and
unauthorized access are more common issues and below are some of the techniques to verify
the security level of the system.
• Injection
• Broken Authentication and Session Management
• Cross-Site Scripting (XSS)
• Insecure Direct Object References
• Security Misconfiguration
• Sensitive Data Exposure
• Missing Function Level Access Control
• Cross-Site Request Forgery (CSRF)
• Using Components with Known Vulnerabilities
• Unvalidated Redirects and Forwards

What is GUI Software Testing?

GUI testing is a testing technique in which the application's user interface is tested whether
the application performs as expected with respect to user interface behaviour.
GUI Testing includes the application behaviour towards keyboard and mouse movements and
how different GUI objects such as toolbars, buttons, menubars, dialog boxes, edit fields, lists,
behavior to the user input.
GUI Testing Guidelines

• Check Screen Validations

• Verify All Navigations
• Check usability Conditions
• Verify Data Integrity
• Verify the object states
• Verify the date Field and Numeric Field Formats
What is Software Quality Assurance in Software Engineering?

Before getting into details of software quality assurance you must know about software quality.
Software quality is an extent to which the developed software meets the requirements specified
by the customers. More the requirements are satisfied better is the quality of the software.

Software quality assurance is the set of actions performed by the SQA group to ensure the
quality of a software. Software quality assurance incorporates:

• SQA process on the software.

• Assuring and controlling the quality of software which include technical evaluation and
testing.
• Applying software engineering practices effectively.
• Managing software work products like an architectural model, document with a
description of the software, source code. Controlling changes made to software work
product.
• Measuring and reporting the quality of the software.

Software Quality Assurance Plan

Abbreviated as SQAP, the software quality assurance plan comprises of the procedures,
techniques, and tools that are employed to make sure that a product or service aligns with the
requirements defined in the SRS(software requirement specification).
The plan identifies the SQA responsibilities of a team, lists the areas that need
to be reviewed and audited. It also identifies the SQA work products.

The SQA plan document consists of the below sections:

1. Purpose section
2. Reference section
3. Software configuration management section
4. Problem reporting and corrective action section
5. Tools, technologies and methodologies section
6. Code control section
7. Records: Collection, maintenance and retention section
8. Testing methodology

Elements of Software Quality Assurance

SQA encompasses a wide range of activities that can be performed to ensure the quality of the
software

1. Standards

We all know that organizations like IEEE, ISO etc. has lined-up a lot up software engineering
standards which should be imposed by the customer and even embraced by software engineers
while developing software.

SQA team must ensure that standards established by the authorized organization are followed
by the software engineers.

2. Review & Audit

The software is evaluated technically by the SQA team in order to discover an error in the
software. While auditing a software the SQA team confirms that good quality guidelines are
followed by the software engineers while developing the software.

3. Testing

The elemental goal of software testing is to identify the bug in the software. SQA team has the
responsibility of planning the testing systematically and conducting it efficiently. This would
raise the possibility of finding the bug in the software if any.

4. Analyzing Error

Software testing reveals bugs and errors in the software which are analysed by the SQA team
in order to discover how the bugs or errors are introduced in the software and also discover the
possible methods required to eliminate those errors or bugs.

5. Change Management

The customer can ask for modifications between the development of the software. The changes
are the most distracting aspect of any software project.

If the implementation of the changes is not properly managed it will cause confusion which
will affect the quality of the software. The SQA team takes care that change management is
practised while developing the software.

6. Education

It is the responsibility of the SQA organization to enlighten the software organizations by

improving their software engineering practices.

7. Vendor Management

It is the responsibility of SQA to suggest software vendors, to accept the quality practices while
developing the software. SQA must also incorporate these quality instructions in a contract
with software vendors.

8. Security Management

With the increase in cybercrime, the government has regulated the software organization to
incorporate policies to protect data at all level. It is the responsibility of SQA to verify whether
the software organization are using appropriate technology to acquire software security.

9. Safety

The hidden defects of any human-rated software (aircraft applications) can lead to catastrophic
events. So, it is the responsibility of SQA to evaluate the effect of software failure and introduce
steps to eliminate the risk.

10. Risk Management

Though it is the job of software organization to analyze and reduce the risk in the software.
But, it is the responsibility of SQA to make sure that risk management actions are properly
conducted and the backup plan has been confirmed.

Software Quality Assurance Activities

#1) Creating an SQA Management Plan:

The foremost activity includes laying down a proper plan regarding how the SQA will be
carried out in your project.

Along with what SQA approach you are going to follow, what engineering activities will be
carried out, and it also includes ensuring that you have a right talent mix in your team.

#2) Setting the Checkpoints:

The SQA team sets up different checkpoints according to which it evaluates the quality of the
project activities at each checkpoint/project stage. This ensures regular quality inspection and
working as per the schedule.

#3) Apply software Engineering Techniques:

Applying some software engineering techniques aids a software designer in achieving high-
quality specification. For gathering information, a designer may use techniques such as
interviews and FAST (Functional Analysis System Technique).

Later, based on the information gathered, the software designer can prepare the project
estimation using techniques like WBS (work breakdown structure), SLOC (source line of
codes), and FP(functional point) estimation.

#4) Executing Formal Technical Reviews:

An FTR is done to evaluate the quality and design of the prototype.

In this process, a meeting is conducted with the technical staff to discuss regarding the actual
quality requirements of the software and the design quality of the prototype. This activity helps
in detecting errors in the early phase of SDLC and reduces rework effort in the later phases.

#5) Having a Multi- Testing Strategy:

By multi-testing strategy, we mean that one should not rely on any single testing approach,
instead, multiple types of testing should be performed so that the software product can be tested
well from all angles to ensure better quality.

#6) Enforcing Process Adherence:

This activity insists the need for process adherence during the software development process.
The development process should also stick to the defined procedures.

This activity is a blend of two sub-activities which are explained below in detail:
(i) Product Evaluation:
This activity confirms that the software product is meeting the requirements that were
discovered in the project management plan. It ensures that the set standards for the project are
followed correctly.
(ii) Process Monitoring:
This activity verifies if the correct steps were taken during software development. This is done
by matching the actually taken steps against the documented steps.

#7) Controlling Change:

In this activity, we use a mix of manual procedures and automated tools to have a mechanism
for change control.

By validating the change requests, evaluating the nature of change and controlling the change
effect, it is ensured that the software quality is maintained during the development and
maintenance phases.

#8) Measure Change Impact:

If any defect is reported by the QA team, then the concerned team fixes the defect.

After this, the QA team should determine the impact of the change which is brought by this
defect fix. They need to test not only if the change has fixed the defect, but also if the change
is compatible with the whole project.

For this purpose, we use software quality metrics which allows managers and developers to
observe the activities and proposed changes from the beginning till the end of SDLC and
initiate corrective action wherever required.

#9) Performing SQA Audits:

The SQA audit inspects the entire actual SDLC process followed by comparing it against the
established process.

It also checks whatever reported by the team in the status reports were actually performed or
not. This activity also exposes any non-compliance issues.

#10) Maintaining Records and Reports:

It is crucial to keep the necessary documentation related to SQA and share the required SQA
information with the stakeholders. The test results, audit results, review reports, change
requests documentation, etc. should be kept for future reference.

#11) Manage Good Relations:

In fact, it is very important to maintain harmony between the QA and the development team.

We often hear that testers and developers often feel superior to each other. This should be
avoided as it can affect the overall project quality.
Statistical Quality Assurance (SQA)

Traditional compliance testing techniques can sometimes provide limited pass/fail information,
which results in insufficient measurements on the batch’s quality control, identification of the
root cause of failure results and overall quality assurance (QA) in the production process.
Intertek combines legal, customer and essential safety requirements to customize a workable
QA process, called Statistical Quality Assurance (SQA). SQA is used to identify the potential
variations in the manufacturing process and predict potential defects on a parts-per-million
(PPM) basis. It provides a statistical description of the final product and addresses quality and
safety issues that arise during manufacturing.
SQA consists of three major methodologies:

1. Force Diagram - A Force Diagram describes how a product should be tested. Intertek
engineers base the creation of Force Diagrams on our knowledge of foreseeable use,
critical manufacturing process and critical components that have high potential to fail.
2. Test-to-Failure (TTF) - Unlike any legal testing, TTF tells manufacturers on how
many defects they are likely to find in every million units of output. This information
is incorporated into the process and concludes if a product needs improvement in
quality or if it is being over engineered, which will eventually lead to cost savings.
3. Intervention - Products are separated into groups according to the total production
quantity and production lines. Each group then undergoes an intervention. The end
result is measured by Z-value, which is the indicator of quality and consistency of a
product to a specification. Intervention allows manufacturers to pinpoint a defect to a
specific lot and production line; thus saving time and money in corrective actions.

Software Reliability

Software Reliability means Operational reliability. It is described as the ability of a system

or component to perform its required functions under static conditions for a specific period.

Software reliability is also defined as the probability that a software system fulfils its assigned
task in a given environment for a predefined number of input cases, assuming that the hardware
and the input are free of error.

Software Reliability is an essential connect of software quality, composed with functionality,

usability, performance, serviceability, capability, install ability, maintainability, and
documentation. Software Reliability is hard to achieve because the complexity of software
turns to be high. While any system with a high degree of complexity, containing software, will
be hard to reach a certain level of reliability, system developers tend to push complexity into
the software layer, with the speedy growth of system size and ease of doing so by upgrading
the software.

For example, large next-generation aircraft will have over 1 million source lines of software
on-board; next-generation air traffic control systems will contain between one and two million
lines; the upcoming International Space Station will have over two million lines on-board and
over 10 million lines of ground support software; several significant life-critical defence
systems will have over 5 million source lines of software. While the complexity of software is
inversely associated with software reliability, it is directly related to other vital factors in
software quality, especially functionality, capability, etc.

Quality Standards

SEI = 'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S.

Defense Department to help improve software development processes.

CMM = 'Capability Maturity Model', developed by the SEI. It's a model of 5 levels of
organizational 'maturity' that determine effectiveness in delivering quality software. It is geared
to large organizations such as large U.S. Defense Department contractors. However, many of
the QA processes involved are appropriate to any organization, and if reasonably applied can
be helpful. Organizations can receive CMM ratings by undergoing assessments by qualified
auditors.

Level 1 - characterized by chaos, periodic panics, and heroic efforts

required by individuals to successfully complete projects. Few if any
processes in place; successes may not be repeatable.
Level 2 - software project tracking, requirements management, realistic
planning, and configuration management processes are in place;
successful practices can be repeated.
Level 3 - standard software development and maintenance processes are
integrated throughout an organization; a Software Engineering Process
Group is is in place to oversee software processes, and training programs
are used to ensure understanding and compliance.
Level 4 - metrics are used to track productivity, processes, and
products. Project performance is predictable, and quality is consistently
high.
Level 5 - the focus is on continuous process improvement. The impact
of new processes and technologies can be predicted and effectively
implemented when required.
Perspective on CMM ratings: During 1997-2001, 1018 organizations
were assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at
3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62%
were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median
size of organizations was 100 software engineering/maintenance
personnel; 32% of organizations were U.S. federal contractors or
agencies. For those rated at Level 1, the most problematical key process
area was in Software Quality Assurance.
ISO = 'International Organization for Standardization' - The ISO 9001:2000 standard
(which replaces the previous standard of 1994) concerns quality systems that are
assessed by outside auditors, and it applies to many kinds of production and
manufacturing organizations, not just software. It covers documentation, design,
development, production, testing, installation, servicing, and other processes. The full
set of standards consists of: (a)Q9001-2000 - Quality Management Systems:
Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and
Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for
Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses
an organization, and certification is typically good for about 3 years, after which a
complete reassessment is required. Note that ISO certification does not necessarily
indicate quality products - it indicates only that documented processes are followed.

IEEE = 'Institute of Electrical and Electronics Engineers' - among other things,

creates standards such as 'IEEE Standard for Software Test Documentation'
(IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit Testing (IEEE/ANSI
Standard 1008), 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI
Standard 730), and others.

ANSI = 'American National Standards Institute', the primary industrial standards

body in the U.S.; publishes some software-related standards in conjunction with the
IEEE and ASQ (American Society for Quality).