Software Quality Engineering

by
Dr. Rizwan
 Quality Concerns of Customer
• The basic quality concern of a user are that a software system performs useful functions
as it is specified
• Performs right functions as specified, which, hopefully fits the user’s needs (fit for use)
• Performs these specified functions correctly over repeated use or over a long period of time (Performs its
functions reliably)
• Customers view can vary
• For some (novice) users – useability is most important factor
• For other (expert) users –reliability is more important than useability
• Customer vs User Expectations?
• The basic quality expectations of a customer are similar to that of a user, with the additional concern for the cost
of the software or service
• This additional concern can be reflected by the so‐called value‐based view of quality, that is, whether a customer
is willing to pay for it
 Quality Concerns of Developers
• For software developers, the most fundamental quality concern is to
• Fulfill their contractual obligations by producing software products that conform to
product specifications or
• Providing services that conform to service agreement
• Usability and Modifiability may be most important for people involved with
software service,
• Maintainability for maintenance personnel
• Portability for third‐party or software packaging service providers
• Profitability and customer value for product marketing
• For product and service managers, adherence to pre‐selected software
process and relevant standards, proper choice of software methodologies,
languages, and tools may be closely related to quality
 So, What is Quality?

• It has may include many different attributes and

• It can be defined and perceived differently based on Customer
and user having different roles and responsibilities.
 Software Quality Attributes

• Software quality attributes are characteristics that define how

well software meets the needs of its users OR
• Software quality attributes are measurable or testable
properties of a software system used by quality architects.
• These properties help to determine whether the software satisfies the
stakeholders' requirements and needs
• List of Software Quality Attributes
• Reliability
• Maintainability
• Usability
• Performance
• Security
 Software Quality Attributes
• Reliability
• The ability of a system or component to perform its required
functions under stated conditions for a specified period of time
• It's about the consistency and dependability of a system's
performance over time
• Metrics
• Failure Rate Analysis: Measure the frequency at which a system or
component fails over a given period
• E.g. Tracking the number of failures against the total operating time or the number of cycles
• Mean Time Between Failures (MTBF): Calculate the average time elapsed
between failures
• Widely used in reliability engineering to assess the expected reliability of a system
 Software Quality Attributes
• Usability
• Is a quality attribute that refers to the ease with which users can
learn to operate a system, perform tasks, and achieve their goals
efficiently and effectively
• Metrics
• Time to complete tasks: Measure the time it takes for users to
accomplish specific tasks within the system
• Shorter task completion times generally indicate higher usability
• Error rates: Track the frequency and severity of user errors while
interacting with the system
• Lower error rates typically indicate better usability
 Software Quality Attributes
• Maintainability
• A quality attribute that refers to the ease with which a system can
be modified, updated, repaired, and extended over time
• Metrics
• Code complexity: Measure the complexity of the codebase using metrics
such as cyclomatic complexity, lines of code, or nesting depth
• High complexity can indicate potential difficulties in understanding and modifying the code
• Code readability: Assess the clarity and comprehensibility of the code by
analyzing factors such as naming conventions, comments, indentation,
and code structure
• Tools like static code analyzers can provide insights into code readability metrics
 Software Quality Attributes
• Performance
• Performance is a quality attribute that refers to the speed,
responsiveness, throughput, and efficiency of a system under
various workload conditions
• Metrics
• Response time: Measure the time taken for the system to respond to user
requests or events
• Throughput: Assess the number of transactions, requests, or operations
that the system can handle per unit of time
• Throughput can be measured in terms of requests per second, transactions per
minute, or other relevant units depending on the nature of the system
 Defect Management and Quality Engineering

• For most software organizations, ensuring quality means dealing

with defects
• Three generic ways to deal with defects:
• Defect prevention
• Defect detection and removal (defect reduction)
• Defect containment
• Defect measurements are used to provide feedback to software
development process
• Quality engineering can be viewed as defect management
 Quality Frameworks and ISO‐9126

• ISO 9126 is the software product evaluation standard from International

Organization for Standardization (ISO)
• ISO‐9126 provides a hierarchical framework for quality definition, organized into
quality characteristics and sub‐characteristic
• There are 6 top‐level quality characteristics, with each associated with its own
exclusive (non‐overlapping) sub‐characteristics
 Quality Frameworks and ISO‐9126
• Functionality:
• The set of attributes that bear on the existence of a set of functions and their specified
properties
• The functions are those that satisfy stated or implied needs
• Reliability:
• The set of attributes that bear on the capability of software to maintain its level of performance
under stated conditions for a stated period of time
• Usability:
• The set of attributes that bear on the effort needed for use, and on the individual assessment of
such use, by a stated or implied set of users
• Efficiency:
• Is the set of attributes that bear on the relationship between the level of performance of the
software and the number of resources used, under stated conditions
• Maintainability:
• Is the set of attributes that bear on the effort needed to make specified modifications
(correct/improve)
• Portability:
• is the set of attributes that bear on the ability of software to be transferred from one
environment to another
 Quality Frameworks and ISO‐9126
Quality Sub‐Characteristics
• Functionality:
• Suitability‐ to the user’s needs
• Accuracy‐ of results
• Interoperability‐ with other systems
• Security‐ against unintended access
• Reliability:
• Maturity‐ frequency of failures
• Fault Tolerance‐ performance in case of faults
• Recoverability‐ of functionality and data loss
 Quality Frameworks and ISO‐9126
Quality Sub‐Characteristics Cont’d
• Maintainability:
• Analyzability‐ effort for diagnosis
• Changeability‐ ease of modification
• Stability‐ after change
• Testability‐ effort required for testing after change
• Portability:
• Adaptability
• Install ability
• Conformance
• Replaceability
 Quality Frameworks and ISO‐9126
• Usability:
• Understandability – The amount of effort required to understand software
• Ease of learning – The degree to which user effort required to learn how to
use the software is minimized
• Operability – The degree to which the effort required to perform an operation
is minimized
• Communicativeness – The degree to which software is designed in
accordance with the psychological characteristics of users

16
 FURPS
• FURPS is an acronym used in software engineering to
represent a set of characteristics or quality attributes that are
often considered when evaluating and defining software
requirements
• Functionality: This category encompasses the features and
capabilities of the software
• It includes requirements related to specific functions, user
interactions, data processing, and system behavior
• Usability: Usability refers to the ease of use and user
experience provided by the software
• It includes requirements related to user interfaces, navigation,
learnability, efficiency, and user satisfaction.
• Reliability: Reliability concerns the dependability and stability
of the software
• It includes requirements related to error handling, fault tolerance,
availability, and the ability to recover from failures.
 FURPS Cont..
• Performance: Performance relates to the speed, responsiveness,
and efficiency of the software
• It includes requirements related to response time, throughput, scalability,
resource utilization, and efficiency.
• Supportability: Supportability focuses on the ease of maintenance,
deployment, and support for the software throughout its lifecycle
• It includes requirements related to documentation, modularity,
configurability, testability, and compatibility with other systems
 CUPRIMDSO
• CUPRIMDSO categories
• Capability
• Functionality
• Usability
• Performance
• Reliability
• Install ability
• Maintainability
• Documentation/information
• Service
• Overall
 Software Process Improvement and Capability
Determination (SPICE)
• The SPICE model is a framework for assessing and improving
software development processes
• It provides a set of standards and guidelines for evaluating the
maturity and capability of software development processes within
organizations
• Developed by the International Organization for Standardization
(ISO) and the International Electrotechnical Commission (IEC) and
is based on the ISO/IEC 15504 standard
• The primary goal of SPICE is to help organizations understand,
measure, and improve their software development processes to
enhance product quality, reduce costs, and increase efficiency
 CMMI
• CMMI stands for Capability Maturity Model Integration
• It is a process improvement framework that provides
organizations with the essential elements of effective
processes
• CMMI helps organizations improve their processes for
developing and delivering products and services, thereby
enhancing their overall performance and competitiveness
 CMMI Objectives
• Improve process efficiency and effectiveness
• CMMI helps organizations identify weaknesses in their processes and implement
improvements to enhance productivity, quality, and customer satisfaction
• Establish best practices
• CMMI provides a set of best practices and guidelines for process improvement based
on industry standards and real-world experience
• Achieve consistency and repeatability
• CMMI helps organizations establish standardized processes that can be consistently
applied across projects and teams, leading to predictable and repeatable results
• Support organizational learning and innovation
• CMMI encourages organizations to continuously learn from their experiences and
innovate their processes to adapt to changing business needs and technological
advancements
 CMMI Levels
• Initial
• Processes are unpredictable, poorly controlled, and reactive
• Managed
• Basic project management processes are established to track cost,
schedule, and functionality
• Defined
• Processes are well characterized and understood, and are tailored to
specific projects
• Quantitatively Managed
• Processes are controlled using statistical and quantitative techniques
to improve process performance
• Optimizing
• Continuous process improvement is enabled through quantitative
feedback and innovation
 Why do we need Quality Models

• Bridges the gap between users and developers.

• Reflects both users’ views and developers’
priorities.
 Defects
• Failure – The inability of a system or component to perform its
required functions within specified requirements (External behavior)
• Deviation from expected/specified behavior
• Fault – Internal characteristics ‐ cause for failures
• An incorrect step, process, or data definition in a computer
program
• Error – Refers to a missing or incorrect human action such as human
misconceptions, misunderstandings, etc. resulting in certain fault(s)
being injected into a software
• During design, coding and data entry
 Defects
• Defect
• Generally, refers to some problem, either with external behavior
or with internal characteristics
• Error/fault/failure are collectively referred to as defects
 Defects
• The consumer’s concern is that the ‘No failure, or few failures with min
impact’
• This concern can be captured by
• Failure Properties:
• Information about specific failures, what they are, how they occur etc.
• Failure count, distribution, density etc.
• Failure Likelihood
• How often or how likely
• Failure Severity Measurement
• Failure impact
 Disadvantages
• Provides only a high‐level (vague) description of quality and does not
show how it will be implemented at program‐level .
• Focuses precisely on the measurement of high‐level software quality
attributes, lacking the details to understand the attributes deeply .
• Does not guide how to build, confirm, evaluate, or measure quality
attributes .
• Does not show whether they are achieved in developed software product
or not .
• Much focus is at the high‐level attributes but a little consideration of the
low‐level attributes (tangible properties) of the software product.
• Lacks how the low‐level quality attributes influence the high‐level quality
attributes .