SOFTWARE

QUALITY
ENGINEERING
LECTURE NO: 2
BY: NAZISH NOUMAN
Beyond Functionality
The Intro…
There’s more Users also have expectations, often unstated, about how well the product will
work.
to software
success than
just Such expectations include how easy it is to use, how quickly it executes, how
delivering rarely it fails, how it handles unexpected conditions and perhaps, how loud it is.
the right
functionality.
Such characteristics, collectively known as quality attributes, quality factors,
quality requirements, quality of service requirements, or the “–ilities,” constitute
a major portion of the system’s nonfunctional requirements
 They are however quite simple; they
are the restrictions or constraints to
be placed on the system and how to
build it.
Non- Functional
Requirements
Their purpose is to restrict the
number of solutions that will meet a
set of requirements.
IEEE Definition of NFR ..
 Product requirements: These
requirements specify how software
product performs

Non-Functional
Organizational requirements: These
Requirements: requirements are derived from the
Product policies and procedures of an
organization.
Requirements
External requirements: These
requirements include all the
requirements that affect the software
or its development process externally.
Software Quality Attributes
One way to classify quality attributes distinguishes those characteristics
that are discernible through execution of the software (external quality)
from those that are not (internal quality)
External quality factors are primarily important to users, whereas internal
qualities are more significant to development and maintenance staff.

Internal quality attributes indirectly contribute to customer satisfaction by

making the product easier to enhance, correct, test, and migrate to new
platforms.
External Quality Attributes
 External quality attributes
describe characteristics that
are observed when the
software is executing.
External quality
attributes
They profoundly influence the
user experience and the user’s
perception of system quality.
1. Availability
Availability is a measure of the planned up time during which the
system’s services are available for use and fully operational.
• Formally, availability equals the ratio of up time to the sum of up time and down time.
• Availability equals the mean time between failures (MTBF) for the system divided by the
sum of the MTBF and the mean time to repair (MTTR) the system after a failure is
encountered.
• Scheduled maintenance periods also affect availability.
• Availability is closely related to reliability and is strongly affected by the maintainability
subcategory of modifiability.
Availability Example

AVL-1. The system shall be at least AVL-2. Down time that is excluded
95 percent available on weekdays from the calculation of availability
between 6:00 A.M. and midnight consists of maintenance scheduled
Eastern Time, and at least 99 during the hours from 6:00 P.M.
percent available on weekdays Sunday Pacific Time, through 3:00
between 3:00 P.M. and 5:00 P.M. A.M. Monday Pacific Time
Eastern Time.
Availability
 Software is not useful
until it is installed on the Downloading apps to a phone or
tablet; moving software from a
appropriate device or PC onto a web server;
platform.

Some examples of
software installation Updating an operating system;
are:
2. Installability
It describes how easy is it installing a huge commercial
to perform these system, such as an enterprise
operations correctly. resource planning tool;

Increasing a system’s installability

reduces the time, cost, user
disruption, error frequency, and
skill level needed for an
installation operation.
Instability Requirements (Example)
INS-1. An untrained user shall be able to successfully perform an initial
installation of the application in an average of 10 minutes.

INS-2. When installing an upgraded version of the application, all

customizations in the user’s profile shall be retained and converted to the new
version’s data format if needed.

INS-3. Installing this software on a server requires administrator privileges.

INS-4. Following successful installation, the installation program shall delete all
temporary, backup, obsolete, and unneeded files associated with the
application
3. Integrity

Integrity deals with preventing information loss and preserving the

correctness of data entered into the system.
➢Integrity requirements have no tolerance for error: the data is either in good
shape and protected, or it is not.
➢ Data needs to be protected against threats such as accidental loss or
corruption, ostensibly identical data sets that do not match, physical damage to
storage media, accidental file erasure, or data overwriting by users.
➢Security sometimes is considered a subset of integrity, because
some security requirements are intended to prevent access to data by
unauthorized users
 INT-1. After performing a file backup, the
system shall verify the backup copy against the
original and report any discrepancies.

Integrity INT-2. The system shall protect against the

unauthorized addition, deletion, or modification
Example of data.

INT-3. The system shall confirm daily that the

application executable have not been by the
addition of unauthorized code
 4. Interoperability

Interoperability indicates how

To assess interoperability, you
readily the system can
need to know which other
exchange data and services
applications the users will
with other software systems
employ in conjunction with
and how easily it can integrate
your product and what data
with external hardware
they expect to exchange.
devices.

IOP-1. The Chemical Tracking System shall be able to import

any valid chemical structure from the ChemDraw (version 13.0
or earlier) and MarvinSketch (version 5.0 or earlier) tools.
 5. Performance

Performance is one of the quality attributes that users often will bring up
spontaneously. Performance represents the responsiveness of the system to various
user inquiries and actions.
 All users want their applications to run instantly, but
the real performance requirements will be different
for a spell-check feature than for a missile’s radar
guidance system.

Satisfying performance requirements can be tricky

because they depend so much upon external factors
Performance… such as the speed of the computer being used,
network connections, and other hardware
components.

Performance is an external quality attribute because

it can be observed only during program execution.
It is closely related to the internal quality attribute
of efficiency, which has a big impact on the user-
observed performance.
 PER-1. Authorization of an ATM withdrawal request
shall take no more than 2.0 seconds

Performance PER-2. Webpages shall fully download in an average of

3 seconds or less over a 30 megabits/second Internet
Example connection.

PER-3. At least 98 percent of the time, the trading

system shall update the transaction status display
within 1 second after the completion of each trade.
6. Usability

usability is the degree to which a software can be used by

specified consumers to achieve quantified objectives with
effectiveness, efficiency, and satisfaction in a quantified
context of use

➢Analysts and developers shouldn’t talk about “friendly” software but rather
about software that’s designed for effective and unobtrusive usage.
Ease of learning Ease of use
Rich, customizable menus and
Wizards
toolbars
Multiple ways to access the same
Visible menu options
function
Meaningful, plain-language
Auto completion of entries
Possible
messages
Help screens and tooltips Auto correction of errors
design
Similarity to other familiar systems
Keyboard shortcuts approaches
Limited number of options and Ability to carry over information
for ease of
widgets displayed from a previous transaction
learning and
ease of use
 USE-1. A trained user shall be able to submit a request
for a chemical from a vendor catalog in an average of
three minutes, and in a maximum of five minutes, 95
percent of the time.

USE-2. All functions on the File menu shall have

Usability shortcut keys defined that use the Control key pressed
simultaneously with one other key. Menu commands
Examples that also appear in Microsoft Word shall use the same
default shortcut keys that Word uses.

USE-3. 95 percent of chemists who have never used

the Chemical Tracking System before shall be able to
place a request for a chemical correctly with no more
than 15 minutes of orientation.
 The average time needed for a How many transactions the user What percentage of a set of
specific type of user to complete can complete correctly in a given tasks the user can complete
a particular task correctly. time period. correctly without needing help.

Usability
indicators How many errors the user
How many tries it takes the user
to accomplish a particular task,
The delay or wait time when
like finding a specific function
include: makes when completing a task.
buried somewhere in the
menus.
performing a task.

The number of interactions (mouse clicks,

keystrokes, touch-screen gestures) required to get to
a piece of information or to accomplish a task.
7. Robustness

Robustness is the degree to which a system continues to function

properly when confronted with invalid inputs, defects in connected
software or hardware components, external attack, or unexpected
operating conditions
 Other attribute terms associated with
robustness are fault tolerance are:

Other Attribute • fault tolerance (are user input errors caught and
Terms of corrected?)
Robustness • survivability (can the camera experience a drop
from a certain height without damage?)
• recoverability (can the PC resume proper
operation if it loses power in the middle of an
operating system update?)
Robustness Example

ROB-1. If the text editor fails before

the user saves the file, it shall ROB-2. All plot description
recover the contents of the file parameters shall have default
being edited as of, at most, one values specified, which the
minute prior to the failure the next Graphics Engine shall use if a
time the same user launches the parameter’s input data is missing or
application. invalid.
 Safety requirements deal with the need to
prevent a system from doing any injury to people
or damage to property

➢Safety requirements might be dictated by government

regulations or other business rules, and legal or certification
8. Safety issues could be associated with satisfying such requirements.

➢Safety requirements frequently are written in the form of

conditions or actions the system must not allow to occur.
 SAF-1. The user shall be able to see a list of
all ingredients in any menu items, with
ingredients highlighted that are known to
cause allergic reactions in more than 0.5
percent of the North American population.

Safety Example
SAF-2. The system shall terminate any
operation within 1 second if the measured
tank pressure exceeds 90 percent of the
specified maximum pressure.
 Under what conditions could a human be harmed by the use
of this product? How can the
system detect those conditions? How should it respond?

What is the maximum allowed frequency of failures that have

the potential to cause injury?
When eliciting safety
requirements you might
need to interview SME What failure modes have the potential of causing harm or
who are very familiar property damage?
with the operating
environment or people
who have thought a lot What operator actions have the potential of inadvertently
causing harm or property damage?
about project risks

Are there specific modes of operation that pose risks to

humans or property?
 Security deals with blocking unauthorized access to system
functions or data, ensuring that the software is protected
from malware attacks, and so on.

9. Security
➢Security is a major issue with Internet software
➢As with integrity requirements, security requirements have
no tolerance for error.
 User identification and
authentication (password
User authorization or
construction rules,
privilege levels (ordinary
password change
user, guest user,
frequency, security
administrator) and user
questions, forgotten logon
access
name or password
controls.
procedures, biometric

Following are
identification..)

some Data privacy (who can

considerations create, see, change, copy,

print, and delete what
information)
Deliberate data destruction,
corruption, or theft

to examine
when eliciting
security Protection against viruses,
worms, Trojan horses,
spyware, rootkits, and other
Firewall and other network
security issues
requirements: malware

Building audit trails of

operations performed and
Encryption of secure data access attempts
 SEC-1. The system shall lock a user’s account
after four consecutive unsuccessful logon
attempts within a period of five minutes.

Security SEC-2. The system shall log all attempts to

Examples access secure data by users having insufficient
privilege levels.

SEC-3. A user shall have to change the

temporary password assigned by the security
officer to a previously unused password
immediately following the first successful logon
with the temporary password.
 SEC-4. A door unlock that results from a
successful security badge read shall keep the
door unlocked for 8.0 seconds, with a tolerance
of 0.5 second.

Security SEC-5. The resident antimalware software shall

Examples quarantine any incoming Internet traffic that
exhibits characteristics of known or suspected
virus signatures.

SEC-6. The magnetometer shall detect at least

99.9 percent of prohibited objects, with a false
positive rate not to exceed 1 percent.
10. Reliability
The probability of the software executing without failure for a specific
period of time is known as reliability

➢Reliability problems can occur because of improper inputs, errors in the software code
itself, components that are not available when needed, and hardware failures.
➢Robustness and availability are closely related to reliability.
➢ Ways to specify and measure software reliability include the percentage of operations
that are completed correctly, the average length of time the system runs before failing
(mean time between failures, or MTBF), and the maximum acceptable probability of a
failure during a given time period.
 Failure exists in varying degrees (e.g. partial or total failure)
Failure simply means that a system, component, or device
can no longer produce specific desired results.

Even if a piece of manufacturing equipment is still running

and producing items, it has failed if it doesn’t deliver the
expected quantities.

Failure Metrics
To effectively manage failures, there are several critical
metrics that should be monitored.

Mean time to repair / Mean time to recover

(MTTR)
It includes: Mean time between Failure (MTBF)
Mean time to Failure (MTTF)
 Mean Time To Repair (MTTR) refers to
the amount of time required to repair a
system and restore it to full
functionality.

What is Mean
Time To Repair The MTTR clock starts ticking when the
repairs start, and it goes on until
(MTTR)? operations are restored.

This includes repair time, testing

period, and return to the normal
operating condition
MTTR Calculation

To calculate MTTR, divide

the total maintenance
time by the total number of
maintenance actions over a
given period of time.
Example
if you have a pump that fails four times in one workday and you
spend an hour repairing each of those instances of failure, your
MTTR would be ???
 Mean Time To Recovery is a measure of
the time between the point at which the
Mean Time To failure is first discovered until the point at
which the equipment returns to operation.
Repair vs Mean
Time To
Recovery So, in addition to repair time, testing
period, and return to normal operating
condition, it captures failure notification
time and diagnosis.
Mean Time To Repair vs Mean Time To Recovery
What is Mean Time Between Failures (MTBF)?

MTBF measures the predicted time that passes between one

previous failure of a system to the next failure during normal
operation. Or, the time between one system breakdown and
the next.

So, while MTTR measures availability, MTBF measures

availability and reliability. The higher the figure of the MTBF, the
longer the system will likely run before failing.
Mean Time Between Failure
MTBF (mean time between failures) is the average time
between repairable failures of a technology product.
The metric is used to track both the availability and
reliability of a product.
The higher the time between failure, the more reliable the
system.
Time between one failure to another
 MTBF
Calculation
To calculate MTBF, divide the
total number of operational
hours in a period by the
number of failures that
occurred in that period.
MTBF is usually measured in
hours.
MTBF = # of operational
hours ÷ # of failures MTBF = Total uptime / # of breakdowns
Example
For example, an asset may have been operational for 1,000
hours in a year.
Over the course of that year, that asset broke down eight
times.
Therefore, the MTBF for that piece of equipment is 125
hours.
How ?
Example to Solve
Looking at the example of the pump we mentioned under
MTTR, out of the expected runtime of ten hours, it ran for
nine hours and failed for one hour spread over three
occasions.
So, MTBF = ??
 Mean Time To Failure (MTTF) is a very basic
measure of reliability used for non-
repairable systems. It represents the length
What is Mean of time that an item is expected to last in
operation until it fails.
Time To Failure
(MTTF)?
MTTF is what we commonly refer
to as the lifetime of any product or
a device.
Mean Time To Failure (MTTF)
In the manufacturing industry, MTTF is one of the many metrics commonly used to
evaluate the reliability of manufactured products.
MTBF is used only when referring to repairable items, MTTF is used to refer to
non-repairable items.
 MTTF Calculation

MTTF is calculated as the

total hours of operation,
divided by the total
number of items being
tracked.
Example
Let’s assume we tested three identical pumps until all of them failed.
The first pump system failed after eight hours, the second one failed
at ten hours, and the third failed at twelve hours. MTTF in this
instance would be …..
Example
Let’s assume we tested three identical pumps until all of them failed.
The first pump system failed after eight hours, the second one failed
at ten hours, and the third failed at twelve hours. MTTF in this
instance would be (8 + 10 + 12) / 3 = 10 hours.
➢This would lead us to the conclusion that this particular type and model of the
pump will need to be replaced, on average, every 10 hours.
➢The only surefire way to increase MTTF is to look for higher-quality items made
from more durable materials.
 REL-1. The mean time between
failures of the card reader component
shall be at least 90 days.

REL-2. The system defect rate shall be

Example less than 1 failure per 1000 hours of
operation.

REL-3. No more than 1 per 1000000

transactions shall result in a failure
requiring a system restart.
 How would you judge whether this system was reliable enough?

What would be the consequences of experiencing a failure when performing

certain
operations with the system?

Following are What would you consider to be a critical failure, as opposed to a nuisance?
some questions
to ask user Under what conditions could a failure have severe repercussions on your
representatives business operations?

when you’re
eliciting reliability No one likes to see a system crash, but are there certain parts of the system
that absolutely have to be super-reliable?

requirements:
If the system goes down, how long could it stay offline before it significantly
affects your business operations?
How to Relate MTBF to System Availability
Availability metrics are expressed in terms of MTBF and MTTR.
Think of it as calculating the availability based on the actual time that the
machine is operating—excluding the time it takes for the machine to recover
from breakdowns.
This is quantified by the following equation:
Availability = MTBF / (MTBF + MTTR)
Internal Quality
Attributes
Internal quality attributes
Internal quality attributes are not directly observable during
execution of the software.

➢They are properties that a developer or maintainer perceives while looking at

the design or code to modify it, reuse it, or move it to another platform.

➢Internal attributes can indirectly affect the customer’s perception of the

product’s quality if it later proves difficult to add new functionality or if internal
inefficiencies result in performance degradation.
1. Efficiency
Efficiency is a measure of how well the system utilizes
processor capacity, disk space, memory, or communication
bandwidth. If a system consumes too much of the
available resources, users will encounter degraded
performance.
Efficiency is closely related to the external quality attribute
of performance
Efficiency Example
EFF-1 At least 30 percent of the processor capacity and memory
available to the application shall be unused at the planned peak load
conditions.

EFF-2. The system shall provide the operator with a warning message
when the usage load exceeds 80 percent of the maximum planned
capacity.
 What is the maximum number of
concurrent users now and anticipated in
the future?

The BA must
By how much could response times or
ask these type other performance indicators decrease
of questions before users or the business suffer
adverse consequences?

How many operations must the system

be able to perform simultaneously
under both normal and extreme
operating conditions?
2.Modifiability
Modifiability addresses how easily the software designs
and code can be understood, changed, and extended.

➢It is closely related to verifiability.

➢If developers anticipate making many enhancements, they can choose
design approaches that maximize the software’s modifiability.
➢High modifiability is critical for systems that will undergo frequent
revision, such as those being developed by using an incremental or
iterative life cycle.
Some aspects of Modifiability
Modifiability encompasses several other quality attribute terms that relate to
different forms of software maintenance
Modifiability Examples
MOD-1. A maintenance programmer experienced with the system shall be able to modify
existing reports to conform to revised chemical-reporting regulations from the federal
government with 10 hours or less of development effort.

SUP-1. A certified repair technician shall be able to replace the scanner module in no more than
10 minutes.

SUP-2. The printer shall display an error message if replacement ink cartridges were not
inserted in the proper slots
3.Portability
The effort needed to migrate software from one operating
environment to another is a measure of portability.

Portability has become increasingly important as

applications must run in multiple environments, such as
Windows, Mac, and Linux; iOS and Android; and PCs,
tablets, and phones.
Data portability requirements are also important.
Portability Examples
POR-1. Modifying the iOS version of the application to run on
Android devices shall require changing no more than 10 percent of
the source code.

POR-2. The user shall be able to port browser bookmarks to and

from Firefox, Internet Explorer, Opera, Chrome, and Safari.

POR-3. The platform migration tool shall transfer customized user

profiles to the new installation with no user action needed.
When you are exploring portability, questions like
the following might be helpful:

➢What different platforms will this software need to run on, both now and in the
future?

➢What portions of the product need to be designed for greater portability than other
portions?

➢What data fles, program components, or other elements of the system need to be
portable?

➢ By making the software more portable, what other quality attributes might be
compromised?
4. Reusability
Reusability indicates the relative effort required to convert a software
component for use in other applications

Reusable software must be modular, well documented, independent of a

specific application and operating environment, and somewhat generic in
capability.

Numerous project artifacts offer the potential for reuse, including

requirements, architectures, designs, code, tests, business rules, data models,
user class descriptions, stakeholder profiles, and glossary terms
Reusability Examples
REU-1. The chemical structure input functions shall be reusable at
the object code level in other applications.

REU-2. At least 30 percent of the application architecture shall be

reused from the approved reference architectures.

REU-3. The pricing algorithms shall be reusable by future store-

management applications.
5. Scalability

Scalability requirements address the ability of the application to

grow to accommodate more users, data, servers, geographic
locations, transactions, network traffic, searches, and other
services without compromising performance or correctness

Scalability is related to modifiability and to robustness, because one

category of robustness has to do with how the system behaves when
capacity limits are approached or exceeded.
Scalability..
Scalability has both hardware and software implications.

Scaling up a system could mean acquiring faster computers, adding

memory or disk space, adding servers, mirroring databases, or
increasing network capacity.
Software approaches might include distributing computations onto
multiple processors, compressing data, optimizing algorithms, and
other performance-tuning techniques.
Scalability Examples

SCA-1. The capacity of the emergency telephone system must be able to

be increased from 500 calls per day to 2,500 calls per day within 12
hours.

SCA-2. The website shall be able to handle a page-view growth rate of 30

percent per quarter for at least two years without user-perceptible
performance degradation.

SCA-3. The distribution system shall be able to accommodate up to 20

new warehouse centers.
6. Verifiability

More narrowly referred to as testability, verifiability refers to how well

software components or the integrated product can be evaluated to
demonstrate whether the system functions as expected.
Designing for verifiability is critical if the product has complex algorithms
and logic, or if it contains subtle functionality interrelationships.
Verifiability is also important if the product will be modified often,
because it will undergo frequent regression testing to determine whether
the changes damaged any existing functionality
Verifiability Examples

VER-1. The development environment configuration shall be identical to

the test configuration environment to avoid irreproducible testing
failures.

VER-2. A tester shall be able to configure which execution results are

logged during testing.

VER-3. The developer shall be able to set the computational module to

show the interim results of any specified algorithm group for debugging
purposes
Certain attributes are of particular
importance on certain types of projects:
❖Embedded systems: performance, efficiency, reliability, robustness,
safety, security, usability

❖ Internet and corporate applications: availability, integrity,

interoperability, performance, scalability, security, usability

❖ Desktop and mobile systems: performance, security, usability

❖ Gaming companies might want to capture emotional requirements

for their software
Implementing quality attribute requirements
Designers and programmers will have to determine the best way to satisfy each quality
requirement.
Although these are nonfunctional requirements, they can lead to derived functional
requirements, design guidelines, or other types of technical information that will produce the
desired product characteristics.
For example,
A medical device with stringent availability and reliability
requirements might include a backup battery power supply
(architecture), along with functional requirements to indicate when
the product is operating on battery power, when the battery is
getting low, and so forth.
This translation from external or internal quality requirements into
corresponding technical information is part of the requirements
analysis and high-level design processes.