Lecture 2

Organization and Process Benchmarking In Software Quality Engineering

• The uniqueness of software quality assurance
• The environments for which SQA methods are
developed
• High complexity
• Invisibility of the product
• Limited opportunities to detect defects (“bugs”)
• Being contracted
• Subjection to customer-supplier relationship
• Requirement for teamwork
• Need for cooperation and coordination with other development
teams
• Need for interfaces with other software systems
• Need to continue carrying out a project while the team changes
• Need to continue maintaining the software system for years
 Attendance
control
system

Input interface Monthly attendance report,

including overtime calculations
Salary
processing
system

Output interface Money transfers to employees’

bank account accounts
Bank
information
systems
• What is software?
• Software errors, faults and failures
• Classification of the causes of software errors
• Software quality – definition
• Software quality assurance – definition and
objectives
• Software quality assurance and software
engineering
Software is:
Computer programs, procedures, and
possibly associated documentation and
data pertaining to the operation of a
computer system.
Software development process

software error
software fault
software failure
The nine causes of software errors are:
1. Faulty requirements definition
2. Client-developer communication failures
3. Deliberate deviations from software requirements
4. Logical design errors
5. Coding errors
6. Non-compliance with documentation and coding instructions
7. Shortcomings of the testing process
8. User interface and procedure errors
9. Documentation errors
Software quality is:

(1) The degree to which a system, component, or

process meets specified requirements.
(2) The degree to which a system, component, or
process meets customer or user needs or
expectations.
Software quality is :

Conformance to explicitly stated functional and

performance requirements, explicitly documented
development standards, and implicit characteristics
that are expected of all professionally developed
software.
Software quality assurance is:

1. A planned and systematic pattern of all actions

necessary to provide adequate confidence that an
item or product conforms to established technical
requirements.
2. A set of activities designed to evaluate the process
by which the products are developed or
manufactured. Contrast with: quality control.
Software quality assurance is:

A systematic, planned set of actions necessary to provide

adequate confidence that the software development
process or the maintenance process of a software system
product conforms to established functional technical
requirements as well as with the managerial
requirements of keeping the schedule and operating
within the budgetary confines.
(1) Assuring an acceptable level of confidence that the
software will conform to functional technical
requirements.
(2) Assuring an acceptable level of confidence that the
software will conform to managerial scheduling and
budgetary requirements.
(3) Initiation and management of activities for the
improvement and greater efficiency of software
development and SQA activities.
(1) Assuring an acceptable level of confidence that the
software maintenance activities will conform to the
functional technical requirements.
(2) Assuring an acceptable level of confidence that the
software maintenance activities will conform to
managerial scheduling and budgetary requirements.
(3) Initiate and manage activities to improve and increase the
efficiency of software maintenance and SQA activities.
• The need for comprehensive software quality
requirements
• Classification of requirements into software quality
factors
• Product operation factors
• Product revision factors
• Product transition factors
• Alternative models of software quality factors
• Who is interested in defining quality requirements?
• Software compliance with quality factors
 Software quality factors

Product operation factors

Product revision factors

Product transition factors

Requirement Document
• A comprehensive definition of requirement
• The need to define requirement that belong to each quality factor
• Requirement documents are expected to differ in the emphasis
placed on the various factors. (Sometimes for comparisons)
• Not all the factors will be represented.
• Correctness
• Reliability
• Efficiency
• Integrity
• Usability
Correctness
• Defined in a list of the software system’s required
outputs.
• Output specifications are usually
multidimensional, including:
- output mission
- required accuracy of the outputs
- the completeness of the output information
- the up-to-dateness of the information
- the availability of the information
- the standards for coding & documenting
Reliability
• Deals with failures to provide services.
• Usually refer to the maximum allowed software system failure rate
that can be the entire system or one or more of its functions.
 Efficiency

• Deals with the hardware resources needed to perform all the

functions
• May include the maximum values at which the hardware resources
will be applied in the software system.
• May also deal with the time between recharging of the system’s units.
Integrity

• Deals with the system security, namely, the prevention of the access
to unauthorized persons.
• Also deals with distinguishing the privilege (read, copy, write, …
permit) to the information of the personnel.
Usability
• Deals with the scope of staff resources needed to train a new
employee and to operate the system.
- Deals with software maintenance activities: corrective
maintenance, adaptive maintenance, perfect maintenance.

• Maintainability
• Flexibility
• Testability
Maintainability
• Determines the efforts that will be needed by users and maintenance
personnel to identify the reasons for failures, to correct the failures,
and to verify the success of correctness.
• Refers to modular structure (size), program documentation
(standards), manuals, etc.
Flexibility
• Refers to the capabilities & efforts required to support adaptive
maintenance & perfect maintenance, including human resources,
extents of activities, etc.
• E.g., The system should be suitable for teachers of all subjects and all
school levels.
Testability
• Deals with the testing of the system as well as with the operation.
- For ease of testing, such as providing predefined intermediate results and log
files.
- For operation, such as automatic diagnostics performed by the system prior to
starting the system; automatic generating report about detected faults.
- For maintenance, automatic diagnostic checks applied by maintenance
technicians to detect the causes of failures.
- Refers to adaptation to other environments and
interaction with other systems.

• Portability
• Reusability
• Interoperability
Portability
• Refers to adaptation to other environments consisting of different
hardware, operating systems, and so forth.
Reusability

• Deals with the use of software modules originally designed for one
project in a new software project being developed.
• Can save resources, shorten time and provide higher quality modules.
Interoperability

• Focuses on creating interfaces with other software systems or with

other equipment firmware.
• Can specify the names of the software or firmware for which interface
is required.
• Can also specify the output structure accepted as standard.
 Alternative factor models
No. Software quality McCall’s classic Evans and Deutsch and
factor model Marciniak model Willis model
1 Correctness + + +
2 Reliability + + +
3 Efficiency + + +
4 Integrity + + +
5 Usability + + +
6 Maintainability + + +
7 Flexibility + + +
8 Testability +
9 Portability + + +
10 Reusability + + +
11 Interoperability + + +
12 Verifiability + +
13 Expandability + +
14 Safety +
15 Manageability +
16 Survivability +
Verifiability
• Defines design and programming features.
• Refer to modularity and simplicity, and adherence to documentation
and programming guidelines.
• Improves the design reviews and software tests.
Expandability
• Refers to future efforts that will be needed to serve larger
populations.
Safety
• To eliminate conditions hazardous to operations of equipment as a
result of errors, including providing alarm signals.
Manageability
• Refers to the administrative tools that supports software modification,
such as CM.
Survivability
• Refers to the continuity of service. These define the minimum time
allowed between failures, and the maximum time permitted for
recovery of service.
Who is interested in defining quality
requirements?
• A project will be carried out according to two
requirement documents:
- The client’s requirement document
- The developer’s additional requirement document
Software compliance with quality
factors
• Examine the development process by reviews, inspections, tests, etc.
• Verify and validate the quality by measuring.
• Sub-factors are defined for quality factors.
• Please see Table 3.3 suggested by Evans and Marciniak (1987).
 Why do we need a wide range
of SQA components ?
• Multitude of source of errors
- various style of source of errors will affect the SQA
components

* The environment in which software development &

maintenance is undertaken directly influences the
SQA components.
 Six Classes of SQA Components
• Pre-project components
• Software project life cycle components
• Infrastructure components for error prevention and improvements
• Management SQA components
• SQA standards, system certification and assessment components
• Organizing for SQA – the human components
• Contract reviews
- To assure commitments have been properly defined under
resources, schedule and budget constraints.
- detailed examination of the project proposal draft
- detailed examination of the contract draft
- include: requirements, schedule, resource, staff’s capacity,
customer capacity, risks.
• Development and quality plans
- To assure plans have been correctly determined.
Development and
quality plans
• Development plan
- Schedule
- manpower & resources
- risk evaluation
- organizational related issues
- project methodology, development tools;
- reuse plans
• Quality plan
- quality goal
- criteria for starting & ending of stage
- list of reviews, tests, scheduled V&V activities
 - The development and the operation-maintenance stages

• Reviews (Formal design reviews & Peer reviews)

• Expert opinions (Formal design reviews)
• Software testing
• Software maintenance components
• Assurance of the quality of external participants’
work
 - Use of computerized and automatic tools

• Procedures and work instruction (based on experience

and knowledge)
• Templates and checklists (supporting devices)
• Staff training, retraining and certification
• Preventive and corrective actions
• Configuration management
• Documentation control
 Documentation control
activities
SQA requires the application of measure to ensure
long-term availability of major documents

• Def. of the types of controlled documents

• Specification of the formats, identification methods
• Def. of review and approval processes for each
controlled document
• Def. of archive storage methods.
• Project progress control
• Software quality metrics
• Software quality costs
Project progress control
Objective: To detect the appearance of any situation
that may induce deviation from the project plan.

Focus on:
1. resource usage
2. schedule
3. risk management activities
4. the budget
Software quality metrics
• Quality of software development and maintenance
activities
• Development teams’ productivity
• Help desk and maintenance teams’ productivity
• Software faults density
• Schedule deviations.
Software quality costs
• Cost of control (prevention costs, appraisal costs, managerial
preparation and control costs) + costs of failure
• Expanding the resources allocated to control activities yields much
larger savings in failure costs while reducing total quality costs
Appraisal Cost
• Appraisal costs are a specific category of quality control costs.
Companies pay appraisal costs as part of the quality control process
to ensure that their products and services meet customer
expectations and regulatory requirements.

• These costs could include expenses for field tests and inspections.
Failure Cost
• Cost of quality (COQ) is defined as a methodology that allows an
organization to determine the extent to which its resources are used for
activities that prevent poor quality, that appraise the quality of the
organization's products or services, and that result from internal and
external failures.
1. External Failure Cost: Cost associated with defects found after the
customer receives the product or service. Example: Processing customer
complaints, customer returns, warranty claims, product recalls.
2. Internal Failure Cost: Cost associated with defects found before the
customer receives the product or service. Example: Scrap, rework, re-
inspection, re-testing, material review, material downgrades
Prevention Cost:
• Prevention Cost: Cost incurred to prevent (keep failure and appraisal
cost to a minimum) poor quality.
• Example: New product review, quality planning, supplier surveys,
process reviews, quality improvement teams, education and training.
 • Project process standards (focus on “how”)
• Quality management standards (focus on “what”)

Objectives:
 Utilization of international professional knowledge
 Improvement of coordination with other organizations’
quality systems
 Objective professional evaluation and measurement of the
organization’s SQA achievement
• Management’s role in SQA
• The SQA unit
• SQA trusties
• SQA committees
• SQA forums
Considerations guiding
construction of organization’s
SQA system
• The SQA organizational base
• The SQA components to be implemented within
the organization and the extent of their use
The main considerations affecting
the use of the components (1/2)
Organizational considerations
• Type of software development clientele
• Type of software maintenance clientele
• Range of software products
• Size of the organization
• Degree and nature of cooperation with other organizations carrying out
related projects
• Optimization objective (quality, productivity, efficiency, savings)
Project and maintenance service considerations
• Level of complexity and difficulty
• Degrees of experience with the project technology
• Extent of software reuse in the new projects
 (2/2)
Professional staff considerations
• Professional qualifications
• Level of acquaintance with team members
 The Software Quality Shrine
Pre-
proj
s ec t SQ
p o nent A co
co m mpo
QA ne n
j ec t S Project ts
pro Development plan
Pre-
Contract review and Quality Plan
Ch.5 Ch.6

Project Life Cycle SQA components

SQA of External Participants

Formal Design Reviews

Software Maintenance
Experts Opinion

Software Testing
Peer Reviews

Chs. 9-10
Sec. 8.2

Sec. 8.3

Sec. 8.5

Ch. 11

Ch 12
Quality Infrastructure components Quality Management Standards
Document- Project
Supporting Training Preventive Configuration Project Software Software Quality Process
Management
ation Progress Quality Quality Management Standards
Procedures Devices Instruction Actions Control Control Metrics Costs Standards
Ch. 14 Ch. 15 Ch. 16 Ch.17 Ch. 18 Ch. 19 Ch.24
Ch. 20 Ch. 21 Ch. 22 Ch. 23

Organizational Base – Human components

Management - Ch. 25 SQA Unit - Sec. 26.1 SQA Trustees – Sec. 26.2 SQA Committees – Sec. 26.2 SQA Forums – Sec 26.4
 Case Study 1
Ibrahim is thinking of automating the proposed new system further by
contracting the backup procedure out to an Internet-based backup and
archiving company. The hotel would send a copy of all its data files over
the Internet to this company for backup storage. Ibrahim has been given
the names of two highly professional companies that offer this service.
Apart from the cost of this service, discuss two important factors that
Ibrahim should investigate before signing a contract with either
company.
Case Study 2
• Elegant Software Solutions has been employing the Build-And-Fix
approach to software development for the past 3 years. This has
resulted in several lawsuits for late delivery and poor-quality software.
To address this issue senior management has hired you as a quality
assessment consultant. Explain the source of the problem and
document an alternative software development strategy. The
proposed strategy must be fully explained, giving any inherent
advantages and disadvantages.