Software Testing & Quality

Engineering Chapter 3

Standards and
Models

Linda Westfall
Quality Press
Objectives

Standards and Models:

• ISO 9000 standards

• IEEE software engineering standards

• SEI Compatibility Maturity Model Integration (CMMI)

ISO
 Explanation of ISO.

 What is ISO-9000?

 Different family members of ISO-9000.

 Which family member of ISO-9000 relates to Software Industry.

 When to use ISO-9000.

 ISO-9000 certification
What is ISO. . .?
ISO stands for the International Standardization organization. It is an
organization which standardized the things on international level so things become
easy to judge.

Examples of Standards:
• Go any where in the world you will find an hour of 60 minutes this is standard
thing.
• Go any where in the world you will find 1Gb of memory equal to 1024Mb no
one can violate this rule no one will create a USB drive of 512mb and say it is
1GB for me.
ISO 9000 is based on documentation and is based on the following:
• Document what you do.
• Do what your document.
• Prove it and improve it.
 What is ISO 9000
 ISO 9000 describes the fundamental features of a Quality Management System (QMS) and its
terminology. ISO 9000 describes how a QMS can be applied to the creation of products and provision
of services. ISO 9004 applies to process improvement.
 Standards such as ISO 9000 series try to ensure that a monitoring and control system to check quality
is in place. They are concerned with the certification of the development process, not of the end-
product.

• ISO 9000 is a family of quality based standards by the International

Standardization Organization (ISO).
• This ISO 9000 family consist on 5 family members from 9000 to 9004.
• All these family members are targeting the quality from different perspectives
and they are applicable to different things and fields of life.
• No one family member of ISO 9000 is specific to any particular product.
ISO 9000 Family members
ISO Standard Description of standards

ISO-9000 Tell details about 9001,2 and 3 which one is best for you.

ISO-9001 Deal with quality assuring in design, development, installation and

maintenance.

ISO-9002 Assesse the production and installation process.

ISO-9003 Evaluation of final inspection and test phase.

ISO-9004 It defines 20 concepts about the quality which are used in the above three
standards 9001,9002 and 9003

ISO 9000-3 It is a standard for quality software systems.

What is ISO 9000
From the family of ISO 9000 the family member ISO-9001 relates the
most to software industry.

ISO-9001
• ISO-9001: It deals with the quality assurance in design,
development, production, installation and servicing.

• The all above mentions things of ISO9001 are related to Software

industry so we chose it the most in software industry.
When to use ISO 9000
• When organization want to regulate the quality system in the organization
so they chose one of the family member of ISO 9000 which relates the
most with their organization.

• Example: In our example if we are a Software industry then to

regulate the quality system in our industry we will chose the ISO 9001
as it relates the most to our software industry.

• During a contract between two parties when an organization want to check

the quality of the suppliers supply.
ISO 9000-3: Principal areas of quality focus

• management responsibility • control of inspection, measuring,

• quality system requirements and test equipment
• inspection and test status
• contract review requirements
• control of nonconforming
• product design requirements products
• document and data control • corrective and preventive actions
• purchasing requirements • handling, storage, and delivery
• customer supplied products • control of quality records
• product identification and traceability • internal quality audit
requirements
• process control requirements
• training requirements
• inspection and testing • servicing requirements
• statistical techniques
ISO-9000 Certification
You have to get ISO-9000 certifications to prove you are delivering the quality

• Certification also help to obtain more business because

many customers require that organizations obtain certain
standards.

• Certification is granted by an external body. Third party

certification bodies provide independent confirmation
that organizations meet the requirements of a certain
standard and hence a certificate is given to that
organizations
An overview of ISO-9000 QMS requirements
The standard is built on foundation of the following principles.
1. Customer Focus
2. Leadership
3. Involvement of People
4. Process Approach
5. Systematic Approach to Management
6. Continual Improvement
7. Factual Approach to Decision-Making
8. Mutually Beneficial Supplier Relationships*
An overview of ISO-9000 QMS requirements - continued
Aforementioned principles are applied through cycles which involve the following
activities.
1. Determining the needs and expectations of the customers.
2. Establishing a Quality policy i.e. a framework which allows the organization’s
objectives in relation to quality to be defines.
3. Design of process which will create the products (or deliver the services) which will
have the qualities implied in the organization’s quality objectives.
4. Allocation of the responsibilities for meeting these requirements for each element
of each process.
5. Ensuring that resources are available to execute the process properly.
6. Design of methods for measuring the effectiveness and efficiency of each process.
7. Gathering measurements.
8. Identification of any discrepancies between actual and target values.
9. Analyze and elimination of causes of discrepancies.
An Overview of the IEEE Software Engineering Standards
• Customer Standards: Standards that describe the
interaction between the customer and supplier of a
software engineering project.
• Process Standards: Standards that describe processes
spanning the life cycle of a software product or service,
including acquisition, supply, development,
maintenance, operations, and measurements.
• Product Standards: Standards that explain the
requirements for classes of software products
characteristics, measurements, evaluations, and
specifications.
• Resource Standards: Standards that recommend proper
documentation, methods, models and tools for a well-
managed software program and its related processes. The SESC object model of software engineering
IEEE standards (SESC the IEEE Software Engineering Standards Committee)
The framework specifies six layers and—for
the middle three layers—four stacks, as
pictured in Figure
1. Terminology: Documents prescribing terms and
vocabulary.
2. Overall Guide: One document providing overall guidance
for the entire collection.
3. Principles: Documents that describe principles or
objectives for use of the standards in the collection.
4. Element Standards: Standards that are the basis for
conformity.
5. Application Guides and Supplements: Guides and
supplements that give advice for using the standards in
various situations.
6. Toolbox of Techniques: Descriptions of techniques that
might be helpful in implementing the provisions of the
higher-level documents. The SESC framework model.
 SESC Framework

Issues / Problems.
This is evidenced by numerous standards having secondary assignments.
For example, IEEE Std. 829 (Software Test Documentation) has a primary
classification as a Resource standard, and a secondary classification as a
Process standard. Similarly, IEEE Std. 730 (Software Quality Assurance Plans)
has a primary classification as a Process standard, and a secondary
classification as a Product standard. In total, there are 15 such secondary
classifications of standards and guides.
The SESC alternative model suffers the same classification ambiguity
problem. For example, IEEE Std. 1058 (Software Project Management
Plans), is assigned to the Project Management category, however, being a
plan, can also be in the Plans category.
A more important short-coming is that it don’t tell you how to get started.
The models don’t specify a phase-in strategy, and they don’t specify core
standards for just getting started.
What is CMMI
• CMMI (Capability Maturity Model Integration) is a proven
industry framework to improve product quality and
development efficiency for both hardware and software
– Sponsored by US Department of Defence in
cooperation with Carnegie Mellon University and the
Software Engineering Institute (SEI)
– Many companies have been involved in CMMI
definition such as Motorola and Ericsson
– CMMI has been established as a model to improve
business results
• CMMI, staged, uses 5 levels to describe the maturity of the
organization, same as predecessor CMM
– Vastly improved version of the CMM
– Emphasis on business needs, integration and
institutionalization
How can CMMI Help
• CMMI provides a way to focus and manage hardware and
software development from product inception through
deployment and maintenance.
– ISO/TL9000 are still required. CMMI interfaces well
with them. CMMI and TL are complementary - both
are needed since they address different aspects.
• ISO/TL9000 is a process compliance standard
• CMMI is a process improvement model
• Behavioral changes are needed at both management and staff
levels. Examples:
– Increased personal accountability
– Tighter links between Product Management, Development,
SCN, etc.
• Initially a lot of investment required – but, if properly managed,
we will be more efficient and productive while turning out
products with consistently higher quality.
Purpose of CMMI
The purpose of CMM Integration is to provide guidance for improving
organization’s processes and your ability to manage the development, acquisition,
and maintenance of products or services.”

Covers the development of software systems

Focus on:
• Applying systematic, disciplined, and quantifiable approaches to development.
• Operation.
• Maintenance of software.

Models of software engineering:

• Process management.
• Project management.
• Support.
• Engineering process areas.
CMMI Staged Representation - 5 Maturity Levels

Level 5 Process performance

continually improved through
incremental and innovative
Optimizing technological improvements.
Level 4

yit
ur
Processes are controlled using

at
Quantitatively
statistical and other quantitative

M
Managed
techniques.
ss
ce
o
Level 3
Processes are well characterized and
Pr

understood. Processes, standards,

Defined
procedures, tools, etc. are defined at the
Level 2 organizational (Organization X ) level.
Proactive.
Managed Processes are planned, documented, performed,
monitored, and controlled at the project level. Often
Level 1 reactive.

Initial Processes are unpredictable, poorly controlled, reactive.

 Maturity Level 1
Initial

• Maturity Level 1 deals with performed processes.

• Processes are unpredictable, poorly controlled, reactive.
• The process performance may not be stable and may not meet
specific objectives such as quality, cost, and schedule, but useful
work can be done.
 Maturity Level 2
Managed at the Project Level

• Maturity Level 2 deals with managed processes.

• A managed process is a performed process that is also:
– Planned and executed in accordance with policy
– Employs skilled people
– Adequate resources are available
– Controlled outputs are produced
– Stakeholders are involved
– The process is reviewed and evaluated for adherence to
requirements
• Processes are planned, documented, performed, monitored, and
controlled at the project level. Often reactive.
• The managed process comes closer to achieving the specific
objectives such as quality, cost, and schedule.
Maturity Level 3
Defined at the Organization Level

• Maturity Level 3 deals with defined processes.

• A defined process is a managed process that:
– Well defined, understood, deployed and executed
across the entire organization. Proactive.
– Processes, standards, procedures, tools, etc. are
defined at the organizational (Organization X )
level. Project or local tailoring is allowed, however
it must be based on the organization’s set of
standard processes and defined per the
organization’s tailoring guidelines.
• Major portions of the organization cannot “opt out.”
Maturity Level 4
• Sub-processes are selected that significantly contribute to overall process performance
• As criteria in managing process the quantitative objects for quality are established
• Quantitative objectives are based on:
– Needs of a customer
– End users
– Organization
– Process implements
• For these processes, detailed measures of process performance are collected and statistically
analyzed
Maturity Level 5
• Focuses on continually improving process performance
through:
– Incremental technological improvements
– Innovative technological improvements
• Both processes are the organization’s set of
measurable improvement activities
 Conclusion
CMMI Benefits

• CMMI product users can expect to:

– Efficiently and effectively improve and
assess multiple disciplines across their
organization
– Reduce costs (including training)
associated with improving and assessing
processes
– Deploy a common, integrated vision of
process improvement that can be used
as a basis for enterprise-wide process Improved
Processes
improvement efforts.

Capability Maturity Model Integration 25/97