Software Quality Assurance

(ITC- 704)
Lecture 02
Software Defects

Dr. Zulfikar Ahmed Maher

2
Effects of Software Defects - 1
 Bugs can have a wide variety of effects,
with varying levels of inconvenience to
the user of the software. Some bugs
have only a subtle effect on the
program’s functionality, and may thus
lie undetected for a long time. More
serious bugs may cause the software to
crash or freeze leading to a denial of
service
3
Effects of Software Defects - 2
 Others qualify as security bugs and
might for example enable a malicious
user to bypass access controls in order
to obtain unauthorized privileges

4
Effects of Software Defects - 3
 The results of bugs may be extremely
serious
 In 1996, the European Space Agency’s
US $1 billion prototype Arian 5 rocket
was destroyed less than a minute after
launch, due a bug in the on-board
guidance computer program

5
Effects of Software Defects - 4
 In June 1994, a Royal Air Force Chinook
crashed into the Mull of Kintyre, killing
29 people. An investigation uncovered
sufficient evidence to convince that it
may have been caused by a software
bug in the aircraft’s engine control
computer

6
Effects of Software Defects - 5
 In 2002, a study commissioned by the
US Department of Commerce’ National
Institute of Standards and Technology
concluded that software bugs are so
prevalent and detrimental that they cost
the US economy and estimated US $59
billion annually, or about 0.6 percent of
the gross domestic product

7
 Software Defects in Six
Application Size Ranges
 1 function point or 125 C statements
 10 function points or 1,250 C statements
 100 function points or 12,500 C statements
 1,000 function points or 125,000 C
statements
 10,000 function points or 1,250,000 C
statements
 100,000 function points or 12,500,000 C
statements 8
Categories of Software
Defects
 Errors of commission
 Errors of omission
 Errors of clarity and ambiguity
 Errors of speed or capacity

9
Errors of Commission
 Something wrong is done
 A classic example at the code level
would be going through a loop one time
too many or branching on the wrong
address

10
Errors of Omission
 Something left out by accident
 For example, omitting a parentheses in
nested expressions

11
Errors of Clarity and Ambiguity
 Different interpretations of the same
statement
 This kind of error is common with all
natural language requirements and
specification documents and user
manuals, too.

12
Errors of Speed and Capacity
 Application works, but not fast enough

13
 Software defects can be found in any of the
documents and work products including very
serious ones in cost estimates and
development plans
 However, there are seven major classes of
software work products where defects have a
strong probability of triggering some kind of
request for warranty repair if they reach the
field

14
Software Defect Origins
 Errors in Requirements
 Errors in Design
 Errors in Source code
 Errors in User Documentation
 Errors due to “Bad fixes”
 Errors in Data and Tables
 Errors in Test Cases
15
 We’ll discuss all of them in detail, when
we talk about different processes of
software development life cycle

16
Defect Discovery
 Defects are discovered by developers &
testers (usually) before release
 Defects are discovered by customers
and users (usually) after release
 Defects discovered after release can be
embarrassing for the development team

17
Defect Discovery by
Customers
 Rule 1: Defect discovery is directly
related to the number of users
 Rule 2: Defect discovery is inversely
related to the number of defects

18
Software Defect Elimination
Strategies
 Effective defect prevention
 High levels of defect removal efficiency
 Accurate defect prediction before the
project begins
 Accurate defect tracking during
development
 Useful quality measurements
 Ensuring high levels of user-satisfaction
19
Defect Prevention and
Removal
 Both defect prevention and removal
techniques are used by the “best-in-
the-class” companies
 Defect prevention is very difficult to
understand, study, and quantify. We’ll
talk about defect prevent in a later
lecture
 Both non-test and testing defect
removal techniques must be applied
20
Typical Defect Removal
 Inspections
 Direct fault detection and removal
 Testing
 Failure observation and fault removal

21
Inspections - 1
 Inspections are critical examinations of
software artifacts by human inspectors
aimed at discovering and fixing faults in
the software systems

22
Inspections - 2
 Inspections are critical reading and
analysis of software code or other
software artifacts, such as designs,
product specifications, test plans, etc
 Inspections are typically conducted by
multiple human inspectors, through
some coordination process. Multiple
inspection phases or sessions may be
used
23
Inspections - 3
 Faults are detected directly in
inspection by human inspectors, either
during their individual inspections or
various types of group sessions
 Identified faults need to be removed as
a result of the inspection process, and
their removal also needs to be verified

24
Inspections - 4
 The inspection processes vary, but typically
include some planning and follow-up activities
in addition to the core inspection activity
 The formality and structure of inspections
may vary, from very informal reviews and
walkthroughs, to fairly formal variations of
Fagan inspection, to correctness inspections
approaching the rigor and formality of formal
methods

25
Non-Test Defect Removal
Methods
 Requirement inspections
 Design inspections
 Code inspections
 Test plan reviews
 Test-case inspections
 User documentation editing or reviews

26
Testing Defect Removal
Methods
 Unit test by individual programmers
 New function testing
 Regression testing
 Performance testing
 Integration testing
 System testing
 Field test (external beta test)
27
Defect Removal
 Not all defects are equal when it comes
to removal
 Requirements errors, design problems,
and “bad fixes” are particularly difficult

28
Software Defect Origins &
Defect Removal Effectiveness
Req. Design Code Document Perf.
Defects Defects Defects Defects Defects

Reviews /
Inspections Fair Excellent Excellent Good Fair

Prototypes Good Fair Fair N/A Good

Testing (all
forms) Poor Poor Good Fair Excellent

Correctness
Proofs Poor Poor Good Fair Poor

29
Defect Removal Efficiency
 Accumulation of defect statistics for
errors found prior to delivery, and then
for a predetermined period after
deployment (usually one year)
 US averages: 85%
 Best projects in best US companies:
99%

30
Summary
 In today’s lecture, we talked about
software defects and where are they
introduced in the software product
 We discussed the approaches to
eliminating these defects

31
References
 Software Quality: Analysis and
Guidelines for Success by Capers Jones
 Software Quality Engineering: Testing,
Quality Assurance, and Quantifiable
Improvement by Jeff Tian

32