Lecture 1 Introduction to Testing
Lecture 1 Introduction to Testing
Software Testing
What is software Testing?
Define
i. software testing
ii. Computer bug
iii. Computer fault
Bugs a.k.a. …
• Defect • Failure
• Fault • Inconsistency
• Problem • Product
• Error Anomaly
• Incident • Product
• Anomaly Incidence
• Variance • Feature :-)
Defective Software
• We develop programs that contain
defects
– How many? What kind?
• Hard to predict the future, however…
it is highly likely, that the software we
(including you!) will develop in the
future will not be significantly better.
Sources of Problems
• Requirements Definition: Erroneous, incomplete,
inconsistent requirements.
• Design: Fundamental design flaws in the software.
• Implementation: Mistakes in chip fabrication, wiring,
programming faults, malicious code.
• Support Systems: Poor programming languages,
faulty compilers and debuggers, misleading
development tools.
Sources of Problems (Cont’d)
• Inadequate Testing of Software:
Incomplete testing, poor verification,
mistakes in debugging.
• Evolution: Sloppy redevelopment or
maintenance, introduction of new flaws
in attempts to fix old flaws, incremental
escalation to inordinate complexity.
Adverse Effects of
Faulty Software
• Communications: Loss or corruption of
communication media, non delivery of
data.
• Space Applications: Lost lives, launch
delays.
• Defense and Warfare: Misidentification
of friend or foe.
Adverse Effects of Faulty
Software (Cont’d)
• Transportation: Deaths, delays,
sudden acceleration, inability to brake.
• Safety-critical Applications: Death,
injuries.
• Electric Power: Death, injuries, power
outages, long-term health hazards
(radiation).
Adverse Effects of Faulty
Software (Cont’d)
• Money Management: Fraud, violation of
privacy, shutdown of stock exchanges and
banks, negative interest rates.
• Control of Elections: Wrong results
(intentional or non-intentional).
• Control of Jails: Technology-aided escape
attempts and successes, accidental release
of inmates, failures in software controlled
locks.
• Law Enforcement: False arrests and
imprisonments.
Bug in Space Code
• Project Mercury’s FORTRAN code had the
following fault:
DO I=1.10 instead of ... DO I=1,10
• The fault was discovered in an analysis of
why the software did not seem to generate
results that were sufficiently accurate.
• The erroneous 1.10 would cause the loop to
be executed exactly once!
Military Aviation Problems
• An F-18 crashed because of a missing
exception condition:
if ... then ... without the else clause
that was thought could not possibly
arise.
• In simulation, an F-16 program bug
caused the virtual plane to flip over
whenever it crossed the equator, as a
result of a missing minus sign to
indicate south latitude.
Year Ambiguities
• In 1992, Mary Bandar received an
invitation to attend a kindergarten in
Winona, Minnesota, along with others
born in '88.
• Mary was 104 years old at the time.
Year Ambiguities (Cont’d)
• Mr. Blodgett’s auto insurance rate
tripled when he turned 101.
• He was the computer program’s first
driver over 100, and his age was
interpreted as 1.
• This is a double blunder because the
program’s definition of a teenager is
someone under 20!
Dates, Times, and Integers
• The number 32,768 = 215 has caused
all sorts of grief from the overflowing of
16-bit words.
• A Washington D.C. hospital computer
system collapsed on September 19,
15
1989, 2 days after January 1, 1900,
forcing a lengthy period of manual
operation.
Shaky Math
• In the US, five nuclear power plants
were shut down in 1979 because of a
program fault in a simulation program
used to design nuclear reactor to
withstand earthquakes.
• This program fault was, unfortunately,
discovered after the power plants were
built!
Bank Generosity
• A Norwegian bank ATM consistently
dispersed 10 times the amount
required.
• Many people joyously joined the queues
as the word spread.
Bank Generosity (Cont’d)
• A software flaw caused a UK bank to
duplicate every transfer payment
request for half an hour. The bank lost
2 billion British pounds!
• The bank eventually recovered the
funds but lost half a million pounds in
potential interest.
Discussion …
• Have you heard of other software bugs?
– In the media?
– From personal experience?
• Does this embarrass you as a future
software engineer?
Specification
“if you can’t say it, you can’t do it”
• You have to know what your product is before
you can say if it has a bug.
• A specification defines the product being
created and includes:
– Functional requirements that describes the
features the product will support. E.g., on a word
processor
• Save, print, check spelling, change font, …
– Non-functional requirements are constraints on
the product. E.g,
• Security, reliability, user friendliness, platform, …
A software bug occurs when at
least one of these rules is true
• The software does not do something that the
specification says it should do.
• The software does something that the
specification says it should not do.
• The software does something that the
specification does not mention.
• The software does not do something that the
product specification does not mention but
should.
• The software is difficult to understand, hard to
use, slow …
Most bugs are not because of
mistakes in the code …
• Specification (~= 55%)
• Design (~= 25%)
• Code (~= 15%)
• Other (~= 5%)
Relative cost of bugs
“bugs found later cost more to fix”
• Cost to fix a bug increases exponentially (10x)
– i.e., it increases tenfold as time increases
• E.g., a bug found during specification costs $1 to
fix.
• … if found in design cost is $10
• … if found in code cost is $100
• … if found in released software cost is $1000
Bug Free Software
• Software is in the news for the wrong reason
– Security breach, Mars Lander lost, hackers getting
credit card information, etc.
• Why can’t software engineers develop
software that just works?
– As software gets more features and supports more
platforms it becomes increasingly difficult to make
it create bug-free.
Discussion …
• Do you think bug free software is
unattainable?
– Are their technical barriers that make this
impossible?
– Is it just a question of time before we can
do this?
– Are we missing technology or processes?
Goal of a software tester
• … to find bugs
• … as early in the software development
processes as possible
• … and make sure they get fixed.