Fundamentals of Testing

Basic Concepts

Software Quality Assurance

Telerik Software Academy
http://academy.telerik.com
 The Lectors
 Snejina Lazarova
Project Manager
BI & Reporting Team

 Dimo Mitev
QA Architect
Backend Services Team

2
 Table of Contents
1.Why is Testing Necessary?
 Software Systems Context
 Causes of Software Defects
 Role of Testing in Software Development,
Maintenance and Operations
 Testing and Quality
 How Much Testing is Enough?
2.What is Testing?
3.Seven Testing Principles
3
Why is Testing Necessary?
 Software Systems Context
 Software systems are integral part of every
sphere of our life Education
Industry Business

Industry
Military

Entertainment

5
 Causes of Software Defects
 A human being can make an
error (mistake)
 Errors produce defects
 Defects are faults / bugs in the
program code, or in a document
 If a defect in code is executed,
that might cause a failure:
 Fail to do what it should do
 Do something it shouldn’t
6
 Causes of Software Defects (2)
 The human factor
 Humans make mistakes
 Poor training
 Time pressure
 Code complexity
 Complexity of infrastructure
 Changing technologies

7
 Causes of Software Defects (3)
 Organizational factors
 Inefficient communication
 Unclearly defined requirements
 Environmental conditions
 Radiation, Magnetism, Electronic fields,
Pollution, Etc.
 These can change the hardware conditions

8
 Definitions
 Anomaly
 Any condition that deviates from expectation
based on requirements specifications, design
documents, user documents, standards, etc. or
from someone’s perception or experience
 Error (mistake)
 A human action that produces an incorrect
result

9
 Definitions (2)
 Bug/Defect/Fault/Problem
 A flaw in a component or system that can cause the
component or system to fail to perform its required function
 A defect, if encountered during execution, may cause a
failure of the component or system.
 Failure
 Actual deviation of the component or system from its
expected delivery, service or result
 Defect/fault masking
 An occurrence in which one defect prevents the detection of
another

10
 What is Software Quality?
 Definition found in IEEE Standard Glossary Of
Software Engineering Terminology
 The degree to which a system, component, or
process meets specified requirements
 The degree to which a system, component, or
process meets customer or user needs or
expectations

11
 The Role of Testing
 Role of testing in software development,
maintenance and operations
 Reduce the risk of problems
 Reduce long-term defect-related costs
 Contribute to the quality of the software
 Help meeting standards:
 Contractual or legal requirements
 Industry-specific standards

12
 Testing and Quality
 Testing can give confidence in the quality of
the software if it finds few or no defects
 If defects are found, the quality increases
when those defects are fixed
 Lessons learnt from previous mistakes improve
future performance

13
 How Much Testing is Enough?
 How much testing should be done is a matter
of risk:
 Too much testing can delay the product release
and increase the product price
 Insufficient testing hides risks of errors in the
final product

14
What Is Testing?
 What is Testing?
 What is testing?
 The process of exercising software
 To verify that it satisfies specified requirements
and to detect errors
 The process of analyzing a software item
 To detect the differences between existing and
required conditions (that is, bugs),
 To evaluate the features of the software item

16
 What is Testing?
 What is testing?
 The process of operating a system or
component under specified conditions
 Observing or recording the results
 Making an evaluation of some aspect of the
system or component

17
 Main Test Activities
 Testing is not just running tests, but also:
 Planning and control
 Choosing test conditions
 Designing and executing test cases
 Checking results
 Evaluating exit criteria
 Reporting on the testing process and system
under test
 Finalizing or completing closure activities
18
 Main Objectives in Testing
 Testing pursues several objectives:
 Finding defects
 Gaining confidence about the level of quality
 Providing information for decision-making
 Preventing defects

19
 Different Viewpoints

 Objectives of testing differ according to the

point of view:
 Developers
 Are perceived as very creative - they write code
without which there would be no system
 Are rarely good communicators
 Can often specialize in just one or two skills (e.g.
VB, C++, JAVA, SQL)

20
 Different Viewpoints

 Objectives of testing differ according to the

point of view:
 QA testers
 Are perceived as destructive - only happy when they
are finding faults!
 Usually require good communication skills, tact &
diplomacy.
 Normally need to be multi-talented (technical,
testing, team skills).

21
 Debugging VS Testing
 Testing
 The activity that initially finds failures in a
software item
 Debugging
 The development activity that finds, analyses
and removes the cause of the failure
 Subsequent re-testing by a tester
 Ensures that the fix does indeed resolve the
failure

22
Seven Testing Principles
 Seven Testing Principles
1.Testing shows presence of defects
 Testing can show that defects are present
 Cannot prove that there are no defects
 Appropriate testing reduces the probability for
defects

24
 Seven Testing Principles (2)
2. Exhaustive testing is impossible
 All combinations of inputs and preconditions
are usually almost infinite number
 Testing everything is not feasible
 Except for trivial cases
 Risk analysis and priorities
should be used
to focus testing efforts

25
 Seven Testing Principles (3)
3. Early testing
 Testing activities shall be started as early as
possible
 And shall be focused on defined objectives
 The later a bug is found – the more it costs!

26
 Seven Testing Principles (4)
4.Defect clustering
 Testing effort shall be focused proportionally
 To the expected and later observed defect
density of modules
 A small number of modules usually contains
most of the defects discovered
 Responsible for most of
the operational failures

27
 Seven Testing Principles (5)
5.Pesticide paradox
 Same tests repeated over and over again tend
to loose their effectiveness
 Previously undetected defects remain
undiscovered
 New and modified test cases
should be developed

28
 Seven Testing Principles (6)
6. Testing is context dependent
 Testing is done differently in different contexts
 Example: safety-critical software is tested
differently from an e-commerce site

29
 Seven Testing Principles (7)
7.Absence-of-errors fallacy
 Finding and fixing defects itself does not help in
these cases:
 The system built is unusable
 Does not fulfill the users’ needs and expectations

30
 Fundamentals of Testing

?
?

?
Questions?
?

?
?
?
?
?
 Exercises
1. A test team consistently finds between 90% and
95% of the defects present in the system under
test. While the test manager understands that
this is a good defect-detection percentage for
her test team and industry, senior management
and executives remain disappointed in the test
group, saying that the test team misses too
many bugs. The users are generally happy with
the system and the failures which have occurred
have generally been low impact.
Continues …
32
 Exercises (2)
Which of the following testing principles is most
likely to help the test manager explain to these
managers and executives why some defects are
likely to be missed?
a)Exhaustive testing is impossible
b)Defect clustering
c) Pesticide paradox
d)Absence-of-errors fallacy

33
 Exercises (3)
2. Search the Internet and get familiar with some
famous examples of software failures and the
costs that have occurred from them
 E.g., you can find some information here:
http://www.devtopics.com/20-famous-software-
disasters/
3. Reproduce the following bugs in windows 7:
 http://www.askvg.com/microsoft-windows-seve
n-bug-report/

34
 Free Trainings @ Telerik Academy
 C# Programming @ Telerik Academy
 csharpfundamentals.telerik.com
 Telerik Software Academy
 academy.telerik.com
 Telerik Academy @ Facebook
 facebook.com/TelerikAcademy
 Telerik Software Academy Forums
 forums.academy.telerik.com