18CSE033T

SOFTWARE TESTING
UNIT I - INTRODUCTION
COURSE OUTCOME: UNDERSTAND THE TEST PLAN, DESIGN TEST
CASES AND TEAM DEVELOPMENT. (CO1)
TOPICS:
Testing as an Engineering Origins of Defects
Activity Cost of defects
Testing as a Process Defect Classes
Testing Maturity Model Defects life cycle
Testing axioms Using Bugzilla for logging and
Basic definitions tracing defects
Software Testing Principles
Tester's Role in a Software
Development Organization
Testing as an Engineering Activity
Software products are
Built on time
Within budget
High quality
Using an engineering approach to software development means the following
 The development of the process is well understood.
 Projects are planned.
 Life cycle models are defined and adhered to.
 Standards are in place for product and process
 Measurements are employed to evaluate product and process quality.
 Components are reused.
Validation and Verification process play a key role in quality determination. Engineers should have
proper education, training and certification.
A test specialist is one whose education is based on the principles, practices and processes that
constitute the software engineering discipline and whose specific focus is on one area of that
discipline, software testing.
A test specialist who is trained as an engineer should have knowledge on the following
 Test related principles,
 Processes,
 Measurements,
 Standards,
 Plans,
 Tools and methods
 How to apply them to the testing tasks.
 Testing as a Process
Software development is an engineering activity for a
quality product
Testing is not just a phase in SDLC - runs parallel to SDLC
Once the Software Requirement Specification (SRS)
document is prepared, testing process starts
Software testing process must be planned, specified,
designed, implemented, and quantified
Testing must be governed by the quality attributes of the
software product
Testing is a dual-purpose process, as it is used to detect
bugs as well as to establish confidence in the quality of
software
To ensure better quality software
Testing process should be organized - there is enough time for important and critical features
of the software
Testing techniques should be adopted such that these techniques detect maximum bugs.
Quality factors should be quantified so that there is a clear understanding in running the
testing process.
Testing procedures and steps must be defined and documented.
There must be scope for continuous process improvement.
Testing Maturity Model
 Test maturity model is based on capability maturity model specifies an increasing series of
levels of a software development organization.
 The higher the level, the better the software development process, hence reaching each level
is an expensive and time-consuming process
 It can be complemented with any process improvement model or can be used as a STAND
ALONE model.
 TMM model has major two components
1. A set of 5 levels that define testing capability
2. An Assessment Model
TMM Levels Goals An objective of TMM levels
At this level, no process areas are identified
Software should run An objective of testing is to ensure that software is working fine
Level 1: Initial
successfully This level lacks resources, tools, and trained staff
No Quality Assurance checks before software delivery
This level distinguish testing from debugging & they are considered distinct activities
Develop testing and
Level 2: Testing phase comes after coding
debugging goals
Defined A primary goal of testing is to show software meets specification
and policies
Basic testing methods and techniques are in place
Testing gets integrated into an entire life cycle
Integration of
Level 3: Based on requirements test objectives are defined
testing into the
Integrated Test organization exists
software lifecycle
Testing recognized as a professional activity
Level 4: Testing is a measured and quantified process
Establish a test
Management Review at all development phases are recognized as tests
measurement
and For reuse and Regression Testing, test cases are gathered and recorded in a test database
program
Measurement Defects are logged and given severity levels
Testing is managed and defined
Testing effectiveness and costs can be monitored
Testing can be fine-tuned and continuously improved
Level 5: Test process
Quality control and Defect prevention are practiced
Optimized optimization
Process reuse is practiced
Test related metrics also have tool support
Tools provide support for Test Case design and defect collection
Testing axioms
1. It is Impossible to Test a Program Completely
2. Software Testing is One of the Risk Based Exercise
3. Testing cannot show the absence of bugs
4. The more bugs you find, the more bugs there are
5. Not all bugs found will be fixed
6. we can't say that when a bug is indeed a bug
7. The specifications mentioned by the user should never fail
8. Software Testers are Not the Most Popular Members in a Project
9. Software Testing is Somewhat Disciplined and Technical Profession
 1. It is Impossible to Test a Program Completely
There may be a large number of possible inputs.
There may be a large number of possible outputs.
There may be different paths to design a software based on users requirements.
It is completely based on specifications of the software.
2. Software Testing is One of the Risk Based Exercise
If you test the software completely it increases the cost for testing
If you leave some aspects in the testing there may be chances of occurring of errors
If your skipped part has errors it may cause a risk these risks includes:
Financial loss, security loss, money loss or loss of a life! We can't predict the type of risk.
If you test too little of the software, the probability of software failure may increase.
Software testing takes so much time.
 3. Testing cannot show the absence of bugs
We can't guarantee that a software has not bugs
In every software there may be minute bugs that may be visible or invisible.
Can you guarantee that there are no bugs?
4. The more bugs you find ,the more bugs there are
May be programmers sometimes have bad days
If you try to fix one bug,there may be chances of gettings some other bugs
Usually programmers make some mistakes in overflow that may lead to new bugs.
We will take care of these things when you are testing a software.
 5. Not all bugs found will be fixed
There is not enough time
-too few people to code and test
-not enough room to left them
-must have software in time
It is really not a bug
It is common that we misunderstood that even if there is no error
If we change the user requirements it may results in test errors
It is too risk to fix
As we discussed earlier it's a risk based exercise we can't fix those problems
It's just not worth it
We usually don't consider the bugs that occur infrequently
Bugs that are appeared in infrequently used features
A user can avoid or prevent those bugs.
6. we can't say that when a bug is indeed a bug
If there is some problem in the software, but we can't find the bug in the entire software we
can't say that it’s a bug.
7. The specifications mentioned by the user should never fail
One of the advantage in software field is that software can be easily changed we change the
code if required
But in other domains it may be impossible
Construct a software that is fair and not copied from any of the other previous systems or
softwares.
Competition is high in the software industry
There are so many new technologies that are released everyday in the market.
So that our software should be unique to others to succeed.
 8. Software Testers are Not the Most Popular Members in a Project
Do’s of a tester:
Finding bugs in the software
Finding the bugs correctly and in the less time
Make sure that all bugs can be fixed as early as possible.
Don’ts of a tester:
Don't post the bad news to the developing team earlier
Make sure to double check even though you can't find if you can inform them.
Act like a professional to other team members
Control your enthusiasm if you are a tester
9. Software Testing is Somewhat Disciplined and Technical Profession
We can say that it is a disciplined activity because it goes through step by step procedure
If we miss one step we may get the test errors
We can say it is a technical profession because we are using
sophisticated tools
advanced technologies
rewarding career growth
If software is simple and more manageable software testers are untrained,we can't find bugs in the software
If we put millions of money on the team we couldn't get a single test error in it.

Basic definitions
Error:
An error is a misunderstanding or mistake or misconception on the part of the software developer.
Fault:
The result of the error is termed as the fault.
It is an anomaly in the software it causes incorrect behavior of the software
It behaves not according to the specifications
Failures:
We can say that a failure is the inability of the software to work in accordance with the software that
is mentioned in the SRS document.
Failures and defects we sometimes call bugs
Software Testing Principles
Principle-1:
Testing is the process of exercising a software component using a selected set of test cases,
with the intent of
Revealing defects, and
Evaluating quality
Explanation:
This principle supports:
Testing is an execution based test to identify the defects in the software.
The separation of testing from debugging is required since the latter thing we do is to identify
the defects to repair the software.
Principle-2:
When the objective of testing is to detect the defects, then a good test case is one that has a high
probability of revealing undetected defect for now.
Explanation:
Careful designing of the test suites by using efficient and effective strategies results in the
identification of undetected defects.so be careful at the time of designing the test cases.
Principle-3:
Test results should be inspected meticulously
Explanation:
Testers need to evaluate and examine the test results whether we get the same output or not
according to the given testing conditions.
 Principle-4:
A test case must contain the expected result or output.
Explanation:
As we discussed above that the each test case must exhibit some functionality so that we can specify the
expected result and the output
Predicting output is one of the important goals in software Testing.
Principle-5:
Test cases must be developed for valid and invalid input conditions
Explanation:
As we know that for better detection of defects we must need to develop test case for inavalid input
conditions also
Sometimes some test cases in some cases gives the correct execution for the invalid entries if that is the
case our software quality must be decreased.
Our software must work for the valid input conditions.
 Principle-6:
The probability of existing defects in a software component is directly proportional to the number of
defects already detected in that software component.
Explanation:
We can say that a defect in some part of the software component may results in more defects in that
software module or component.
If we test the software rigorously there might be chances of entering new defects.
Principle-7:
Testing should be carried out by a team that is independent of the development group.
Explanation:
This principle is true for psychological as well as practical related reasons.
It is difficult to convince the developer that he/she to convenience that the software created by them is
faulty.
Also they thought in the same way as they developed it so that it would be difficult to detect the defects.
So the developing team must be differ from the testing team.
Principle-8:
Testing must be repeatable and reusable.
Explanation:
If the test case is reusable it would be easy for other developers to understand it and make
changes to it.
And also it reduces the time needed to create the test cases.
Principle-9:
Testing should be planned
Explanation:
Testing is successful when we plan it thoroughly starting from the requirements phase to the
end of the testing phase.
Otherwise we will miss the quality of our software product that we are working on.
Principle-10:
Testing activities should be included in the software development life cycle(SDLC)
Explanation:
As we discussed earlier in the axioms it should be must to include the testing in the entire
software development process
Otherwise it would be difficult to us to know where the errors might come from
May be the errors may come from the requirements phase that passes throughout the code
If it is the case we backtrack to every phase and we try to find errors
This would be difficult for the programmers.
Tester's Role in a Software
Development Organization
The tester’s job is to
 Reveal defects,
 Find weak points,
 Inconsistent behaviour,
 Circumstances where the software does not work as expected.
It is difficult for developers to effectively test their own code. A tester needs very good
programming experience in order to understand how code is constructed, and to know where
and what types of, defects could occur.
A tester should work with the developers to produce high-quality software that meets the
customers’ requirements.
Teams of testers and developers are very common in industry, and projects should have a
correct developer/tester ratio.
The ratio will vary depending on
 Available resources,
 Type of project,
 TMM level.
 Nature of the project
 Project Schedules
A testers also need to work with requirements engineers to make sure that requirements are
testable, and to plan for system and acceptance test.
Testers also need to work with designers to plan for integration and unit test.
Test managers need to cooperate with project managers in order to develop reasonable test
plans, and with upper management to provide input for the development and maintenance of
organizational
◦ Testing standards, Polices, Goals.
 Testers also need to cooperate with software quality assurance staff and software
engineering process group members.
 Testers are a part of the development group. They concentrate on testing. They may be part
of the software quality assurance group. Testers are specialists, their main function is to plan,
execute, record, and analyse tests. They do not debug software. When defects are detected
during testing, software should be returned to the developers.
 The developers locate the defect and repair the code. The developers have a detailed
understanding of the code, and they can perform debugging better.
 Testers need the support of management. Testers ensure that developers release code with
few or no defects, and that marketers can deliver software that satisfies the customers’
requirements, and is reliable, usable, and correct.
 Revealing defects
 Finding weak points
 Inconsistent behavior
 Constructs where the software does not work as expected.
 To be most effective a tester requires extensive programming experience in order to
understand what kind of errors are detected and where likely those detects are defected.
 Testers also need to work with requirements engineers to ensure that requirements are
testable, and plan for system and acceptance tests.
 Testers also work with designers to plan for integration and unit tests. Test Managers will
need to work with project managers in order to develop the test case and to provide inputs
from a very large domain and also to maintain the organizational rules and policies.
 Finally, testers also need to cooperate with software quality assurance staff and software
engineering process group members.
 Their main function is to plan, execute, record and analyze tests.
 They do not debug software
Origins of Defects
Variance from specifications: A defect from the perspective of the builder of the product.
Variance from what is desired: A defect from a user’s (or customer’s) perspective. Testers need
to identify both types of defects. Defects generally fall into one of the following three
categories:
Wrong The specifications have been implemented incorrectly. This defect is a variance from
what the customer/user specified
Missing A specified or wanted requirement is not in the built product. variance from
specification, an indication that the specification was not implemented, or a requirement of the
customer identified during or after the product was built
Extra A requirement incorporated into the product was not specified. This is always a variance
from specifications, but may be an attribute desired by the user of the product. However, it is
considered a defect.
 Defect is a flaw in the system, it has no negative impact until it affects the operational system
 A defect that causes an error in operation or negatively impacts a user/customer is called a
failure
 Defects turn into failures - that damages the organization
 Single defect can cause millions of failures
Why Are Defects Hard to Find?
Not looking: Tests not performed because particular test condition is unknown. Some parts of
system go untested because developers assume software changes don’t affect them.
Looking but not seeing: Developers become so familiar with their system that they overlook
details, which is why independent verification and validation should be used to provide a fresh
viewpoint.
Defects typically found in software systems are the results of the following circumstances:
IT improperly interprets requirements
The users specify the wrong requirements
The requirements are incorrectly recorded
The design specifications are incorrect
The program specifications are incorrect
There are errors in program coding
There are data entry errors
There are testing errors
There are mistakes in error correction
The corrected condition causes another defect
 Defects have negative effects on software users. Software engineers work very hard to
produce high-quality software with a low number of defects.
Education: The software engineer did not have the proper educational background to prepare
the software artifact.
Communication: The software engineer was not informed about something by a colleague.
Oversight: The software engineer omitted to do something.
Transcription: The software engineer knows what to do, but makes a mistake in doing it.
Process: The process used by the software engineer misdirected his/her actions.
The impact of defect on the user ranges from a minor inconvenience to rendering the software
unfit for use. Testers have to discover these defects before the software is in operation. The
results of the tests are analysed to determine whether the software has behaved correctly.
A tester develops hypotheses about possible defects. Test cases are then designed based on the
hypotheses. The hypotheses are used to,
 Design test cases.
 Design test procedures.
 Assemble test sets.
 Select the testing levels suitable for the tests.
 Evaluate the results of the tests.
Cost of defects
The cost of defects can be measured by the impact of the defects and when we find them.
Earlier the defect is found lesser is the cost of defect.
For example if error is found in the requirement specifications during requirements gathering
and analysis, then it is somewhat cheap to fix it. The correction to the requirement
specification can be done and then it can be re-issued.
In the same way when defect or error is found in the design during design review then the
design can be corrected and it can be re-issued.
But if the error is not caught in the specifications and is not found till the user acceptance then
the cost to fix those errors or defects will be way too expensive.
If the error is made and the consequent defect is detected in the requirements phase then it is
relatively cheap to fix it.
 Similarly if a requirement specification error is made and the consequent defect is found in
the design phase then the design can be corrected and reissued with relatively little expense.
 The same applies for construction phase. If however, a defect is introduced in the
requirement specification and it is not detected until acceptance testing or even once the
system has been implemented then it will be much more expensive to fix.
 This is because rework will be needed in the specification and design before changes can be
made in construction; because one defect in the requirements may well propagate into
several places in the design and code; and because all the testing work done-to that point will
need to be repeated in order to reach the confidence level in the software that we require.
 It is quite often the case that defects detected at a very late stage, depending on how serious
they are, are not corrected because the cost of doing so is too expensive.
Defect Classes
Important-adapt a single classification scheme & apply it to all projects
Some defects will fit into more than one class or category developers, testers & SQA - try to be
consistent when recording defect data
Defect types & frequency of occurrence → guide test planning & test design
Execution based testing strategies - strongest possibilities of detecting particular types of
defect
Tests for new & modified software-designed to detect the most frequently occurring defects
Defect classes focusing execution based testing:
 Requirement/specification defect class
 Design defect class
 Coding defect class
 Testing defect class
 1. Requirement/specification Defect classes:.
Defects injected in early phase persist & very tough to remove at later phases
Requirement document
- Written in natural language with specification
- Occurrence of ambiguous, contradictions, unclear, redundant &imprecise requirements Specification
document – similar
- Format specification language with tool
- To prevent incorrect description of system behaviour
Functional Description defects-
•Overall description of what the system does & how the system should behave is incorrect, ambiguous
& incomplete
Feature defects
•Feature descriptions-missing, incorrect, incomplete superfluous
•Distinguishing character of a software component/system
Feature Interaction defects
-Incorrect description of how the features should interact
-Explain the example
Interface description defects
-Defects that occur in the description of how the target software is to interface with external hardware,
software & users
Blackbox based testing-at unit, integration, system & acceptance levels
Feature interaction & Interface description defects-detected using BB test at integration & system level tests
2. Design defects:
• Occur when system components, interaction between component to component, component to outside
software/hardware are incorrectly designed
• Covers defects in design of algorithm, control, logic, data elements, module interface description
• Detailed design description - at pseudo level with processing steps, data structures, input output
parameters & control structures
 Algorithmic & Processing defects:
- Processing steps – incorrect
- Calculation - not in order-missing/duplicated Algorithm reuse-chosen an inappropriate algorithm
Control, logic & sequence defects
- Control defects occur when logic flow – incorrect
-Branching-soon, late, incorrect
-Unreachable psuedocode, improper nesting, improper procedure calls
-Incorrect use of logic operators <, >, Boolean expression in branching operation
Data defects
-Incorrect design of data structures
- Record - lacking a field, incorrect type for variable in record
- Array-not have proper number of elements assigned
-Reviews/data dictionary - reveal these defects
Module Interface description defects
- Incorrect/inconsistent parameter types
- Incorrect number of parameters/ ordering of parameters
Functional description defects
- Design may not properly describe the functionality of a module
- Incorrect/missing/unclear design elements
- Detected during design review
External Interface description defects
- Incorrect design descriptions for interfaces external software, database & hardware devices
- Improper sequence of commands, lack of proper messages & feedback messages
3. Coding defects
• From errors in implementing the code
• Closely related to design defect classes
• Not understanding programming constructs
• Transcription/omission
 Algorithmic & Processing defects
- Unchecked overflow, underflow conditions
- Comparing inappropriate data types
- Conversion of datatype
- Precedence of operators
- Misuse of parentheses
- Incorrect use of signs
Control, logic and sequence defects
Case statements, iteration of loops, missing paths
Typographical defects-syntax error
Initialization defects - omission/incorrect initialization statements
Dataflow defects
Variable should be initialized before its use
Don't initialize twice before intermediate use
Data defects
- Incorrect implementation of data structures
-record, array, flags, indices & constant sets
Module Interface defects - similar to design defects
Code documentation defects
-Code documentation – doesn't reflect what the program actually does Incomplete/
ambiguous/incorrect/out of date
External hardware, software interface defects
-System calls, links to database, input output sequences, memeory usage, resource usage,
interrupts, protocols & interfaces with build files
•Design & coding defects- whitebox testing-dataflow, control, logic - Blackbox testing - decision
tables, Boolean expression
4. Testing defects
• Defects in test plan, test cases, test harnesses, procedures
Test Hareness defect
- Test hareness/Scaffolding code - auxiliary code developed to test the software at the unit and
integration levels
- - Should be carefully designed & tested – reuse
- - Subject to all other types of defects
Test case design/ Test procedure defects
- Incorrect/incomplete, missing, inappropriate test cases & procedures
 Defects life cycle
 Defect life cycle is a cycle which a defect goes through during its lifetime. It starts when defect is found
and ends when a defect is closed, after ensuring it’s not reproduced. Defect life cycle is related to the
bug found during testing.
 The defect life cycle can vary from organization to organization and also from project to project based
on several factors like organization policy, software development model used (like Agile, Iterative),
project timelines, team structure etc.
New: When a defect is logged and posted for the first time. It’s state is given as new.
Assigned: After the tester has posted the bug, the lead of the tester approves that the bug is genuine
and he assigns the bug to corresponding developer and the developer team. It’s state given as assigned.
Open: At this state the developer has started analyzing and working on the defect fix.
Fixed: When developer makes necessary code changes and verifies the changes then he/she can make
bug status as ‘Fixed’ and the bug is passed to testing team.
Pending retest: After fixing the defect the developer has given that particular code for retesting to the
tester. Here the testing is pending on the testers end. Hence its status is pending retest.
Retest: At this stage the tester do the retesting of the changed code which developer has given to him to
check whether the defect got fixed or not.
Verified: The tester tests the bug again after it got fixed by the developer. If the bug is not present in the
software, he approves that the bug is fixed and changes the status to “verified”.
Reopen: If the bug still exists even after the bug is fixed by the developer, the tester changes the status
to “reopened”. The bug goes through the life cycle once again.
Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in
the software, he changes the status of the bug to “closed”. This state means that the bug is fixed, tested
and approved.
Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one
bug status is changed to “duplicate“.
Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state
of the bug is changed to “rejected”.
Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next
releases. The reasons for changing the bug to this state have many factors. Some of them
are priority of the bug may be low, lack of time for the release or the bug may not have major
effect on the software.
Not a bug: The state given as “Not a bug” if there is no change in the functionality of the
application. For an example: If customer asks for some change in the look and feel of the
application like change of colour of some text then it is not a bug but just some change in the
look of the application.
 Using Bugzilla for logging and tracing
defects
 Bugzilla is a bug tracking tool that helps to track the
issues related to their product.
 Bugzilla tool is written in Perl language, and it uses
MySQL database.
 It is a bug tracking tool. However, it can also be used
as a test management tool because it can be linked
with other Test case management tools such as
Quality Center, Testlink, etc.
Advanced Search capabilities
Bugzilla has two forms of search:
It has Google-like bug search which is simple to use, and it also searches the full text of a bug.
It provides you a very advanced search system where you can create any type of search that
you want such as time-based searches
Email Notifications controlled by user preferences
Bug lists in multiple formats
Scheduled reports by email
Automatic Duplicate Bug Detection
File/Modify Bugs by email
Time Tracking
Request System
ove Bugs Between Installations
"Watch" Other Users
Reports and Charts
Components of Bugzilla
1) New Bug
2) Search - Simple search & Advanced search
3) Preferences- a) Account Information, b) General Preferences, c) Email Preferences, d) Saved
searches, e) Permissions
4) Reports – a) Graphical reports b) Tabular reports c) Duplicates
https://www.javatpoint.com/bugzilla#Components
THANK
YOU…!