GDC VN – WPLC Team

Writing Effective Defect Report

Thang Ho – WPLC Team

© 2008 IBM Corporation

 GDC VN – WPLC Team

Course Prerequisite

 N/A

2 © 2008 IBM Corporation

 GDC VN – WPLC Team

Course Timetable

 Course presentation (3.0 hours)

3 © 2008 IBM Corporation

 GDC VN – WPLC Team

Agenda

 Some situations need to consider

 Objectives
 Characteristics of Defect Report
 Tips for creating a report
Q&A

4 © 2008 IBM Corporation

 GDC VN – WPLC Team

Situations
 Someone said that: “Defect writing is an effortless and
elementary job and one just needs to write whatever is
being observed and convey it in some form. Is is correct?”
The answer is NO. You will know the reason why after
going through out the course.

 Tester’s responsibility is not only to find out and report

bugs, but also to manage and handle bugs efficiently until
those are closed.
 Whenever creating a report of defects, tester should be careful
and conscientious, as it is need formal, accurate and organized
writing skills.

5 © 2008 IBM Corporation

 GDC VN – WPLC Team

… Situations

 Every individual has his own style of writing, so the

statements to express a defect may vary ways when two
persons report the same defect.
 Defect report is written manifestation of a defect and expected
not only to be viewed by the developer, but also, in many cases
may be looked at by someone at the higher level or may be a
review committee.
 Erroneous way of writing a defect report creates
misinterpretation and result in unnecessary delay in the fixing
process.

6 © 2008 IBM Corporation

 GDC VN – WPLC Team

Objectives
 This course is designed for testers who will
fact to write defect report during their
executing tests.
 After this course, You will know:
 Characteristics of defect report
 How to create an effective defect report

7 © 2008 IBM Corporation

 GDC VN – WPLC Team

Characteristics of Defect Report

8 © 2008 IBM Corporation

8
 GDC VN – WPLC Team

Characteristics of Defect Report

 Condense
 Accurate
 Neutralize

 Precise
 Isolate
 Generalize
 Recreate
 Impact
 Debug
 Evidence
9 © 2008 IBM Corporation
 GDC VN – WPLC Team

Condense

 You should re-read defect report at least one time,

focusing on eliminating extraneous steps or words.
 Cryptic or vague commentary is not the goal, but neither
should the report wear out its welcome by droning on
endlessly about irrelevant details nor steps which one
need not perform to repeat the failure.

Describe defect Clearly but Briefly

10 © 2008 IBM Corporation

 GDC VN – WPLC Team

Condense - Example
 I was setting up a test whose real intent was to detect memory errors. In the
process I noticed a new GUI field that I was not familiar with. I decided to exercise
the new field. I tried many boundary and error conditions that worked just fine.
Finally, I cleared the field of any data and attempted to advance to the next
screen, then the program halted. Several retries revealed that anytime there is not
any data for the "product description" field you cannot advance to the next screen
or even exit or cancel without halting.

Too much information, most of which is not helpful.

 The "exit", "next", and "cancel" functions for the "Product

Information" screen didn’t work correctly when the "product
description" field is empty or blank

Clearly, Have Enough and Briefly

11 © 2008 IBM Corporation

 GDC VN – WPLC Team

Accurate
–Is it really a defect? Could it be user error, setup problem
etc.?
–Before writing up the problem consider
• Is there something in the setup that could have caused this?
• Could an incomplete cleanup, incomplete results, or manual
interventions from a previous test cause this?
• Could this be the result of a network or some other
environmental problem?
• Do you really understand how this is supposed to work?

Checking all related things may cause the defect!

12 © 2008 IBM Corporation

 GDC VN – WPLC Team

Accurate - Example
 Situation: There is a build and it was installed successful
–On side of test, the server is Windows NT server 4.0, the
program cannot work well – some functions like search, Add
new record, etc did not work correctly. (Wrong results after
searching, Lost data when saving …)

On side of developing, the server is Windows 2000 server,

the program does not fact to issue like this.
Are there are different between Windows NT server and
Windows 2000 Server?
After changing the server on Test side to Windows 2000
server, the issue is still there.
 In fact, there is missing a step in installation process,
MDAC 2.7 must be installed.
13 © 2008 IBM Corporation
 GDC VN – WPLC Team

Neutralize
 It is a well-known fact that software testers carry bad news about
the software and therefore they sometimes face unpleasant
reactions.
 Don’t attacking developers, criticizing the underlying error,
attempting humor, or using sarcasm can create ill with
developers and divert attention from the bigger goal: increasing
the quality of the product.
 The cautious tester confines defect reports to statements of fact.

Report - Being a technical document, no Humor, no Sarcasm

14 © 2008 IBM Corporation

 GDC VN – WPLC Team

Neutralize - Example
 I logged to system as Administrator. I went to Manage User screen. Although
there is no User, but the “Delete”, “Details” buttons are still enabled. I have
never seen any screens like this. I don’t understand a reason why someone
can implement a screen like this and I hope I never see any funny defect like
this again!
 I tried to click on them, nothing happen for “Details” button, but for “Delete”
button, I got run error message and the program was terminated. An expert
programmer can prevent any defect like this soon.

You are not Tester, maybe you are a boss!

But even the boss does not criticism the member

 Log into system as Administrator. Go to Manage User screen. Although there

is no User, but the “Delete”, “Details” buttons are still enabled.
 Try to click on them, nothing happen for “Details” button, but for “Delete”
button, got run-time error message and the program was terminated.

Just describe the facts

15 © 2008 IBM Corporation

 GDC VN – WPLC Team

Precise

 The tester should go through the report to make

sure it is not become to misinterpretation.
 The tester must avoid words or phrases that are
vague, misleading, or subjective.

Goal should be clear, indisputable statements of fact.

16 © 2008 IBM Corporation

 GDC VN – WPLC Team

Precise - Example
 Issuing a cancel print when job is in PRT state (job is already in the printer and
server is waiting to receive print complete from printer) causes the port to not
time out. The printer never returns to a READY state and indefinitely displays
"PRINTING IPDS FROM TRAY1" in the op-panel.

It is hard to tell if the problem is

(1) the port not timing out or
(2) the printer not returning to ready or
(3) the message on the op panel

 Canceling a job while it is printing causes the printer to hang.

A short summary of exactly

what you perceive the problem to be.

17 © 2008 IBM Corporation

 GDC VN – WPLC Team

Isolate
 After trying to reproduce the failure, the tester should then
proceed to isolate the defect. It will help developers a head
start on debugging.
 Consider the following when isolating problems
–Try to find the shortest, simplest set of the steps required to reproduce the failure
–Ask yourself if anything external to the specific code being tested contributed to
the problem
–If your test has multiple input conditions, changing certain variables, such as
system configuration, boundary of values or something like that, that may alter
the symptom of the failure.

Isolate defect to save time for debugging

18 © 2008 IBM Corporation

 GDC VN – WPLC Team

Isolate - Example
 Using Windows NT, Launch the application, Open some screens such as Users,
Facilities, Event Rooms, … Then Close all of them. Click on “X” button at the top
right of screen to close the program. It seems the program was not really
terminated. I had to kill the program by opening Task Manager and select “End
Process” function.

How about if user did not open any screens?

And how about with other platform?

 In fact the issue only relates to Close function of the program. If user just launch
the program then Close it right away, the same error.
 The issue is only occurred with Windows NT platform. The program was
terminated well with Windows XP, or Windows 2000 Pro, Windows 98.

Defect isolation needs a deep thought and understanding

about the application that you are testing.

19 © 2008 IBM Corporation

 GDC VN – WPLC Team

Generalize

 After the tester has an isolated and reproducible

failure, he/she should try to generalize the problem.
 Ask yourself does the same failure occur in other
modules or locations? Or Are there more severe
occurrences of the same failure?

The same failure will be addressed in other modules

20 © 2008 IBM Corporation

 GDC VN – WPLC Team

Generalize - Example
 I went to User screen, View details of an user, make some changes. Then I
went to other screens. I got message with title “CnB”
 At the User screen, I selected to delete a message, I got message with title
“C&B”
 … I got message with title “Conference & Banqueting Module”
 Please make sure that We just using one Title in this screen, That is
“Conference & Banqueting Module”

This defect was not generalize because maybe

it will occur on other screens

 There are a lot of Title on common message like "CnB", or "C&B System", or
"C&B Module" or "Conference & Banqueting Module". Just use one and only
one public variant for this Title and everyone must follow, Please see
attachment to know more details.

The defect report was generalized

21 © 2008 IBM Corporation

 GDC VN – WPLC Team

Re-create
 Testers should check reproducibility of a defect before writing a
defect report.
 A good rule of thumb is three attempts to repeat the defect before
writing the report.
 Consider:
–Should explain exactly what is required to do the re-create.
• List all the steps, include the exact syntax, file names, sequences, etc.
• Document the shortest, easiest means of re-creation.
• Gather all the relevant information that may provide useful information to the person who has to
try and fix the problem.

Defect is really existing and repeatable

22 © 2008 IBM Corporation

 GDC VN – WPLC Team

Re-create - Example
 Trashed contents of new file that I created by formatting some text in Arial font.

Too short, and maybe developer cannot know

what is the editor which has the defect like this

 Steps to reproduce:
1. Started SpeedyWriter editor, then created new file.
2. Typed four lines of text, repeating “The quick fox jumps over the lazy brown
dog” each time.
3. Highlighted all four lines of text, then pulled down the font menu and
selected Arial.
4. All text became corrupted into control characters, numbers, and random
binary data.

Say exactly step by step what editor has defects

23 © 2008 IBM Corporation

 GDC VN – WPLC Team

Impact

 How impact to customer if the defects were to surface in customer

environment?
 Consider
–Make your best judgment
–Don’t oversell
–Use current impact system in PVCS Tracker (Fatal, Major, Minor, or
Cosmetic)

Have right evaluate about impacting of defect to system

24 © 2008 IBM Corporation

 GDC VN – WPLC Team

Impact – Pre-define list

 Cosmetic: The product is not working as it was designed, but this is a
cosmetic or trivial problem that has little or no business impact on the
customer.
 Minor: Any defect that affects enhanced program features from working as
designed, a work around is available
 Major: Any defect that affects standard program functionality from working as
designed, a work around is available
 Fatal: Any defect that stops standard functionality of program from working
as designed, no work around is available.

25 © 2008 IBM Corporation

 GDC VN – WPLC Team

Impact - Example
 I logged to system as Administrator. I went to Manage User screen. There are
some users, I selected one and tried to click on “Delete” button. I got run-time
error message and the program was terminated.

Missing an evaluate about how this defect impact to

 Log into system as Administrator. Go to Manage User screen. There are some
users, Select one and tried to click on “Delete” button. Got run-time error
message and the program was terminated.
 This defect should be considered a Fatal defect and need to fix as soon as
possible before the program go to next phrase.

Using the current impact system of our company

26 © 2008 IBM Corporation

 GDC VN – WPLC Team

Debug

 What does the developer need to debug this?

–Are there traces, dumps, logs, and so forth that
should be captured and made available with this
defect report?

Give more defect details to easy find out root cause

27 © 2008 IBM Corporation

 GDC VN – WPLC Team

Debug - Example
 Log into system as Administrator. Go to Manage User screen. There are some
users, Select one and tried to click on “Delete” button. Get run-time error
message and the program was terminated.
 This defect should be considered a Major defect and need to fix as soon as
possible before the program go to next phrase.

If any, tester should show details of runtime error as below

This will help developer

in debugging and has
solution to fix

28 © 2008 IBM Corporation

 GDC VN – WPLC Team

Evidence
 What will prove the existence of the error? Documentation or
images?
 Consider
– Provided both the expected results and the actual results? Is there documentation that
supports that expected results?
– Don’t assume everyone sees things the same way you do.
– Don’t expect people to read between the lines and draw the same conclusions as you.
– Don’t assume that 3 weeks from now you will remember why you thought this was a bug.
– Think about what it is that convinced you that this is a bug and include that in the report.

Using evidence when you think it is necessary

29 © 2008 IBM Corporation

 GDC VN – WPLC Team

Evidence - Example
 I logged to system as Administrator. I created a reminder (Its status is now Not
Completed). When the time of reminder is reached, I got an alert message for
the reminder, Then I went to see the Status of the reminder, It is still “Not
Completed”. It should be changed to “Over due”

Who say about this rule? Where is evidence for this?

 Log into system as Administrator. Create a reminder (Its status is now Not
Completed). When the time of reminder is reached, Get an alert message for
the reminder, Then go to see the Status of the reminder, It was still “Not
Completed”. It should be changed to “Over due” – Please refer to Configuration
Reminder Use Case version 1.2, at section 5.2

State clearly about the evidence

30 © 2008 IBM Corporation

 GDC VN – WPLC Team

Summary about Characteristics

 Condense
Describe defect Clearly but Briefly
 Accurate
Checking all related things may cause the defect!
 Neutralize
Being a technical document, no Humor, no Sarcasm
 Precise
Goal should be clear, indisputable statements of fact.
 Isolate
Isolate defect to save time for debugging
 Generalize
The same failure will be addressed in other modules
31 © 2008 IBM Corporation
 GDC VN – WPLC Team

… Summary about Characteristics

 Recreate
 Defect is really existing and repeatable
 Impact
 Have right evaluate about impacting of defect to
system
 Debug
 Give more defect details to easy find out root
cause
 Evidence
 Using evidence when you think it is necessary

32 © 2008 IBM Corporation

 GDC VN – WPLC Team

Writing an effective defect report

33 © 2008 IBM Corporation

33
 GDC VN – WPLC Team

Writing an effective defect report

 At this time, you understand about

characteristics of defect report.
 Now I will show you some tips for writing defect
report

34 © 2008 IBM Corporation

 GDC VN – WPLC Team

Defect report

 What are defect report content?

–Title
–Steps to reproduce defect
–Expected result
–Additional Info
–Comments/Discussion History
–Attachments
–… and some more information depend on project, such as Build
number, Use Case/Req, etc

35 © 2008 IBM Corporation

 GDC VN – WPLC Team

Title

 Each defect needs to have a good one-line title. This is a

kind of skill similar to writing a good attention-grabbing
media add campaign. Can you really make out from the
headline about what it is going to talk about?
 Ideally, a bug title should not carry more than 10 words.
 You can put Title for Each defect at Title in PVCS tracker.

36 © 2008 IBM Corporation

 GDC VN – WPLC Team

Steps to Reproduce
 Are you really providing the exact information how to reproduce a defect?
Now, what is meant be ‘exact information’?
 Most of the time a tester thinks that it will be easy for the developer to
understand whatever he assumes and with this assumption, many
important steps are sometimes skipped.
 Correct terminologies and appropriate naming conventions should be
used while writing the steps. One should always keep it in mind about the
clarity in providing the information and systematically note down the
steps accordingly.
 Ask yourself: can this defect be reproduced by anyone who reads your
defect report?
 You need to state clearly about what is happening while performing the
steps mentioned in the defect report. This has to be extremely specific
and all the results how the software behaves should be mentioned
without missing anything.

37 © 2008 IBM Corporation

 GDC VN – WPLC Team

Expected results

 You need to have a well guided decision as to ‘what is expected’

needs to be taken.
 A tester may take guidance from anyone who has knowledge or
experience in the area you are testing. You can also get useful
information if a previous version of the application you are
testing exists or if there are any competitive industry standard
applications available.

38 © 2008 IBM Corporation

 GDC VN – WPLC Team

Additional Information

 This is important to mention in a defect report if you

have any other relevant observation associated with
the problem observed, which may be of some help to
the developer to establish some correlation with the
actual issue.

39 © 2008 IBM Corporation

 GDC VN – WPLC Team

Comments/Discussion History

 A tester needs to be careful while making his comments as all

these comments or the discussion history is important.
 Imagine a situation where you are asked to do testing for
someone who has left the organization. You have to rely totally
on the discussion history to really understand how to proceed
from that stage. Also, when a fix is ready from development and
now it’s your job to verify and close the issue.

40 © 2008 IBM Corporation

 GDC VN – WPLC Team

Attachments

 Sometimes to have a better understanding of any situations,

apart from simply writing down the error, a test file or a
snapshot should be provided along with the bug report.
 This proves to be very effective as it can be treated as
evidence of existence of a problem. It also saves lot of time
as the developer don’t have to waste his time in recreating
the error, which on the other way can be detected and
observed in the test case provided by the tester.

41 © 2008 IBM Corporation

 GDC VN – WPLC Team

Example - Defect report

 Title: Username was truncated without warning message
 Steps to reproduce:
–At the User screen, Click on Create button to create new one
–Input Value to Username field with string 45 chars long
–Input all data as required and click on Save button
 Expected result:
–Program had to save exactly what I inputted in Username field (String with 45 chars long). In fact, the
username was truncated to 18 chars long without warning message.
–Or at the Username field, there is limitation in length (User can only input maximum 25 chars long as required
in Use case).
–{This defect need to setting as Minor} – Select Minor in Severity field
 Note:
–The same error for other fields like First name, Last name and when Updating user’s details.
–There is only one screen in the system has this defect.

42 © 2008 IBM Corporation

 GDC VN – WPLC Team

Summary
 Report the defect
–Getting the problem fixed with the least amount of effort
–The proper information is provided is more important
than superior writing skills

Credibility and value-add are better

because of your defect reports

43 © 2008 IBM Corporation

 GDC VN – WPLC Team

Questions and Answers

44 © 2008 IBM Corporation