Test Notes

I don't make software; I make it better
posts - 78 , comments - 60 , trackbacks - 616

Bug Reporting Best Practices

Here is the excellent post written by Marie Hagman.  This is one of the post where you can find comprehensive details about “how to file a good bug report”.  Update the post a bit to make more generic. 

Which Bugs Get fixed?

To make the most impact on the product you want to spend the most time on the issues that are most likely to be fixed.

 

Bugs that get fixed:

  • Affect many customers
  • Are embarrassing or do not convey professionalism. Ex: fit and finish bugs
  • Are easy to fix

Bugs that less likely to be fixed:

It would be great to fix every bug in the product, but it’s also great to ship J. In prioritizing which issues to fix, here are the some factors that cause certain bugs to miss the triage bar. Bugs that are least likely to be addressed:

  • Are not reproducible or very hard to reproduce. This may be because the problem occurs intermittently or there are not enough details in the bug report
  • Have strange and complex steps to introduce failure
  • Have no perceived customer impact
  • Are edge cases
  • Risk introducing greater instability through the bug fix then the bug itself causes. Especially late in the product cycle, the bar for these types of bugs is very high because the bugs that may be introduced by the fix we will likely not have time to fix.

Found a bug? Dig deeper…

To create the highest quality bug report which will save developers time and increase the likelihood the bug is fixed, a little extra work goes a long way:

  • Try to find a more general repro
  • Are all of the steps required?
  • Is the bug tied to a single setting or is there a small range of cases? Identify the range.
  • Is the bug more serious than it appears?
  • What other user actions cause the error to occur?
  • Does the error occur with different settings or options selected?
  • Look for follow-up or related errors

How to file a good bug

Here are some tips on how to create a great bug report that will enable your customer to action on it quickly. Often bug reports submitted lack key peices of information so triage teams send bugs back to customers to request more information. We don’t want to mistakenly resolve and issue as By Design, Won’t Fixed or Postponed, so providing a complete bug report helps ensure the best response from triages. We want to spend as much time as possible fixing customer bugs – to that end, high quality bug reports help us spend less time deciphering the issue and more time resolving it.

To file a great bug report:

  • Use clear and minimal repro steps
  • Include only one issue per report.  If you have multiple issues together, file separate bugs, even if they are related.  This allows us to deal with them separate.  Even if you encountered them in the same action, they need to be treated seperately internally.  
  • If possible, try variations to see if you can better nail down the issue.  Do any of the Tools Options settings affect what happens with the bug?  Example: internal vs. external help.
  • Example: In the Help system, help can be internal or external.  Does it repro with both?
  • Avoid jargon or terms that may be difficult to understand.
  • Make sure other customers will easily understand the bug

Bug Titles

Bug titles should completely and succinctly describe the issue. Internally, we spend a lot of time querying the bug database and looking at lists of bug titles. It saves time when we don’t need to open the whole bug report to know what the problem is. 

Here are some tips on writing bug titles:

  • Fully capture the essence of the bug.  We see a lot of partial titles that require more information.  Example: “Autosave function” is not a good enough title.  What about autosave?  When the bug comes in, we put it in a bucket.  The title should provide enough information to differentiate it from other bugs in the same bucket.
  • Think about it from the perspective of other customers trying to find your bug.
  • You can start by considering the keywords that you used to search for dupes before filing your bug.
  • Include the specific name of the components involved in your bug in the title.  Toolwindow, dialog, designerExamples of Bug Titles
  • Most of the improved titles are longer, but not overly long. Don’t include unnecessary commentary either.

 

Original Title

Improved Title

 

“Search selected text”

“Find and Replace dialog should default to last-searched term”

 

“Control Tab Files”

“Had to press Ctrl-Tab twice for toolwindow selection dialog to raise”

 

“Change filename in Solution Explorer”

“Changing case of a filename in Solution Explorer doesn’t take effect”

Repro Steps

Steps to reproduct the bug are the most import part of a bug. Investing a little extra time in this area helps developers immensely. When creating repro steps:

  • Always try to recreate the bug before writing the report
  • Walk through the steps of your bug and make sure you didn’t leave out a step.
  • Could someone reproduce the issue without familiarity with the feature?
  • Start your repro steps from empty IDE. Don’t assume that a project is open, or a dialog is open.  Don’t assume that the title has already been read when you write the repro steps.  Often times, if a bug occurs on a dialog or an options pane, the reader of the bug won’t know how to get there.

Repro Steps Example

  1. Choose Help->Search
  2. Search for "integer"
  3. Right click on a result and uncheck "Show Abstract"
  4. Scroll down to the bottom of the list

Result: White space appears, because the scroll region didn't recalculate after hiding abstracts

Expected: After hiding abstracts, scroll region adjusts to match the search result display region

Descriptions

Descriptions should provide all the other useful information about and issue that is not captured in the repro steps some of which may include:

  • Does the bug repro on other hardware or software configurations
  • Are there configuration dependencies?  Install options, Tools Options settings, etc.
  • Is the bug new in this version?

Attachments

Attachments are extremely helpful, especially for fit and finish UI bugs.

  • A picture is worth 1000 words. Screenshots with comments are priceless.
  • A picture doesn’t replace repro steps. The bug should still be captured in text even if a screenshot is attached.

Print | posted on Wednesday, October 6, 2004 12:33 PM | Filed Under [ Software Testing CSTE ]

Feedback

Gravatar

# re: Bug Reporting Best Practices

Sudhakar,

Am a regular tob your blogs. Good work.
Am referring your CSTE compilation link to my blogs too.

Thanks.
10/21/2004 1:08 PM | Devankur
Gravatar

# re: Bug Reporting Best Practices

Devankur,

Thanks Devankur. You blog is so nice and I am checking it almost everyday. btw, my name is 'Siva', not 'Sudhakar'

Thanks,
Siva
10/21/2004 3:39 PM | Siva
Gravatar

# Bug Reporting Best Practices

hey guys.. u r doing really great jib.. keep it up..but it would be a little more helpful if u could provide us with templates...
1/12/2005 3:07 PM | Aamir
Gravatar

# re: Bug Reporting Best Practices

I really like the Process for Reporting the bug i.e step and then Bug description...I also like the way you suggest which type of Bugs needs reporting...Gr8 Remarks
8/5/2005 2:31 PM | Sachin
Gravatar

# re: Bug Reporting Best Practices

I want to apply for CSTE exam. But i don't have any notes. Can you please send me those notes on skaustubh22@gmail.com
4/10/2007 1:51 PM | Kaustubh
Gravatar

# re: Bug Reporting Best Practices

Hi,

I want to give a presentation on any topic related to Software testing. Kindly suggest me a good topic on which i can give a small but good presentation. Also give me the documents in case if u have any related to that topic or any good site on which i can search for it.
Thanks
Ujwala
6/4/2007 5:41 AM | Ujwala
Gravatar

# re: Bug Reporting Best Practices

Hi,

This article is really good.

I would like to add a point in this,
A picture is worth 1000 words but there are times where explaining about a bug becomes difficult through screenshots in that case I think attaching a video file of the bug becomes useful. We can zip (using WinZip or WinRar softwares) the video file and attach it in the bug to decrease the size of the attachment.

Thanks & Regards,
Smita Singh
1/27/2008 6:44 PM | Smita Singh
Post A Comment
Title:
Name:
Email:
Comment:
Verification:
 

Powered by: