The ability to write a good bug report is so much more important than the bug report itself.
More often than not, a shoddy bug report is going to get closed offhand, despite the validity of the bug itself. Most developers don’t have the time to parse through and guess at what the report actually means, and they certainly don’t have the ability to “fix” something if they don’t know what it is they’re fixing. This is especially true if you ever try to open an issue with an open source project — for better or for worse.
Yes, there will be a dev out there who will take the time to figure it out. Conversely, there’s also devs out there who will just try to fix the problem as described without second thought.
Our goal is to eliminate the need for that first option, and the problems that arise with the second.
You want to write something that is clear and concise. It describes only what the problem is, with no frivolous details, and only what the true, expected outcome is. This ensures correctness when it comes to developing, testing, and documentation later.
As always with advice, I’m actually just talking about myself. But all of these things are steps I make sure I follow each and every time, and that I find immeasurably helpful.
Isolate the Bug
Before clicking that hot New Ticket button, make sure you know about the issue you’re going to file. Knowing “when I open a large file it’s slow” isn’t necessarily enough. Approach the problem like it was another person coming to you. Ask yourself questions to narrow the scope of the issue. Isolate it, so that you can file the ticket correctly.
One Bug Per Ticket
Only report one bug per ticket. If there are multiple related bugs surrounding the same feature, then break every single one into their own ticket.
This prevents anyone looking at that ticket from overlooking one of the reported issues. It also helps later on when you search through your ticket system, when you create release notes, when you plan sprints, and more.
So one bug per ticket.
Define the Bug – Ticket Summary
How would you describe this bug in roughly ten words? This is the title or summary of the ticket.
The ticket summary is referenced everywhere. When you search through your backlog, when others look through the ticketing system when they have a related issue, when your future self can’t remember what was done to resolve the issue.
Good documentation is sorely lacking everywhere, and the only way to fix this is to do it yourself. It starts with writing good tickets.
It’s important that the ticket summary is concise and states exactly what the problem is. Do your best not to be vague.
Here are a couple of real-world ticket summaries (with specifying data scrubbed) as examples of the difference between a “good” ticket summary and a “bad” ticket summary.
Good Summary: Validation needed in $APPNAME to check IIS before stating an Instance is operational
Bad Summary: Manager/ IIS validation for an instance
Good Summary: Add UI validation to $PAGENAME to prevent duplicate rolling limits from being configured
Bad Summary: App:PK: Enforce a unique limit
It’s easy to go overboard when you’re trying to be brief, and this can sometimes be tricky. I think it’s always worth the time to do it right, however.
Provide a Test Plan
Test plans are essential information in bug reports. Without a test plan, something that isn’t broken might become “fixed” and make it through an entire development cycle — then the customer will receive a “fixed” version with the original bug intact.
Having a test plan also proves the issue you’re reporting is actually a bug and can be reproduced willingly. This is required for both dev and QA to effectively do their jobs.
Test plans should be brief and clear — just like the summary. Include only the steps required to reproduce the bug, and nothing else. Sometimes you may have to run through your steps more than once to eliminate unnecessary actions.
Make sure that, after the summary, the test plan is the very next information you fill you.
Steps to writing a good test plan:
- Indicate whether you can reproduce the bug at will, occasionally, or not at all*
- Include the version number of the software, if applicable, and any pertinent environmental variables (operating system, web browser, etc.)
- The exact steps to reproduce the bug, and only the necessary actions
- The Actual Result (the behavior you observed) and the Expected Result (how you think it should behave)
Typically I’d say if you can never repro the bug, don’t file the ticket. Realistically, there are times where you can’t get eyes on your issue without a ticket.
Write a draft up and get someone to help you, or try again to repro the issue from another angle. Include all the information you can — still list out all the steps you took, or all the steps reported by the customer, and include as much other pertinent data as you can.
- When writing out steps in the test plan, try to list the actions in the present tense. Pretend you’re instructing someone while standing over their shoulder. This is more a matter of personal preference and style. When I’m reading a test plan it helps to have fewer things for my mind to decode (past tense -> present tense, for example) so I can just focus on the issue at hand.
- Limit conjecture. Only include it after all of the facts have been stated.
- Be precise! If there’s more than one way to do something, be sure you’ve specified exactly how you did it. For example, using Ctrl+S or clicking File > Save to save a file. Sometimes the difference does matter.
- If you performed additional exploration after discovering a bug, report what you did — but not at the expense of the facts about the bug itself.
- Document in the ticket if any promises were made to a customer regarding the bug in the ticket.
Remember: without a test plan, anyone looking at this ticket can’t know if they’re actually reproducing the bug.
Here is an example of a good test plan next to a bad test plan. As above, this was taken from a real-world scenario, with identifiable info scrubbed.
Good Test Plan:
- Process the attached file through [$SYSTEMNAME]
- Once processed, open the File Details page for this file
- Click Menu > Review File Full Contents
- Click on a row inside the file and use the arrow keys to navigate through the rows at least twelve times
Actual Result / What Happens: After navigating back and forth about twelve times, the application will stop responding for 30 seconds, and then a SQL timeout exception occurs
Expected Result / What Should Happen: The user should be able to navigate through the file’s full contents at will without running into a SQL exception
Bad Test Plan:
Load a large file and scroll through its items on the file details page
Sometimes you need more space to describe the issue than the Summary allows. In these cases, think of the Summary as a Title, and the Description as the Body. Flesh out your explanation — maybe this ticket is asserting that there is a design flaw (yes — these can be bugs, too), and you’re including business use cases to explain why.
A design flaw is a bug; don’t let anyone tell you otherwise. They are more insidious and often harder to untangle than bugs. This will be the topic of another post, soon.
Always remember: never copy and paste customer correspondence to use as a Description of a bug report.
Rephrase the customer email in your own words. If you come up with questions while doing so, this is a good sign. Ask the customer these questions and use those details in the ticket.
Take Your Time
“Saving time” while writing a ticket so it just gets out there will invariably lead to extra time being spent on this ticket later. Don’t ever count on your own memory; it’s a liar. And don’t count on someone else being able to parse your words if you didn’t take the time to be clear.
Think of this as an investment. The more time you put into writing a quality ticket, the more you get out of it. You will spend less time explaining yourself later when someone comes to ask you what you meant by so-and-so in the test plan. You will save dev and QA time when they don’t have to figure out a good test plan for a bug. You will save time when it comes to writing release notes. The benefits go on and on — a larger upfront time investment always saves more time down the road.
As with most things, if it’s worth doing, then it’s worth doing right.
Remember that, above all else, we all just want things to get better. We’re all on the same team. Remember there’s another human on the other side of this screen.