Chapter 3: How to classify your bug report?
Any brilliant QA superhero should be able to classify the detected bugs into the right categories of report. This piece of information is key to help the product manager or the development team understand how serious the bug is. This information will also help them choose what to read first and which bugs need to be prioritized based on its severity, reproducibility and type.
In this article, we will try to clarify as much as possible the meaning of each category of severity, reproducibility and type of a bug.
Severity
Severity is the level of impact the bug has on the product.
There is something subjective about severity: when you choosing a level, always think about the product first. How much will the bug affect the core purpose of the product?
A bug can be a major bug on a A-product but a minor bug on a B-product.
On WE ARE TESTERS, we consider 4 levels of severity:
Block
Definition:
The bug prevents the user from doing the intended action. He can’t go further. He’s stopped. In 99% of the cases, a bug with “block” severity is also a functional bug (see “type of bug” classification below).
Examples:
- the link is not working
- the page is not reachable
- impossible to change the password
- the form can’t be submitted
- a valid information was not saved after submitting the form
- the order is not confirmed and the user is redirected to the home page
- the application crashes every time an action is done
- upon an action, the page/app freezes
- impossible to install the mobile app
- the reset password is not received
- payment for an order is refused
- the item can’t be added to cart
…
Major
Definition:
The bug is highly disturbing for the user but doesn’t prevent him from doing the action.
Examples:
- some HTML code is visible on the page
- an information pop-up doesn’t contain text
- the page needs 10 sec to load
- an ad covers the page content but can be closed
- the image of the product is stretched
- the page is displayed as plain HTML text
- sound is missing (video platform)
- social shares are not working
- the field is not marked as mandatory (but appears to be in fact)
- a text field is displayed twice
- there is neither a title nor a placeholder to indicate what information should be provided in the field
- a part of the website is in another language
- wrong IP localisation (identified in China while navigating from Spain)
…
Minor
Definition:
The bug has low impact on the user browsing the website or the mobile application. However it is important to report these bugs. It is up to the product manager to decide whether to prioritize them.
The bug-fixing usually has a low direct ROI (Return On Investment) but in some cases it is essential to do it for the brand image.
Examples:
- spelling mistakes
- blurred image / pictogram
- non-aligned items
- font types with no logic through the pages
- an anchor leads to the wrong part
- mouseover on a link doesn’t highlight it
- font differs from pages
- branding issues (colors, backgrounds,…)
- wrong description of an item (eshop)
- untranslated content (minor value content)
Suggestion
Definition:
A suggestion is not a bug. Which means the product works faultlessly and in accordance with the product manager’s expectations. BUT you perceive it as confusing and/or you think it does not conform the web standards.
Ok now let’s be honest: when the next release is at stake, product managers are rarely interested in this information. Don’t be disappointed. It’s just not the best moment for it. However, when the release is done, your feedback becomes way more valuable to them. So keep suggesting improvements that could make the product more user-friendly!
Examples:
- provide dedicated and precise error messages in each situation
- indicate number of reviews
- filters : change category “N stars and more” by just “ N stars” and allow to select more than one value at once.
- use a standard Captcha system
- increase font size on a mobile app to help the user click precisely on a link (mobile app)
- allow to go to previous/next page by swiping right/left
Types of bugs
The type of a bug is the nature of the bug.
This categorization is objective: a bug will always have the same nature for whatever product. Nevertheless, some bugs are tricky and you may hesitate between 2 categories.
We consider 5 types of bugs and we will provide a maximum of examples below to help you recognize them in the future.
Functional bug
Definition:
A functional bug is a dynamic bug related to an action your are doing. You can only find it while doing an action on the product. The product’s reaction is not as expected.
Examples:
- scrolling is not possible on a page although there is content below
- horizontal scrolling is possible but in this case not relevant
- a website leads to its staging version
- a button is not active
- the expected email never arrives
- the alphabetic keyboard is opened by default instead of the numeric one
- login is not possible
- a button redirects to a 404 page error
- the loading bar never ends
- the user is sent back to the homepage while trying to access another page
- the mobile app doesn’t work when the phone is on flight mode
- the tap is inactive
- the sound of a video is missing
- the pop-up can’t be closed
- the page freezes
- the autocompletion feature provides partial information
- an error message with this type of text: “an error has occurred, please try again later” but not with this: “please enter a valid email address” unless it’s a misleading message.
- a missing error message like “please enter a valid email address”.
- the zoom in or zoom out is not working
- inactive return button (on the page)
- inactive geo-localization
- wrong geo-localization
- impossible to register
- inactive footer links
- inactive filters
- items not matching the filters
- wrong anchors
- non-functional social and email shares
- outdated information (stock for example, delivery date and time)
- impossible to write in a text field
- static jumbotron
- lack of translation
Graphical bug
Definition:
A graphical bug is a static bug related to layout and UI (User Interface) issues.
Examples:
- blurred image
- misalignments
- missing or huge or inconsistent margins between elements of the page
- fields getting out of the screen resolution
- abnormal button display based on brand guidelines
- stretched pictures or videos
- overlaps (text, image, buttons, links, advert, etc.)
- unexpected colors / font type / font size based on brand guidelines and mockups.
- missing favicon
- field label is cut
Wording bug
Definition:
A wording bug is related to the text content of the website or the mobile app. On many websites/app, the product manager has access to edit the content himself thanks to a CMS (Content Management System). In these cases, it will be very easy to fix the bugs and it will be released instantly.
Examples:
- bad translation
- different texts between mockup and live product
- special characters not encoded
- missing accents
- spelling mistakes
- grammatical mistakes
- absurd sentences
- misleading text
- missing words
- uppercase and lowercase inconsistencies
- unusual titles for cart, checkout process, buttons
Ergonomics
Definition:
An ergonomic issue is related to the user experience. Although some may be bugs, the majority of the issues will probably be suggestions.
Examples:
- too many clicks necessary to see a product detail page
- logo is not where logo are commonly located on the page
- navigation bar is too big and covers half of the page above the fold.
- font size is too small
- font and background colors are not contrasted enough
- the website is not responsive when browsing on mobile or tablet.
- the buttons are placed in unusual location
- clicking on a link should open the information on a new tab (on a particular case).
- mandatory fields should be indicated on the form
- loading time should be designed so the user knows he has to wait (on a particular case).
- no brand guidelines seem to be implemented
- popins and popups should contain a closing pictogram.
- irreversible action should be preceded by a message “do you confirm you want to DO THIS ACTION?”
- successful actions should be followed by a message “action confirmed”.
- registration should be blocked for users selecting a date of birth younger than XX years old.
- Green, orange and red are commonly used for success, alerts and failure, colors are mixed when providing a transactional message.
- textfield is too small / too large
- Radio buttons should be implemented instead of multiple choice buttons
Performance bug
Definition:
A performance bug is an issue related to the technical environment that damages the perceived quality of the product.
Examples:
- long loading time on all pages
- the page is displayed bit by bit
- loading page /download was interrupted
- page could not appear as maximum loading time was up
- the app crashes when opening
- the app crashes randomly
- the app crashes every time an action is done
- video streaming is in very bad quality
- video is buffering
- some pictures are not loading
Reproducibility
Reproducibility refers to the frequency the bug can be repeated.
Always:
A large majority of the bugs you’ll discover can be reproduced again and again and again (until someone fixes it). Which is good! The steps to reproduce are probably easy to identify and to write. The development team will understand the origin of the bug easily or at least they will know where to start digging.
Randomly:
These bugs are the most challenging ones and they can either drive you crazy or excite you. They appear only sometimes meaning that the conditions to reproduce the bugs are not yet identified. So if you want to unravel the mystery you should be patient and investigate step by step by changing one thing at the time until you finally get it and feel like a superhero.
Once:
It occurred only once. Either it’s a real bug but the conditions to reproduce it are not known yet or it’s a transitory bug. It can be interesting to report it for the record but the product manager will not pay very much attention to it unless it can be reproduced a second time.
The above definitions and examples are here to help the members of our WE ARE TESTERS community to better classify the bug reports when working on a mission. However, these definitions can apply to QA (Quality Assurance) in general. The values for each category are standard values across the QA market.
< previous chapter: description — —— — next chapter: steps to reproduce>
