Bug severity guide

What is Severity?

The severity of a bug report reflects the impact of that particular issue on the software under testing. The severity of a bug report can also be defined as the impact the issue has on the user’s ability to interact with the app and its features.

When trying to understand the severity of the issue you encountered, ask yourself: How much of an impact does this issue have on the application and my ability to interact with it? 

Here’s a rundown of the different severities you can select when reporting a bug on the Tester Work platform:

1. Critical

A defect that completely hampers or blocks testing of the product/ feature is a critical defect. An example would be in the case of UI testing where after going through a social media sharing flow, the UI displaying “Your photo was shared successfully” remains present and does not allow the user to close it or progress further in the app. Another example of a critical defect is a situation where a feature that needed testing is missing from the app.

For any reason, if the application crashes or it becomes unusable while going through normal user flows, the defect should be classified under critical severity.

Note: If the user needs to perform very specific actions in order to be able to reproduce the issue, it will not be classified as critical! Critical issues are the ones which are easily reproducible by performing normal user behavior in the app under testing

2. High

Any feature implemented that is not meeting its requirements/use case(s) and behaves differently than expected can be classified under High Severity.

High defects are the issues where an important feature of the app isn’t performing as expected and the user is prevented from using the app as they should be able to.

For example: In the email service provider like Yahoo or Gmail, when you are not allowed to add more than one recipient in the CC section, this defect is classified as a High defect as the major functionality of the application is not working properly. What is the expected behavior of the CC section in the mail, it should allow the user to add multiple users. So when the major functionality of the application is not working properly or when it behaves differently than expected, it is a major defect. 

Also adding to the High severity are some Critical issues that require very specific repro steps and/or have a lower than 100% repro rate. This category could include issues that imply the user performing steps that are out of the ‘usual user behavior’ category: multiple taps at the same time, opening and closing features multiple times, weird interrupts while performing an action, a long list of specific steps that need to be performed in order to reproduce the issue. 

3. Medium

Any feature implemented that is not meeting its requirements/use case(s) and behaves differently than expected but the impact is negligible to some extent or it doesn’t have a major impact on the application, can be classified under Medium severity.

A moderate defect occurs when the product or application doesn’t meet certain criteria or still exhibits some unnatural behavior, however, the functionality as a whole is not impacted.

For example: In the email service provider like Yahoo or Gmail, there is a section called “Terms and Conditions” and inside there will be multiple links regarding the terms and conditions of the website. When one of the multiple links is not redirecting the user anywhere, it is called Medium severity as it doesn’t have a major impact over the application.

This type of defect results in minimal loss of functionality or user experience. 

Also adding to the Medium severity are some High issues that require very specific repro steps, that affect less important features (medium importance features) and/or have a lower than 100% repro rate.

4. Low

Any cosmetic defects including spelling mistakes or alignment issues or font casing can be classified under Low Severity.

A low severity bug occurs when there is almost no impact on the functionality but it is still a valid defect that should be corrected. Examples of this could include spelling mistakes in error messages printed to the user.

For example: In the email service provider like Yahoo or Gmail, You can notice the “License page”; if there are any spelling mistakes or misalignment on the page, this defect is classified as Low.

5. Usability

A usability problem is an aspect of the system and/ or a demand on the user which makes it unpleasant, inefficient, inconvenient or impossible for the user to achieve their goals in typical usage situations.

A Usability problem occurs when all the functionalities of the app work but the intended user story can have a more fluid flow if some things would change. Examples of this could include having more specific information when an error message is triggered, changing the background color in order for the app/web site to have a more appealing design or rearranging the options from a menu so that the user can easily select the more common options.

A Usability issue is a defect in the experience the user has while interacting with the app without affecting the user’s ability to use the features of the app.

For example: In the email service provider like Yahoo or Gmail, you can notice that for selecting all emails you have to perform 2-3 actions as opposed to a single action. The user experience would be better if you only had to perform one action instead of 2-3.

Please note that most of the testing projects inside Tester Work do not support Usability testing. The details provided here for Usability issues are strictly aimed at providing a better understanding of what a Usability issue is, as opposed to Functional issues.

+1 Useful tips

When you’re about to report a defect, always ask yourself “How much of an impact does this issue have on the application and my ability to interact with it?” in order to correctly identify the Severity level of your bug report.

Severities are also influenced by the level of probability one bug can be reproduced in a Live environment by a normal user:

Example #1: Consider that there is a situation where the user finds a mistake in the naming of the product itself or some problem with the UI documentation. A tester would normally open a minor/cosmetic defect and may be very simple to fix, but when it comes to the product look and feel / User experience, it could cause a serious impact. 

In this case, a Low/Medium severity issue can go up to a High severity due to the high importance of that spelling mistake (the name of the app). 

Example #2: There could be certain conditions under which a particular defect occurs which may be an extremely rare or no possibility to hit in the customer environment. Even though functionality-wise this may seem like a high severity defect to a tester, considering its rarity of occurrence and high cost to fix – this would be classified as a low severity issue.

In this case, a Critical bug (the app crashes) will be set as High or even Medium severity if the repro steps are more exotic (either a lot of repetitive actions are needed or the steps imply that the user performs additional actions that are not related to the normal user behavior) or the reproducibility of the bug itself is very low (less than 3 times out of 5 tries).