Prioritization in Quality Assurance — How we do it
Five levels of prioritization in COBE
For the outsiders, Quality Assurance might be only about finding issues and writing steps to reproduce. In reality, what makes a good QA is not only to have an eye for catching bugs, but to know how to recognise the importance of each of them. There are bugs a QA Engineer might see as irrelevant, but for the Product Owner it might be a showstopper.
The thing is, each fixed bug doesn’t have the same value to the customers, PO or QA, and that is why aligning priorities is so important.
Defining the different levels of priority might be a challenge for newly setup QA teams, since the internet is full of different and often contradicting information. We also did a research before, and practically each page we’ve opened had its own definitions and explanations. That’s when we decided to create our own categorisation, explained in the following text.
5 types of prioritisation in COBE
Bugs found (like we have them :bug:) in our projects are now being split into 5 categories: Trivial, Minor, Major, Critical and Blocker.
The definition of each type is based on the way this bug impact on the functionality of the app. Let’s dive in!
Definition: Trivial bug doesn’t affect functionality and progress. It doesn’t affect the user usage of the app.
Example: In the Login screen there is a button for the Login. Text on the button is misspelled to ‘Logun’.
Explanation: Yes, this is definitely a bug. But it doesn’t affect on functionality. User can log in in the app which is the main purpose of this feature. And this misspelling doesn’t affect on the users understanding of the usage of that button.
This kind of bugs are lowest in the priority list. They will be fixed if there is enough time, or in the period of polishing the app.
Definition:Minor bug has the potential to affect progress. The part of a feature is malfunctioning or is unable to function. There is a simple workaround to avoid this behavior.
Example:User needs to click on the button to do some action. In order to trigger this action, the user needs to click 2 times on the button.
Explanation:As a user, if something is not working you will definitely click one more time to be sure that you have clicked. There is a simple workaround to avoid this problem. It won’t be a trivial bug, because it can be pretty annoying if this issue appears on the button that is ‘important’ in the app.
This kind of bugs should be resolved after all the serious bugs are fixed.
Definition:There is a workaround but it’s hard to go through it and there is a chance that this bug will affect the progress.
Example:One of the features in the app is deleting the item in the list. The item can be deleted from the list, but also it can be deleted from the details page of that item. Let’s say that deleting from the list is not working.
Explanation:This is the major bug because one of the features is not working at all so it’s affecting the progress. But, because there is a more than one way of deleting the item, a user has a workaround and he can do this action.
The bugs with the highest prioritisation? This is it. It’s not a show stopper, because there is a way for a user to finish the action. But this bug should be fixed because it’s not harmless.
Definition:This type of bug affects the usage of the app and there’s no workaround. It’s a show stopper which means the functionality cannot be delivered unless that bug is cleared.
Example:In the application, there is a contact form. User is able to send his data in order to apply for a job. After he has fulfilled the whole form he has clicked on the submit button. The form is not sent to the employer.
Explanation:The main function of this feature is not working. User is able to fill out everything (that’s why it’s not a blocker), but he is not aware that his form has never been sent.
If the bug has label critical, that means that something is seriously broken. Usually, the feature is unusable, so this should be fixed immediately.
Definition:The entire app or just some of its features are broken and they aren’t usable. Users or testers are blocked.
Example:One of the features in the application is ‘Edit profile’ where the user can change his data. When he taps on the edit button, nothing happens.
Explanation:As it’s written in the definition, we are blocked. In the edit profile section can be more than one feature. For example, a user can edit his name, email or image. None of these features can’t be tested because there is no path user can take to get there.
Same as critical bug, this should be fixed immediately.
Please note!There are circumstances in which the same bug can jump up or down on the priority list. For example, the trivial bug above mentioned — misspelled word “Logun”, during the production phase is only trivial. But if the team is one day away from the release point, this bug would become more important as its quite embarrassing to go live with a mistake in the first steps of the app!
Two more factors affecting the priority
It is all about the context. If you are building an app where the most important thing is the look and feel of the app, then the UI bugs will get the highest priority. In this case, maybe the one-pixel difference will be enough to launch the bug into the major priority list. In some other cases, the one-pixel bug will be a trivial bug and the team might decide to leave it be.
The other thing that affects the priority of a bug is the personal opinion of the tester who is reporting the bug. There is a small difference between trivial and minor bug, or critical and blocker. The examples that are stated are pretty straight forward, but sometimes the difference will be not that obvious and the priority will depend on the opinion of the person who is reporting it.
We didn’t forget severity but…
By looking at the prioritisation literature and online info, the term severity will probably show up. The definitions online say that:
Severity stands for the impact that the bug has on the functionality of the app.
Priority defines the order in which the bug should be fixed.
In our team, we use the combination of both and we call it prioritisation. Our projects are mostly done using scrum or Kanban. Implementation is divided into 2 weeks iteration, with a mini release after each of them.
Therefore, we are dealing with the one set of the bugs for one iteration. Our mindset is that most of the bugs need to be fixed anyway. First, the team needs to deal with the bugs that have the highest prioritisation, and if there is a time they are going to fix the bugs that are not that much important (in an ideal world each bug is important, I know). At this point, the labeling both severity and priority would be an overhead.
How do you handle priority in your company? Do you have the same categories as we do? Feel free to share your experience, we would like to hear it!
Nikolina is a master of math, QA engineer and a Scrum Master at COBE. While her responsibilities mostly focus on QA, she is strongly influencing and challenging the whole team, changing our process from scratch. As a former basketball player, she shows impeccable work habits and contagious energy that often reflects on the team.