5. Outcomes: The fate of your issue

So, you’ve found a pesky little problem in the app under test, and you’ve submitted your bug report, what now?

Well, first of all, congratulations! Thank you for your work, our team will now review your bug. One of the moderators in charge of your test cycle will periodically review any new issues submitted on the project. The review process can occur at any time during the testing interval based on our team’s workload and other factors. We recommend that after you submit your first issue on the project you continue testing and try to find other bugs as the review does not always take place immediately after you hit the submit button.

Eventually, however, your bug reports will always receive a moderation outcome based on the quality of your report.

A good outcome will be rewarded with rank points and you will receive payment for your effort.

A bad one, however, will decrease your rank and will also bring you no payment for the time and effort you invested.

Here are all the decisions you might encounter as a tester:

• Confirmed - your issue was accepted without edits. That means your report was perfect or at the very least good enough that it did not warrant any modification. You should be proud of yourself and should continue reporting at this level.

• Confirmed (with edits from Test Manager) - Your issue was accepted but some edits had to be made. When the moderator needs to change any of the fields present in the bug report but overall considers the issue valid then this will be the resolution.

• Invalid - Not tester error - Your issue was rejected for some reason, but you will still receive payment for it. This means that the reason for the rejection was not the tester’s fault. Either a miscommunication occurred, some information was vague, or the app’s server caused certain problems during the test. Whatever the reason, the bug can not be accepted but the tester’s work is still valued. The reason for this outcome will usually be mentioned in the bug report’s comment section.

• Needs more information - Unlike the others, this is not a final resolution. In this case, the moderator does not consider that you provided enough context or thinks that something vital to the report is missing. You will receive a notification urging you to provide the missing info. 

However, you must understand that this is more of courtesy from the moderator to the tester. When submitting an issue you are expected to turn in a complete and adequate report from the start. The ‘Needs more information’ resolution is given simply because the moderator saw some value in the bug report and wants to give the tester a second chance to get it right. The moderation team could just as easily reject an incomplete bug report with a ‘Poor Quality’ resolution, as it is the responsibility of the tester to ensure his or her report contains everything required.

• No tester response - Unlike the previous resolution, this one is negative, final, and is directly derived from a tester failing to respond to the moderator’s request. This resolution is usually set at the end of the time allocated for that particular test when all issues need to be moderated. 

As previously mentioned, the tester is fully responsible for providing a complete report, therefore any delays in providing the additional information required will be penalized with this decision, even in the unfortunate event of the issue being reviewed late in the cycle period, and the ‘No tester response’ coming shortly after. We, therefore, advise all our testers to keep an eye out for ‘Information Requests’ and being prepared to double-check their issues during the entire testing period.

• Not in scope - Another negative resolution, the ‘not in scope’ decision is given when a tester submits an issue that goes against the conditions and rules set out in the Tester Spec. The moderator will always leave a response detailing why the issue has been marked as such, but in most cases, the simple truth is that the tester did not read the Test Spec Document carefully enough, especially the sections detailing what is and what is not in Scope for the Test.

• Duplicate - Moving further down on the rejection scale, a ‘duplicate’ resolution is rather self-explanatory. The moderator has noticed that the issue you have submitted has already been reported, either in the Known Issues List that each tester is required to review before submitting an issue, or earlier on in the same test cycle by another tester. In both cases, the result is the same and the duplicate entry will be rejected with a link pointing the tester in cause towards the first issue.

• Expected behavior - This type of rejection usually stems from the tester’s failure to completely read and/or understand the provided documentation regarding the app. This resolution is set when the moderator considers that the action described by the tester in the bug report is normal and/or correct. The most common misunderstanding about the app’s proper functionality is derived from the tester’s expectations. These expectations, however, are not always in line with the intended behavior of the app. Similarly to ‘not in scope’ the moderator will always provide the reasoning behind this decision in the comments section of the bug report page.

• Poor quality report - Finally, the worst resolution a bug report can get is that of a ‘poor quality report’. When receiving this resolution it means that a tester has utterly failed to properly create an acceptable bug report, so much so that we dedicated an entire chapter on this result that you can find reading this guide further.