The "Not In Scope" rejection reason under the loop

We assign a “Not In Scope” resolution to any issue which, although it may be considered a valid bug objectively, does not fall within the scope of a test cycle.

As such, there are many reasons as to why a bug can be marked as “Not In Scope”, such as a bug which is in a different section or affects a different feature other than the ones which are specifically in scope, it is a Usability issue, it is an issue caused by Unusual User Behaviour, or the bug is caused by a 3rd party element. 

Let's see each of them a little bit more detailed:

1. Not In Scope - based on documentation

There are two potential cases when an issue falls outside the scope of a Test Cycle according to available documentation, which you can find detailed below.

• The issue affects a feature or occurs in a section which does not concern the test cycle’s scope

To avoid these situations as much as possible, we recommend that you always double-check the Tester Spec, as some projects may have large Scope lists.

•  The issue does not concern a Test Case’s script and the test cycle does not have an expanded Exploratory scope

If you’ve submitted an issue while executing a Test Case, but the issue is not generated by performing the Test Case’s instructions or occurs in a section which is not covered by it, this is considered “Not In Scope”. Always make sure that you perform the actions provided within Test Cases as strictly as possible.

2. Not In Scope - Usability

Any issue which describes a suboptimal functionality, extremely minor visual defect, or non-uniform design of the application, is considered a Usability issue, which will be rejected as “Not In Scope”.

• Best practices
• If you’re unsure whether an issue is a bug or a Usability issue:
• Ask yourself if the application is broken by the described behavior, or if it could only be improved;
• Make sure you read our expanded documentation on the subject of Usability issues for more details.

3. Not In Scope - Unusual User Behaviour 

Any issue which is caused by uncommon in-app actions, spamming buttons, abusing navigation, or any other behavior which would not be encountered during normal usage of the app, is considered “Not In Scope”. 

• Best practices
• If the issue is particularly severe (eg. tapping the same button 5 times causes an app to crash), then submit it with reduced severity. The fact you need to perform that many taps in order to reproduce the issue mean it’s not critical.
• Keep in mind that not all unorthodox behaviors are Unusual User Behaviour - performing 2-3 extra taps if an app freezes for a second is common, thus the issue you’re encountering may be valid.

4. Not In Scope - 3rd party

If an issue occurs as a result of a 3rd party element (another application or setting which is not standard issue for a device), then it is considered “Not In Scope”. As long as it does not concern interactions with habitual elements (calls, messages, notifications, etc.), it is not considered a valuable issue, as nobody can account for every application on the market.

• Best practices

If the issue is generated by standard features (such as the native OS battery saver, system notifications, OS actions like updates, phone calls and/or video calls on the most common applications), the issue is valid, as it is reasonable to expect that an application must function correctly when interacting with the operating system, for example.

• Examples of unacceptable 3rd party elements:
• Network conditioners
• 3rd party RAM managers (Note: native RAM optimization processes, such as for Huawei and OnePlus devices are acceptable as long as they’re not abused for testing)
• 3rd party battery savers (other than the native battery save process)
• Rooted devices and root interactions (unless specifically requested for a test cycle)
• Custom Launchers with heavy UI changes