Even though the topic of our ranking system has already been discussed, we would also like to tackle this subject from a payment point of view. Many of you have questions about how exactly the final payment is calculated and how we explain major ranking drops that happen overnight.
First of all, please check the ranking rules that every tester should be aware of:
- The “30 days” rule: this rule refers to the fact that the ranking is calculated based on the performance of each tester within the last 30 days
- The “UK time” rule: the rank of the testers is updated every day at midnight UK time, without exception
- The “Relativity” rule: even if no issues were reported and accepted/rejected by a tester, the ranking could still change if other testers performed better/worse (ex.: if another tester performs better, your rank points won’t change, BUT you will be placed on a lower position on the leaderboard)
- The “Bug report time” rule: the value of a bug report is determined by the rank a tester has at the time of submitting an issue
- The “Test runner” rule: Test Cases, including our test runner runs, are not subject to the rank multiplier. Only bug reports submitted on the tracker will benefit from this.
- The “Payout boost” rule: the different payout boosts offered by test managers for certain projects apply only to the base payment of the bug, not to the ranked value of the bug
- The “Rejected bug” rule: rejected bugs are also counted and can influence the rank of the tester. Try to avoid rejections. You can find valuable content about this topic here and here.
💡 Example of a ticket raised because of the 30 days rule:
“Yesterday my rank was Diamond but I checked it now and it seems that it dropped to Silver. How is this possible?”
- The tester reported very good bugs in several test cycles, but unfortunately, many of these were reported more than 30 days ago. This is why the rank has dropped so dramatically.
💡 Example of a ticket regarding major rank drop overnight (involving the rejected bug rule):
“My rank dropped from platinum to silver in 24 hours. In the last 30 days, I've reported over 40 bugs, so I think there is a problem with your ranking system.”
- The ranking system worked as expected. Indeed, the tester reported many bugs in the last 30 days, but the rejected issues outnumbered the valid ones. In this situation, rank change is inevitable.
💡 Example of a ticket raised due to the UK time rule, bug report time rule, and time zone gaps
“My current ranking is Platinum, but I didn’t receive full rank multipliers for all the bugs reported during project [...]. I received my new ranking before the cycle of this app ended and the errors were confirmed.”
- In the above case, the tester notes that the ranking multiplier for Platinum rank wasn’t added to all his/her bug reports. What happened here?
The test cycle in question was 2-days long, so our tester managed to report issues on day 1 and day 2 as well. However, the ranking system is updated every single day at midnight UK time, thus on day 1 the tester had a Gold rank, but on the second day, he reached Platinum.
In this case, the value of the issues reported on the first day is calculated with the Gold multipliers, while the issues reported on day two received the Platinum multiplier, even though all of them were reported during the same test cycle.
Note, that similar situations can happen even if the test cycle is only a few hours long because of the time zone gaps.
For example, a Mexican tester receives a test invitation in which it is stated that the project has to be done between 11:00 and 20:00 Mexican time. During these hours, midnight will pass in the UK (18:00 in Mexico), thus the rank of the tester will be recalculated. All the bugs reported before 18:00 MX time will be calculated with the old rank, and for all the bugs reported after the mentioned hour, the new multiplier will be applied.
💡 Example of the payout boost rule:
“I took part in a test cycle in which we should have received a 100% payment boost for bug reports, however, I believe that it wasn’t calculated correctly, as I also have Diamond rank and the amounts were lower than I expected. Please check.”
- The payment boosts offered by the test managers are calculated independently using the base payment and do not factor in the rank multipliers.
❌ This is what the testers’ interpretation of the formula looked like:
Base payment x Ranking multiplier x Payment Boost = Final payment
✔️And here you can find the correct formula we apply:
(Base payment x Payment Boost) + (Base payment x Ranking multiplier) = Final payment