Before closing a test or closing down any variations, we want to ensure that the test results are stable and not likely to shift dramatically.
When a variation has less than 200 users, as well as less than 30 installs, just a couple of users can dramatically shift the performance results, as each individual user holds more weight than when the sample size is larger. Before those thresholds are met, it is likely that you would see high confidence one day and a dramatic decline the next.
We know in the initial days of a test, page performance can be more volatile due to the quality of users who are initially sent in through your traffic campaign.
As such, we require 3 full days of traffic before we can start recommending any action.
If you have any further questions, please feel free to contact us at email@example.com