Will testers override experiment assignment after experiment started?

Young Sun

If I turn on an experiment with distribution, let’s say, 50%/50% control vs treatment, and 50% rollout, and then I set some testers in the testers tab for treatment, will the testers all be valid treatment? i.e. testers are overriding my distribution with higher priority?

Yosimy.Cortes

Hi Young,

Thanks for reaching out! Yes, you're correct. If you set some users as testers for a specific variant (in this case, treatment) in the testers tab, these users will be assigned to the treatment group regardless of the distribution settings. This means that the testers tab has priority over the distribution settings.

So, even if your experiment is set up with a 50%/50% control vs treatment distribution, the users you've specified as testers for the treatment group will be part of the treatment group.

This feature is designed to allow you to test the experiment on specific users or devices by manually assigning them to a variant. It's important to note that changes made in the Testing section will only affect the specified users and will not impact the overall distribution of users between variants.

I hope this helps!

Best,
Yosimy


P.S. Checkout upcoming events and user meetups on our events page.