In my product, users can send each other messages. If we want to experiment on how messaging works, we can’t run a typical A/B experiment because we need both the sender & recipient to be in the same variant. One standard way of addressing this is running experiments in different geographic locations, with the assumption that each location has a mostly-closed network. This hasn’t worked for us, unfortunately, as our separate locations aren’t very similar in the first place so we can’t compare them. Some companies have the ability to expose experiments based on network clustering. Is that a feature that Amplitude has?
Side note: I have bet on with my engineer that the answer is no. =D So you won’t disappoint me if that’s the case!
The best alternative I can think of is to use BigQuery to manually separate users into clusters, normalize on relevant attributes, try to pair similar clusters to assign to different variants, and upload those assignments as a user property, but I have no way of ensuring similar levels of exposure. It seems like there’s just too many factors where things can go wrong. =/
Question
Can Amplitude do network experimentation?
Reply
Welcome to the Amplitude Community!
If you don't have an Amplitude account, you can create an Amplitude Starter account for free and enjoy direct access to the Community via SSO. Create an Amplitude account. You can also create a Guest account below!
If you're a current customer, select the domain you use to sign in with Amplitude.
analytics.amplitude.com analytics.eu.amplitude.comor
Welcome to the Amplitude Community!
If you don't have an Amplitude account, you can create an Amplitude Starter account for free and enjoy direct access to the Community via SSO. Create an Amplitude account. Want to sign up as a guest? Create a Community account.
If you're a current customer, select the domain you use to sign in with Amplitude.
analytics.amplitude.com analytics.eu.amplitude.comor
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.