Skip to main content

We are trying to set up A/B testing for anonymous users that is targeted specifically for iPhone users. Amplitude, however, seems to be unable to detect iPhones for the variant as per our testing. For reasons unknown the variant is not returned in the request, because Amplitude does not recognize iPhone via the user agent.

We are using the JavaScript SDK and setting it up as per documentation, and I’ve done some debugging to see what type of user agent string is being delivered via network requests from a phone:

Granted, this is from an iPhone simulator, but based on the information I gathered from the web, it is exactly as a real device, which also do not work. We had QA engineers try to enable this experiment as well with real iPhones on our website and we couldn’t get it to work.

Here’s a snippet of the JS code:

const amplitudeExperiment = window.experiment;

if (amplitudeExperiment) {
var div = document.getElementById('variant-info');

amplitudeExperiment.fetch().then(function() {
const experimentKey = 'experiment-key';
const variant = amplitudeExperiment.variant(experimentKey);

if (variant.value === 'treatment') {
div.innerHTML += 'Treatment enabled';
} else {
div.innerHTML += JSON.stringify(navigator.userAgent)
}
}).catch(function(e) {
// Silently fail
});
}

I am testing this by inputting some content in an element because I don’t have developer tools on an iPhone simulator. Real logic is of course going to be different, once we can get this to work. At this point is looking like we have to do device detection ourselves and leave this out of the experiment on Amplitude’s end.

We’ve have set the experiment to allocated users, with a segment filter that defines either Device Type to be Apple iPhone, or Device Family to be Apple iPhone. Neither seems to work. The variant is not returned as the “user does not match segment”

We’ve also tried other variations of these segments, including adding all iPhone versions here. This seem to work on a Macbook if I set the Device Type to also contain Mac, but iPhone just doesn’t.

So what’s wrong here?

I dig in to the user data (which is really bothersome, having to find the actual user to see the event and not finding the event through experiment monitoring), and seems like all device info is empty:

 

I even tried setting this up manually in the fetch request, but all I get is empty data here.


Ah, alright. This was caused by an outdated version of Analytics SDK that the application was running. I updated it and also integrated it with the Experiment SDK here. Seems to work now. I assume the Analytics SDK sends in more data to parse the device information.

This is now solved!


@Ossi Pesonen amazing!! Apologies for the late reply and huge props for sticking with this. Thanks for reporting and resolving this issue!


Reply