We are trying to set up A/B testing for anonymous users that is targeted specifically for iPhone users. Amplitude, however, seems to be unable to detect iPhones for the variant as per our testing. For reasons unknown the variant is not returned in the request, because Amplitude does not recognize iPhone via the user agent.

We are using the JavaScript SDK and setting it up as per documentation, and I’ve done some debugging to see what type of user agent string is being delivered via network requests from a phone:

Granted, this is from an iPhone simulator, but based on the information I gathered from the web, it is exactly as a real device, which also do not work. We had QA engineers try to enable this experiment as well with real iPhones on our website and we couldn’t get it to work.

Here’s a snippet of the JS code:

const amplitudeExperiment = window.experiment;

if (amplitudeExperiment) {
  var div = document.getElementById('variant-info');

  amplitudeExperiment.fetch().then(function() {
    const experimentKey = 'experiment-key';
    const variant = amplitudeExperiment.variant(experimentKey);

    if (variant.value === 'treatment') {
        div.innerHTML += 'Treatment enabled';
    } else {
        div.innerHTML += JSON.stringify(navigator.userAgent)
    }
  }).catch(function(e) {
    // Silently fail
  });
}

I am testing this by inputting some content in an element because I don’t have developer tools on an iPhone simulator. Real logic is of course going to be different, once we can get this to work. At this point is looking like we have to do device detection ourselves and leave this out of the experiment on Amplitude’s end.

We’ve have set the experiment to allocated users, with a segment filter that defines either Device Type to be Apple iPhone, or Device Family to be Apple iPhone. Neither seems to work. The variant is not returned as the “user does not match segment”

We’ve also tried other variations of these segments, including adding all iPhone versions here. This seem to work on a Macbook if I set the Device Type to also contain Mac, but iPhone just doesn’t.

So what’s wrong here?