A large number of users contain only one experimental event


Userlevel 1
Badge +2

Hi, Guys.

We found that there are a large number of users with only one event([Experiment] Assignment) in our project and no action after. Such users are close to 100,000 per day, But in this project we have less than 100 users.

So we want know: 

   1.how these users are generated?

   2.Why these users are generated and what is the role?

   3.how we handle?

data link: https://analytics.amplitude.com/carsome/chart/new/4ii7idc

user as picture:

 


3 replies

Userlevel 6
Badge +8

Hi @xiaomeng.zhang ,

 

I hope you are well! So the [Experiment] Assignment event will fire when the user calls the fetch() function to get the appropriate variants belonging to an experiment. This is done in the code. I see you have several active experiments so it seems users are being “fetched” for these experiments in your code. For example here the user seems to be genuinely interacting with the app and simply was assigned to the experiment because he fit a certain targeted group. I do see users like this one and this one, they seem to be calling fetch but not activating any events. They may have opened the app and immediately closed it for example or they may have possible been sent it manually. I would speak with your developers to see why so many are being triggered and if it is possible to test the websiote and segment integration. I would also be sure to upgrade to the latest SDK. 

 

So we want know: 

   1.how these users are generated?

This is difficult for us to determine on our end but the most likely cause is that users are going to the website and fetch is being called, they get assigned, and then leave the website. However, the more likely reason is your integration with Segment might be slightly incorrect and this could be causing segment to send a bunch of assignment events to be fired for users. I hope this article can help!


   2.Why these users are generated and what is the role?

See my above reasoning but I do suspect the Segment integration here. I suspect it was assigned to all users, literally, even if the user was not in Amplitude and once we got the event for those users, it was their first event. I would try to see if there is an integration with the Segment + Experiment that is possibly causing issues. 

   3.how we handle?

For now, I would highly recommend upgrading to the latest JS SDK, send me the websiteURL if you could, and also make contact with Segment to see if they are possibly calling fetch on our SDK multiple times for users. 


I hope this helps with the initial investigations!

 

Kind Regards,
Denis

Userlevel 1
Badge +2

Hi,Denis Holmes

We already upgrad to the latest JS SDK, And we think these abnormal users and events should be sent from the server. 

Userlevel 6
Badge +8

Hi @xiaomeng.zhang ,

Have you tested the website with the Amplitude Instrumentation Explorer Add-On for Chrome? That can often help with troubleshooting these issues. The Assignment will always come from the server I believe but the user must have called fetch() at some point. Did you get a chance to ask Segment with their SDK if there was possibly something causing the fetch() to be triggered for all users?
With Segment, are you using their cloud or device mode?

Reply