Solved

Weekly cohorts / unbounded retention: why o they get worse over time?

  • 28 September 2021
  • 4 replies
  • 261 views
A
Rank
Badge
  • Asja
  • Conversationalist
  • 5 replies

Hi everyone! 

I performed a weekly retention analysis (unbounded retention) based on 

  • Starting Event: Account Verified (user is assigned a known User ID)
  • Return Event: Any Event 

    Now what I see is that older cohorts (we only launched the app on May 27th) show much better W1, W2, W3 etc. retention rates than the younger ones (especially with the cohort at the end of July there’s a big drop). What could be the reason? Is ‘any event’ a good return event at all, as it also includes end_session ? 
     

    Unbounded weekly retention by cohorts 

    Thanks in advance!

    Asja

     
icon

Best answer by MikkoKarvonen 28 September 2021, 15:50

View original

4 replies

MikkoKarvonen
Rank
Userlevel 5
Badge +3

This is the expected result when using Unbounded Retention.

In Unbounded Retention, the user is considered retained if they have performed the follow-up event (in your case, any event) at any point after the starting event, and they’ll marked as retained for all the weeks, up until the point in time they performed the latest follow-up event.

Let’s say you have 100 new users in one week.

During Week 1, 20 of them come back. Your retention for Week 1 is at 20%.

During Week 2, 25 of them come back, but 10 of them are same as during Week 1. So that’s 15 new users. Your retention for Week 1 is now at 35%, and your retention for Week 2 at 15%.

During Week 3, 20 users come back, 5 for the first time. Your retention for Week 1 is now at 40%.

Essentially, the longer it has been since the starting date, the more time the users have had time to come back, and more your retention creeps up. Up until it reaches some saturation point.

If you want to compare weeks to weeks, use N-Day Retention instead. It gets rid of this factor.

Both approaches have their place, but it’s good to be aware of how they’ll affect your results.

Hope this helps!

A
Rank
Badge

Hi @MikkoKarvonen , thank you so much. So basically that means then that from the May 24 cohort, only 0.9% have churned (never returned again) until today, and the rest performed any other event? 

Would it be possible, that, among those ‘any events’, there are also end_session events? Which would not say much about the stickiness of our app. 

Would it be smarter to add a specific event to the return event, such as start_session ? or something of higher value, to have more significant results here? 

MikkoKarvonen
Rank
Userlevel 5
Badge +3

Hi @MikkoKarvonen , thank you so much. So basically that means then that from the May 24 cohort, only 0.9% have churned (never returned again) until today, and the rest performed any other event? 

Would it be possible, that, among those ‘any events’, there are also end_session events? Which would not say much about the stickiness of our app. 

Would it be smarter to add a specific event to the return event, such as start_session ? or something of higher value, to have more significant results here? 


@Asja, we don’t track the end session events in our app, so I’m not familiar how they interact with this, but it sounds entirely feasible that they are included.

99,1% retention does sound like a utopia (if you had such figures and even half-decent way to monetize, you should really be talking to investors and not us!), so something like that is likely affecting these results. If you are using push messages and sending events about them to Amplitude, that could be one option.

I’d suggest one of two options:

  1. Use some other event, as you suggested yourself. Your app should have some key activity that define whether the users are actually interacting with it as intended. Tracking that would give you more relevant results.
  2. Under Govern → Events you can define which events are counted as Active (i.e. triggering any of them counts as being active user) and which event are Inactive (I’d drop at least things like Start Session and End Session there, as well as anything else that is not related to the user actively interacting with your app). After doing that definition, you could switch Any Event → Any Active Event in your chart and get more relevant results.
A
Rank
Badge

@MikkoKarvonen Thank you very much, that sounds very sensible. I will try the suggested and see how it changes. Have a great day, Asja

Reply


Terms & ConditionsCookie settings