I'm a big fan of Amplitude and I find it really amazing tool. Surprisingly I have an issue with the simplest report (as it supposed to be). I'm talking about Users: New vs Returning report.
How can I properly compare the segments of New users vs Returning users?
I need to analyze the behavior patterns for new vs returning users
I've used tons of variants an no one seems reliable enough. What is the right way to do this?
Thanks a lot !
Best answer by MikkoKarvonen
So essentially a new user is someone who visits your website for the first time (as far as the tracking is concerned, people changing browsers, clearing their cookies etc will always distort this data) and returning user is someone who comes back after their first visit.
One reason this is challenging is that Amplitude and Google Analytics approach things slightly differently. Strictly speaking Amplitude does not have a concept of a returning user, as it focuses on active users instead. Also the definition of a visit is bit different.
Based on this assumptions I’m creating two different segments:
- All users
who performed New User = 1
any time in each day,
- All users
who performed Any Active event > 1
any time during last 30 days
The users in your segment 1 are also included in your segment 2, since they would also have performed an active event during the last 30 days. This may be intentional, but I’m getting the impression that this is not what you are looking for.
- Huge data discrepancy b/w Amplitude and GA4 (Amplitude shows two times more the amount of new users daily)
First things that comes to mind: have you had Google Analytics and Amplitude tracking set up for the same period of time? If you’ve had GA before and are now adding Amplitude, Amplitude is naturally considering some users new that are actually returning, since it has not seen those users before.
Also, is the instrumentation the same on both tools?
- When I compare 1) all users (new user event) with 2) all new users (who performed new user=1 any time in each day) I see 124 vs 175 unique events. But I expect to see the same number because the only thing I did is just moved New User event from the left side to segment attribution. It’s better to show on a screen. Attached the report for a specific day.
This is indeed curious. I did some experiments on our data, and it looks like there is a difference between setting the first cohort up as New User = 1 time vs New User >= 1 time. If I use the >= version, the numbers match.
If I had to guess, this is probably related to the way time zones are handled, but perhaps some Amplitude representatives would have more insight?
So, these two thing make me feel that I cannot rely on the data.
Keep in mind that to some extent you are comparing apples to oranges, since GA and Amplitude define some things in different ways.
There is a chance that in the case of anonymous website visitors GA may be better in identifying the same user on different devices, if they use Chrome on both devices. That’s something Amplitude just can’t do.
On the other hand, Amplitude will give you much more flexibility to analyse your data than GA does.