Skip to main content
Solved

Can you update historical events with a new user property

  • 7 November 2022
  • 5 replies
  • 511 views

Hi there,

We added a new user property and I was hoping that this would propagate to historical events.

However, this has not occurred and this user property is still showing as `(none)` for events created before this user property was set.

I was reading this somewhat related topic where @Yuanyuan Zhang notes:

Once a user property is set, it will be applied to all events going forward until it gets updated. For example, you first have user property email = abc@gmail.com, all events will have this user property value. Then you send in another value today email = def@gmail.com, all events from that point onwards will have the user property value def@gmail.com.

 

And:

You can update the same user prop "email" by including "email" in the payload with the new value and specify that it will be overwritten for the user properties at the top and the new value will propagate to further events. Historical events will not get updated.

 

Does this confirm that it is not possible to update the user properties of historical events? Or is there a way to do this?

Would like to emphasise that this is a new user property that did not previously exist (in case that makes any difference).

Any advice is greatly appreciated!

Lachy

5 replies

Yuanyuan Zhang
Rank
Userlevel 5
Badge +5

Hi @Lachy 

Thank you for this post!

It is correct that it is NOT possible to update the user properties of historical events at Amplitude. User properties will only apply to events going forward. Our current architecture is based on pre-aggregated sets by the hour, day, week and month for users and events. While this allows us to scale well, a limitation is that historical data is immutable.

Hope this clarifies!

Best,

Yuanyuan from Amplitude
James Lindsay

The user properties being ‘time based’ is actually quite nice in Amplitude, since it can do more in terms of reporting than what you get in the likes of Mixpanel.

Although I do find one thing lacking here. 

Say I want to import historical user properties. 

What I’m getting from doing that is the user properties only present on users who trigger any event after the historical import onwards. That’s okay… I guess for event-based reports like Segmentation. 

Although when I try and do an analysis using the User Composition I’m still only seeing the user properties for users who triggered an event from import onwards. What we’re lacking here is a historical view of the user properties when importing historically… which seems like a feature ‘bug’?

This could perhaps be remedied by sending another event after the historical user properties import but that isn’t ideal.

 

One other issue I find is when importing data, say using the BigQuery connector or even Hightouch. If we’re importing user profiles with properties and events… wouldn’t we then need to make sure we’re syncing the user profiles with their properties before we sync their associated events? 

Going forwards I’ll be adding user properties to their relevant events when syncing (such as Lifetime Value in the Purchase event) and reserving user profile updates to non-event-specific properties.

James B. Lindsay
Yuanyuan Zhang
Rank
Userlevel 5
Badge +5

@James Lindsay 

 

Although when I try and do an analysis using the User Composition I’m still only seeing the user properties for users who triggered an event from import onwards. What we’re lacking here is a historical view of the user properties when importing historically… which seems like a feature ‘bug’?

This could perhaps be remedied by sending another event after the historical user properties import but that isn’t ideal.

 

I understand this might be confusing, but this is not a bug. User Composition looks at the user properties, not events, but the values of user properties are drawn from active events of users. This behavior may not fit all use cases, so please feel free to submit feature requests. 

One other issue I find is when importing data, say using the BigQuery connector or even Hightouch. If we’re importing user profiles with properties and events… wouldn’t we then need to make sure we’re syncing the user profiles with their properties before we sync their associated events? 

 

It is correct that you would need to sync user properties before syncing events because user properties only apply to events going forward. 

Yuanyuan from Amplitude
S

Hi — wondering if cohorts would solve this problem, i.e. defining cohorts on the current value of a user properties, then comparing cohorts on historical events. Would that work?

James Lindsay

Hi — wondering if cohorts would solve this problem, i.e. defining cohorts on the current value of a user properties, then comparing cohorts on historical events. Would that work?

Unfortunately, I don’t think that would work either.

I ended up changing how we import user properties and made it, so the relevant events synced all the user properties (i.e. lifetime spending, etc).

This would still pose an issue for ‘CRM like’ properties which simply aren’t attached to a specific event. It would also pose an issue if the user property were across multiple events (like total spending across online and offline POS sources). 

In this case, sending a ‘dummy event’ may be the only solution. Another option is to use computed properties… although this is only available for 365 days windows and is a paid add-on. 

James B. Lindsay

Reply


Terms & ConditionsCookie settings