Skip to main content

We are bumping into our property limit and need to clean things up. Admittedly our data governance has been lacking for awhile and has caused us to have a lot of events/properties that showed up years ago for a bit and are no longer used.

In my naive attempt to resolve this issue I went and found all properties not seen > 2 years ago and deleted them. This does seem to have removed them from the property limit count but after digging around a bit in docs I realized there is a nuanced behavior to deleted events/properties in that they are actually ‘blocked’ as well. As such, if a ‘new’ event is sent that has a property that matches one of these deleted property it won’t be associated w/ the event or saved in Amplitude. It does seem like these properties can be undeleted to solve that but the delete behavior is a bit problematic because it creates a ‘gotcha’ that could confuse neophytes or be missed and impact our data collection. Property names are usually rather simplistic like ‘button_name’ and could actually have a significant likelihood of coming back over really long running projects. IMHO, this seems like undesired behavior but I would assume it has to do with current system storage and processing limitations.

Is there an alternative approach I can take here that will get the outcomes I want? I basically want to get under our limits by removing old unused events and properties yet allow them to become available again  automatically if they are sent by one of our systems in the future.

There is this time to live (ttl) concept. Does that expiration remove old events and properties when the expiration hits?

There is also transformations concept but does doing that reduce property counts? I also don’t know if really helps us here w/ our desired outcomes either.

I did see someone mention a dump, transform, and reload into a different project could be an option but that pain is probably worse than dealing w/ the delete behavior.

Hey Lance,

Great questions! There are a few here so let me address them below:

You're correct in your understanding that deleting properties in Amplitude will block them from future ingestion. This is indeed due to the way our system is designed, but would you be able to elaborate a bit on what you would be hoping to achieve by _deleting_ a property and still ingesting it?

Depending on the current taxonomy your organization has, you could potentially drive down the overall volume by enriching certain events with more properties and reducing the number of overall events (which will typically coincide with a drop in total user/event properties). An example would be if you have multiple 'clicked' or 'page view' events then you can isolate all of those events into a single event and pass in the necessary properties such as 'button clicked' = 'spring_campaign_cta'. You can drive down properties by aligning your events to have consistent properties throughout. Generally this drives down associated property numbers if you can keep properties consistent throughout these types of events.

As for your question about Time to Live (TTL), this feature is about data retention rather than property count. If you have enabled the TTL feature, data outside of the retention period is permanently deleted, but this is set at the Amplitude Organization level and impacts all event data.

Transformations, on the other hand, allow you to modify your event data structure retroactively, but they too do not reduce property counts.

One option you might consider is using the Export API to pull all the data from your project, sanitize the data (remove properties), and then perform a Self Backfill and use Batch API to push events into a new Amplitude project. However, please note that backfilling data will increase the event volume in your organization, which may incur costs. Deleting the old project will not reduce the total ingested volume of events, but this could be helpful for creating an ideal environment for your desired event/property taxonomy.

I hope this information helps. Let me know if you have any other questions.

Best,
Jarren


P.S. Checkout upcoming events and user meetups on our events page.

Thanks for the reply. Sound like no real great options here and we will just have to keep in mind that previous property deletion might be why we don’t see a property on our future events.

elaborate a bit on what you would be hoping to achieve by _deleting_ a property and still ingesting it?

Probably best to do this via an example:

Let’s say we had an event in our system called ‘Article Created’ with a property ‘article_id’. Our system has evolved and this event is no longer sent so we decide to delete `article_id` to get our property count down. A few years after this delete we start sending a new event called ‘Resource Viewed’ and it too has a property `article_id`. Given `article_id` is still a deleted property it will not be ingested or show up in Amplitude drop downs for this new `Article Viewed` event. This is the ‘gotcha’ I am referring to. We need to have knowledge around how deleted properties behave to diagnose this as it is not intuitive, if it even gets noticed at all.

Hi Lance,

Thank you for the clarification. I understand your concern. In the scenario you described, if you delete the `article_id` property and later want to use it again for a different event, you would indeed need to undelete it first. This is because when a property is deleted in Amplitude, it is blocked from future ingestion.

This is a crucial aspect of data governance in Amplitude. The idea is to prevent the accidental reuse of property names, which could lead to confusion or incorrect data analysis. However, I understand that this might not be intuitive in all situations.

If you find yourself in a situation where a deleted property needs to be reused, you can always undelete the property. Once it's undeleted, it will be available for ingestion again. You can find more information on how to do this in our help documentation.

I hope this helps clarify things. Let me know if you have any other questions.

Best,
Jarren


P.S. Checkout upcoming events and user meetups on our events page.

Reply


Terms & ConditionsCookie settings