Solved

Client Event Timestamp vs Client Upload Timestamp

  • 5 August 2021
  • 4 replies
  • 3438 views

Hello,

Looking at various Event Streams of my users, I’ve noticed that the Client Event Time & Client Upload Time have various time spans between them. Sometimes they are seconds apart, sometimes hours, sometimes days. Seconds or hours isn’t necessarily a big deal, but events uploading days later is causing real problems for us.

 

I understand that the Amplitude SDK has a batching mechanism so I don’t expect the timestamps to match exactly.

 

My question is there a way in amplitude to graph the skew of client event and client upload time? I want to get an idea of on average how far behind is the client upload from the client event time.

 

I found this old blog post with a chart that I want to recreate with our data (https://amplitude.com/blog/dont-trust-client-data) but the link at the bottom of the post is broken (https://amplitude.com/docs#timestamps).

 

Attached is a screenshot of an event that was uploaded days later:

 

Does anyone have ideas or recommendations on how to visualize this? Thank you!

icon

Best answer by tracy.guo 6 August 2021, 03:33

View original

4 replies

Userlevel 7
Badge +10

Hey @bitfl1p ,

Afaik, there isn’t a native way of accessing the client_event_time and client_upload_time as properties while graphing in Amplitude. So it might be tough to visualize the skew between those two in any of the charts.

The only solution I can think of is having access to your raw event data in a data warehouse and extracting the difference between those 2 timestamps in SQL. 

You can read the following post for ways to enable SQL with your event data -

 

I’m not sure if it’s normal with the SDK implementation, but I feel the difference of almost 3 days in the sample you shared is quite a bit . You might want to reach out to the platform team if this is causing issues.

Hope this helps!

 

Userlevel 4
Badge +3

Hi @bitfl1p ! Thanks for your question!

First, here’s the updated link for the definitions of each timestamp: https://help.amplitude.com/hc/en-us/articles/229313067#Raw-Data-Fields

The Client Event Time and Client Upload Time timestamps are pulled from the user’s device and Client Upload Time will depend on the user’s internet connection. If the user loses connection or is operating on airplane mode, the events will be stored until they re-establish connection. For most (if not all) Amplitude SDKs, the cadence at which Amplitude SDKs upload events is either every 30 seconds or every 30 events by default. This option can also be configured if you’d like to increase the rate at which events are sent. More information on these options can be found here: JavaScript SDK | iOS SDK | Android SDKLook out for options named eventUploadPeriodMillis, eventUploadThreshold, setEventUploadMaxBatchSize and eventUploadPeriodSeconds etc. 

As @Saish Redkar mentioned, querying on client event and client upload time is not available in Amplitude, but if you have a paid plan, you can  have the option to export your data to Snowflake, Amazon S3, or Google Cloud Storage, you can query on these values using SQL. 

Hope this information helps! Let us know if you have any questions!

@tracy.guo Hello, I would like to ask when we’re actually making charts in Amplitude which timestamp out of the ones I listed below is being used to plot the data points? 


client_event_time
client_upload_time
server_received_time
server_upload_time
event_time

 

 

Userlevel 7
Badge +10

Pulling this from the doc -  “Daily exported files are based on server_upload_time and all dashboards are based on event_time. We recommend that queries on raw data use event_time.”

Reply