Issue with piping data from Snowflake to Amplitude pipeline via Hightouch
Background
We pipe events from Snowflake into Amplitude via a Hightouch sync.
Sometimes the syncs work perfectly. Meaning Hightouch queries Snowflake and returns the exactly correct number of new events. Amplitude successfully receives the events.
Sometimes the syncs work imperfectly but are close. Meaning Hightouch queries Snowflake and returns the exactly correct number of new events. Amplitude almost receives all the events but is short a few in almost every daily sync.
Sometimes the syncs completely fail. Meaning Hightouch queries Snowflake and returns the exactly correct number of new events but Amplitude fails to receive any events.
Issue
In order for our stakeholders to use this data reliably, we need the syncs to work correctly every time. Has anyone had this issue and can you please help us debug this issue? Because Hightouch’s numbers always match the counts in Snowflake, we believe this is an issue on Amplitude’s side of failing to receive all the events that were sent.
