Skip to main content

I am new to Amplitude, I have configured a source to GCS for event data import. It imported first time when I have configured, but later whenI am pushing another files to GCP bucket with same extension and format it is not reading that file.

Hi @anandraprinc welcome to the community! Happy to help troubleshoot this. First off, when importing data from a Google Cloud Storage, we have some documentation here: https://www.docs.developers.amplitude.com/data/sources/google-cloud-storage/ 
 
Would you be able confirm that all pre-requisites are met for this integration and also where you might be seeing the sync behavior that you've described? 
 
After getting a bit more information around the behavior you're experiencing, I may be able to help further.


Hello @Esther Trapadoux Thanks for your response.

I have followed the same documentation given here  https://www.docs.developers.amplitude.com/data/sources/google-cloud-storage/ 

I have done all the required setup and It also picked up data first time. 

But when the new files comes to GCP Bucket , It did not picked up.

I am not understanding the configuration given here https://www.docs.developers.amplitude.com/data/sources/google-cloud-storage/#storage-organization-requirements I think I might have missed something.

Thanks

Awaiting your response


I am running into the same issue. I followed the storage organization requirements to the best that I could. My files are in my bucket organised as follows:


<bucket-name>/amplitude/2023/04/19/11/whatthetest.json

My folder is amplitude. so that is the prefix. But the data is still not getting imported.


Hi @anandraprinc and @mukkkkkk got it, and thanks for following the steps in the documentation. It could be a matter of event mapping as well, have you also seen the Guided Converter Creation section to see if there are any steps that may have been missed during setup: https://www.docs.developers.amplitude.com/data/sources/google-cloud-storage/?h=gc#guided-converter-creation 
 
A few more things to check otherwise:
 
Make sure you’ve taken care of some prerequisites.

  • Make sure you have admin permissions for your Amplitude org.
  • Make sure you have a GCS service account with the appropriate permissions. Learn more.
  • Make sure that a project exists to receive the data. If not, create a new project.
  • Make sure your GCS bucket has data files ready to be ingested. They must conform to the mappings that you outline in your converter file.
  • Make sure the data in your GCS bucket follows the format outlined in Amplitude's HTTP API v2 spec.

Thanks @Esther Trapadoux . It worked . I was doing mistake while storage organization requirements configuration. 

Thanks you so much.

 


@anandraprinc YAY!!!! Glad you got it to work. 🎉


Worked for me as well. It just took a day or so for it to start working. Not sure why that was the case but I just had to wait


Reply