Hi all!
Questions:
- What is the expected variance between reporting in Amplitude vs source of truth? I would like to confidently state that x% to y% of variance is normal and that any variances outside of those parameters should be investigated.
- How can I prove that such variance is the best recommendation?
- What could be the reasons for the variance and how to prove it?
Context:
We are an e-commerce website with a non-authenticated experience (ie, visitors or purchasing customers are not required to login). Some years ago, there was an audit to identify the variances between the orders completed reported by our operational system (source of truth) and Amplitude and there was a ~10% variance. A couple of years ago, this same audit was performed and the variance had increased to ~15%; more recently, it is suspected to have climbed up to ~18%. One explanation could be the increase of ad blocker usage which seems to be aligned with the incremental variation observed throughout the years but I can’t think of a way to prove that this is the most likely reason.
Thank you!!!
