Skip to main content

Product Experimentation

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences.

  • 107 Topics
  • 184 Replies
107 Topics
P
Conversationalist
peckerbidsConversationalist
asked in Product Experimentation
Switch between experiment and flag

Hello,If we switch an experiment to a flag and then back to an experiment, does the data get lost from the first experiment?

1582
P
Conversationalist
6 months ago
D
Conversationalist
dargeliesConversationalist
asked in Product Experimentation
Experiment Flag Slack Integration / Webhook

Hey there,Recently we had an incident that was caused by a feature flag being toggled. The incident commander was not aware of the feature flag being toggled to understand what changed in the system. Typically, we push system changes like this to Slack with automations to make it more visible. I noticed there is a Slack integration for some Amplitude features and webhooks for custom monitors, but didn’t see this. What we’re looking for is an integration or webhook that will notify us when a feature flag state is changed. Is there any way to do something like this? Thanks

2243
jarren.patao
Community Manager
6 months ago
L
Helpful Hero
liukaichiHelpful Hero
asked in Product Experimentation
Question about using Bucketing Slat

I know this is documented quite well here: https://www.docs.developers.amplitude.com/experiment/general/evaluation/implementation/#activationBut this experiment is very important, so I just want to make sure I have this right. We have 3 flags that we want to ensure get to the same users. If we use the same Bucketing Salt for each of the flags, then if we keep the segments and rollout numbers the same, then the same users should receive all 3 flags. Is this correct? Are there any pitfalls that we should be aware of when doing something like this? Thanks in advance.

2022
T
Team Member
7 months ago
A
Alexandra ResslerNew Member
asked in Product Experimentation
Can Amplitude do network experimentation?

In my product, users can send each other messages.  If we want to experiment on how messaging works, we can’t run a typical A/B experiment because we need both the sender & recipient to be in the same variant.  One standard way of addressing this is running experiments in different geographic locations, with the assumption that each location has a mostly-closed network.  This hasn’t worked for us, unfortunately, as our separate locations aren’t very similar in the first place so we can’t compare them.  Some companies have the ability to expose experiments based on network clustering.  Is that a feature that Amplitude has?Side note: I have bet on with my engineer that the answer is no. =D  So you won’t disappoint me if that’s the case!The best alternative I can think of is to use BigQuery to manually separate users into clusters, normalize on relevant attributes, try to pair similar clusters to assign to different variants, and upload those assignments as a user property, but I have

2563
J
Experienced
7 months ago
M
Micah BlackburnNew Member
asked in Product Experimentation
New Developer Here With Some UTM Questions

Hello, I am developing amplitude with my ecommerce website smartbettor.ai. I have currently implemented amplitude to track events using the java script sdk. I track payments with stripe and want to track sign ups. After a successful stripe checkout users are redirected to a registration form. The only way to complete sign up is to complete this registration form so I have our ‘sign up’ event being the submission of this form. There are 2 problems with this currently.  Numbers between what stripe says the customer sign ups are and what amplitude is saying are very different. Why would amplitude not be able to track every time this event happens and have significantly less than what stripe says? We want to track utm source. We are starting ad campaigns and want to know what website the user came from. I added these lines which I assumed would track: includeUtm: true, // Track UTM parameters includeReferrer: true, // Track referrer includeGclid: true, // Track Google Click Identifier incl

1130
M
7 months ago
M
Mohammad GhasemiNew Member
asked in Product Experimentation
attribution currently is unavailable

when i want to use attribution, it shows me : attribution currently is unavailablehow can i do for that?

840
M
9 months ago
L
Liron BurrackNew Member
asked in Product Experimentation
amplitude/experiment-js-client SDK vs amplitude-js SDK?

Does amplitude-js npm package includes the amplitude/experiment-js SDK inside it?what is the difference between them? 

930
L
9 months ago
Rune Andersen
Well-Versed
Rune AndersenWell-Versed
asked in Product Experimentation
Any Payload size limit in a variant ?

Questions is:Are there a limit for how large (line, characters, mb) a JSON Payload can be as a part of Feature flag.For a client im implementing Experiment/Feature Flags for IOS and Android. One of the Feature flag planned are to be used as a “All Users 100%” with a payload with info about shops etc. Example of this is an object of all shops, and info for each shop.Intention is to make only one “App Data” feature flag that will grow overtime.For now it’s only 74 lines, and est 1200 characters. But it’s intended to grow over time.  

1380
Rune Andersen
Well-Versed
9 months ago
F
Federico HuneeusNew Member
asked in Product Experimentation
Trouble tracking exposure events. Assignment events work, exposure doesn't

Hey guys, can someone give me a hand. I’ve followed pretty much every step in the guides to track exposure, but nothing gets registered on my dashboard. What I’ve done is loading the experiment.start() and then running the line const variant = experiment.variant(‘experiment-one’) [experiment-one is the flag], but it doesn’t work.i tried both the Amplitude and Segment integration, but no luck. This is one way I tried (with Segment → )export async function loadScript() { const promise = new Promise((resolve, reject) => { const script = document.createElement("script"); script.type = "text/javascript"; // script.async = true; // script.defer = true; script.text = ` !function(){var analytics=window.analytics=window.analytics||[];if(!analytics.initialize)if(analytics.invoked)window.console&&console.error&&console.error("Segment snippet included twice.");else{analytics.invoked=!0;analytics.methods=["trackSubmit","trackClick","trackLink","trackForm","page

1360
F
9 months ago
D
Daniel BarriaNew Member
asked in Product Experimentation
Evaluation REST API not working

Hey everyone! Here is a situation that I am facing beign new using amplitude. We’re trying to use amplitude Evaluation REST API to get users’ variant for a feature flag we’ve created. All seems to be alright, deployment key is created, feature flag is created and active but for some reason when I try to get user variant it always returns an empty objecthttps://www.docs.developers.amplitude.com/experiment/apis/evaluation-api/#exampleI’ve tested from that page and also doing a cURL. Any help would be awesome! Thanks

2753
D
10 months ago
N
NitishKumarNew Member
asked in Product Experimentation
In the funnel of "Play Song or Video" to "Purchase Song or Video", what percentage of users do "Play Song or Video" 3 times, before conversion (Purchase Song or Video)

I have created this chart https://app.amplitude.com/analytics/demo/chart/6yds5ago/edit/rz913e1d, but want to know if this is correct, please help.

3901
Saish Redkar
Expert
10 months ago
N
NitishKumarNew Member
asked in Product Experimentation
 What are the Average and Median times taken to do "Send Message" after "User Sign Up"? What is the distribution chart of "Time Taken" for this conversion?

This is in B2B Chat Analytics, I tried but don’t know if my answer is correct. Hope someone can help me with this. https://app.amplitude.com/analytics/demo/chart/hjqmr0zx/edit/cbmsp21w

4471
Saish Redkar
Expert
10 months ago
N
NitishKumarNew Member
asked in Product Experimentation
 What are the Average and Median times taken to do "Send Message" after "User Sign Up"? What is the distribution chart of "Time Taken" for this conversion

This is in B2B Chat Analytics, I tried but don’t know if my answer is correct. Hope someone can help me with this. https://app.amplitude.com/analytics/demo/chart/hjqmr0zx/edit/cbmsp21w

4531
Saish Redkar
Expert
10 months ago
P
Prashant PadhyNew Member
asked in Product Experimentation
What is the conversion from "Add Item to Wishlist" to "Complete Purchase" completed within 1 day, from Japan users, for "Item_ID" = 100, in June?

E - COMMERCE TOOL

1840
P
10 months ago
N
NitishKumarNew Member
asked in Product Experimentation
What is the true conversion(Event level conversion) from "Select Song or Video" to "Play Song or Video", when the Song or Video is selected from "Main Landing Screen". Compare this conversion number by different "Sources" , in the same chart.

Hi, I feel my answer is a little close https://app.amplitude.com/analytics/demo/chart/hjqmr0zx, Is this correct, if not I need little assistance.

4721
Saish Redkar
Expert
10 months ago
N
NitishKumarNew Member
asked in Product Experimentation
What is the conversion from "Play Song or Video '' to "Purchase Song or Video '' , if the user follows a playlist (Follow Playlist) within 24 hours of first use?

Hi, I tried to do it this way https://app.amplitude.com/analytics/demo/chart/new/3w9uxdfi, the problem I am not able to figure out this part,  “if the user follows a playlist (Follow Playlist) within 24 hours of first use”. Hoping that the chart above is the correct answer for the question (if not please help me on what to change). How to implement this 24 hours part?

4953
N
10 months ago
D
Dave PorterNew Member
asked in Product Experimentation
Separate Amplitude Experiment deployment or not for SSR server?

BackgroundThe examples in the server side rendering guide for Amplitude Experiment have…a “server” deployment key for initialising the node SDK used to fetch initial variants, see https://www.docs.developers.amplitude.com/experiment/guides/server-side-rendering/#initialize-the-server-sdka “client” deployment key for initialising the javascript SDK ExperimentClient with those initial variants, also used on the server. See first “if” branch here https://www.docs.developers.amplitude.com/experiment/guides/server-side-rendering/#initialize-the-client-sdk-on-render. The “client” deployment key matches that used to initialise the ExperimentClient in the browser in the second “if” branch.However, the Amplitude Experiment Node SSR Demo app here https://github.com/amplitude/experiment-node-server/tree/main/packages/ssr-demo uses an identical “client” deployment key for all three. (https://github.com/amplitude/experiment-node-server/blob/main/packages/ssr-demo/lib/experiment.ts and https://githu

1720
D
11 months ago
C
Casey HNew Member
asked in Product Experimentation
Redirect A/B test with Amplitude

Hi, Has anyone ever succeeded in settng up a redirect A/B test using Amplitude Experiment? We have been looking for instructions to set up redirect test but couldn’t find any. 

1750
C
1 year ago
V
Valentin IvanovNew Member
asked in Product Experimentation
Missing User ID in Start Session - GTM

Hi everyone, for my project I use GTM template “Amplitude Analytics Browser SDK” to initialise the Amplitude but I set userId and identify user in the project itself(by calling the newly created amplitude object), because GTM doesn’t have access to the information needed. Also I set timer with 30 retries in case that object is not initialised yet.Now what happens is that for majority of users but not all(around 70%) I can’t see information from identify, as well as User ID, in the user properties of Start Session event . However in the future events of same session the information I send with identify method is present but User ID is still missing. This causes the table in User Look-Up to show Amplitude ID instead of user Id for these events(but if you open the page of the user, he has correct Id). This is applicable only for the 70% of users I mentioned at the beginning, the rest are receiving identify data and userId in Start Session, so further events have userId set.Does anyone kno

3070
V
1 year ago
T
Thomas GroendalNew Member
asked in Product Experimentation
How do I correlate time in platform to retention?

We’re working on a tool that is a platform for investigations in a virtual machine. We have a thesis that more time spent in the machine itself is going to be indicative of happy customers, as we meet a broader range of their investigative tasks in platform. I want to test this by seeing if time in platform is predictive of retention. I can measure time in platform on average, but I can’t figure out how to link it back to retention.I’m also open to criticism of the idea in general. 

1801
Saish Redkar
Expert
1 year ago
N
Novita OliveraNew Member
asked in Product Experimentation
Change experiment key name

Can we change the experiment key name for existing Amplitude experiment?

1490
N
1 year ago
L
Novice
lanceNovice
asked in Product Experimentation
Session metrics in experiments

Is it possible to track session metrics (e.g., Time spent per user, Avg length, etc.) in an Experiment Results chart? In the past we’ve tried using User Session charts but we aren’t cleanly able to create separate treatment and control segments and it does not have the same functionality in leveraging the only after Exposure event… feature in an Experiment Results chart 

1310
L
Novice
1 year ago
J
Jacky LaiNew Member
asked in Product Experimentation
What is the best way to differentiate the path in funnel analysis?

I have one button, and one dropdown on the same page. There will be 3 events for each:1. button shown → button clicked → target page2. dropdown shown → dropdown clicked → target pageMy question is,Q1.what’s the granularity of time interval amplitude is using?Let’s say, if the granularity is 1 minute, within 1 minutebutton shown → dropdown shown → button clicked → dropdown clicked → target pageSince all the events happen in one minute, how to tell which scenarios (button or dropdown) completed the flow?Thanks.

1821
dangrainger
Expert
1 year ago
M
Manuel FenklNew Member
asked in Product Experimentation
Amplitude Integration via Kotlin Multiplatform (cross-platform)

Hey folks, we are currently on the way towards migration from GA towards Amplitude. We have one iOS and one Android App, and we want to take this opportunity to integrate Amplitude via Kotlin Multiplatform Mobile (KMM).Goal: One business logic for the whole analytics integration in order to reduce the overhead of maintaining it on 2 platform and - most important - consistency among the platforms.Here is a video from GDG in Melbourne last week, talking about that in a more general way:  Does someone here in the community maby has experiences with such an integration? If yes, I would be very happy to get some insights about it. How did go, pitfalls, would you do it again, etc :)Thanks a lot!Best, Manuel 

5231
M
1 year ago
R
Rohin MadhavanNew Member
asked in Product Experimentation
Integrating Appsflyer attribution data to Amplitude

We have been trying to send the Appsflyer integration data to Amplitude but we are unable to see the events on the Amplitude dashboard. We have followed documentations provided by both Appsflyer and Amplitude but we do have a few doubts which might be causing the integration to fail.We are using Amplitude-swift for getting the user analytics and initially we were to collecting the IDFA value from the user but for this integration to work I assume we require IDFA from both Amplitude and Appsflyer so we implemented the IDFA collection plugin as we are not able to use adSupportBlock method mentioned in the documentation. Would this be enough to get the IDFA collected? Could you clarify how to match the events in both Appsflyer and Amplitude? Any two events with the same IDFA value would suffice or should we map the events to amplitude while setting this up in Appsflyer?

2430
R
1 year ago

Recent Tips & Tricks

Terms & ConditionsCookie settings