Skip to main content

Product Experimentation

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences.

  • 107 Topics
  • 184 Replies
107 Topics
Amplitude Admin
Community Manager
Amplitude AdminCommunity Manager
published in Product Experimentation
Welcome to the product experimentation forum!

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences This is where you can ask and get feedback on questions such as: ✅ Am I able to export the data I have in Experiment similar to Analytics?✅ How can I re-assign variants to my users?✅ Why are my users not getting targeted the right variant?✅ What happens if we merge projects in Experiment? Helpful resources: ❓Overview of Amplitude Experiment❓The experimentation workflow: from configuring to troubleshooting your experiment❓Amplitude Experiment key terms ❓Set up mutually-exclusive experiments❓Understand the Experiment Analysis view❓Advanced topics in Experiment: duration estimates, sequential testing for statistical inference, MDE and confidence intervals, randominzation, performance and scaling, sample ratio mismatches❓Amplitude Academy course: Getting Started with ExperimentStill have questions? Post them here! 

4941
T
1 year ago
A
Ahmed zoheiryNew Member
asked in Product Experimentation
How to change experiment allocation/ targeting trigger ?

In the current experimentation UI in amplitude. we have the option to target all user or based on user property, but what we need is to target based on specific event.lets imagine that we have a funnel and Viewed landing page (first event in the funnel)  Amplitude normally assign users on variants before or after this event get fired  Clicked on product carousel  this is the event that we want to start assigning when it happen to include only in the experiment users who have done this event and the variants (treatment / control) should get exposed after firing this event.How to work on implementing solution 2. is there is a way from amplitude ui or it should be done using code ? and if with code, how to do it ?       

81
B
Conversationalist
1 day ago
T
Tom HubenNew Member
asked in Product Experimentation
Delay on clicks

when i added the snippet to my web site. All the clicks starting adding 3 to 5 second delay.I assume i am doing something incorrect? for example a click to open Modal took 5 seconds. Take away Amplitude and instant

90
T
1 day ago
S
Novice
Son JoohoNovice
asked in Product Experimentation
Do Experiment and Session Replay work in Flutter platform?

Hi Experts, I would like to have experts’ advice how Experiment and Session Replay works in Flutter Platform.I couldn’t find any doc and SDK on this topic.Please give your advice. Cheers,Jooho

1002
L
6 days ago
J
Jiseong KimNew Member
asked in Product Experimentation
Why number of active user is same with new User

"Despite having retention, why are the numbers of new users and active users coming out the same?" 

40
J
15 days ago
M
milleryoonNew Member
asked in Product Experimentation
Visual Editor doesn't work

I was making an Experiment, But I can’t run visual editor.When I run visual editor, I met a screen like below.I supposed that it caused by Arc Browser, but I faced same screen at Chrome Browser.Can you help me to do experiment? Thanks.

190
M
20 days ago
L
Novice
lanceNovice
asked in Product Experimentation
Exposure tracking provider for NodeJS

Hi! We currently utilize the NodeJS SDK to handle server-side analytics and experiments. We were wondering if there are any plans to build an exposure tracking provider for NodeJS, similar to the JS SDK implementation

2345
I
1 month ago
J
Japser YooNew Member
asked in Product Experimentation
How to use reverse proxy for Experiment SDK

We have integrated the Experiment SDK (JavaScript) into our React application and are currently utilizing both the experiment and feature-flag functionalities.To fetch flags data, we initialize the SDK as follows:```javascriptthis.experiment = Experiment.initializeWithAmplitudeAnalytics(apiKey);await this.experiment.start();```Then, we retrieve the data using this.experiment.variant(variantKey).However, we are encountering an issue where the flags data is blocked by ad blockers or in certain regions, preventing us from accessing the data. To resolve this, we attempted to use a reverse proxy URL like below:```javascriptExperiment.initializeWithAmplitudeAnalytics(apiKey, { flagServerUrl: "proxy-url" });```Unfortunately, the SDK automatically appends `sdk/v2/flags` to each request, which makes it difficult to bypass the blockage. Ultimately, we are trying to achieve a setup similar to how we apply a reverse proxy to Amplitude event requests, where we can set a reverse proxy URL for flags

530
J
1 month ago
Y
Yiji HuangNew Member
asked in Product Experimentation
Duplicated events with different ip addresses

Hi here, We find that sometimes we will receive duplicated events (same event id, same session id). Looks like the ips are different. The client time are the same but one client upload time has huge lag (several days to even two weeks)Is this expected behavior? It brings us some trouble counting user events, and we found about 7% of the events will be duplicated

880
Y
1 month ago
N
Niharika ParabNew Member
asked in Product Experimentation
Multiple First Name/Last Name Values in Amplitude

We are experiencing an issue where Amplitude is showing multiple values for first names, last names, etc., although we are sending single values (first name, last name, etc.) through Rudderstack from our side.Assumption: We believe Amplitude might be retaining previous login details and appending them to the new data. The support replied by saying:Amplitude's user property is sticky, so once the custom user property is set for users, it will associate with all subsequent events unless explicitly unsetting them.Please help us out by letting us know how to unset users.

260
N
2 months ago
M
Michael McNicholasNew Member
asked in Product Experimentation
Does anyone have any experience using Amplitude Experiment and Google Tag Manager together?

Does anyone have any experience using Amplitude Experiment and Google Tag Manager together? Our goal is to have Amplitude Experiment set-up like a typical client side tool since it would be difficult to integrate our servers-side code due to some vendor issues. (long-term addressable, but not in the near term) So the basic thought is1> Fire Amplitude Analytics in GTM with the  “Amplitude Analytics Browser SDK” GTM community template. 2> Fire a script tag to initialize the Expierment. <script src=“https://unpkg.com/@amplitude/experiment-js-client@1.9.0/dist/experiment.umd.js”></script> 3> Create a Google Tag Manager Variable which is a custom javascript variable and captures which the experiments and variations the visitor is in. 4> Use those values in the GTM variables to execute or not, the code in our experiments. This is not pretty.  But I haven’t come across much content on Amplitude Experiment, since it is a newer product and not for the lower tier plans. a

3393
C
Team Member
2 months ago
N
Navneet YadavNew Member
asked in Product Experimentation
Amplitude Logger [Error]: Unexpected token 'F', "Fixed Resp"... is not valid JSON

In the given screenshot I’m using domain proxy /amplitude-ts URL is redirecting to https://api2.amplitude.com/2/httpapi but /amplitude-ts is giving 404 as you can see in the screenshot.  

250
N
2 months ago
H
Heejae LeeNew Member
asked in Product Experimentation
The 400 response code from Amplitude when Adjust sends a callback

We recently integrated Adjust, and Adjust is successfully sending callbacks to Amplitude. However, it appears that Amplitude is returning a 400 response. Could you provide any suggestions on how to resolve this issue?

420
H
2 months ago
E
eve weinbergNew Member
asked in Product Experimentation
What happens if i leave my javascript snippet after I end my A/B test?

Hello, I just ended by A/B test and decided to ‘roll back’ - not use the variant I tested.However, i can’t remove the javascript from the page header right now due to other reasons. Is this ok?Will any users be affected, or should I just remove the js script when I can and it’s not really that harmful.

583
eddie.gaona
Team Member
3 months ago
E
eve weinbergNew Member
asked in Product Experimentation
Why am i getting a 'URL is required' warning, when building my experiment

I can’t get past this step, when making my experiment: 

640
E
3 months ago
T
Tobias Bold TechNew Member
asked in Product Experimentation
Experiments Api - "Create" endpoint returns 500

Hey, when I try to create an experiment using the API I just get Error code 500 "Internal Server Error" as response. This happens I call the API from my retool application, but also when using curl from my local machine.My networking tab shows that I send this request:"request": {    "url": "https://experiment.amplitude.com/api/1/experiments",    "method": "POST",    "body": "{\"key\":\"test-api-call\",\"name\":\"Test\",\"description\":\"Try to create an experiment using the API\",\"projectId\":******}",    "headers": {      "Content-Type": "application/json",      "User-Agent": "Retool/2.0 (+https://docs.tryretool.com/docs/apis)",      "Accept": "application/json",      "Authorization": "---sanitized---",      "ot-baggage-requestId": "undefined",      "x-datadog-trace-id": "5857767016106236026",      "x-datadog-parent-id": "5648445407639375651",      "x-datadog-sampling-priority": "0",      "x-datadog-tags": "_dd.p.dm=-1",      "traceparent": "00-0000000000000000514af810bc03187a-4e634

640
T
3 months ago
D
DevNew Member
asked in Product Experimentation
User Property has not been received

I have created a user property called client and have hit it multiple times using the JavaScript SDK (Server) and also shows flag is on or off for the filters but in the feature flag it is still showing that property has not been received. User Property - client The code for the user properties is here:const user = { user_properties: { client: 'uber', }}const variants = await experiment.fetchV2(user); 

640
D
3 months ago
K
Kamal MittalNew Member
asked in Product Experimentation
In React Native Android app has issue with gradle 8.2.1

Hi I am using amplitude react native latest sdk @amplitude/analytics-react-native in react native. It is working fine with iOS, but when i try to build with Android, it is return error like as below:A problem occurred configuring project ':amplitude_analytics-react-native'.> defaultConfig contains custom BuildConfig fields, but the feature is disabled.  To enable the feature, add the following to your module-level build.gradle:  `android.buildFeatures.buildConfig true` Can you guys suggest how we can resolve that issue?Here, I tried to add this buildConfig flag but the issue is still there.Thanks 

4563
K
3 months ago
R
ramonagoodwinNew Member
asked in Product Experimentation
lotto game sites

Best Capabilities of Most Online Lottery ToolsLotteries have been around for centuries, and they show no indications of disappearing any time soon. Online lotteries are ending up being more and more popular every day. It is due to the fact that they offer a number of benefits over standard lotteries. This post will talk about the top attributes of most online lottery systems. We will certainly additionally supply some ideas on selecting the right system for you. For instance, your selected site ought to be certified. So, without further ado, let's get started. Among the very best features of online lottery systems is that they supply cost-free trials. You can try out the site before you commit to it. In addition, it is a great method to see if the platform is right for you. You can additionally utilize the trial period to find out more info how to use the platform and obtain a feel for it. Most cost-free tests last for a week or 2, so you will certainly have lots of time to decide if t

740
R
4 months ago
C
CrisNew Member
asked in Product Experimentation
A/B testing the price - how do we set up the user property?

We have a funnel consisting of 14 steps and we’d like to do A/B testing for the funnel to test the price, which assigns on step 1 (50/50, randomly picked). What we don’t understand is how do we add user properties for the price variation A and B? What we fail to understand is how to properly add the properties to the users? The piece of code looks like this:amplitude.logEvent('Price_Variation', { price: value }); How do we set it up properly?

780
C
4 months ago
A
Apurva ChobitkarNew Member
asked in Product Experimentation
time tracing for long session

I am not able to see the duration of event if its more than 12 hours or event started at yesterday and ended today.Amplitude capturing start & end session for above mentioned scenarios but not showing time line(spent hours.)

550
A
4 months ago
A
AaronMNew Member
asked in Product Experimentation
Tracking conversion time in an experiment

Hi!I’m running an experiment and want to see if there is a significant difference between my control and treatment group in the time it takes for users to perform a specific action. When creating a funnel graph I see that the average time to convert is a pre-calculated metric. Is there a way to track this metric in my experiment? Or is there any other way to add conversion time between two events as a custom metric to my experiment?

1092
A
4 months ago
S
Shubham WadhwaNew Member
asked in Product Experimentation
how to filer user id by odd and even logic for an event

how to filer user id by odd and even logic for an event

1061
S
5 months ago
A
Astara InnovationNew Member
asked in Product Experimentation
Does flutter works in developer mode?

Hi, I have been trying to setup the Flutter client, and right now nothing is working as expected. I have followed the examples in the documentation, even copying code from your GitHub example flutter app, but it looks like nothing is working.I’m struggling to get the Amplitude events logged in your EU instance (I’ve tried with USA and the same happens). Thus, I’m a bit lost. I’m trying to debug everything running flutter locally, could this be the problem? Does Amplitude SDK only work when built for distribution? If the answer is yes, how can I debug it?Just to be clear, I have setup the client to have a threshold of 1 event, and submit them every 5 seconds, but I cannot see anything on the console or any event in my account. Any help will be appreciated.

690
A
5 months ago
M
Martin ENew Member
asked in Product Experimentation
Experiment Assignment and Exposure discrepancies

We have recently integrated Amplitude analytics and have more recently been testing Amplitude Experiments on our website. For our first test experiment, the ratio between assignment and exposure is roughly 2:1, so there is double the amount of assignments as there is exposures and we don’t know why that is?We also checked in Pathfinder and could see the following problem: Sometimes exposures seem to occur before the assignment event, is that an issue?  We included the Amplitude browser and experiment sdks via the npm modules @amplitude/experiment-js-client and @amplitude/analytics-browser We then initialize the experiment sdk like so: private _experiment = Experiment.initializeWithAmplitudeAnalytics( environment.amplitudeDeploymentKey, { fetchOnStart: true } );as well as the browser SDK:amplitude.init(environment.amplitudeApiKey, { defaultTracking: true });and here is the code that checks the variant with additional experiment.start() call just in case: hasVariantValue( exp

1150
M
5 months ago

Recent Tips & Tricks

Terms & ConditionsCookie settings