Skip to main content

Product Experimentation

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences.

  • 121 Topics
  • 205 Replies
121 Topics
Amplitude Admin
Community Manager
Amplitude AdminCommunity Manager
published in Product Experimentation
Welcome to the product experimentation forum!

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences This is where you can ask and get feedback on questions such as: ✅ Am I able to export the data I have in Experiment similar to Analytics?✅ How can I re-assign variants to my users?✅ Why are my users not getting targeted the right variant?✅ What happens if we merge projects in Experiment? Helpful resources: ❓Overview of Amplitude Experiment❓The experimentation workflow: from configuring to troubleshooting your experiment❓Amplitude Experiment key terms ❓Set up mutually-exclusive experiments❓Understand the Experiment Analysis view❓Advanced topics in Experiment: duration estimates, sequential testing for statistical inference, MDE and confidence intervals, randominzation, performance and scaling, sample ratio mismatches❓Amplitude Academy course: Getting Started with ExperimentStill have questions? Post them here! 

5561
T
K
Kick AmaNew Member
asked in Product Experimentation
Best IPTV Service Providers in 2025

LOOKING TO Enjoy Top Quality Streaming With TalktomyTV SERVICESTART YOUR FREE TRIAL : https://talktomytv.com✔ Smooth, Buffer-Free Streaming – No lag, even on live sports and HD channels.✔ Huge Channel Selection – Covers international, sports, movies, and premium networks.✔ Multi-Device Compatibility – Works on Firestick, Android, Smart TVs, and more.✔ 24/7 Support – Quick and responsive customer service.✔ 24-Hour Free Test – Try it before you commit!

992
T
R
Raghav MahajanNew Member
asked in Product Experimentation
Session Replay not showing any sessions

Session replays are not showing if sampling rate is set to 0 locally and enableRemoteconfig is true.We are setting the sampling rate on amplitude to 100% but that doesn’t seem to work.FYI: Works fine when sampling rate is set locally.This is our code  

130
R
D
DataDaze
asked in Product Experimentation
Do users get re-assigned from experiments if they no longer match targeting criteria?

I have two experiments, Experiment_A and Experiment_BI have them set up with Flag Prerequisites https://amplitude.com/docs/feature-experiment/advanced-techniques/flag-prerequisites Experiment_A has targeting like platform = MobileExperiment_B is dependent on Experiment_A = off and has targeting like platform = Desktop Let’s say a multi-platform user is on Mobile at first and they are put into Experiment_A, but on a different session they are on Desktop.  Which outcome is true?The user is put into Experiment_A because of their past assignment The user is put into Experiment_B because they do not currently match Experiment_A, but they do match Experiment_B The user is put into no experiment, because They no longer match A, but have a past variant from A and would not meet the flag prerequisite (Experiment_A = off) to be in Experiment_B  

332
jarren.patao
Community Manager
S
Sameer KhaladkarNew Member
asked in Product Experimentation
iOS SDK upgrade

HelloI am trying to upgrade iOS SDK, after including latest XCFreamwork, I am getting following error while build the APP. “Failed to build module 'AmplitudeSwift'; this SDK is not supported by the compiler (the SDK is built with 'Apple Swift version 5.9.2 (swiftlang-5.9.2.2.56 clang-1500.1.0.2.5)', while this compiler is 'Apple Swift version 6.0.2 effective-5.10 (swiftlang-6.0.2.1.2 clang-1600.0.26.4)'). Please select a toolchain which matches the SDK.”I used XCFramework from following link which gave me the error reported below.https://github.com/amplitude/Amplitude-Swift/releases I also tried building it using Carthage like below by modifying cart filegithub "amplitude/Amplitude-Swift" ~> 1.11.7 I think Amplitude need to give XCFramework with newer version of SWIFT.Please advise.

213
sydney.koh
Team Member
D
DmitryNew Member
asked in Product Experimentation
Onboarding experiment - iOS SDK

Hi,We are integrating the Experiments SDK into our existing iOS app, which already uses Amplitude Analytics.Our experiment involves testing two different onboarding flows, and we want to measure which performs better.Current approach: We initialize the SDK using initializeWithAmplitudeAnalytics. Our configuration sets fetchOnStart(false). We call experiment.start(user). We implement a short delay (displaying a splash screen briefly) to allow sufficient time to retrieve the variant assignment. Could you please help clarify the following points? Is this approach (using local evaluation with a slight delay) suitable for instantly evaluating onboarding experiments? To avoid targeting existing users who've already completed onboarding, we're considering setting a user property (e.g., finished-old-onboarding = true). Is this the recommended method? Regarding app reinstalls: Each reinstall generates a new device ID. For consistent testing, should we manually persist the device ID (f

250
D
N
Nick MurphyNew Member
asked in Product Experimentation
How can I get all users in a flag/variant via the API?

I need to programmatically get the list of users falling under a flag variant. I have tried using the Experiment Management API “Get Variant Inclusions”, but this seems to only return the users stored under “Testing”, rather than percentage-assigned via “Assignment”. How can I do this without having to inefficiently loop through all my users and check their status via Remote Evaluation?

210
N
K
karthickNew Member
asked in Product Experimentation
Issue with @amplitude/storybook-addon-amplitude Angular Storybook Project

HI ,after installing following package@amplitude/storybook-addon-amplitudeam not able this package to my preview.ts file.am getting like below errorplease suggest me to how to resolve this..thanks

200
K
T
Tuyen NguyenNew Member
asked in Product Experimentation
Web experiment does not run

I set up the first web experiment 2 days ago but now I don't receive any data for this experiment. Don’t know why? 

271
T
C
ccixdNew Member
asked in Product Experimentation
question regarding surveys SDK

Dear Support Team, We have added surveys to our product and tested them internally. However, we encountered some bugs and unexpected behavior where surveys cannot be snoozed or dismissed even after users have completed them. We have triple-checked the limits and settings of the surveys we created, but the issue persists. Could you please assist us in resolving this? 

250
C
N
NitishKumarNew Member
asked in Product Experimentation
What is the true conversion(Event level conversion) from "Select Song or Video" to "Play Song or Video", when the Song or Video is selected from "Main Landing Screen". Compare this conversion number by different "Sources" , in the same chart.

Hi, I feel my answer is a little close https://app.amplitude.com/analytics/demo/chart/hjqmr0zx, Is this correct, if not I need little assistance.

9342
H
V
VMGMNew Member
asked in Product Experimentation
Missing events make it impossible to trust funnel chart

Hello, I’m logging an event for every action my user does in onboarding. No actions are skippable and they are always in the same order. However, about 20% of the time there are a few missing events, meaning the user goes from Screen A → Screen B → Screen C but there are only events for Event A → Event C. These missing events are ruining my funnel charts, making it look like my conversion is much lower than it is.What can I do to fix this?

380
V
B
BKMNew Member
asked in Product Experimentation
After signout, Next set of amplitude events till the next login session are capturing under the previous logged out user event stream even after reset

To stop tracking events under the same UserID after logging out of the application used reset method.Analytics.shared().reset()With this change, a new session is initiated by closing the previous session, and the userID appears as "-" in User Properties. However, the events are still being displayed under the same old user's event stream. instead of displaying under the anonymous user. 

661
B
Well-Versed
M
Matthew DoNew Member
asked in Product Experimentation
How to pull the best blog post of the day based off most likes/comments

am working on a social video platform, and i use amplitude for my analytics. i made a segmentation graph charting the amount of blog posts we get uploaded per day, and i am trying to figure out how i can pull the blog post onto my dashboard? like what steps would i need to take or is this even possible

584
Saish Redkar
Expert
R
Raphael Rodrigues SilvaNew Member
asked in Product Experimentation
How to use A/B Tests with Amplitude Experiment for Server-Side Email Campaigns and Unauthenticated Users

I need to do an A/B test so that 50% of people receive a certain email and the other 50% receive another email. The email is sent on the server and I am using the experiment flag to decide which email the person should receive. Through email, these people have a link and when they click, they access the same page on my app. On this page there are some events that I want to monitor and compare the email conversion rate. Even though the person is not authenticated, I can know the email because this information exists in the url.In this case, how should I proceed?I have seen that when I fetch the flags providing the user_id it generates the [Experiment] Assignment event but if I don't provide the user_id it doesn't. What is the impact on the experiment of not having this [Experiment] Assignment generated?Do I need to verify the user in order for the experiment to work properly? I'm asking this because we verify the user after they logged in, but at this point the user is not logged in.I'v

1101
liuaud
Peer Moderator
A
Ahmed zoheiryNew Member
asked in Product Experimentation
How to change experiment allocation/ targeting trigger ?

In the current experimentation UI in amplitude. we have the option to target all user or based on user property, but what we need is to target based on specific event.lets imagine that we have a funnel and Viewed landing page (first event in the funnel)  Amplitude normally assign users on variants before or after this event get fired  Clicked on product carousel  this is the event that we want to start assigning when it happen to include only in the experiment users who have done this event and the variants (treatment / control) should get exposed after firing this event.How to work on implementing solution 2. is there is a way from amplitude ui or it should be done using code ? and if with code, how to do it ?       

1032
C
Team Member
L
Novice
lanceNovice
asked in Product Experimentation
Consistent bucketing across unique amplitude experiments

Hi there!We're working on configuring three experiments, one for each platform (web, ios and android) but we want the bucketing to be consistent across all platforms (i.e., a user that sees variant A on web should also see variant A on iOS and Android). If the rollout %, # of variants and variant weight all remain the same, could we apply the same bucketing salt string to all three experiments and expect consistent bucketing for all of our users in these experiments? 

4154
C
L
Laura JelenichNew Member
asked in Product Experimentation
iOS SDK manually changing the experiment variant on the client side

I’m working on an iOS app using the Amplitude SDK, and multiple developers are testing experiments. However, not all testers can access the Amplitude dashboard to change the assigned variants. To facilitate testing without using the dashboard, I’m looking for a way to override or manually set the variant for specific experiments directly on the client side.Does the iOS SDK currently support something like this?

984
liuaud
Peer Moderator
T
Tom HubenNew Member
asked in Product Experimentation
Delay on clicks

when i added the snippet to my web site. All the clicks starting adding 3 to 5 second delay.I assume i am doing something incorrect? for example a click to open Modal took 5 seconds. Take away Amplitude and instant

800
T
S
Novice
Son JoohoNovice
asked in Product Experimentation
Do Experiment and Session Replay work in Flutter platform?

Hi Experts, I would like to have experts’ advice how Experiment and Session Replay works in Flutter Platform.I couldn’t find any doc and SDK on this topic.Please give your advice. Cheers,Jooho

2332
L
J
Jiseong KimNew Member
asked in Product Experimentation
Why number of active user is same with new User

"Despite having retention, why are the numbers of new users and active users coming out the same?" 

510
J
M
milleryoonNew Member
asked in Product Experimentation
Visual Editor doesn't work

I was making an Experiment, But I can’t run visual editor.When I run visual editor, I met a screen like below.I supposed that it caused by Arc Browser, but I faced same screen at Chrome Browser.Can you help me to do experiment? Thanks.

940
M
L
Novice
lanceNovice
asked in Product Experimentation
Exposure tracking provider for NodeJS

Hi! We currently utilize the NodeJS SDK to handle server-side analytics and experiments. We were wondering if there are any plans to build an exposure tracking provider for NodeJS, similar to the JS SDK implementation

3875
I
J
Japser YooNew Member
asked in Product Experimentation
How to use reverse proxy for Experiment SDK

We have integrated the Experiment SDK (JavaScript) into our React application and are currently utilizing both the experiment and feature-flag functionalities.To fetch flags data, we initialize the SDK as follows:```javascriptthis.experiment = Experiment.initializeWithAmplitudeAnalytics(apiKey);await this.experiment.start();```Then, we retrieve the data using this.experiment.variant(variantKey).However, we are encountering an issue where the flags data is blocked by ad blockers or in certain regions, preventing us from accessing the data. To resolve this, we attempted to use a reverse proxy URL like below:```javascriptExperiment.initializeWithAmplitudeAnalytics(apiKey, { flagServerUrl: "proxy-url" });```Unfortunately, the SDK automatically appends `sdk/v2/flags` to each request, which makes it difficult to bypass the blockage. Ultimately, we are trying to achieve a setup similar to how we apply a reverse proxy to Amplitude event requests, where we can set a reverse proxy URL for flags

1500
J
Y
Yiji HuangNew Member
asked in Product Experimentation
Duplicated events with different ip addresses

Hi here, We find that sometimes we will receive duplicated events (same event id, same session id). Looks like the ips are different. The client time are the same but one client upload time has huge lag (several days to even two weeks)Is this expected behavior? It brings us some trouble counting user events, and we found about 7% of the events will be duplicated

1360
Y

Recent Tips & Tricks

Terms & ConditionsCookie settings