Skip to main content

User Groups
Events
Documentation
Blog
Product Updates
Academy

Homepage
Using Amplitude
Product Experimentation

Product Experimentation

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences.

110 Topics
194 Replies

When you subscribe we will email you when there is a new topic in this category

110 Topics

AaronMNew Member

asked in Product Experimentation

Tracking conversion time in an experiment

Hi!I’m running an experiment and want to see if there is a significant difference between my control and treatment group in the time it takes for users to perform a specific action. When creating a funnel graph I see that the average time to convert is a pre-calculated metric. Is there a way to track this metric in my experiment? Or is there any other way to add conversion time between two events as a custom metric to my experiment?

Astara InnovationNew Member

asked in Product Experimentation

Does flutter works in developer mode?

Hi, I have been trying to setup the Flutter client, and right now nothing is working as expected. I have followed the examples in the documentation, even copying code from your GitHub example flutter app, but it looks like nothing is working.I’m struggling to get the Amplitude events logged in your EU instance (I’ve tried with USA and the same happens). Thus, I’m a bit lost. I’m trying to debug everything running flutter locally, could this be the problem? Does Amplitude SDK only work when built for distribution? If the answer is yes, how can I debug it?Just to be clear, I have setup the client to have a threshold of 1 event, and submit them every 5 seconds, but I cannot see anything on the console or any event in my account. Any help will be appreciated.

Martin ENew Member

asked in Product Experimentation

Experiment Assignment and Exposure discrepancies

We have recently integrated Amplitude analytics and have more recently been testing Amplitude Experiments on our website. For our first test experiment, the ratio between assignment and exposure is roughly 2:1, so there is double the amount of assignments as there is exposures and we don’t know why that is?We also checked in Pathfinder and could see the following problem: Sometimes exposures seem to occur before the assignment event, is that an issue? We included the Amplitude browser and experiment sdks via the npm modules @amplitude/experiment-js-client and @amplitude/analytics-browser We then initialize the experiment sdk like so: private _experiment = Experiment.initializeWithAmplitudeAnalytics( environment.amplitudeDeploymentKey, { fetchOnStart: true } );as well as the browser SDK:amplitude.init(environment.amplitudeApiKey, { defaultTracking: true });and here is the code that checks the variant with additional experiment.start() call just in case: hasVariantValue( exp

Shubham WadhwaNew Member

asked in Product Experimentation

how to filer user id by odd and even logic for an event

how to filer user id by odd and even logic for an event

peckerbidsConversationalist

asked in Product Experimentation

Switch between experiment and flag

Hello,If we switch an experiment to a flag and then back to an experiment, does the data get lost from the first experiment?

dargeliesConversationalist

asked in Product Experimentation

Experiment Flag Slack Integration / Webhook

Hey there,Recently we had an incident that was caused by a feature flag being toggled. The incident commander was not aware of the feature flag being toggled to understand what changed in the system. Typically, we push system changes like this to Slack with automations to make it more visible. I noticed there is a Slack integration for some Amplitude features and webhooks for custom monitors, but didn’t see this. What we’re looking for is an integration or webhook that will notify us when a feature flag state is changed. Is there any way to do something like this? Thanks

liukaichiHelpful Hero

asked in Product Experimentation

Question about using Bucketing Slat

I know this is documented quite well here: https://www.docs.developers.amplitude.com/experiment/general/evaluation/implementation/#activationBut this experiment is very important, so I just want to make sure I have this right. We have 3 flags that we want to ensure get to the same users. If we use the same Bucketing Salt for each of the flags, then if we keep the segments and rollout numbers the same, then the same users should receive all 3 flags. Is this correct? Are there any pitfalls that we should be aware of when doing something like this? Thanks in advance.

Alexandra ResslerNew Member

asked in Product Experimentation

Can Amplitude do network experimentation?

In my product, users can send each other messages. If we want to experiment on how messaging works, we can’t run a typical A/B experiment because we need both the sender & recipient to be in the same variant. One standard way of addressing this is running experiments in different geographic locations, with the assumption that each location has a mostly-closed network. This hasn’t worked for us, unfortunately, as our separate locations aren’t very similar in the first place so we can’t compare them. Some companies have the ability to expose experiments based on network clustering. Is that a feature that Amplitude has?Side note: I have bet on with my engineer that the answer is no. =D So you won’t disappoint me if that’s the case!The best alternative I can think of is to use BigQuery to manually separate users into clusters, normalize on relevant attributes, try to pair similar clusters to assign to different variants, and upload those assignments as a user property, but I have

Micah BlackburnNew Member

asked in Product Experimentation

New Developer Here With Some UTM Questions

Hello, I am developing amplitude with my ecommerce website smartbettor.ai. I have currently implemented amplitude to track events using the java script sdk. I track payments with stripe and want to track sign ups. After a successful stripe checkout users are redirected to a registration form. The only way to complete sign up is to complete this registration form so I have our ‘sign up’ event being the submission of this form. There are 2 problems with this currently. Numbers between what stripe says the customer sign ups are and what amplitude is saying are very different. Why would amplitude not be able to track every time this event happens and have significantly less than what stripe says? We want to track utm source. We are starting ad campaigns and want to know what website the user came from. I added these lines which I assumed would track: includeUtm: true, // Track UTM parameters includeReferrer: true, // Track referrer includeGclid: true, // Track Google Click Identifier incl

Mohammad GhasemiNew Member

asked in Product Experimentation

attribution currently is unavailable

when i want to use attribution, it shows me : attribution currently is unavailablehow can i do for that?

Liron BurrackNew Member

asked in Product Experimentation

amplitude/experiment-js-client SDK vs amplitude-js SDK?

Does amplitude-js npm package includes the amplitude/experiment-js SDK inside it?what is the difference between them?

Rune AndersenWell-Versed

asked in Product Experimentation

Any Payload size limit in a variant ?

Questions is:Are there a limit for how large (line, characters, mb) a JSON Payload can be as a part of Feature flag.For a client im implementing Experiment/Feature Flags for IOS and Android. One of the Feature flag planned are to be used as a “All Users 100%” with a payload with info about shops etc. Example of this is an object of all shops, and info for each shop.Intention is to make only one “App Data” feature flag that will grow overtime.For now it’s only 74 lines, and est 1200 characters. But it’s intended to grow over time.

Federico HuneeusNew Member

asked in Product Experimentation

Trouble tracking exposure events. Assignment events work, exposure doesn't

Hey guys, can someone give me a hand. I’ve followed pretty much every step in the guides to track exposure, but nothing gets registered on my dashboard. What I’ve done is loading the experiment.start() and then running the line const variant = experiment.variant(‘experiment-one’) [experiment-one is the flag], but it doesn’t work.i tried both the Amplitude and Segment integration, but no luck. This is one way I tried (with Segment → )export async function loadScript() { const promise = new Promise((resolve, reject) => { const script = document.createElement("script"); script.type = "text/javascript"; // script.async = true; // script.defer = true; script.text = ` !function(){var analytics=window.analytics=window.analytics||[];if(!analytics.initialize)if(analytics.invoked)window.console&&console.error&&console.error("Segment snippet included twice.");else{analytics.invoked=!0;analytics.methods=["trackSubmit","trackClick","trackLink","trackForm","page

Daniel BarriaNew Member

asked in Product Experimentation

Evaluation REST API not working

Hey everyone! Here is a situation that I am facing beign new using amplitude. We’re trying to use amplitude Evaluation REST API to get users’ variant for a feature flag we’ve created. All seems to be alright, deployment key is created, feature flag is created and active but for some reason when I try to get user variant it always returns an empty objecthttps://www.docs.developers.amplitude.com/experiment/apis/evaluation-api/#exampleI’ve tested from that page and also doing a cURL. Any help would be awesome! Thanks

NitishKumarNew Member

asked in Product Experimentation

In the funnel of "Play Song or Video" to "Purchase Song or Video", what percentage of users do "Play Song or Video" 3 times, before conversion (Purchase Song or Video)

I have created this chart https://app.amplitude.com/analytics/demo/chart/6yds5ago/edit/rz913e1d, but want to know if this is correct, please help.

NitishKumarNew Member

asked in Product Experimentation

What are the Average and Median times taken to do "Send Message" after "User Sign Up"? What is the distribution chart of "Time Taken" for this conversion?

This is in B2B Chat Analytics, I tried but don’t know if my answer is correct. Hope someone can help me with this. https://app.amplitude.com/analytics/demo/chart/hjqmr0zx/edit/cbmsp21w

NitishKumarNew Member

asked in Product Experimentation

What are the Average and Median times taken to do "Send Message" after "User Sign Up"? What is the distribution chart of "Time Taken" for this conversion

This is in B2B Chat Analytics, I tried but don’t know if my answer is correct. Hope someone can help me with this. https://app.amplitude.com/analytics/demo/chart/hjqmr0zx/edit/cbmsp21w

Prashant PadhyNew Member

asked in Product Experimentation

What is the conversion from "Add Item to Wishlist" to "Complete Purchase" completed within 1 day, from Japan users, for "Item_ID" = 100, in June?

E - COMMERCE TOOL

NitishKumarNew Member

asked in Product Experimentation

What is the true conversion(Event level conversion) from "Select Song or Video" to "Play Song or Video", when the Song or Video is selected from "Main Landing Screen". Compare this conversion number by different "Sources" , in the same chart.

Hi, I feel my answer is a little close https://app.amplitude.com/analytics/demo/chart/hjqmr0zx, Is this correct, if not I need little assistance.

NitishKumarNew Member

asked in Product Experimentation

What is the conversion from "Play Song or Video '' to "Purchase Song or Video '' , if the user follows a playlist (Follow Playlist) within 24 hours of first use?

Hi, I tried to do it this way https://app.amplitude.com/analytics/demo/chart/new/3w9uxdfi, the problem I am not able to figure out this part, “if the user follows a playlist (Follow Playlist) within 24 hours of first use”. Hoping that the chart above is the correct answer for the question (if not please help me on what to change). How to implement this 24 hours part?

Dave PorterNew Member

asked in Product Experimentation

Separate Amplitude Experiment deployment or not for SSR server?

BackgroundThe examples in the server side rendering guide for Amplitude Experiment have…a “server” deployment key for initialising the node SDK used to fetch initial variants, see https://www.docs.developers.amplitude.com/experiment/guides/server-side-rendering/#initialize-the-server-sdka “client” deployment key for initialising the javascript SDK ExperimentClient with those initial variants, also used on the server. See first “if” branch here https://www.docs.developers.amplitude.com/experiment/guides/server-side-rendering/#initialize-the-client-sdk-on-render. The “client” deployment key matches that used to initialise the ExperimentClient in the browser in the second “if” branch.However, the Amplitude Experiment Node SSR Demo app here https://github.com/amplitude/experiment-node-server/tree/main/packages/ssr-demo uses an identical “client” deployment key for all three. (https://github.com/amplitude/experiment-node-server/blob/main/packages/ssr-demo/lib/experiment.ts and https://githu

Casey HNew Member

asked in Product Experimentation

Redirect A/B test with Amplitude

Hi, Has anyone ever succeeded in settng up a redirect A/B test using Amplitude Experiment? We have been looking for instructions to set up redirect test but couldn’t find any.

Valentin IvanovNew Member

asked in Product Experimentation

Missing User ID in Start Session - GTM

Hi everyone, for my project I use GTM template “Amplitude Analytics Browser SDK” to initialise the Amplitude but I set userId and identify user in the project itself(by calling the newly created amplitude object), because GTM doesn’t have access to the information needed. Also I set timer with 30 retries in case that object is not initialised yet.Now what happens is that for majority of users but not all(around 70%) I can’t see information from identify, as well as User ID, in the user properties of Start Session event . However in the future events of same session the information I send with identify method is present but User ID is still missing. This causes the table in User Look-Up to show Amplitude ID instead of user Id for these events(but if you open the page of the user, he has correct Id). This is applicable only for the 70% of users I mentioned at the beginning, the rest are receiving identify data and userId in Start Session, so further events have userId set.Does anyone kno

Thomas GroendalNew Member

asked in Product Experimentation

How do I correlate time in platform to retention?

We’re working on a tool that is a platform for investigations in a virtual machine. We have a thesis that more time spent in the machine itself is going to be indicative of happy customers, as we meet a broader range of their investigative tasks in platform. I want to test this by seeing if time in platform is predictive of retention. I can measure time in platform on average, but I can’t figure out how to link it back to retention.I’m also open to criticism of the idea in general.

Novita OliveraNew Member

asked in Product Experimentation

Change experiment key name

Can we change the experiment key name for existing Amplitude experiment?

1
2
3
4
5

Page 2 / 5

Recent Tips & Tricks

Improving Data Quality with Collaboration
Tag Manager Vs. SDK
Insights Are Only As Good As Your Schema

Powered by Gainsight

Terms & Conditions Cookie settings

Welcome to the Amplitude Community!

If you don't have an Amplitude account, you can create an Amplitude Starter account for free and enjoy direct access to the Community via SSO. Create an Amplitude account. You can also create a Guest account below!

If you're a current customer, select the domain you use to sign in with Amplitude.

analytics.amplitude.com analytics.eu.amplitude.com

or

Username *

E-mail address *

Password *

I accept Terms & Conditions.

loginBox.register.email_repeat

Welcome to the Amplitude Community!

If you don't have an Amplitude account, you can create an Amplitude Starter account for free and enjoy direct access to the Community via SSO. Create an Amplitude account. Want to sign up as a guest? Create a Community account.

If you're a current customer, select the domain you use to sign in with Amplitude.

analytics.amplitude.com analytics.eu.amplitude.com

or

Sign in as a guest with Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK