Product Experimentation

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences.

  • 48 Topics
  • 133 Replies

48 Topics

Amplitude Admin
Team Member
Amplitude AdminTeam Member
published in Product Experimentation

Welcome to the product experimentation forum!

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences This is where you can ask and get feedback on questions such as: ✅ Am I able to export the data I have in Experiment similar to Analytics?✅ How can I re-assign variants to my users?✅ Why are my users not getting targeted the right variant?✅ What happens if we merge projects in Experiment? Helpful resources: ❓Overview of Amplitude Experiment❓The experimentation workflow: from configuring to troubleshooting your experiment❓Amplitude Experiment key terms ❓Set up mutually-exclusive experiments❓Understand the Experiment Analysis view❓Advanced topics in Experiment: duration estimates, sequential testing for statistical inference, MDE and confidence intervals, randominzation, performance and scaling, sample ratio mismatches❓Amplitude Academy course: Getting Started with ExperimentStill have questions? Post them here! 

1541
T
2 months ago
P
Polina MokretsovaNew Member
asked in Product Experimentation

Distinct Property Values ​​Per User in the Event Segmentation

Hello! I’m a product analyst, studying the Amplitude for work.I got acquainted with the documentation on the functionality, but didn’t find a detailed description and application of Distinct Property Values ​​Per User in the Event Segmentation report.Can you explain where this functionality can be applied? And what does it mean?I will be grateful for your help!

3
Saish Redkar
Expert
4 days ago
R
Ricardo Mighty HealthNew Member
asked in Product Experimentation

Chart query error

I'm trying to do a single request (Manually, without any type of automation, just one time) but I sill receive the error: HTTP 429: Too many requests (Too Many Requests: Exceeded rate limit with query of cost 16425)Anyone faced this problem before?Regards,Schieck.

2
R
4 days ago
B
Ben KloesterNew Member
asked in Product Experimentation

What permissions does LaunchDarkly Cohort sync really need?

The instructions for setting up syncing of Amplitude cohorts to LaunchDarkly suggest creating an access token with Writer permissions, which has write access to pretty much everything except account members and billing. That seems like much more than is needed to do what the sync needs, as AFAICT it just creates and updates Segments in LD. Does anyone know what access is actually required, to what objects, in order for the sync to work? I’m considering creating a custom role that only has write permissions to Segments, for instance.

3
Esther Trapadoux
Community Manager
4 days ago
F
Novice
Florent VilmartNovice
asked in Product Experimentation

Exposure tracking through segment

Hi there!For Amplitude experiments, it is recommended to emit an exposure event.We are using segment.io and are wondering how we can track those events properly.Whenever we call `track` it ends up being an event with name `$exposure`.Segment doesn’t seem to wanna pass `event_type` to the destination.The alternative is to use the `identify` call, but I wonder if there are side effects compared to using the $exposure call.

7
F
Novice
17 days ago
V
Vania AlyaNew Member
asked in Product Experimentation

Error Message when Saving Experiment

Hi amplitude team, I wanted to raise an error message that I encountered when saving changes to my experiment. If i may know what triggers this error? Because this happened to a couple of experiments with the same error message and we don’t use any value ($1,$2,$3) that was stated in the error message. Thank you!   

2
Y
Team Member
20 days ago
O
Ossi PesonenNew Member
asked in Product Experimentation

Amplitude Experiment device type for iPhone 14 not working

We are trying to set up A/B testing for anonymous users that is targeted specifically for iPhone users. Amplitude, however, seems to be unable to detect iPhones for the variant as per our testing. For reasons unknown the variant is not returned in the request, because Amplitude does not recognize iPhone via the user agent.We are using the JavaScript SDK and setting it up as per documentation, and I’ve done some debugging to see what type of user agent string is being delivered via network requests from a phone:Granted, this is from an iPhone simulator, but based on the information I gathered from the web, it is exactly as a real device, which also do not work. We had QA engineers try to enable this experiment as well with real iPhones on our website and we couldn’t get it to work.Here’s a snippet of the JS code:const amplitudeExperiment = window.experiment;if (amplitudeExperiment) { var div = document.getElementById('variant-info'); amplitudeExperiment.fetch().then(function() { co

3
Esther Trapadoux
Community Manager
27 days ago
L
Novice
lanceNovice
asked in Product Experimentation

Exposure tracking provider for NodeJS

Hi! We currently utilize the NodeJS SDK to handle server-side analytics and experiments. We were wondering if there are any plans to build an exposure tracking provider for NodeJS, similar to the JS SDK implementation

4
L
Novice
1 month ago
A
anandraprincNew Member
asked in Product Experimentation

Unable to fetch new file from GCS Bucket

I am new to Amplitude, I have configured a source to GCS for event data import. It imported first time when I have configured, but later whenI am pushing another files to GCP bucket with same extension and format it is not reading that file.

7
M
1 month ago
N
Novita OliveraNew Member
asked in Product Experimentation

Difference between Exposure event with blue font and green font

I saw that sometimes my exposure event is in blue one and sometimes it is in green font. I read that this is due to the event that is outside the session.Can I interpret that blue exposure is for front end experiment and green exposure is for back end experiment? What conditions that make the exposure event is outside the session?Reference: https://github.com/segmentio/analytics-react-native/issues/654 

2
Esther Trapadoux
Community Manager
1 month ago
N
Novita OliveraNew Member
asked in Product Experimentation

Best practices for using custom exposure in Amplitude experiments

Our aim is to utilize custom exposure in Amplitude experiments to make sure the users who see the changes is expected as the definition that we want to target. But we have observed a problem of receiving some exposures without any assignment, even after a few weeks of implementation. Is there any suggestion or guidelines of best practices for using custom exposure effectively in Amplitude?https://analytics.amplitude.com/tiketcom/chart/new/denicz1 

1
Jeremie Gluckman
Team Member
1 month ago
I
Novice
invictusNovice
asked in Product Experimentation

Exposure Events Tracking

Hi Amplitude CommunityI am trying to launch my first experiment and trying to track exposure eventHowever the docs seem to not have been updated.What is the correct code to track exposure event.I am using this amplitude.track( "$exposure", (Map<String, ? extends Object>) new Properties() .put("exposure.getFlagKey()",exposure.getVariant()));} but I feel this is not the right wayCould someone please help me out?  

7
J
Novice
2 months ago
L
Helpful Hero
liukaichiHelpful Hero
asked in Product Experimentation

Flag-Dependent Experiment Allocation

I have a question regarding experiment allocations.We have two flags that we want to be dependent on each other. For example "For users who are shown this, give them another experiment".For example using two flags, "show_tabs" and "show_new_shiny_tab". I would like to configure an experiment that is "For users who see the tabs, perform an A/B Test to users to test a new shiny tab". So to achieve behavior, we select on the "show_new_shiny_tab" experiment the following: "Users where show_tabs = true/false".  Will this work as expected? From what we see, there is room for doubt, but overall it looks like it is working. Is there any concerns with this approach? 

1
Jeremie Gluckman
Team Member
2 months ago
J
jacoob_mNew Member
asked in Product Experimentation

Flag management by API.

Hi,I use a flag to enable user features in my app. I add user ids to the segments by UI.I did it according to the doc: Working with feature flags and feature rolloutsBut it is inconvenient with more users.I would like to do it by API. I got acquainted with Experiment customers: New endpoints available for the Management API but I did not find an answer there on how to manage the list of users belonging to the segment. I can pull a segment for a flag but I can't change the segment.Can I do it with the current state of the API? Maybe there are some other ways to enable/disable the flag for users via the API.

1
Jeremie Gluckman
Team Member
2 months ago
S
Soumya MohanNew Member
asked in Product Experimentation

Retention based on user performing event multiple times in a period

Hi, is it possible to create a retention chart where we only a count a user as retained if they performed an event atleast X times in a period like a week or atleast on X days of a week?  The reason we want to do this is that we believe our product should atleast be used 3-5 times a week and given how it works, it very easy for the user to take an action once in the week by mistake (without a lot of intention to use it). 

2
Saish Redkar
Expert
2 months ago
F
Fran6New Member
asked in Product Experimentation

Cohort with specific behaviour every month

I would like to create a cohorte of people who performed a certain number of actions per month (like listen to < 20 songs in each month last year). But what if someone listen to 20 songsa month and then 21 in the other one ? In that case I would like to take him out of my cohorte and only keep the people who performed less than 20 songs every month during a year. Is it possible to get these people in a cohorte ?  Thanks in advance

1
Saish Redkar
Expert
2 months ago
0
00rladkfk00New Member
asked in Product Experimentation

An error occurred while embedding Amplitude data into Google Sheets

Hello, community membersI have a question about integrating with Google Sheets.I followed the documentation on the Amplitude blog and tried to integrate with Google Sheets.I followed the instructions in the blog, but when I pressed the "Add" button, I got the error message and it didn't go any further.(* message -> ["Add Chart Failed: <html><title>500: Internal Server Error</title><body>500: Internal Server Error</body></html> ) I have searched within the community for information on this, but have not been able to find any.Does anyone know about this case. Any help would be great for me.Thank you 😀 

2
Jeremie Gluckman
Team Member
2 months ago
Anthony
Well-Versed
AnthonyWell-Versed
asked in Product Experimentation

Android device blocked from sending events to Amplitude

I think my specific Android device, which has our Android app with the Amplitude Android SDK, is being blocked from sending any events to Amplitude. How can I check that? 

3
Anthony
Well-Versed
2 months ago
A
aprbirdkimNew Member
asked in Product Experimentation

Can I delete my account and sign-up again?

Hi, I created my personal account with an organisation, and I realised that I have to re sign-up with a business account. Would it be okay to delete my current account and re-create it with the same organisation? Thank you :) 

4
belinda.chiu
Community Manager
2 months ago
T
tcaprioNew Member
asked in Product Experimentation

Holdout Group option missing from UI

Why don’t I see an option to set up a holdout group in my account? https://help.amplitude.com/hc/en-us/articles/13508918823835

2
T
2 months ago
I
Novice
IanNovice
posted in Product Experimentation

Snowflake data source not shown in Events as source

I added a full sync with valid SQL query from Snowflake, but the new data source cannot be found in Events, and cannot be loaded into Amplitude.

5
Jeremie Gluckman
Team Member
2 months ago
N
neoNew Member
posted in Product Experimentation

HTTP V2 API "Connection reset by peer" error

There is no problem with the event sent to Braze (have the same structure), but the “Connection reset by peer” error occurs only in the event sent to Amplitude. (Occurs in about 5%) Since we are not using a separate proxy, there is no path for the error to occur on our side. What could be the cause? 

2
Jeremie Gluckman
Team Member
2 months ago
R
reyanalystNew Member
posted in Product Experimentation

Snowflake as Source is not working

I’ve tested 4 scenarios of different configurations to set Snowflake as source and all of them failed:Event + Time Based Configuration - successful creation of source BUT will have a Status of “No Data” even after performing the scan. In the docs, it states the first scan is supposed to do a Full sync, but no activity can be seen in the “Ingestion Jobs” tab. The “Edit Import Config” shows an error stating that the file has not been pulled with the configuration settings, when I did not even do anything to it yet. User Prop + Time Based Configuration - same issue as the first one. User Prop + Full Sync Configuration - successful creation of source and Status will show as Connected. “Ingestion Jobs” tab shows that data is indeed ingested. BUT, there are 2 problems I have observed. First, the count of rows ingested does not match the table row count in Snowflake: ingested 31,220 when the table has 38,873 in Snowflake. Some observation I had:In the “Edit Import Config” tab, time-based conf

3
Jeremie Gluckman
Team Member
2 months ago
R
reyanalystNew Member
posted in Product Experimentation

Snowflake Source not showing up in Event sources

This is a follow up to my concern in the link below:  I mentioned that Time-based ingestion for Snowflake integration is not working, but how about for the full-sync ingestion? Still does not work. Upon creating an event, you can click on the event to connect it to existing sources, in my case, I have 4 test snowflake sources. However, it is not showing. The number of sources say that I have 0, when I have 4. I actually tried to click add source to create another hoping it would attach the source to the event, but it would not. 

1
Amplitude Admin
Team Member
3 months ago
L
Novice
lanceNovice
asked in Product Experimentation

Consistent bucketing across unique amplitude experiments

Hi there!We're working on configuring three experiments, one for each platform (web, ios and android) but we want the bucketing to be consistent across all platforms (i.e., a user that sees variant A on web should also see variant A on iOS and Android). If the rollout %, # of variants and variant weight all remain the same, could we apply the same bucketing salt string to all three experiments and expect consistent bucketing for all of our users in these experiments? 

3
Amplitude Admin
Team Member
3 months ago

Terms & ConditionsCookie settings