Product Experimentation

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences.

  • 32 Topics
  • 79 Replies

32 Topics

Amplitude Admin
Team Member
Amplitude AdminTeam Member
published in Product Experimentation

Welcome to the product experimentation forum!

Amplitude Experiment is our product that allows teams to remotely modify and configure the product experiences for unique audiences This is where you can ask and get feedback on questions such as: ✅ Am I able to export the data I have in Experiment similar to Analytics?✅ How can I re-assign variants to my users?✅ Why are my users not getting targeted the right variant?✅ What happens if we merge projects in Experiment? Helpful resources: ❓Overview of Amplitude Experiment❓The experimentation workflow: from configuring to troubleshooting your experiment❓Amplitude Experiment key terms ❓Set up mutually-exclusive experiments❓Understand the Experiment Analysis view❓Advanced topics in Experiment: duration estimates, sequential testing for statistical inference, MDE and confidence intervals, randominzation, performance and scaling, sample ratio mismatches❓Amplitude Academy course: Getting Started with ExperimentStill have questions? Post them here! 

851
T
5 days ago
0
00rladkfk00New Member
asked in Product Experimentation

An error occurred while embedding Amplitude data into Google Sheets

Hello, community membersI have a question about integrating with Google Sheets.I followed the documentation on the Amplitude blog and tried to integrate with Google Sheets.I followed the instructions in the blog, but when I pressed the "Add" button, I got the error message and it didn't go any further.(* message -> ["Add Chart Failed: <html><title>500: Internal Server Error</title><body>500: Internal Server Error</body></html> ) I have searched within the community for information on this, but have not been able to find any.Does anyone know about this case. Any help would be great for me.Thank you 😀 

0
0
12 hours ago
T
tcaprioNew Member
asked in Product Experimentation

Holdout Group option missing from UI

Why don’t I see an option to set up a holdout group in my account? https://help.amplitude.com/hc/en-us/articles/13508918823835

1
Jeremie Gluckman
Community Manager
3 days ago
I
Novice
IanNovice
posted in Product Experimentation

Snowflake data source not shown in Events as source

I added a full sync with valid SQL query from Snowflake, but the new data source cannot be found in Events, and cannot be loaded into Amplitude.

5
Jeremie Gluckman
Community Manager
10 days ago
N
neoNew Member
posted in Product Experimentation

HTTP V2 API "Connection reset by peer" error

There is no problem with the event sent to Braze (have the same structure), but the “Connection reset by peer” error occurs only in the event sent to Amplitude. (Occurs in about 5%) Since we are not using a separate proxy, there is no path for the error to occur on our side. What could be the cause? 

2
Jeremie Gluckman
Community Manager
10 days ago
R
reyanalystNew Member
posted in Product Experimentation

Snowflake as Source is not working

I’ve tested 4 scenarios of different configurations to set Snowflake as source and all of them failed:Event + Time Based Configuration - successful creation of source BUT will have a Status of “No Data” even after performing the scan. In the docs, it states the first scan is supposed to do a Full sync, but no activity can be seen in the “Ingestion Jobs” tab. The “Edit Import Config” shows an error stating that the file has not been pulled with the configuration settings, when I did not even do anything to it yet. User Prop + Time Based Configuration - same issue as the first one. User Prop + Full Sync Configuration - successful creation of source and Status will show as Connected. “Ingestion Jobs” tab shows that data is indeed ingested. BUT, there are 2 problems I have observed. First, the count of rows ingested does not match the table row count in Snowflake: ingested 31,220 when the table has 38,873 in Snowflake. Some observation I had:In the “Edit Import Config” tab, time-based conf

3
Jeremie Gluckman
Community Manager
10 days ago
Anthony
Experienced
AnthonyExperienced
asked in Product Experimentation

Android device blocked from sending events to Amplitude

I think my specific Android device, which has our Android app with the Amplitude Android SDK, is being blocked from sending any events to Amplitude. How can I check that? 

2
Anthony
Experienced
11 days ago
R
reyanalystNew Member
posted in Product Experimentation

Snowflake Source not showing up in Event sources

This is a follow up to my concern in the link below:  I mentioned that Time-based ingestion for Snowflake integration is not working, but how about for the full-sync ingestion? Still does not work. Upon creating an event, you can click on the event to connect it to existing sources, in my case, I have 4 test snowflake sources. However, it is not showing. The number of sources say that I have 0, when I have 4. I actually tried to click add source to create another hoping it would attach the source to the event, but it would not. 

1
Amplitude Admin
Team Member
18 days ago
L
lanceNew Member
asked in Product Experimentation

Consistent bucketing across unique amplitude experiments

Hi there!We're working on configuring three experiments, one for each platform (web, ios and android) but we want the bucketing to be consistent across all platforms (i.e., a user that sees variant A on web should also see variant A on iOS and Android). If the rollout %, # of variants and variant weight all remain the same, could we apply the same bucketing salt string to all three experiments and expect consistent bucketing for all of our users in these experiments? 

3
Amplitude Admin
Team Member
22 days ago
J
JohnNew Member
posted in Product Experimentation

Flags and variants for anonymous users

Hello I have this case where we need to rollout a feature slowly for anonymous users, but we have found out that variants doesn’t work for those users if we set them different percentages, for example I have a flag with 2 variants, A and B I set A to 90% and B to 10%, test this with an anonymous user and he get neither of those! he gets the OFF flag, anonymous users only receive a flag if I set only one variant to 100%

4
Amplitude Admin
Team Member
23 days ago
C
chris.meyerNew Member
asked in Product Experimentation

Approval System for Experiment?

Is there an approval system within Amplitude Experiment we can configure for experiment changes in a specific environment? In an ideal world I am looking for an approval system when changing targeting, rollout %, etc. where you can request reviewers for your changes and submit for approval.

1
Jeremie Gluckman
Community Manager
23 days ago
C
ccreaserNew Member
asked in Product Experimentation

Way to chart the time between two specific events per user?

Hello everybody!I am writing this question to see if it is feasible to perform analysis in amplitude on the period of time between two specific events performed by individual users.It would be great to be able to show the trends of this outlined time period between a user baseFor an example: say we have two events: login event sign out eventWould it be possible to chart the time period between the latest two of these respective events for each user? For example to determine how long a typical user may remain logged in before signing out?Thanks for any responses! 

2
Jeremie Gluckman
Community Manager
23 days ago
I
Novice
IanNovice
asked in Product Experimentation

Time-based sync for Snowflake has no jobs started

I am currently ingesting data from Snowflake, using a valid SQL query.​​​​​The sync job does not even start for the time-based sync. How long would I have to wait before the data is synced?

8
belinda.chiu
Community Manager
24 days ago
D
Novice
Dimple BNovice
asked in Product Experimentation

How can I view screen/ event view time in the report?

While applying the filter for events, the report gives the tap count/ view count for the screen/ event. However, instead of tap count. I want to view the time for which user has been on any particular screen.Example: Dashboard screen view is an event, and users have visited this page for 2 hr in a day and tapped for about 120 times during the day. When filter is applied for usertype = ABC & event name= Dashboard screen view, this gives tap count say 90, however, I would like to see, page view time say1.2 hr.I have tried this with ‘Segmentation’ and ‘User sessions’ both. How can I achieve the same ?

6
D
Novice
1 month ago
R
richaNew Member
posted in Product Experimentation

Cannot login - how to contact support?

I can’t login - see attached screenshot.I am trying to find a way to contact support but all I see is help articles and this community. 

2
belinda.chiu
Community Manager
1 month ago
0
00rladkfk00New Member
asked in Product Experimentation

What are the exact event definition criteria for Pathfinder charts?

What are the exact event definition criteria for Pathfinder charts?(Subtitle: Pathfinder Chart vs Funnel Chart) Hello everyone, I have a question about Pathfinder charts. I tried drawing the same event using the Pathfinder chart and the funnel chart, but it occurred with completely different results on the two charts. Even if I search the guide, the exact event criteria of the Pathfinder chart are not defined. Does anyone know about the event definition criteria of the Pathfinder chart? Even if I try again and again, I get completely different results depending on the usage chart when measuring the same event. I would like to know why. Does anyone know about this case. Any help would be great for me  Thank you 😀

1
T
Team Member
1 month ago
A
Novice
AngelShamiehNovice
asked in Product Experimentation

Filter by proporties

Hello, Im new to Amplitude and Im still learning. recently, the team added the events and properties requested by product.  Now Im trying to see all events related to a specific property, but for some reason when I filter by “Any event” Im not able to find the properties were added by the team.   Waiting to hear from you,Thanks a lot Angel

3
A
Novice
1 month ago
K
Novice
Kevin SimpsonNovice
asked in Product Experimentation

React Native Experiment SDK, Android Emulator, and M1 Macs

We’re seeing an interesting issue with employees on M1 MacBooks who are running our app in development utilizing an Android Emulator through Android Studio. The initialization of the Experiments SDK is crashing the development client. It seems exclusive to M1 MacBooks, as folks on Intel i7 machines seem to experiencing no problems.Anyone else experiencing similar issues? Ideas on solutions?

4
K
Novice
1 month ago
T
techtransportsimpleNew Member
asked in Product Experimentation

Not able to access User Session

I cannot search for a User ID beyond one day in Search User field. Also I cannot see user event history beyond 1 day.

1
Z
Team Member
2 months ago
Amplitude Admin
Team Member
Amplitude AdminTeam Member
published in Product Experimentation

How Burger King’s app reached #1

"We are a burger company; that is our product. It's not a website. But to stay relevant, we have to do much more than sell Whoppers. Today more than ever, it's important for us to think about experiences people will have with the Burger King brand that will be relevant and establish a meaningful, emotional connection."  - Elie Javice, former VP of Tech and Product Management at Restaurant Brands International. When Burger King launched its "Whopper Detour Campaign" in 2019, the fast-food giant used Amplitude to understand behavior, target its messaging, and calculate impact. The total sales value sold through the app increased 3x during the promotion. And that streak continued after the promotion when mobile app sales were double what they used to be. The Whopper Detour put the app on the map, making it a popular Burger King staple by building new habits and loyalty among customers. Building on this, Burger King used Amplitude for its BK Cafe Subscription project, leveraging the platfo

890
Amplitude Admin
Team Member
3 months ago
L
Helpful Hero
luigi7upHelpful Hero
asked in Product Experimentation

SplitIO and Amplitude ab testing

Hi, we’ve decided to use SplitIO for AB testing in my company, but since we also use Amplitude, I’d really like to have the ability of analysis based on ab tests user segments in Amplitude tooSo, we send a getTreatment(“test_name”) call to SplitIO that returns the test and the version that we should serve to the client. We then serve different versions to the client based on that. However we also want to register that in Amplitude so that we can do our full funnel analysis, retention analysis, stickiness analysis based on the cohort users belong to.My approach would be to register each test and variant as a user property in Segment/Amplitude and this way we could easily do every analysis we want by simply grouping the users by the version that was served to them. Would that be the best practice?In one tutorial (albeit by Mixpanel) they suggest that we should fire an event when the test is served to the user. I really dislike this approach because IMHO the test version served to the use

3
Denis Holmes
Team Member
4 months ago
E
ElReydelSEONew Member
asked in Product Experimentation

Test AB via Redirections

Hi, Hope everything goes wellQuestion: In the future, will Amplitude be able to run ab tests via redirects?Best regards

1
Jeremie Gluckman
Community Manager
4 months ago
L
LirooNew Member
asked in Product Experimentation

400 and CORS Errors on a Experiment Route

Hey !Since 1 or 2 weeks I start to have a CORS / 400 Error on a specific route which is:https://api.lab.amplitude.com/sdk/vardataIt happens only in dev / staging env (both same API KEYS) where it doesn’t happen in my production env.I tried a lot of stuff: generating new API KEYS, removing all staging experiments, updating SDK versions, changing pc, network but nothing seems to change that, it seems to be related to the staging project/deployment on Amplitude Experiment.It’s really annoying for my devs and me as it’s spamming constantly the console and as a dev, having errors in the console is just frustrating.If you have any clue on that subject, please help me, thanks! :)

1
Yuanyuan Zhang
Team Member
5 months ago
X
Well-Versed
xiaomeng.zhangWell-Versed
asked in Product Experimentation

The impact of the merger project on Skylab

Hi, Guys.We are going to create a new project that will contain three countrys, The data should be come from different websites.We have the following questions：1.Do we need to create a new Deployments in Skylab for merger project?2.In the merged project, we need to distinguish the websites of three countries, how should we set up skylab?3.If in a merged project we only want to set abtest for my-website, how can we set it?4.How we differentiate between countries to distribute traffic？5.If we change the Deployments key, will the same device_id get the same version before and after the change?6.If we update the deployments key, assuming we update the key in the latest version of the app, will this affect users of the older version?

3
eddie.gaona
Team Member
6 months ago
X
Well-Versed
xiaomeng.zhangWell-Versed
posted in Product Experimentation

A large number of users contain only one experimental event

Hi, Guys.We found that there are a large number of users with only one event([Experiment] Assignment) in our project and no action after. Such users are close to 100,000 per day, But in this project we have less than 100 users.So we want know:    1.how these users are generated?   2.Why these users are generated and what is the role?   3.how we handle?data link: https://analytics.amplitude.com/carsome/chart/new/4ii7idcuser as picture: 

3
Denis Holmes
Team Member
9 months ago

Terms & Conditions