Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing
-
Upload
cheng-luo -
Category
Technology
-
view
424 -
download
1
description
Transcript of Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing
![Page 1: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/1.jpg)
#mceconf #mce-addingknobs 11 Jan 2014 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Adding Knobs to your App – Modifying & Measuring Behavior in the Wild (without Republishing) Cheng Luo, Amazon Appstore 11th January, 2014 @chengluo
![Page 2: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/2.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Agenda • Why Amazon Mobile SDK • How to use A|B testing • Add your A|B testing in 10 minutes • Demo
![Page 3: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/3.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
30% 25%
45%
<$50 $50-‐$200 $200+
Monthly Unique Visitors (US):
97M
6.6 Active Global Customer
Accounts:
200M Avg. Daily Unique Visitors
(US):
12.9M
Note: Active Customer Accounts are unique email addresses that have placed an order in the last 12 months Sources: 1. Amazon Investor Relations; 2. ComScore PlanMetrix January, 2013.
10%
38%
52%
<$25K $25K-‐$75K $75K +
1 2 2 2
8%
32%
18%
42%
0-‐18 18-‐34 35-‐44 45+
Household Income of Amazon Unique Visitors2
Age Composition of Amazon Unique Visitors2
% of Amazon Shoppers Who have Made an Online Purchase2
Amount Spent
Avg. Visits per Visitor per
Month (US):
Active Users
![Page 4: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/4.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
• Amazon Appstore runs Android apps
• 75% of Android apps work without any need for customisaTon • Simple APIs to publish free-‐to-‐user, freemium and premium apps
A/B TesTng Service API
GameCircle API
Maps API Device Messaging API
In-‐App Purchasing API
Simplicity
![Page 5: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/5.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
The latest Kindle Fire HDX
Kindle Fire HDX 7" Tablet $229, with 24x7 May Day support
![Page 6: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/6.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
What A|B Testing can do for you
Precisely control who is affected
Change in the wild; No new APK needed
Safely test back-‐end load
![Page 7: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/7.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Case Study - Air Patriots Used A/B Testing to decide: • Include in-game advertising • Provide purchasing short-cuts • Notification frequency • Level difficulty • Promotional imagery
![Page 8: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/8.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Case 1 – Image
![Page 9: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/9.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Case 2 – Ads placing
![Page 10: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/10.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Case 3 - Notification Notification appears • x1 day • x 3 days → Sweet spot • x 7 days
![Page 11: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/11.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Case 4 - User Retention
Level: 1 Difficulty: Easy
Level: 5 Difficulty: Medium
![Page 12: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/12.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Air Patriots Sample Results
![Page 13: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/13.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add A/B Testing to Your App
![Page 14: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/14.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add A/B Testing to Your App
![Page 15: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/15.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add A/B Testing to Your App
![Page 16: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/16.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Initialize A/B Testing in Your App private static final String APP_KEY = "876c809ba7084b9fbe2c08a22e5acda1"; private static final String PRIVATE_KEY = "9nxxk4//qL89ymXl/P5aL7w8uhs3DVDrLcyYViG6yn0="; AmazonInsights insightsInstance; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); . . . InsightsCredentials credentials = AmazonInsights.newCredentials(APP_KEY, PRIVATE_KEY); insightsInstance = AmazonInsights.newInstance(credentials, getApplicationContext()); }
![Page 17: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/17.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Create an A/B Testing Project
![Page 18: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/18.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Create an A/B Testing Project
![Page 19: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/19.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Create an A/B Testing Project
![Page 20: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/20.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add a New Launch
![Page 21: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/21.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add a New Launch
![Page 22: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/22.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add a New Launch
![Page 23: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/23.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Multiple Projects Are Allowed
private static final String AB_PRJ_ACC = "Acceleration"; //Project name private static final String AB_VAR_MAXACC = "maxAcceleration"; //Variable name private static final String AB_PRJ_VEL = "Velocity"; //Project name private static final String AB_VAR_MAXVEL = "maxVelocity"; //variable name
![Page 24: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/24.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Load Variation Variables private static final int DEF_MAXACC = 5;
private static final int DEF_MAXVEL = 50;
private int maxAcc;
private int maxVel; private void loadTreatment() { ABTestClient abClient = insightsInstance.getABTestClient(); abClient.getVariations(AB_PRJ_VEL, AB_PRJ_ACC).setCallback(new InsightsCallback<VariationSet>() { @Override public void onComplete(VariationSet variations) { Variation variation = variations.getVariation(AB_PRJ_ACC); maxAcc = variation.getVariableAsInt(AB_VAR_MAXACC, DEF_MAXACC); variation = variations.getVariation(AB_PRJ_VEL); maxVel = variation.getVariableAsInt(AB_VAR_MAXVEL, DEF_MAXVEL); } }); }
![Page 25: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/25.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Changing Values on the Fly
![Page 26: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/26.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Changing Values on the Fly
![Page 27: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/27.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Changing Values on the Fly
![Page 28: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/28.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Changing Values on the Fly
![Page 29: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/29.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add a Segment
![Page 30: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/30.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add a Segment
![Page 31: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/31.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Add a Segment
![Page 32: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/32.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Use the Segment
//get user age from the app settings SharedPreferences myPrefs; myPrefs = getSharedPreferences("com.example.testingapp", Context.MODE_PRIVATE); int userAge = myPrefs.getInt("com.example.testingapp.USER_AGE", ""); //Initial the segments here
UserProfile userProfile = insightsInstance.getUserProfile(); userProfile.addDimensionAsNumber("age", userAge);
Use App Segngs to idenTfy your user segments
![Page 33: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/33.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Demo
![Page 34: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/34.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Wrap up • A/B Testing
– Free cross platform services – Precise measuring behaviours using custom
segments – Modifying app behaviors without
republishing
![Page 35: Amazon AB Testing - Modifying & Measuring App Behaviors Without Republishing](https://reader033.fdocuments.in/reader033/viewer/2022060108/554d2d36b4c905c5208b5324/html5/thumbnails/35.jpg)
#mceconf #mce-addingknobs 11 Jan 2014
Thank you@AmazonAppDev