2016 the year of machine learning 12.16.2015
Transcript of 2016 the year of machine learning 12.16.2015
![Page 1: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/1.jpg)
2016 The Year of Machine Learning:
Why Bid Algorithms Will Always Outperform Humans
![Page 2: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/2.jpg)
Our Speaker
Bryan Minor, Ph.D.
Chief Scientistat Acquisio
![Page 3: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/3.jpg)
Housekeeping• The webinar is recorded and will be
made available by email
A• The slides will also be available by email
• Q&A session at the end of the webinar
• Use the Chat box to submit your questions at any time
For those that would like a trial or demo in Portuguese or Spanish, and are from a Latam country, please contact: Ghislain Nadeau, [email protected] • For anywhere else please contact: [email protected]
![Page 4: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/4.jpg)
Poll QuestionAre you currently using a bid optimization solution?
a) Yesb) Noc) I’m looking for one
![Page 5: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/5.jpg)
Agenda• What is Machine Learning?• Machine Learning at Acquisio
• Bid & Budget Management• Gamification
• Predictions for 2016 driven by Machine Learning• Conclusions
![Page 6: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/6.jpg)
What is Machine Learning?Machine learning is a subfield of computer science that evolved from the study of pattern recognition and computational learning theory in artificial intelligence. Machine learning explores the construction and study of algorithms that can learn from and make predictions on data. – Wikipedia (https://en.wikipedia.org/wiki/Machine_learning)
![Page 7: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/7.jpg)
Bid and CPC comparison
Campaign Type SubType (Bid/CPC) Number Campaigns
Search Others 2.33 11,087
Search Brand 7.13 1,919
Search Dynamic Search 1.40 294
Search RLSA 3.71 826
Display Others 1.89 1,118
Display Remarketing 1.86 762
![Page 8: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/8.jpg)
Mobile Optimization Problem• No pure Mobile campaigns• Can only set Bid at the device level
Mobile Other (Computer and Tablet)
• Mobile bid modifier -100% to +300%
• Budget shared across all devices in Campaign Controlling mobile spend
![Page 9: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/9.jpg)
Using Machine Learning in solving BBM problem
• Setting Daily Budget• Setting Bid every 30 minutes• Managing Mobile bidding • Anomaly detection (ensuring success)• Allocation of Budget across Publishers (AdWords, Bing,
Yahoo!Japan,…)• Day of week % of spend allocation
![Page 10: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/10.jpg)
BBM Problem:For a fixed Budget for budget period (month)
With a group of Campaigns (Budget Group)Make Daily Budget last whole DayMaximum Average CPC per day limitFairly compete Campaigns based on value of Clicks (conversions)Maximize Clicks (conversions)
![Page 11: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/11.jpg)
Continuous SEM OptimizationFeatures:
1. Examination and adjustment of Bids in regular intervals many times per day
2. Examination of Budget spend precision many times per day with hyper accurate control
3. Updating of modeling parameters in algorithms on a longer characteristic time scales
4. Auto detection and dealing with anomalies
![Page 12: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/12.jpg)
Algorithm model
• Cruise missile model• Dynamic Non-linear optimization• Small steps more often
![Page 13: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/13.jpg)
BBM Theory
C B
AAA
minCPC
0 2 4 6 8 1 00
5
1 0
1 5
2 0
2 5
3 0
3 5
C P C
Clicks
day
![Page 14: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/14.jpg)
Continuous SEM Optimization
• B graph – Daily Budget spent
• C graph – Daily Budget not spent
• A – location of maximum number of Clicks for a fixed Daily Budget obeying constraints
• minCPC – Lowest value of CPC produces Clicks
![Page 15: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/15.jpg)
Experimental ABC data #1
![Page 16: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/16.jpg)
Results: X-graph #1
Start Clicks Start CPC End Clicks End CPC
1,066 $0.51 1,848 $0.27
![Page 17: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/17.jpg)
Results: X-graph #2
Start Clicks Start CPC End Clicks End CPC
29 $1.24 56 $0.63
![Page 18: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/18.jpg)
Results: X-graph #3
![Page 19: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/19.jpg)
Results: ABC-graph #3
![Page 20: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/20.jpg)
Results: X-graph #4
![Page 21: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/21.jpg)
Results: ABC-graph #4
![Page 22: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/22.jpg)
Virtual Auctions
![Page 23: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/23.jpg)
BBM Spend - Nov 2015
![Page 24: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/24.jpg)
BBM Constraint Obedience - Nov 2015
Constraint
Constraint
day
CPC
CPC CPC
CPC
![Page 25: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/25.jpg)
Proving Machine Learning works:• 20,000 Campaigns in AdWords
• 12,000 on BBM• 8,000 not on BBM
• June 2015
![Page 26: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/26.jpg)
BBM Search - Daily Budget spend
Case (Spent Daily Budget %) times (Not BBM %)BBM 3.6BBM (3.7+ grade) 4.0
![Page 27: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/27.jpg)
BBM Search performance – Daily Budget Spent case
Case Imp Share IS LTB IS LTR CPC Avg. Pos.Not BBM 47.28% 32.49% 29.14% $6.46 2.306BBM 55.42% 18.43% 26.13% $4.04 2.417BBM (3.7+ grade) 54.99% 15.84% 20.23% $2.95 2.484
Case Imp Share % IS LTB % IS LTR % CPC % Avg. Pos. %BBM 17.2% -43.3% -10.3% -37.5% -4.8%BBM (3.7+ grade) 16.3% -51.2% -30.6% -54.4% -7.2%
![Page 28: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/28.jpg)
BBM Search performance – Daily Budget not Spent case
Case Imp Share IS LTR CPC Avg. Pos.Not BBM 64.51% 35.49% $4.29 2.50BBM 77.18% 22.82% $4.05 2.22BBM (3.7+ grade) 73.41% 26.59% $2.91 2.35
Case Imp Share % IS LTR % CPC % Avg. Pos. %BBM 19.6% -35.7% -5.7% 11.1%BBM (3.7+ grade) 13.8% -25.1% -32.3% 6.4%
![Page 29: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/29.jpg)
Daily Budget Spend (%)
![Page 30: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/30.jpg)
Distributions of Daily Budget spent (%) - Human
![Page 31: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/31.jpg)
Distributions of Daily Budget spent (%) – BBM no Skynet
![Page 32: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/32.jpg)
Distributions of Daily Budget spent (%) – BBM with Skynet
![Page 33: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/33.jpg)
Gamification• Continuously coaching user to better results • Enhances user brand loyalty
o Autonomyo Masteryo Connection
• Currently doing Anomaly detection daily o Setup problemso Budget underspendingo Warnings
• Machine Learning based
![Page 34: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/34.jpg)
2016 Predictions• Year of Machine Learning in AdTech/MarTech causing:
1. Continued suppression of CPC2. Accelerated consolidation of Platforms3. New quality advertising volume external to Google AdWords4. Greatly increase verticalization of technology stack available
to advertisers Exponential growth in Algorithm economy offerings via SOA
(Service Orientated Architectures) with RESTful API Lowering of skills necessary to use these ML algorithm services
(IFTTT)5. Leveling of the Playing Field for SMB advertisers
![Page 35: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/35.jpg)
Acquisio and Machine Learning• Machine Learning driving innovation in AdTech/MarTech• BBM offers Machine Learning optimization of Bid & Budget within and
across publishers (AdWords, Bing, Yahoo!Japan) • Machine Learning is the cornerstone of Gamification
• Required for Self-Service BBM
• Cross Publisher optimization will greatly increase in 2016• Google AdWords, Bing, Facebook, Yahoo!Japan
![Page 36: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/36.jpg)
ReferencesAdvancements in Machine Learning: Acquisio Summit Keynoteo YouTube video: http://tinyurl.com/p6s85f2o SlideShare: http://tinyurl.com/owwn2ow
Bid vs. Pay: A Case for Automated Optimizationo http://www.acquisio.com/blog/ppc-marketing/bid-vs-pay-case-automa
ted-optimizationPay vs. Bid: Optimizing for Mobile and Non-Mobileo http://www.acquisio.com/blog/mobile/pay-vs-bid-optimizing-mobile-an
d-non-mobile
![Page 37: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/37.jpg)
Poll Question
Are you interested in learning more about Acquisio’s bid optimization solution:
a) Yesb) Noc) I’m ok for now
![Page 38: 2016 the year of machine learning 12.16.2015](https://reader031.fdocuments.in/reader031/viewer/2022030222/5883cb941a28abb7308b55eb/html5/thumbnails/38.jpg)
Faster. Smarter. Better.
Questions?