LEARN HOW LENOVO Tesco MAKE ADOBE ...LEARN HOW LENOVO & Tesco MAKE ADOBE & ENTERPRISE DATA...
43
Ashish Braganza Global Business Intelligence Director @Lenovo Simon Ricketts Data, Analytics and Optimisation Strategist @Tesco David Searle General Manager, EMEA @SyntasaCo LEARN HOW LENOVO & Tesco MAKE ADOBE & ENTERPRISE DATA ACTIONABLE IN HADOOP
Transcript of LEARN HOW LENOVO Tesco MAKE ADOBE ...LEARN HOW LENOVO & Tesco MAKE ADOBE & ENTERPRISE DATA...
Ashish BraganzaGlobal Business Intelligence Director
@Lenovo
Simon RickettsData, Analytics and Optimisation Strategist
@Tesco
David SearleGeneral Manager, EMEA
@SyntasaCo
LEARN HOW LENOVO & TescoMAKE ADOBE & ENTERPRISE
DATA ACTIONABLE IN HADOOP
Don’t forget to tweet using
Log in to your account using the details sent to you when you registeredEnable social and select the room for your session e.g. Room CS8Ask questions and answer polls
Ashish BraganzaGlobal Business Intelligence Director
@Lenovo
Simon RickettsData, Analytics and Optimisation Strategist
@Tesco
David SearleGeneral Manager, EMEA
@SyntasaCo
LEARN HOW LENOVO & TescoMAKE ADOBE & ENTERPRISE
DATA ACTIONABLE IN HADOOP
4
Predictive Behavioural Analytics
99
29%
17% Product / Service Innovation
17% Customer Service Across All Touchpoints
Source : E-Consultancy : Digital Intelligence Brief – 2017 Digital Trends
Gartner - https://www.gartner.com/marketing/customer-experience
1010
17% Product /
Service Innovation
17% Customer
Service
Across All
Touchpoints
Source - E-Consultancy : Digital Intelligence Brief – 2017 Digital Trends
Gartner Customer Experience - https://www.gartner.com/marketing/customer-experience
89%
Source : E-Consultancy : Digital Intelligence Brief – 2017 Digital Trends
Gartner - https://www.gartner.com/marketing/customer-experience
14
15 15
16 16
17 17
ACHIEVING the single digital customer viewSimon Ricketts
Data, Analytics and Optimisation Strategist
Who We Are
470,000colleagues
11countries
310,000+
colleagues
in the UK
3,500stores in the UK
78 million
shopping trips
weekly
Our PurposeServing Britain’s shoppers a little better every day
c
Clickstream Data
BigQuery
Clickstream Data
BigQuery
Building the single Digital Customer View
Tesco
Analytics
Platform
Clickstream Data
BigQuery
+
Tesco Analytics
Platform
Datafeed
BigQuery
• Integrate enterprise data
• Map Adobe Schema to Tesco Behavioral Schema
• Consolidate multiple clickstreams
• Overlay Adobe classifications
• Consolidate disparate RSID’s
WHERE WE ARE NOW
LET THE Algorithms DO THE TALKINGAshish Braganza
Director, Global Business Intelligence
Lenovo 2017
Business Question:
Can we identify today’s visitors that will
account for 80% of future purchases?
Lenovo 2017
Use Case:
Display Retargeting Optimization
Lenovo 2017
So Why Algorithmic Retargeting?
Lenovo 2017
Algorithmic vs Rule-Based
Rule-Based Algorithmic
• Segments are heuristic
• Building rules are manual and can get
complicated very quickly
• Managing rules requires ongoing attention
• Hard to control rule-based audience size
o Some rules create small audience
sizes e.g. Cart Abandoners
o Other rules pick up everyone e.g.
Homepage or Product pages
• Segments are data driven
• Knowing purchase likelihood scores
enables value-based bidding strategies
• Easy to adjust audience sizes
• Continues to relearn as data changes
• Algorithms manage the complexity
without user intervention
Lenovo 2017
So we first did some fancy math
Lenovo 2017
S....
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
* Test results validated for a 30 day period with +1M visitors with over 95% confidence level Lenovo 2017
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Random
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
• What’s the theoretical best case?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Random
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
Lenovo 2017* Test results validated for a 30 day period with +1M visitors with over 95% confidence level
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
•
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Random
Best Case
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
Lenovo 2017
• What’s the theoretical best case?
* Test results validated for a 30 day period with +1M visitors with over 95% confidence level
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Random
Best Case
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
Lenovo 2017
• What’s the theoretical best case?
• Test 1: How good are rules?
* Test results validated for a 30 day period with +1M visitors with over 95% confidence level
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
• Test 1: How good are rules?
•
Test 1*: Rule-based Retargeting Measurements
SegmentVisitors
(Cumm.)
Future Purchasers (Cumm.)
Conversion Rate
(Cumm.)
Cart Abandoners 8% 21% 7.0%
Product Viewers 44% 42% 2.4%
Home 57% 43% 1.9%
Other 100% 100% 2.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Rule Based
Random
Best Case
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
Lenovo 2017
• What’s the theoretical best case?
* Test results validated for a 30 day period with +1M visitors with over 95% confidence level
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
• Test 2: How good are algorithmic segments?
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Rule Based
Random
Best Case
Lenovo 2017
• What’s the theoretical best case?
* Test results validated for a 30 day period with +1M visitors with over 95% confidence level
• Test 2: How good are algorithmic segments?
•
Test 2: Algorithmic Retargeting Measurements
SegmentVisitors
(Cumm.)
Future Purchasers (Cumm.)
Conversion Rate
(Cumm.)
High (>0.9) 3% 55% 41.3%
Medium (0.5 – 0.9) 9% 80% 21.7%
Low (0.3 – 0.5) 22% 90% 10.5%
Very Low (<0.3) 100% 100% 2.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0% 20% 40% 60% 80% 100%
Fu
ture
Pu
rch
asers
Today’s Visitors
Algorithmic
Rule Based
Random
Best Case
Business Question: Can we identify today’s visitors that will account for 80% of future purchases?
Lenovo 2017
• What’s the theoretical best case?
* Test results validated for a 30 day period with +1M visitors with over 95% confidence level
• Without any rules or algorithms we need to
randomly pick 80% of the visitors
The Constructs of the Algorithm…
• Every hour the algorithm estimates the visitors’ likelihood to make a
future purchase
• The model learns from visitor’s past online behavior (last 7 days)
• Adobe Analytics data used
• All page views
• All events
• The model relearns every week
• Testing – 30 days with +1M visitors
Lenovo 2017
And the results were…
Lenovo 2017
Bigly Huuuge!!
Lenovo 2017
It’s the Dogs…
Lenovo 2017
Rule-based Algorithmic
A/B Test Design and Methodology
50%
6.1M
6 Weeks
50%
6.1M
6 Weeks
Population
Duration
Splits
Metrics Impressions, Conversion, Revenue & Expenses Impressions, Conversions, Revenue & Expenses
Geo United States United States
Population
Duration
Splits
Metrics
Geo
- Anyone who added a product to the shopping cart
- Anyone who viewed a Product Page
(Yoga, X, Y, etc.)
- Anyone who visited the Homepage
- High: 15X over average conversion rate
- Medium: 3X over average conversion rate
- Low: 0.5X over average conversion rate
Lenovo 2017
Results and opportunities for efficiencies identified…
• Reduced display ad costs by over 97% maintaining the same conversion rate.
• Impression counts for algorithmic segments significantly smaller yielding a
nearly equal number of purchasers.
Duration: 6 weeks Geo: United States
Segments Visitors Purchasers Conversion Rate Net Spend Impressions
Algorithmic 6.1M 39.5K 0.6417% $8.3K 2M
Rule-Based 6.1M 39.4K 0.6401% $356K 86M
Total 12.2M 79K 0.6409% $364K 88M
Lenovo 2017
How Algorithmic Retargeting Works
Lenovo Hadoop
Cluster
Adobe Marketing
Cloud
www.lenovo.com
Publishers
Audience Manager
Analytics
DSPsBehavioral Traits
Clickstream
Ads
Impressions
- Behavioral Schema
- Machine Learning
- Audience Manager API
DSPs
Tags
Q&A
#AdobeSummit
Submit your questions using the
Your question, or one similar, may already be in the queue
Don’t forget to tweet using
to your account using the details sent to you when you registeredsocial, the schedule and the session
Complete the
▪
▪
▪
▪
9 Session Prize
