© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21 © 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit 1
Econometric modeling for marketers Dr. Sid Shah | Director, Business Analytics, Adobe Helen Southgate | Online Marketing Controller Strategy & Planning, BSkyB
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
2
Dr. Sid Shah Director, Business Analytics, Adobe [email protected] | @drsidshah
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
3
Helen Southgate Online Marketing Controller Strategy & Planning, BSkyB
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
4
Econometric models specify the statistical relationship between various variables and a phenomenon of interest
What are econometric models?
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
5
Econometrics let you
measure the unmeasurable!
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Econometric
6
Provide predictive insights 1 Answer complex what-if questions 2
Be applied to a wide variety of problems 3
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
A common attribution problem
7
0
2000
4000
6000
8000
10000
12000
0
20000
40000
60000
80000
100000
120000
140000
160000
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
SE
M O
rde
rs
Tv
Sp
en
d
TV Spend vs SEM Orders
TV Spend SEM Orders
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Too Many Moving Parts?
8
No
rma
lize
d
Imp
ress
ion
s Brand Impressions
No
rma
lize
d
Imp
ress
ion
s Fitness Impressions
No
rma
lize
d
Sp
en
d
TV Spend
Nov Dec Jan Feb
Nov Dec Jan Feb
Nov Dec Jan Feb
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
TV spend vs. SEM orders: The econometric analysis
9
Build
Model
Build deterministic econometric model Explicitly state assumptions
Apply Econometrics
Remove seasonal effects (de-trending) Statistically connect impressions with TV spend
Predict
Predict impressions as a function of TV spend Quantify impact of TV on search
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
10
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
The model
11
Assumptions • Additive linear model (OK for small changes) • Generic impressions (fitness) represents the seasonal trend • Brand keywords are usually at position 1 and hence bid is
relatively constant
𝛽𝐵𝑖𝑑𝐵𝑟𝑎𝑛𝑑 = 𝐼𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠𝐵𝑟𝑎𝑛𝑑@𝑃𝑜𝑠1 = 𝐶 𝑓 𝑆𝑒𝑎𝑠𝑜𝑛𝑎𝑙𝑖𝑡𝑦 ~𝐼𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠𝑁𝑜𝑛 𝐵𝑟𝑎𝑛𝑑
𝐼𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠𝑆𝑒𝑎𝑟𝑐ℎ_𝐵𝑟𝑎𝑛𝑑 − 𝑓 𝑆𝑒𝑎𝑠𝑜𝑛𝑎𝑙𝑖𝑡𝑦 = 𝛼𝑆𝑝𝑒𝑛𝑑𝑇𝑉 + 𝐶
𝐼𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠𝑆𝑒𝑎𝑟𝑐ℎ_𝐵𝑟𝑎𝑛𝑑 = 𝑓 𝑆𝑒𝑎𝑠𝑜𝑛𝑎𝑙𝑖𝑡𝑦 + 𝛼𝑆𝑝𝑒𝑛𝑑𝑇𝑉 + 𝛽𝐵𝑖𝑑
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
12
Someone told me that
each equation I included in the book
would halve the sales
Stephen Hawking
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Detrending
13
Ind
exe
d C
ha
ng
e t
o W
ee
k 0
Brand Impressions Change
Ind
exe
d C
ha
ng
e t
o W
ee
k 0
Seasonality
-20%
0%
20%
40%
60%
80%
100%
120%
140%
0%
5%
10%
15%
20%
25%
30%
35%
40%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9
Ind
exe
d C
ha
ng
e t
o W
ee
k 0
TV Attributable Change
Tv Attributeable Change Tv Spend*Data presented is for exposition only
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Regression
14
y = 1E-06x + 1.1351 R² = 0.8628
0
0.5
1
1.5
2
2.5
3
3.5
0
20
0,0
00
40
0,0
00
60
0,0
00
80
0,0
00
1,0
00
,00
0
1,2
00
,00
0
1,4
00
,00
0
1,6
00
,00
0
Imp
ress
ion
Vo
lum
e J
um
p
Media Spend
Impression Volume on Brand terms vs. TV Spend
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
TVs effect on search impressions
15
0
8%
21%
31%
38% 44%
49% 53%
57% 60%
63% 65%
0%
10%
20%
30%
40%
50%
60%
70%
TV Spend
TV Attributable SEM Brand Impressions
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Predict
16
0
500
1,000
1,500
2,000
2,500
3,000
0 5,000 10,000 15,000 20,000 25,000 30,000 35,000
Ord
ers
SEM Budget
SEM Performance for different TV Budgets
TV Spend Baseline TV Spend 4x
TV drives awareness which drives impressions and raises the SEM curve
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Multiplicative models for multi channel marketers
Sophisticated multi channel models
Account for multiplicative channel effects (synergies)
Assume non linearity ( Concave/sigmoidal response)
Account for time lags
Account for feedback mechanisms in inputs and responses ( Vector Auto Regressive Models)
17
𝑶𝒖𝒕𝒄𝒐𝒎𝒆 = 𝐵𝑎𝑠𝑒𝑙𝑖𝑛𝑒 ∗ 𝐷𝑟𝑖𝑣𝑒𝑟1𝛼1 ∗ 𝐷𝑟𝑖𝑣𝑒𝑟2
𝛼2 … ∗ 𝐿𝑎𝑔 𝐹𝑎𝑐𝑡𝑜𝑟𝑠𝛾
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
A Continuous
Process
18
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
19
Econometrics let you
measure the unmeasurable!
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Econometric
20
Provide predictive insights 1 2 Answer complex what-if questions
Be applied to a wide variety of problems 3
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21 © 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit
Adobe Summit BSkyB
UEFA Champions League
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
22
So what is a
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Attribution models
Attribution = a method of assigning credit to a sale across marketing channels
Last click First click Even / linear distribution
Time decay Position based
(First, middle, last)
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Long live the last click model
Just make sure you understand what drives it
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Why do we need to look at attribution?
For our last campaign 83% of orders involved multiple clicks across key digital channels
Single Click 17%
Multiple Click 83%
Nu
mb
er
of O
rde
rs
Number of Clicks
1 2 3 4 5 6 7 8 9 10
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Three key objectives
Identifying inefficiencies
Identifying opportunities
Allocating spend to be most effective
What can attribution analysis tell us?
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Why is it still an unexplored area?
The challenges of attribution analysis C
on
fusi
on
Data Available
Volume of data Cross channel devices Support & knowledge gap
?
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
28
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Earlier participation varies per channel and inclusion of impression data
Channels add value beyond last click
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Affiliate Display Natural Search Paid Search
Not Last Exposure
Last Exposure
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Drill down into affiliate, search term, creative, partner etc.
Conversion
Partner 1
Partner 2
But be careful as within each channel there are variances
Single Click 37%
Multiple Clicks but Pure Channel
16%
Last Click but Multiple Channels
27%
Not Last Click but Involved
20%
Single Click 19%
Multiple Clicks but Pure Channel
10%
Last Click but Multiple Channels
24%
Not Last Click but Involved
47%
Partner 1
Partner 2
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Shows value of search higher up the funnel
Search key word analysis
0
0.5
1
1.5
2
2.5
3
Brand Generic
Ratio of Assist to Every Conversion
0
0.5
1
1.5
2
2.5
3
3.5
Individual keywords ratio of assist to every conversion
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
32
So what actionable insight is there
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Actionable Insight Attribution analysis is pointless unless you do something with it
33
Understand the wider value of some of our marketing partners, particularly in the affiliate channel which has led to differing remuneration models and supported a 5% increase in affiliate channel orders 1
Evaluate the cost effectiveness of generic search terms in terms of the customer purchase journey. Increase / decrease spend on
2 customer journey across all channels 3
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Customer is the heart
Beyond online attribution
© 2013 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
#AdobeSummit @drsidshah @HelenMarie21
Dr. Sid Shah [email protected] Twitter: @drsidshah LinkedIn: Siddharth Shah
Helen Southgate Email: [email protected] Twitter: @HelenMarie21 LinkedIn: http://www.linkedin.com/in/helensouthgate
Top Related