The Science of Audience and Attribution Modeling

#BingThere

#BingThere

Matt FisherSr. Program ManagerDemand MonetizationBing Ads, Microsoft@StuckInTheNet

Kevin KleinInsights ManagerBing Ads, Microsoft@kkwrites

Today’s Speakers

#BingThere

6m 6m 6m

MAFISHER08

Matt FisherSr. Program Manager, Data Science / Ad SolutionsBing Ads, Microsoft@StuckInTheNet

C9H13NO3

Kevin KleinInsights Management, Bing Ads

B.A. in Jazz Guitar Performance

Willfully signs up for perennial heartbreak at the handsof the Washington Capitals

Avid reader of fantasy and science fiction

Blues guitar performer

#BingThere

• Attribution • What is it…..

• Bing UET • How can it help…..

• Remarketing• Tying it all together

Agenda

#BingThere

“In digitial marketing, Attribution is defined as the process of identifying a set of user actions (“events” or "touchpoints") that contribute in some manner to a desired outcome, and then assigning a value to each of these events.”*

#BingThere*Wikipedia

Attribution

#BingThere

#BingThere

Defining Attribution

#BingThere

Understanding attribution through the lens of….Basketball

Does this represent value? Or does this?

Karl Malone, AKA ‘The Mailman’

All time Utah Jazz points leader

Avg. 25 PPG (points per game)

Avg. 3.6 assist per game

John Stockton, AKA ‘Pasty Gangsta’

All time Utah Jazz assist leader

Avg. 10.5 assist per game

Avg. 13.1 PPG (points per game)

#BingThere

Understanding attribution through the lens of….Hockey

#BingThere

Does this represent value? Or does this?

Joe Thorton, AKA ‘Jumbo Joe’

All time San Jose Sharks assist leader

Total Assist w/ Sharks: 718

Avg. Assist per Game: 0.79

Games Played: 903

Patrick Marleau

All time San Jose Sharks points leader

Career Points w/ Sharks: 1,075

Avg. Points per Game: 0.72

Games Played: 1,479

#BingThere

Understanding attribution through the lens of….Soccer

#BingThere

Understanding attribution through the lens of… fantasy stories.

Does this represent the resistance to Voldemort? Or does this?

Descriptortext here

Understanding attribution through the lens of… fantasy stories.

Does this show the cumulative rebel power that destroyed two of the galaxy’s most sinister spacestations?

Or does this?Descriptortext here

Understanding attribution through the lens of… fantasy stories.

Does this show the fellowship responsible for delivering Middle-Earth from the brink of an Age under the influence of malevolent shadow?

Or does this?Descriptortext here

Understanding attribution through the lens of…fantasy stories.

When you don’t think about attribution in your search funnel, you’re failing to acknowledge your:• Hermione Grangers• Ron Weasleys• Han Solos• Princess Leias• Gandalfs• Samwise Gamgees

#Bing There

Choose an attribution model based on your overall marketing objectives

Growth

Efficiency

#Bing There

First Click Last Click Last Non-Direct Click Pole-Based

Attribution Models – The Most Common Attribution Models

Position Based Attribution Models

• Splits value to first & last clicks

• Not good for long purchase cycles

• Top of the Funnel (TOFU)• Favors: Awareness &

Acquisition• Most likely inefficient (low

ROI)

• Bottom of the Funnel (BOFU)

• Favors: Brand & Remarketing

• Highly efficiency (high ROI)

• Similar to Last Click• Removes last non-

paid channel

#BingThere

#Bing There

Attribution Models

Time Decay Reversed Time Decay LinearRegression Models All Touches are Equal

• Distributes across touch points• Uneven distribution for either Bottom of the Funnel or Top of the

Funnel

• Every touch point is created equal

Attribution Models

Data Driven

Adwords OnlyDigital “Fingerprint”

Data Driven Custom Model

• Unique to each company• A lot of effort/time investment • Based on YOUR internal performance data & models

• Single channel view only • Only based off of Adwords

Data

Which is the Perfect Attribution Model

First / Last ClickThe last of the first channel

clicked is awarded 100% for the conversion

Data-DrivenIncludes same level of

scoring – allocates value across assists to

the final conversions

Bing UET for Attribution & Remarketing

#BingThere

Bing Universal Event Tracking (UET) – One Code to Rule Them All

Who is using UET?

Remarketing?

The combination?

UET Tags enable important features across the search experience

Targeting criteria

Advertiser bid

Search query

Click to website

Conversion on

website

Remarketing – Create audience profiles based on past visitors

Auto-bidding – Modify bids to achieve conversion and CPA goals

Conversion tracking – Provides conversion attribution back to ad spend

Website

Set Up Goals to Deliver More Insights – Think Multiple Processes & Funnels

Goal

Goal

Goal Transaction

Actions

Engaged w/Content

Goals Completion

Visits & Visitors

Shipping Details

Add to Cart

Cart Visitors

Billing Details

Completion

Define Multi-Step Processes

Visits Longer than X time Visits More then X pageviews

Visit Duration

Account LoginsVideo ViewsViewed Specific Content/PagesLive Chat / Help

Request More Information / Contact UsDownload MaterialsForm CompletionsCreate an AccountItems Saved to Wishlist

Measuring & Distributing Influence Across Touch Points (Single Channel)

Car Seats

Top Safety Rated Infant Car Seats

Britax B-Safe 35

Britax vs Mesa Car Seats

Infant Car Seats

Query Path

0

0

100

0

0

5

15

50

25

10

50

15

5

10

25Education / Research

Comparison

Transaction

5

8

25

20

17

Use Goals to Create a More Inclusive View of Top to Bottom Performance

Assist shows the number of times each keyword contributed to a conversion that is associated with a different keyword

Download and analyze!

Key Learning:

Keyword example 10 has only produced 2 conversions during this time period, but it has assisted 28 other conversions!

Your To-Do List

Add UET Code to Website

Set-up & Define Goals

Use Assists or Goals to see full value across the funnel

Conversions Micro-ConversionsMulti-step Funnels

Pull keyword performance reports and include assists + goals to customized attribution model

To get a cross-channel view of performance, set-up the same goals from UET in your analytics platform.

Then you can pull the goals data by channel and campaign to understand

interactions.

Sample size : 1,000 users (US)Time : 2016/03/01 ~ 2016/05/31 Filters• Travel relevant domains : orbitz.com,

Tripadvisor.com, Expedia.com, Priceline.com, etc.

• Keywords : hotel, flight, resort, car rental, etc.

• User action : Hotel review, flight search, package search, etc.

Data Profile

#BingThere

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 880

10

20

30

40

50

60

70

80

Weighted Average:37 days

Hotel CDJ Length by User Distribution

Number of Users

#BingThere

Search tripadvisor booking expedia hotels priceline hilton trivago kayak marriott orbitz travelocity ihg bookingbuddy0

20

40

60

80

100

120

140

160

180

0

100

200

300

400

500

600

700

800

900

764

672

501 476422

303 287 278 263 257218 204 187

154

Avg_PV Avg_Time (min.) Users

Top Hotel CDJ Domains

32

Key Takeaways:

1 2 3 4 5 6 7 8 9 100

20

40

60

80

100

120

140

160

180

200

# of domain visits per user

#BingThere

Remarketing & Audiences

#BingThere

With Bid & Target adjustments, All Campaigns could be remarketing

campaigns…….

Include+% BidExclude-% Bid

Layering Audience Lists

Exclude

-% BidInclud

e+% Bid

SEARCH

DISPLAY EMAILS

FACEBOOK

LINKEDIN

SEARCH

DISPLAY EMAILS

FACEBOOK

Think Holistically about Audiences LINKEDINHow can you remarket to audiences other channels have engaged with?

#BingThere

Creating Cross-Channel Audiences:

Simple:Include targeting for other marketing channel:

• Affiliates• Social• Email• Display

Advanced:Onsite actions/behaviors and include specific audiences and demographics targeting by channel:

Social by Demographic Data:Facebook & LinkedIn• Age Range• Job Title• Employer• Interests

Basic:PPC only.Targeting onsite actions and behaviors.

Onsite Behaviors & Actions

Set Up Audience Lists to Tailor Content, Messaging & Optimization

Transaction

Actions

Content Engagement

Completion

Visit Duration

Shipping Details

Add to Cart

Cart Visitors

Billing Details

Completion

Define Multi-Step Processes

Cross Channel VisitorsDuration on Site (Bounces)

Visits, Visitors, Visit Duration

Logged In UserVideo ViewsViewed Specific Content/PagesLive Chat / Help Request More Information / Contact Us

Download MaterialsForm CompletionsCreate an AccountItems Saved to Wishlist

Customized & Target Content to the Audience Behind Your Clicks:

Everything tracked with Goals +

Cross-Channel TrafficCombination of pages visitedPages not visited Abandoned shopping cartsReferral URLDuration of time since last

visitLoyal Consumers

KeywordsAd CopyLanding Pages

Actions & Behaviors:Traditional PPC Targeting

Targeting outside of the Audience Lists

Messaging

Landing Pages

KeywordsTop of the Funnel

Refining the Funnel &

Congruent Customer Journeys

Optimization

Bid Strategy

Your Remarketing To-Do List

Define Target Audiences

Customize Content &PPC Campaigns

Customize Ad Copy, Keywords, Landing PagesTake Customers Through the Purchase Journey

Cross-Channel VisitorsMulti-Step Processes & FunnelsOnsite Behavior & Actions

Bid Optimization Strategy

Non-linearMulti-channel

PersonalizedLong term

Holistic

MessagingAttribution & Value

Digital strategy Purchase path

#BingThere

Q A

© 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation.

Because Microsoft must respond to changing market conditions, it should not be interpreted as a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the dateof this presentation.

Microsoft MAKES NO WARRANTIES–EXPRESS, IMPLIED, OR STATUTORY–REGARDING THE INFORMATION IN THIS PRESENTATION.