Morphing GA into an Affiliate Analytics Monster

Morphing GA into an Affiliate Analytics Monster

By Phil Pearce

#MeasureCamp @PHILPEARCE

London 25th March 2017

Warning / Disclaimer

1. GA is design for audience measurement (not affiliates tracking)

2. Run this on a parallel GA account (not your main GA account) as it will report very different numbers and use _ga2 cookie.

3. This is not a simple process…

Pls Download and import this GTM file: bit.ly/affiliatemonsterdownload Or use this generic login: username: [email protected] password: philrulesgtm7 Select "Affiliate Monster container"

Working Demo

http://bit.ly/affiliatemonsterdownload


…but, I like to push the boundaries of what`s possible

(with GA)…

…and I`m a slightly Mad Analytics Scientist!

Note: It has taken lots of <body>s to get to this point…

GTM <body>`s from the past...

But it worked! and… the data came alive :)

Before vs After

Agenda

Agenda 1. Define the problem

– Business Alignment (custom business logic in affiliate industry)

– 1st party vs 3rd party cookies

– GA is whitelisted on Adblocker plus

– Existing affiliateId=123 params ignored by GA (and utm_id not used by affiliates)

2. Solution

– Mapping affiliateId to utm_id

– Overwriting setLocation & setReferral

– DimensionWidening via CSV upload of affiliateId >> source/medium/campaign

3. Learnings/Mistakes

– Enabling utm campaign cookie to prevent session reset on 2nd page

– Accounting for #utm vs ?utm

4. Future improvements

– Organic and Social organic cookie

– Cross-domain tracking of source and affiliateId (#_ga=xxx.xxx pigybacking)

– GoogleSheet for affiliateId DimensionWidening

5. Questions

This is the story of one such mad Analytics Experiment…

So, new super-affiliate client just signed!

• GA classic >> GA universal migration via GTM • 5 CMS environments

(Desktop, New Mobile, Old mobile site, Tablet, Campaign Site) • utm tags only used on email and banners • Affiliates 90% of traffic but missing utm tags. Hence, Direct was 80% of traffic! • Marketing Channel Analysis not possible in GA!!

Source: builtwith.com/detailed/behemothclient.com

5 CMS platforms: Desktop, New Mobile, Old mobile site,

Tablet and Campaign Site.

Cleaning & Standardise Equipment before the

big operation

http://builtwith.com/detailed/behemothclient.com

Testing on Mice

1. Parallel Tracker 2. Unit Tests & Sinario Tests 3. Query Time utm_id widening

Alive indefinitely

Unit Test / Mice Test

Code Refinements

1 day Live test

Standardise GTM Setup Affiliate

Monster

Project Timeline 3 months bring the beast to life

In Reality…

It was really hard!

Insert Picture of - Phil with no hair

and - lots of dead mice

But we did get there :)

But about those dead mice…

Why not use native GA?

1. setSource utm field (rather than setLocation)

2. Custom Multi-channel funnels

3. BigQuery sessionisation [GA Prem. feature]

4. GA 6 mth sessions setting changed to 30days

5. utm_nooveride=1 not used

Recipe for a Affiliate Monster 1. Have GTM on all pages 2. Declare affiliateTracker name 3. Declare ga2 object 4. Declare ga2 cookie 5. setLocation override in pageview 6. setReferral override in pageview 7. Be on SSL (or enable meta always send referral)

Mistakes to Avoid

1. cleaning/removing existing utm params

2. Accounting for the order of params

3. #utm vs ?utm or both

4. removing double-quotes

5. JS logic: gclid/dclid >> utm >> affiliateId >> organic >> social referral >> referral

Pls Download and import this GTM file: bit.ly/affiliatemonsterdownload Or use this generic login: username: [email protected] password: philrulesgtm7 Select "Affiliate Monster container"

Working Demo



part1: jsv_pageHostnameTLD function(){

var domain={{Page Hostname - www removed}};

var rootDomain = undefined;

if (domain) {

domain.toLowerCase();

}

if (domain.substring(0,4)=="www.") {

domain=domain.substring(4,domain.length);

}

var domParts = domain.split('.');

for(var i=1;i<=domParts.length;i++)

{

document.cookie="gtmtestcookie1=1; path=/; domain="+domParts.slice(i*-1).join('.');

if(document.cookie.indexOf("gtmtestcookie1") != -1){

var rootDomain = domParts.slice(i*-1).join('.');

document.cookie="gtmtestcookie1=1; expires=Thu, 01 Jan 1970 00:00:01 GMT; path=/; domain="+domParts.slice(i*-1).join('.');

break;

}

}

return rootDomain;

}

part2: setCookie for utm_id setTrackingSessionCookie

<script>

function setTrackingSessionCookie(trackingCookieName, cookieValue, cookieDaysDuration) {

var pageHostnameTLD = {{jsv_pageHostnameTLD}};

var cookieExpires = "";

if (cookieDaysDuration) {

var date = new Date();

date.setTime(date.getTime() + (cookieDaysDuration *24*60*60*1000) );

cookieExpires = "; expires=" + date.toGMTString(); // If not provided, it will be a session cookie.

}

// Session SetCookie

if (trackingCookieName && cookieValue) {

document.cookie = trackingCookieName + '=' + cookieValue + cookieExpires + '; domain=' + pageHostnameTLD + '; path=/';

}

};

// Set Based on URL or URL Hash

setTrackingSessionCookie("gclid", {{Page URL Query tracking gclid}} , "30");

setTrackingSessionCookie("dclid", {{Page URL Query tracking dclid}} , "30");

setTrackingSessionCookie("utm_id", {{Page URL Query tracking utm_id}} , "30");

setTrackingSessionCookie("utm_source", {{Page URL Query tracking utm_source}} , "30");

setTrackingSessionCookie("utm_medium", {{Page URL Query tracking utm_medium}} , "30");

setTrackingSessionCookie("utm_campaign", {{Page URL Query tracking utm_campaign}} , "30");

setTrackingSessionCookie("utm_content", {{Page URL Query tracking utm_content}} , "30");

setTrackingSessionCookie("utm_keyword", {{Page URL Query tracking utm_keyword}} , "30");

setTrackingSessionCookie("utm_userid", {{Page URL Query tracking utm_userid}} , "30");

</script>

part3: setLocation

<script>

function setTrackingLocation() {

var pageProtocol = {{Page URL Protocol}};

var pageHostname = {{Page Hostname}};

var pagePath = {{Page Path}};

var pageQuery = {{Page URL Query}};

var referrerHostname = {{Referrer Hostname - www removed}};

var countryCode = {{Lookup CountryCode from Country1 - PP reminder to check DE SH or DESH or SH}};

var deviceType = {{Lookup bespokeDeviceCategory from ContainerID}};

var cookieGoogleClickid = {{cookie gclid}};

var cookieDoubleClickId = {{cookie dclid}};

var cookieUtmId = {{cookie utm_id}};

var cookieUtmSource = {{cookie utm_source}};

var cookieUtmMedium = {{cookie utm_medium}};

var cookieUtmCampaign = {{cookie utm_campaign}};

var cookieUtmContent = {{cookie utm_content}};

var cookieUtmKeyword = {{cookie utm_keyword}};

var cookieAffiliateId = {{cookie affiliateId}};

var cookieIsLandingPageHit = {{cookie session gtm_isLandingPageHit}};

cookieIsLandingPageHit = ( cookieIsLandingPageHit == undefined ) ? "true" : "false";

// 0. Remove double-quote within utm tags... ""textlink"/adcash" to "textlink/adcash"

// 0. Change undefined to empty string so no risk of "utm_source=undefined"

pageQuery = (typeof pageQuery !== 'undefined') ? pageQuery.replace(/["]+/g, '') : "";

cookieAffiliateId = (typeof cookieAffiliateId !== 'undefined') ? cookieAffiliateId.replace(/["]+/g, '') : "";

cookieUtmId = (typeof cookieUtmId !== 'undefined') ? cookieUtmId.replace(/["]+/g, '') : "";

cookieUtmSource = (typeof cookieUtmSource !== 'undefined') ? cookieUtmSource.replace(/["]+/g, '') : "";

cookieUtmMedium = (typeof cookieUtmMedium !== 'undefined') ? cookieUtmMedium.replace(/["]+/g, '') : "";

cookieUtmCampaign = (typeof cookieUtmCampaign !== 'undefined') ? cookieUtmCampaign.replace(/["]+/g, '') : "";

cookieUtmContent = (typeof cookieUtmContent !== 'undefined') ? cookieUtmContent.replace(/["]+/g, '') : "";

cookieUtmKeyword = (typeof cookieUtmKeyword !== 'undefined') ? cookieUtmKeyword.replace(/["]+/g, '') : "";

// 1. Remove internal utm tag that is RESETTING the sessions

cookieUtmSource = (cookieUtmSource === "Internal link") ? "" : cookieUtmSource;

cookieUtmMedium = (cookieUtmMedium === "Internal link") ? "" : cookieUtmMedium;

cookieUtmCampaign = (cookieUtmCampaign === "Internal Campaign") ? "" : cookieUtmCampaign;

cookieUtmCampaign = (cookieUtmCampaign === "Internal Campaign") ? "" : cookieUtmCampaign;

pageQuery = (pageQuery === "utm_source=Internal link&utm_medium=Internal link&utm_campaign=Internal Campaign") ? "" : pageQuery;

pageQuery = (pageQuery === "utm_source=Internal link&utm_medium=Internal link&utm_campaign=Internal Campaign") ? "" : pageQuery;

pageQuery = (pageQuery === "utm_source=Internal%20link&utm_medium=Internal%20link&utm_campaign=Internal%20Campaign") ? "" : pageQuery;

pageQuery = (pageQuery === "utm_source=Internal%20link&utm_medium=Internal%20link&utm_campaign=Internal%20Campaign") ? "" : pageQuery;

// 3. Removed existing utm_source tags in URL - so they are NOT appended TWICE

var cleanPageQuery;

if( pageQuery ) {

console.log("GTM2 utm_id testing - 01. pageQuery: " + pageQuery);

var myRegEx = /^(.*?)([?&#]?(utm_source|utm_medium|utm_campaign|utm_content|utm_keyword|utm_id)=.*?){0,6}([?&#](.*))?$/i;

if(myRegEx.test(pageQuery)===true ) {

var part1 = myRegEx.exec(pageQuery)[1];

var part2 = myRegEx.exec(pageQuery)[5];

part1 = (part1 === undefined) ? "" : part1; // change undefined to empty string

part2 = (part2 === undefined) ? "" : part2; // change undefined to empty string

part2 = (part1 && part2) ? "&" + part2 : part2; // If both part1 and part2 populated - then preappend "&" to part2

cleanPageQuery = part1 + part2;

console.log("GTM2 utm_id testing - 02. cleanPageQuery: " + cleanPageQuery);

} else {

cleanPageQuery = pageQuery;

console.log("GTM2 utm_id testing - 03. cleanPageQuery: " + cleanPageQuery);

}

};

// 4. Declare output variable

var output = undefined;

/******************************************

// BEGIN landing page hits

******************************************/

if(cookieIsLandingPageHit == "true") {

console.log("GTM2 utm_id testing - isThisLandingPage: " + cookieIsLandingPageHit);

// 4a. If Adwords gclid OR dclid present dont send utm tags OR utm_id as it is not needed

if(cookieGoogleClickid || cookieDoubleClickId) {

console.log("GTM2 utm_id testing - 04. cookieGoogleClickid: " + cookieGoogleClickid);

output = pageProtocol + '://' + pageHostname;

if (cleanPageQuery) {

output = output + pagePath + "?" + cleanPageQuery;

dataLayer.push({"location": output, "event":"donewith_affIdScript"});

console.log("GTM2 utm_id testing - 05x. gclid output: " + output);

} else {

output = output + pagePath;


console.log("GTM2 utm_id testing - 06x. gclid output: " + output);

};

return output;

// 4b. If existing utm tag present send these (with OR without the utm_id=cookieAffiliateId)

} else if(cookieUtmSource) {



output = output + pagePath + "?" + cleanPageQuery +

"&utm_id=" + cookieAffiliateId +

"&utm_medium=" + cookieUtmMedium +

"&utm_source=" + cookieUtmSource +

"&utm_campaign=" + cookieUtmCampaign +

"&utm_content=" + cookieUtmContent +

"&utm_keyword=" + cookieUtmKeyword;


console.log("GTM2 utm_id testing - 07x. cookieUtmSource query output: " + output);

} else {

output = output + pagePath +

"?utm_id=" + cookieAffiliateId +

"&utm_medium=" + cookieUtmMedium +

"&utm_source=" + cookieUtmSource +

"&utm_campaign=" + cookieUtmCampaign +

"&utm_content=" + cookieUtmContent +

"&utm_keyword=" + cookieUtmKeyword;


console.log("GTM2 utm_id testing - 08x. cookieUtmSource noquery output: " + output);

};

return output;

// 4c. If affiliateId is present and no utm tags found then construct fake utm tags

} else if(cookieAffiliateId) {

// Remove internal referrals from referrerHostname and change to lowercase

if( referrerHostname ) {

referrerHostname = referrerHostname.toLowerCase();

var myRegEx = /(^|.+\.)(ClientName\.(com|at|de|be|dk|it|sh)|tipdev\.com|local\.dev|localhost|local|nblz\.ru|wbprx\.com|([0-9]{1,3}\.){3}[0-9]{1,3})$/i;

if(myRegEx.test(referrerHostname)===true ) {

referrerHostname = undefined;

}

};

// If referrerHostname is empty change to "aff" + cookieAffiliateId

referrerHostname = (referrerHostname == undefined || referrerHostname == null || referrerHostname === "") ? "aff" + cookieAffiliateId : referrerHostname; // If referrerHostname is empty output cookieAffiliateId

countryCode = (countryCode === "COM") ? "INT" : countryCode; // Replace "COM" with "INT"

deviceType = (deviceType == undefined) ? "" : deviceType; // If utm_content is empty output an empty string

console.log("GTM2 utm_id testing - 09. referrerHostname: " + referrerHostname);

console.log("GTM2 utm_id testing - 10. countryCode: " + countryCode);

console.log("GTM2 utm_id testing - 11. deviceType: " + deviceType);



output = output + pagePath + "?" + cleanPageQuery +

"&utm_id=" + cookieAffiliateId +

"&utm_medium=affiliate" +

"&utm_source=" + referrerHostname +

"&utm_campaign=" + countryCode + "-" + cookieAffiliateId +

"&utm_content=" + deviceType;

// + "&utm_keyword=";


console.log("GTM2 utm_id testing - 12x. AffiliateId query output: " + output);

} else {

output = output + pagePath +

"?utm_id=" + cookieAffiliateId +

"&utm_medium=affiliate" +

"&utm_source=" + referrerHostname +

"&utm_campaign=" + countryCode + "-" + cookieAffiliateId +

"&utm_content=" + deviceType;

// + "&utm_keyword=";


console.log("GTM2 utm_id testing - 13x. AffiliateId noquery output: " + output);

};

return output;

// 4e. Default

} else {





console.log("GTM2 utm_id testing - 14x. default query output: " + output);

} else {



console.log("GTM2 utm_id testing - 15x. default noquery output: " + output);

};

return output;

};

/******************************************

// BEGIN Second page hits

******************************************/

} else {


console.log("GTM2 utm_id testing - isThisLandingPage: " + cookieIsLandingPageHit);




console.log("GTM2 utm_id testing - 16x. secondPage - default query output: " + output);

} else {



console.log("GTM2 utm_id testing - 17x. default noquery output: " + output);

};

return output;

}

// On HitCallback...

// setTrackingSessionCookie("gtm_isLandingPageHit", "false");

};

setTrackingLocation();

</script>

Part4: isLandingPage cookie check

part5: isLandingPageHit

function() {

return function() {

setTrackingSessionCookie("gtm_isLandingPageHit", "false");

}

}

Part5: Bring it all together

Scripts activated here

{{jsv_setLandingPageHit - TRUE}}

Wins

1. No need to email 1000 Affiliates to re-tag links!

2. No IT dept involvement (all done in GTM)

3. Better cookie retention vs Affiliate cookies (esp. on Safari)

4. Better internal trust & actionability in GA data

5. Free!

Next iteration

1. Engagement scoring per AffilaiteID

2. utilise affiliateId/gaClientId cross-domain tracking and better affiliateId retention

3. CRM integration (similar to utmz method)

4. MeasurementProtocol 4 qualified affiliate sales

5. utm_userid respawn on mailchimp email

Questions & Prizes Phil Pearce Senior Web Analyst and GTM specialist

[email protected]

linkedin.com/in/philpearce

Web

Analytics

Exchange

mentor

750 GA

questions answered Tracking

protection

group

Free Prize for the best question(s).

http://www.linkedin.com/in/philpearce

http://www.linkedin.com/in/philpearce

bit.ly/gtmdevguide2 (free) Prize: GTM developer guide

free!! bit.ly/gtmdevguide2

http://bit.ly/gtmdevguide2

Morphing GA into an Affiliate Analytics Monster

Data & Analytics

Transcript of Morphing GA into an Affiliate Analytics Monster