State of digital ad fraud 2017 by augustine fou
-
Upload
dr-augustine-fou-independent-ad-fraud-researcher -
Category
Internet
-
view
2.139 -
download
1
Transcript of State of digital ad fraud 2017 by augustine fou
State of Digital Ad FraudJanuary 1, 2017 Update
January 2017Augustine Fou, [email protected] 212. 203 .7239
Ad Fraud is VeryLucrative and Scalable
January 2017 / Page 3marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How profitable is ad fraud? EXTREMELY
Source: https://hbr.org/2015/10/why-fraudulent-ad-networks-continue-to-thrive
“the profit margin is 99% … [especially with pay-for-use cloud services ]…”
Source: Digital Citizens Alliance Study, Feb 2014
“highly lucrative, and profitable… with margins from 80% to as high as 94%…”
January 2017 / Page 4marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How scalable are fraud operations? MASSIVELYCash out sites are massively scalable
131 ads on pageX
100 iframes=
13,100 ads /page
One visit redirected dozens of timesKnown blackhat technique to hide real referrer and replace with faked referrer.
Example how-to:http://www.blackhatworld.com/blackhat-seo/cloaking-content-generators/36830-cloaking-redirect-referer.html
Thousands of requests per pageSingle mobile app calling 10k impressions
Source: Forensiq
January 2017 / Page 5marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Example – AppNexus cleaned up 92% of impressions
Increased CPM prices by 800%
Decreased impression volume by 92%
Source: http://adexchanger.com/ad-exchange-news/6-months-after-fraud-cleanup-appnexus-shares-effect-on-its-exchange/
260 billion
20 billion
> $1.60
< 20 cents
“pity those advertisers who bought before the cleanup”
Ad Fraud Harms The Digital Ad Ecosystem
January 2017 / Page 7marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Ad fraud/ad spend are hitting all-time highs
Digital ad FRAUD
Digital ad SPENDSource: IAB 2016 F1H Report
$ billions
January 2017 / Page 8marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Ad fraud is now the largest form of crime
$20 billion
CounterfeitGoods U.S.
$18 billion
Somalipirates
$70B 2016E Digital Ad Spending
Bank robberies
$38 million
$31 billionU.S. alone
$1 billion
ATM Malware
Payment Card Fraud 2015
$22 billion
Source: Nilson Report Dec 2016
Source: ICC, U.S. DHS, et.
al
Source: World Bank Study 2013
Source: Kaspersky 2015
$7 in $100$3 in $100
“this is a PER YEAR number”
Digital Ad Fraud
Source: IAB H1 2016
$44 in $100
January 2017 / Page 9marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
CPM/CPC buckets (91% of spend) is most targeted
Impressions(CPM/CPV)
Clicks(CPC)
Search27%
91% digital spend
Display10%
Video7%
Mobile47%
Leads(CPL)
Sales(CPA)
Lead Gen$2.0B
Other$5.0B
• classifieds• sponsorship• rich media
(89% in 2015)Source: IAB 1H 2016 Report
(86% in 2014)
January 2017 / Page 10marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Two key ingredients of CPM and CPC FraudImpression (CPM) Fraud
(includes mobile display, video ads)
1. Put up fake websites and load tons of display ads on the pages
Search Click (CPC) Fraud
(includes mobile search ads)
2. Use fake users (bots) to repeatedly load pages to generate fake ad impressions
1. Put up fake websites and participate in search networks
2. Use fake users (bots) to type keywords and click on them to generate the CPC revenue
screen shots of fake sites
Fake Websites(cash-out sites)
January 2017 / Page 12marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Websites – spectrum from bad to good
Ad Fraud Sites
Click Fraud Sites
100% bot
mostly human
Piracy Sites
Premium Publishers
Sites w/ Sourced Traffic
“fraud sites” “sites w/ questionable practices” “good guys”
“real content that real humans want to read”
January 2017 / Page 13marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Identical sites – fraud sites made by template
100% bot
January 2017 / Page 14marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Countless fraud domains used to commit ad fraudhttp://analyzecanceradvice.comhttp://analyzecancerhelp.comhttp://bestcanceropinion.comhttp://bestcancerproducts.comhttp://bestcancerresults.comhttp://besthealthopinion.comhttp://bettercanceradvice.comhttp://bettercancerhelp.comhttp://betterhealthopinion.comhttp://findcanceropinion.comhttp://findcancerresource.comhttp://findcancertopics.comhttp://findhealthopinion.comhttp://finestcanceradvice.comhttp://finestcancerhelp.comhttp://finestcancerresults.comhttp://getcancerproducts.com
100M+ more sites like these, designed to profit from high value display, video, and mobile ads
Fake Visitors(bots)
January 2017 / Page 16marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Bots are automated browsers used for ad fraud
Headless BrowsersSeleniumPhantomJSZombie.jsSlimerJS
Mobile Simulators35 listed
Bots are made from malware compromised PCs or headless browsers (no screen) in datacenters.
Bots
January 2017 / Page 17marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Bots range in sophistication, and therefore cost
Javascript installed on webpage
Malware on PCsData Center BotsOn-Page BotsHeadless browsers
in data centersMalware installed on
humans’ devices
Less sophisticated Most sophisticated
Source: AdAge/Augustine Fou, Mar 2014 Source: Forensiq Source: Augustine Fou, Oct 2015
“not many people know that the official industry lists of bots catch NONE of these bots, not one.”
1 cent CPMsLoad pages, click
10 cent CPMsFake scroll, mouse movement, click
1 dollar CPMsReplay human-like mouse movements, clone cookies
January 2017 / Page 18marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Any device with chip/connectivity can be used as a botTraffic cameras used as botnet (Engadget, Oct 2015)
mobile devices
connected traffic lights
connected cars
thermostat connected fridge
Security cams used as DDoS botnet (Engadget, Jun 2016)
(TechTimes, Sep 2016)
Bot/Fraud Detection
January 2017 / Page 20marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Three main types of bot / fraud detectionIn-Ad
(ad iframes)On-Site
(publishers’ sites)
• Used by advertisers to measure ad impressions
• Limitations – tag is in foreign iframe, severe limits on detection
ad tag / pixel(in-ad measurement)
javascript embed(on-site measurement)
In-Network (ad exchange)
• Used by publishers to measure visitors to pages
• Limitations – most detailed and complete analysis of visitors
• Used by exchanges to screen bid requests
• Limitations – relies on blacklists or probabilistic algorithms, least info
ad served
bot
human
fraud site
good site
January 2017 / Page 21marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Fraud bots are NOT on any list
10,000bots observed
in the wild
user-agents.org
bad guys’ bots3%Dstillery
“findings from two independent third parties, Integral Ad Science and White Ops”
3.7%Rocket Fuel
“Forensiq results confirmed that ... only 3.72% of impressions categorized as high risk.”
2 - 3%comScore
“most campaigns have far less; more in the 2% to 3% range.”
bot list-matching
“not on any list”disguised as popular browsers – Internet Explorer; constantly
adapting to avoid detection
January 2017 / Page 22marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How Fraud HarmsGood Publishers
January 2017 / Page 23marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Significant ad revenue stolen from publishers
1. Bots collect “cookie” 2. Bots cause ad impressions on fake sites.
www.nejm.org healthsiteproductionalways.com
FOR EXAMPLE ONLY
January 2017 / Page 24marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
http://www.olay.com/skin-care-products/OlayPro-X?utm_source=msn&utm_medium=cpc&utm_campaign=Olay_Search_Desktop
Bad guys pretend to be good publishers’ sites
Click thru URL passes fake source “utm_source=msn”
buy eye cream online(expensive CPC keyword)
1. Fake site that carries search ads
Olay.com ad in #1 position
2. search ad served, fake click
Destination pagefake source declared
3. Click through to destination page
January 2017 / Page 25marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Bad measurements wrongly accuse publishers
Publisher clearly does not have 90% bots and never had
“you have low viewability”
“you have 90% bots”• We want a refund• We won’t pay• We want make-goods
January 2017 / Page 26marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Best Practices of Good Publishers
1. Reduce/eliminate shortcuts – mainstream publisher never sources traffic, never uses audience extension or other practices that artificially inflate impressions
2. Protect data and reputation – news publisher purged 30+ trackers from their sites to minimize “data leakage” and stopped selling remnant/unsold inventory on exchanges
3. Consistently prove ROI – specialty publisher limited ads to 3 per page, lazy loads all ads, filters all known bots by name; better business outcomes proven over time
“hard work and consistency will pay off”
January 2017 / Page 27marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How Fraud HarmsAdvertisers
January 2017 / Page 28marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
How many clicks/sessions/views do you want?
click on links
load webpages tune bounce rate
tune pages/visit
“bad guys’ bots are advanced enough to fake most metrics”
January 2017 / Page 29marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
What click through rates are you shooting for?Programmatic display
(18-45% clicks from advanced bots)Premium publishers(0% clicks from bots)
0.13% CTR(18% of clicks by bots)
1.32% CTR(23% of clicks by bots)
5.93% CTR(45% of clicks by bots)
Campaign KPI: CTRs
January 2017 / Page 30marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Want 100% viewability? 0% NHT (bots)?
Bad guys cheat and stack ALL ads above the fold to make 100% viewability.
“100% viewability? Sure, no problem.”
AD
• IAS filtered traffic, • DV filtered traffic• Pixalate filtered traffic, • MOAT filtered traffic, • Forensiq filtered traffic
“0% NHT? Sure, no problem.”
January 2017 / Page 31marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Best Practices of Savvy Advertisers“don’t assume your agency took care of it”
• Challenge all assumptions – don’t assume someone else “took care of it.” Verify, by demanding line-item detailed reports, because fraud hides easily in averages
• Check your Google Analytics - question anything that looks suspicious; more details that can reveal fraud and waste
• Corroborate measurements – measure different parameters together and see if they still make sense together; reduce false positives or negatives
• Use conversion metrics – CPG client uses click-and-print digital coupons; pharma client uses doctor finder zip code searches, plus clicks to doctor pages; retailers use sales
January 2017 / Page 32marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Ad Fraud Hits NewAll Time Highs
January 2017 / Page 33marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Methbot eats $1 in $6 of $10B video ad spend
Source: Dec 2016 Whiteops Discloses Methbot Research
“the largest ad fraud discovered to date, a single botnet, Methbot, steals $3 - $5 million
per day, $2 billion annualized.”
1. Targets video ad inventory$13 average CPM, 10X higher than display ads
2. Disguised as good publishersPretending to be good publishers to cover tracks
3. Simulated human actionsActively faked clicks, mouse movements, page scrolling
4. Obfuscated data center originsData center bots pretended to be from residential IP addresses
January 2017 / Page 34marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Mobile fraud is much larger than detected
“bad guys’ apps don’t install fraud detection SDKs; so the reported low rate of fraud is due to only good apps being measured.”
Mobile app install fraud research (via mxpresso)• 50 – 70% mobile devices were fake• 40 – 50% of the app installs were fake• 10 – 20% were faked Play Store installs
January 2017 / Page 35marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Implications for Digital Media
January 2017 / Page 36marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Humans block ads; fraud bots don’tComparing high human vs high bot samples
96% bots sample
42% ad blocked
1% ad blocked
93% human sample
Comparing ad blocking vs non-ad blocking samples
ad blocking ON
ad blocking OFF
January 2017 / Page 37marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Ad impressions served mostly to bots, by far
Total Human Users – 115 million
Visitors (U.S. Only)
U.S. Internet – 285 million
Source: eMarketer 2016 estimate Source: Distil Networks 2015
Adblock Users (humans) – 45 million
Source: PageFair / Adobe 2015
“subtracting adblocking humans, your open exchange ad impressions are being served to a population that is disproportionally non-human.”
Non-Human Traffic (NHT) HUMAN VISITORS
ads served
“fraud sites” “sites w/ questionable practices” “good guys”Websites
3% IVT caught by industry lists
39%Ad blocking humans
71% 29%
January 2017 / Page 38marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
No matter how much traffic, bots don’t convert
102,231 sessions
0 sessions
goal events – no change
bot traffic turned off
bot traffic turned off
January 2017 / Page 39marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Other Hidden Dangers
January 2017 / Page 40marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Analytics are messed up by fake data
7% conversion rate 13% conversion rateartificially low actually correct
January 2017 / Page 41marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Real human audiences stolen from publishers
specialized audience:human oncologists
jco.ascopubs.org
specialized audience can be targeted elsewhere
“cookie matching”(by placing javascript on your site)
FOR EXAMPLE ONLY
January 2017 / Page 42marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
In-ad measurements could be entirely wrong
Publisher Webpagepublisher.com
Foreign Ad iFramesadserver.com
Cross-domain (XSS) security restrictions mean iframe cannot:• read content in parent frame• detect actions in parent frame• see where it is on the page
(above- or below- fold)• detect characteristics of the
parent page
1x1 pixeljs ad tags ride along
inside iframe
incorrectly reported as 100% viewable
parent frameforeign iframes
January 2017 / Page 43marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
On-site Javascript poses gaping security risks
Source: https://www.exchangewire.com/blog/2016/05/19/%E2%80%8Bon-site-javascript-trackers-open-gaping-security-holes/
January 2017 / Page 44marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
From our First-hand Data
January 2017 / Page 45marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Visually show differences in quality / humanness
good publishers
ad exchanges/networks
volume bars (green)
Stacked percentBlue (human)Red (bots)
red v blue trendlines
January 2017 / Page 46marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Traffic surges caused by bots vs real humans
Caused by bots
Caused by humans
January 2017 / Page 47marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Publishers taking action to reduce bots
Publisher 1 – stopped buying traffic
Publisher 2 – filtered data center traffic
January 2017 / Page 48marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Stepwise improvement using our data
Period 1 Period 3Period 2
Initial baseline measurement
Measurement after first optimization
Eliminating several “problematic” networks
January 2017 / Page 49marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Advertisers buying low vs high quality media
Traffic to Site from Buying LOW quality media
Traffic to Site from Buying HIGH quality media
January 2017 / Page 50marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Better media leads to way better outcomesMeasure Ads Measure Arrivals Measure Conversions
clean, good media
low-cost media, ad exchanges
346
1743
5
156
30X better outcomes • More arrivals• Better quality
January 2017 / Page 51marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
About the Author
January 2017Augustine Fou, [email protected] 212. 203 .7239
January 2017 / Page 52marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Dr. Augustine Fou – Independent Ad Fraud Researcher2013
2014
Follow me on LinkedIn (click) and on Twitter @acfou (click)
Further reading:http://www.slideshare.net/augustinefou/presentationshttps://www.linkedin.com/today/author/augustinefou
2016
2015
January 2017 / Page 53marketing.scienceconsulting group, inc.
linkedin.com/in/augustinefou
Harvard Business Review – October 2015
Excerpt:
Hunting the Bots
Fou, a prodigy who earned a Ph.D. from MIT at 23, belongs to the generation that witnessed the rise of digital marketers, having crafted his trade at American Express, one of the most successful American consumer brands, and at Omnicom, one of the largest global advertising agencies. Eventually stepping away from corporate life, Fou started his own practice, focusing on digital marketing fraud investigation.
Fou’s experiment proved that fake traffic is unproductive traffic. The fake visitors inflated the traffic statistics but contributed nothing to conversions, which stayed steady even after the traffic plummeted (bottom chart). Fake traffic is generated by “bad-guy bots.” A bot is computer code that runs automated tasks.