Are Bad Bots Destroying Your Conversion Rate and Costing You Money?
-
Upload
distil-networks -
Category
Technology
-
view
196 -
download
0
Transcript of Are Bad Bots Destroying Your Conversion Rate and Costing You Money?
![Page 1: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/1.jpg)
Antoine Zammit Elias Terman
VP of MarketingVP of Technology
Don’t Let Bad Bots
Deflate Your Conversion
Rates and Brand
+28 More Brands!
![Page 2: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/2.jpg)
Good Bots, Bad Bots, and Human Traffic
![Page 3: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/3.jpg)
The Open Web Application Security Project (OWASP) is an important standards body in the application security community. Their annual top
10 threats list is the basis for many web application security programs. They are now expanding their scope to include automated threats -
bots.
SUBSET OF THREATS NAME DEFINING CHARACTERISTICS
Account Aggregation
Account Creation
Credential Cracking
Credential Stuffing
Use by an intermediary application that collects together multiple accounts and interacts on their behalf
Create multiple account for subsequent misuse
Identify valid login credentials by trying different values for username and/or passwords
Mass log in attempts to verify the validity of stolen username/password pairs
Carding
Card Cracking
Cashing Out
Multiple payment authorisation attempts used to verify the validity of bulk stolen payment card data
Identify missing start/expiry dates and security codes for stolen payment card data by trying different values
Buy goods or obtain cash utilising validated stolen payment card or other user account data
Footprinting
Vulnerability Scanning
Fingerprinting
Probe and explore application to identify its constituents and properties
Crawl and fuzz application to identify weaknesses and possible vulnerabilities
Elicit information about the supporting software and framework types and versions
OTHER
Ad Fraud
CAPTCHA Bypass
Denial of Service
Expediting
Scalping
Scraping
Skewing
Sniping
Spamming
Token Cracking
False clicks and fraudulent display of web placed advertisements
Solve anti-automation tests
Target resources of the application and database servers, or individual user accounts, to achieve denial of service
Perform actions to hasten progress of usually slow, tedious or time-consuming actions
Obtain limited-availability and/or preferred goods/services by unfair methods
Collect application content and/or other data for use elsewhere
Repeated link clicks, page requests or form submissions intended to alter some metric
Last minute bid or offer for goods or services
Malicious or questionable information addition that appears in public or private content, databases or user messages
Mass enumeration of coupon numbers, voucher codes, discount tokens, etc
100% OF OWASP AUTOMATED THREATS (BOTS) TARGET TRAVEL INDUSTRY
PAYMENT
CARDHOLDER
DATA
ACCOUNT
CREDENTIALS
VULNERABILITY
IDENTIFICATION
This work is licensed under the Creative Commons Share-Alike License for OWASP Automated Threat Handbook Web Applications by Distil Networks
![Page 4: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/4.jpg)
The bad bot landscape
How bad bots impact the travel industry
Web/screen scraping and spinning (hoarding)
Increased GDS pull costs
Decreased SEO, slowdowns, and downtime
Account takeover, credit card fraud, and points fraud
Skewed conversion metrics and look-to-book ratios
WMPH Vacations Case Study
Q&A
Agenda
![Page 5: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/5.jpg)
Advanced Persistent Bots
Basic scripts running
in command line
Headless browsers,
advanced scripts,
Cycle IPs and User
Agents
Real browser
automation, malware
APBs
75%
![Page 6: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/6.jpg)
More Bad Bots Claim to Be Mobile
The amount of bad bots claiming to be
mobile browsers jumped 42.78% in 2016
![Page 7: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/7.jpg)
Mobile App Tools Used by Bot Operators
Mobile Device Farms Mobile Device Emulators Debugging Software
Mobile device emulators that mimic
human users
Testing systems that mimic human users
on mobile devices (e.g. AWS Device
Farm, Google Firebase Testing Lab)
Debugging software used for
tampering with SDKs/reverse
engineering the app
![Page 8: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/8.jpg)
About Distil Networks
Industry Expertise
● Invented the category
● The recognized leader
● 70 airline customers
The Most Effective Technology
● Wider: Web, API, and Mobile
● Deeper: Catch more bots
● Smarter: Without impacting users
Vigilant and Dedicated Partner
● Not A Solution, Your Solution
● Unprecedented access
● An extension of your team
Bot Defense as Adaptable and Vigilant as the Threat Itself
Travel Industry Leaders Rely on Distil...
![Page 9: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/9.jpg)
True or False?
You have good visibility and control
over unwanted website traffic and
transactions.
Poll
Question
![Page 10: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/10.jpg)
You’ve Been Scraped
OWASP AUTOMATED THREAT: SCRAPING
Scraper Bot Sophistication
![Page 11: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/11.jpg)
![Page 12: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/12.jpg)
CompetitorsContent Theft
Competitive Intel
Price Scraping
AggregatorsStart-ups
Unauthorized Middlemen
Hackers / FraudstersContent for Fake Pages
Search EnginesGoogle
Bing
Yahoo
Baidu
Who is behind Web Scraping?
![Page 13: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/13.jpg)
What Kind of Data is Being Scraped?
Customer data
Pricing info
Editorial content
GDS API pulls
SEO strategies
Booking engine inputs
![Page 14: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/14.jpg)
Spinning (Hoarding) by Unauthorized Middlemen
Middlemen using mobile device emulators to continuously
hold seats in the airline booking engine, but not buying
Resell on a secondary market once a buyer is found
Monetary damage:
➔ Empty seats on planes
➔ Loss of add-on sales like upgrades, travel insurance,
etc. (about $20 to $40 of additional revenue per sale
for airlines*)
AIRLINE
CUSTOMER USE CASE
Spinning via
Mobile App
Emulators
Source: http://www.eyefortravel.com/mobile-and-technology/scraping-single-biggest-threat-travel-industry*
![Page 15: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/15.jpg)
Application Denial of Service
OWASP AUTOMATED THREAT: DENIAL OF
SERVICE
Denial of Service Bot
Sophistication
![Page 16: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/16.jpg)
DDoS vs. Application Denial of Service
Application Denial of Service
Attacks the application directly
Hard to spot because it won’t show up
as an anomaly on your firewall and
may not impact load balancer
DDoS
Attacks the ISP hosting your
application
Easier to spot because it floods
upstream infrastructure to point where
packets never arrive at the web server
![Page 17: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/17.jpg)
Account Takeover, Credit Card
Fraud, and Loyalty Points Fraud
![Page 18: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/18.jpg)
Bad Bots Love Login Pages
OWASP AUTOMATED THREATS:
CREDENTIAL CRACKING, CREDENTIAL STUFFING
Account Takeover Bot
Sophistication
![Page 19: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/19.jpg)
How Credential Stuffing Works
Over 1 billion
usernames, passwords
combinations exist in the
wild
Credential stuffing exploits
our propensity to reuse
passwords across multiple
sites.
![Page 20: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/20.jpg)
Account Based Fraud
OWASP AUTOMATED THREATS:
CARDING, CARD CRACKING, CASHING OUT
Account Exploitation Bot
Sophistication
![Page 21: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/21.jpg)
Travel Rewards Fraud
Dark Web listings that indicate typical price
ranges for airline and hotel loyalty accounts:
Airline loyalty accounts: $3.20 - $208
Hotel loyalty accounts: $1.50 - $45
Source: http://blog.cxloyalty.com/the-cost-of-loyalty-accounts-on-the-dark-web-how-to-protect-members
72 percent of loyalty program managers say they
have experienced an instance of loyalty program
fraud firsthand
![Page 22: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/22.jpg)
Skewed Analytics and Look-to-Book Ratios
OWASP AUTOMATED THREAT: SKEWING
Sophistication level of bots
that skew analytics
![Page 23: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/23.jpg)
Sophisticated Bots Appear as Human in Analytic Data
53% of bots able to load external Assets (e.g. JavaScript)
These bots will skew marketing tools such as (Google
Analytics, A/B testing, conversion tracking, etc.)
![Page 24: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/24.jpg)
Skewed Analytics Leads to Misinformed Business Decisions
Inaccurate analytic data results in
Poor funnel analysis & optimization
Poor conversion rates
Inaccurate KPI tracking
Skewed look-to-book ratios
Difficulty in planning server expansion
![Page 25: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/25.jpg)
The bad bot problem I'm most
concerned about:
A. Web scraping
B. Account-based fraud
C. Skewed analytics / look-to-book
D. Slowdowns and downtime
Poll
Question
![Page 26: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/26.jpg)
About WMPH Vacations
At a Glance
Founded 2004 / 140 employees
More than 600,000 clients booked
9 corporate brands
30 websites
Award-Winning Mobile App
Reservation systems serve both direct
customers and 45 agents
Private label solutions
![Page 27: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/27.jpg)
WMPH Technology Stack
30 different web properties
Mobile iCruise App for IOS & Android
Standardized web application stack
Employee Intranet
10 Virtual Servers on AWS
Cloud-based Phone System using 8x8 technology
Entire company is now over 90% cloud-based
API calls into everything from small cruise lines to
large Global Distribution Systems
![Page 28: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/28.jpg)
WMPH Bot Challenges
Bad Bot Challenges
Aggressive web scraping caused site
slowdowns
API scraping almost took a cruise partner
offline
Constant barrage of SQL injection attack
attempts caused lots of noise in logs
Spam on cruise inquiry forms polluted
backend systems
Bots skewed conversion metrics
![Page 29: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/29.jpg)
Tried Several Approaches to Solve the Problem...
Put CAPTCHAs on Forms Looked for Patterns Blocked IPs in AWS ELB
Creates a poor user experience Bots appear human in logs Defeated by distributed IP attacks
Defeated by advanced bots Labor intensive Defeated by low and slow crawlers
Defeated by CAPTCHA farms Distributed attacks hard to pinpoint Defeated by peer-to-peer / proxies
Reduces conversions rates Reactive in nature Reactive in nature
![Page 30: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/30.jpg)
WMPH Vacations Selection Criteria
Bot Detection and Mitigation Solution Requirements
Block web scrapers without impacting human visitors or
good bots like Googlebot
Increase website availability and speed
Simple setup
Little or no maintenance; “self-optimizing” solution
Protect APIs powering our websites and mobile apps
![Page 31: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/31.jpg)
Protect our web and mobile API servers
Fingerprint device
Verify browser
Verify device
Verify human
Verify Mobile Device ID
Verify mobile app
Verify device
Verify human
Stop bot operators (using mobile device farms,
device emulators, etc.) from accessing the API
servers that power our mobile apps
Prevent scrapers from hitting our
APIs through our website or by going
directly to our API servers
![Page 32: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/32.jpg)
WMPH Results with Distil
40% increase in response times; no slowdowns
since deploying Distil
Improved partner relationships
Leads up 100% – No more spam – Only serving
CAPTCHAs to bots
Conversion rates up 22%
Self-tuning, proactive approach saving 20 hours
per month
Protecting login of company intranet
![Page 33: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/33.jpg)
iCruise.com Traffic Overview
![Page 34: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/34.jpg)
iCruise.com Traffic Overview
![Page 35: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/35.jpg)
iCruise.com Traffic Overview
![Page 36: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/36.jpg)
iCruise.com Click Fraud Report
![Page 37: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/37.jpg)
Best Practices and Lessons Learned
IT and marketing need to partner on solving
the bad bot problem.
Review the Distil logs daily.
Blacklist aggressive bot IP numbers
Report aggressive IPs to their respective
IPSs. Follow up, and follow up, and follow
up.
Distil support will give you a list of urls being
hit by the bad bots. This will help you
determine what they are trying to do.
Don’t whitelist your office IP right away.
![Page 38: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/38.jpg)
www.distilnetworks.com/trial/
Offer Ends: October 31st at 5PM
Two Months of Free Service + Traffic Analysis
![Page 39: Are Bad Bots Destroying Your Conversion Rate and Costing You Money?](https://reader031.fdocuments.in/reader031/viewer/2022022415/5a66768d7f8b9a5e648b464d/html5/thumbnails/39.jpg)
Antoine Zammit Elias Terman
VP of MarketingVP of Technology