Securing Property PortalsLamudi Case Study

Platforms

Portals

Brokerages

MLS Customers

Premium Brands

Distil in Real Estate and Premium Brands

The New Threat Landscape of APBs

Advanced Persistent Bots (APBs)...

AdvancedMimick human behaviorLoad JavaScriptLoad external resourcesSupport cookiesBrowser automation (Selenium, PhantomJS)

Persistent Dynamic IP rotationDistribute attacks across IP addressesHide behind anonymous and peer-to-peer proxies 2015 Distil Bad Bot

Report

Homegrown Solutions Are Ineffective

Creates a poor user experience Bots appear human in logs Defeated by distributed IP attacks

Defeated by advanced bots Labor intensive Defeated by low and slow crawlers

Defeated by CAPTCHA farms Distributed attacks hard to pinpoint

Defeated by peer-to-peer / proxies

Reduces conversions by up to 27%

Reactive in nature Reactive in nature

Web App Security Requires Complementary Solutions

l

DDoS Mitigation Firewall WAF Distil Bot Protection

Core Competency

Volumetric attacks on

infrastructureNetwork layer attacks Application coding

exploitsAutomated abuse, misuse, and

attacks (scraping, fraud, account takeover, etc.)

Techniques Scrubbing centers,

Large pipes

Access Control Lists (ACLs),

Rules-BasedApp layer understanding,

ACLs, Rules-BasedReal-time Analysis, Fingerprinting, Honeypotting, Machine learning,

Behavioral modeling

Survey Respondents

100 real estate executives representing over 600,000 realtors

14 real estate portal operators running 400,000 real estate websites

2015 Real Estate Web Scraping Survey

○ 50% - 75% of bot traffic is from Consumer ISPs

○ 7 of top 10 sources of bad bots are Consumer ISPs

○ Most Consumer ISPs had 1,500+ IPs with bots

Highlights of Bot Sources on Real Estate Websites

The Facts on Scraping Real Estate Data

Top 7 Consumer ISPs with Bot Traffic

1 Comcast

2 Time Warner Cable

3 Verizon FIOS

4 Charter

5 Cox

6 CenturyLink

7 AT&T Uverse

Highlights of Bot Sophistication

○ 18-45% Automated browsers - mimicking humans

○ 14-25% Already in bot database - fingerprinted, known bots

○ 16-42% Slow crawlers - recycling IPs and user agents

About Lamudi

30+ Countries900,000+ Listings660+ Employees

Property portal focused exclusively on emerging markets

Lamudi Bad Bot Challenges

Bad Bot ChallengesBad guys scraped listing data to duplicate listings, impact SEO, and compete w/Lamudi Bots are spamming listing agent/owner contact forms & reducing agent retention & satisfaction15,000 bad bot requests per minute (15x human traffic) caused slowdownsWAF-based IP blocking system used enginering time and was ineffective

Lamudi Selection CriteriaBot Detection and Mitigation Solution RequirementsSupport a complex deployment across several AWS instances with Akamai

Block web scrapers and spammers without impacting human visitors

Accurately identify good bots vs. bad bots

Increase website availability and speed

Detect automated browsing tools

Simple setup for 30+ domains

Little or no maintenance, “self-optimizing” solution

Lamudi Results with DistilResults with ROINo more scraping data → unique listings = better SEO

No more form spam to agents → higher value leads = $$

Less time addressing agent complaints → Rev. Retention = $$

Increased website performance → Faster site = better SEO

Save 100 engineering hours/mo. → More resources! Save $$

“Distil is the best anti-bot and anti-scraper protection solution available, hands down.” Oliver Fiege, CTO, Lamudi

How the Distil Bot Detection Solution Works

As web traffic passes through Distil, the system

1. Fingerprints each incoming connection and compares it to our Known Violators Database

2. If it’s a new fingerprint, validates the browser to determine if it’s a Bot or Not

3. “No Silver Bullet” - Distil randomizes a battery of challenges to find bots and remain spoof-proof from the bot coders

4. Based on your settings, Distil automatically tags, challenges, or blocks the bot

Sticky Bot Tracking With No Impact On Real UsersDevice FingerprintingFingerprints stick to the bot even if it attempts to reconnect from random IP addresses or hide behind an anonymous proxy or peer-to-peer network

Tracks distributed attacks that would normally fly under the radar

Without Distil

With Distil

Without Impacting Users Sharing the Same IPAvoids blocking residential users or organizations that might share the same NAT as the bot or botnet

Browser ValidationDetects all known browser automation tools, such as Selenium and Phantom JS

Protects against browser spoofing by validating each incoming request as self reported

Advanced Bot Detection Increases Accuracy

Behavioral Modeling and Machine LearningMachine-learning algorithms pinpoint behavioral anomalies specific to your site’s unique traffic patterns

Self optimizing algorithms improve bot detection and mitigation without manual configuration

Awards and Analyst Recognition

“Analyzing behavior provides the best chance of detecting and blocking bot-

driven attacks.”

5 Stars across the board.“Verdict: For monitoring the impact of bots on a network this is the tool one

needs.”

The only anti-bot solution to be included in Gartner’s Online Fraud

Detection Market Guide

Ovum puts Distil Networks On The Radar. “Clear innovation compared to

similar services.”

www.distilnetworks.com

QUESTIONS….COMMENTS?C H A R L I E @ D I S T I L N E T W O R K S . C O M

1.703.962.1614OR CALL CHARLIE ON