F5 ASM v12 DDoS best practices

BIG IP ASM V12 DDOS PROFILE

Lior Rotkovitch, NPI

ASM, L7 DDoS & Analytics

Global Service Tech Summit, Seattle Sep, 2015, v3

[email protected]

© F5 Networks, Inc 2

ASM – DDoS Profile

DDOS - HTTP FLOODS ATTACKS

• From single IP to single URL

• From multiple IP’s to single fixed URL

• From multiple IP’s to multiple fixed URL’s

• From multiple IP’s to multiple random URL’s

• From multiple IP’s from a specific country

• Fine Tune your Thresholds & Reporting

DDOS - BOTS

• Simple bots

• Impersonating Bots

• Bots with cookies & JS capabilities

• Bots acting as full browser

• Reporting


HTTP Floods facts:

Servers DatabaseHacktivism

Google Web Bot

Unidentified User

User

Source IP‘s

Users Or Bots

Web Site

Web Bot

• Legitimate Layer 7 requests

• Asking a web page thousands of time instead of one (few) times

• Exhausting backend servers resources: memory, CPU, Disk etc

• Relatively easy to execute with simple tools

• Not easy to detect the offending source nor to prevent it

• Wrong identification will prevent valid users from accessing the site (false positive )


HTTP Floods types

Hacktivism

Google Web Bot

Unidentified User

User

Source IP‘s

Users Or Bots

Web Bot

Requests increase from or/and to URL’s inside the web site

• From single IP to single URL

• From multiple IP’s to single fixed URL



• From multiple IP’s originating from a specific country

Web Site

Servers Database


ASM Detection & Mitigation concept - HTTP Floods

Latency App URL’s &

objects

Hacktivism

Google Web Bot

Unidentified User

User

RPSSource IP‘s

Users Or Bots

Web Bot

Source IP’s

ASM process:

1. Monitoring entities: RPS Latency IP’s URL’s

2. Detecting Increase

3. Activating Mitigation

Web Site

Servers Database


ASM Detection & Mitigation concept – DoS Profile

Location: Security ›› DoS Protection ›› DoS Profiles ›› dos


TPS Based Detection: Transaction Per Second based detection and mitigation

Client: Hey server, can I get the web page ?

ASM: no, you are sending too many requests.

You will have to

• Answer CSID

• Answer CAPTCHA

• Be Rate Limited / Blocked

Server


TPS Based DetectionMonitoring Request Per Second increase form source IP, Geo, URL, Site Wide.Then apply one of the mitigation policies: CSID, CAPTCHA, Rate limit


TPS Based Detection

1. BY Source IP (Detection & Mitigation Polices)

2. Mitigation polices:

a) Client Side Integrity Defense

b) CAPTCHA challenge

c) Request Blocking

3. By Geolocation (Detection & Mitigation Polices)

4. By URL (Detection & Mitigation Polices)

5. By Site Wide (Detection & Mitigation Polices)

6. Prevention Duration

1

abc

3

4

5

2

6


By Source IP: Detection Criteria

Detection

Detection: thresholds for determining DDoS attack - by source IP increase

Mitigation: which mitigation will apply on the offending source IP

Mitigation


Ratio thresholds - measuring ratio with two time interval:

• Long (History Interval): Measure the last 1 hour RPS averages every 10 seconds

• Short (Detection Interval): Measure the last 10 seconds RPS averages every 10 seconds

Detection – Ratio



TPS increased by: ((370 - 50) /50)*100 = 640%

640% > 500% = True


Detection – Ratio

Long (History Interval): 50 TPS

Short (Detection Interval): 370 TPS

Example:



TPS increased by % AND minimum fix number of transactions

640% AND 40 = True

Detection – Ratio

Example:

At least X Transactions:

A minimum condition to

prevent false positive

increase (source IP

starts browsing the site

and goes from 0 to 30

RPS)



(TPS increased by % AND minimum fix number of transactions) OR TPS reached

640% AND 40 OR 200 = True

Detection – Ratio

Fixed

Example:

TPS reached:

Ratio thresholds

OR’ed with fixed TPS

or


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


Client Side Integrity Defense – Concept

User

Web Bot


ASM: no, you are sending too many requests. Are you a browser ?

if a browser:

Yes, I’m a browser

If a bot:

*^lkjdfg@#$

ASM: ok, you are allowed. Here is the web page you asked for.

ASM: Bye Bye – Blocked

Server


• Checking JavaScript capabilities

• A client is considered legitimate if it meets the following criteria:

• The client support JavaScript

• The client support HTTP cookies

• The client should calculate a challenge inside the JS

• If satisfied = legitimate client that can access the site

Client Side Integrity Defense – Concept


Client Side Integrity Defense - Flow

User Browser DoS Profile App

First main page access

HTTP Request (no cookie)

Computational challenge

Solve challenge/

set cookie with time stamp

HTTP Request (cookie) Reconstruct request

Original HTTP Request

HTTP Response (main page)


More object requests (cookie)

Validate cookie: format & time stamp

More object requests

More responsesMore responsesDeliver page

• This is the flow and timeline

of events.

• Transparent to the user,

done under the hood

• Note that request is held at

the ASM and not arriving the

app until checks are

satisfied

• Not all checks are described

here, some are internal IP.

Send JS test


Client Side Integrity Defense –JavaScript sample

• The JS is obfuscated

• From user perceptive this

is transparent action.


• If no reply – No problem for us

• If didn’t solve the challenge but still sending request – Block (RST)

• If did solve the challenge but:

• Cookie is wrong format – Block (RST)

• Time stamp expired – Block (RST)

• If client access a resource (image) without getting the cookie first – Block (RST)

Client Side Integrity Defense – Mitigation summary


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


CAPTCHA Challenge - Concept

User

Web Bot


ASM: no, you are sending too many requests. Please answer this CAPTCHA challenge, show me your human !:

If a user:

OK, I answered

If none user:

Ha ? *^lkjdfg@#$

ASM: ok, you are allowed. Here is the web page you asked for.

ASM: Bye Bye – Block him dood !

Server


CAPTCHA Challenge

Ultimate solution for identifying human or bot

Send challenge to every IP that reached IP detection criteria thresholds

To CAPTCHA or not to CAPTCHA ?

Some argues that CAPTCHA is not a good usability because an innocent user gets CAPTCHA and he will not know why. So, remember that a valid user should pass browser tests. i.e. if a user is blocked (or gets a CAPTCHA) there is a reason and maybe he is not innocent (infected ?) .

“Completely Automated Public Turing test to tell

Computers and Humans Apart”


CAPTCHA – customize response

• Can be customize to the web site look and feel colors via css

• Failure Response page is served if the first attempted fails


CAPTCHA Challenge - Flow


Request mypage.php

GET /mypage.php (no cookie)

CAPTCHA HTML +JS response

Cookie with time stamp

Solve CAPTCHA

CAPTCHA rendered

Submit CAPTCHA

solution

GET /mypage.php + CAPTCHA

cookieVerify CAPTCHA solution

Validate cookie

GET /mypage.php

HTML of mypage.phpHTML of mypage.phpmypage.php

rendered

Send CAPTCHA• While the system is still in a

state of attack the offending source will be presented with another CAPTCHA every 5 min.

• Same as CSID, request is held at the ASM until CAPTCHA is solved


• If didn’t submit the challenge - no request DOSing us

• If didn’t solve the challenge but still sending us attacks – Blocked

• If did solve the challenge but:

• Cookie is wrong format – RST

• Time stamp expired 5 min– RST

CAPTCHA – mitigation summary


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


Request Blocking / Rate limit


ASM: no, you are sending too many requests.

I’m limiting your requests sending rate

While CSID and CAPTCHA try to understand who is the offending source (bots or browsers)

request limiting is indifferent to the “identity” and limits the offending sources.


Request Blocking

Request Blocking:

• Blocking: block all IP’s from the offending source – if a source IP reached thresholds I don’t

want him on my site at this point

• Rate Limit: limit the amount of allowed request from the offending source – if reached

thresholds I can sustain only some of the traffic at this point


Request Blocking – Mitigation Summary

• Block all – blocking all traffic from the offending source (i.e. I don’t want to see any more traffic from this source)

• Rate Limit – rate limit the offending source

Example

If long was 50 TPS And increase in short is 150 TPS

Rate limit to 50 TPS

Rate limit will limit to long (history) TPS rate


TPS based: by source IP – Summary

Client Side Integrity Check

CAPTCHA Challenge

Request Blocking

• Measuring source IP increase

• All source IP’s that reached the thresholds will be presented with the enabled mitigation

• If still increasing , fall back according to the order in the GUI (switching mitigation)


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


HTTP Floods – Geolocation detection and Mitigation

Hacktivism

Google Web Bot

Unidentified User

User

Source IP‘s

Users Or Bots

Web Bot

Servers Database

Web Site

http floods type: From multiple source

IP’s originating from a specific country


• Geolocation – Relative to the whole traffic of the site:

500 % request increase of the whole site from a specific country

AND

At least 10 % of the whole site traffic

Geolocation - Detection


Geolocation – Mitigation

• Client Side Integrity Check

• CAPTCHA Challenge

• Request Blocking

All clients requests arriving from the specific country will be presented with mitigation:

(note that blocking will block

all users from this country)


Geolocation – Black n White listing

• Allows access to the web site regardless of

geolocation detection criteria thresholds only

i.e. other thresholds still apply

• Specifies the countries that the system always blocks

whenever the system is in a state of DDoS detection.

• Done regardless of the thresholds set in the DDoS

profile


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


HTTP Floods – URL Detection and Mitigation

App URL’s &

objects

Hacktivism

Google Web Bot

Unidentified User

User

RPSSource IP‘s

Users Or Bots

Web Bot

Source IP’s

http://site.com/sell.php

Servers Database

Web Site

• Measuring requests increase on a URL

• Floods types:




TPS increase by* AND at least X TPS **

OR TPS reached

URL Detection Criteria

Collecting RPS on URL’s

Calculation:

*Ration of long and short

**Minimum TPS thresholds for detection


URL Detection Criteria– Mitigation



• Request Blocking – Rate limit (No block all)

All clients that access the URL:


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


HTTP Floods – Site Wide Detection and Mitigation

App URL’s &

objects

Hacktivism

Google Web Bot

Unidentified User

User

RPSSource IP‘s

Users Or Bots

Web Bot

Source IP’s

Monitoring: all entities

ServersDatabase

Web Site

Floods types:


• Cases where DDoS attack is under the radar


*TPS increase by AND Minimum TPS thresholds for detection

OR TPS reached

Site-Wide Detection Criteria

• Collecting RPS on the entire website (all entities – URL’s, IP’s)

• In some cases the floods will avoid thresholds for IP based or URL based.

• Site wide provide another layer of detection and prevention

Detection: Ratio

Fixed


Site-Wide Detection Criteria – Mitigation



• Request Blocking - only rate limit no blocking

All clients that access the site:

Prevention polices


TPS Based Detection





c) Request Blocking





1

abc

3

4

5

2

6


Prevention duration



• Request Blocking

De escalate - start from the top

Every 7200 seconds

Escalate top down every 120 second if thresholds are still increasing


Stress Based detection

• Predictive Latency – predict how long it will take to serve a new incoming request

Server: I’m fine, keep on sending them

ASM: Hey server, how many more requests can you handle ?

I’m the server


Stress Based Detection and prevention concept

Client: Hey server , can I get the web page ?

ASM: mmm let me check. The Server can take additional incoming requests. you are allowed

ASM: no, my backend latency is now too high and you are sending too many request.

You will have to:

• Answer CSID or

• Answer CAPTCHA or

• Be rate limit

Client: Hey server, can I get web pages again now ?

I‘m the server

…….

…….

After a while


Stress Based – GUI

• Same concept as TPS based: source IP, Geo, URL, Site wide and their mitigation policies.

• Addition condition of backend latency. i.e. only when the two conditions reach thresholds, then apply mitigation policy.

Note: Can work together (operate in

parallel) with TPS based and act as

layers of protection (e.g. TPS based

does only CSID in alert mode and

Stress based does request blocking in

case of latency increase)


Stress Based Detection & Mitigation

• Similar to TPS based,

Quiz yourself, what each item means ?

1. By Source IP

a) CSID

b) CAPTCHA

c) Request Blocking

2. By Geolocation

3. By URL

4. Site Wide


Stress Based Detection – thresholds condition

Latency threshold

exceeded?

TPS threshold

exceeded?AND

Then: Activate

Mitigation Policy

• Mitigation Is activated when two types of thresholds are reached :

Latency thresholds

AND

TPS thresholds


Stress Based Detection – thresholds condition

• in order to apply a prevention policy, both TPS and Latency thresholds must be exceeded, then the enabled prevention policy is activated.

• Latency thresholds are not visible in the GUI, they are part of automatic detection.

Example:

Automatic stress detection enters a

state of exceeding thresholds. This by

itself will not active the prevention.

Only when the TPS thresholds will

exceed then the prevention policy is

activated.

prevention

TPS thresholds

stress detection


TPS based VS Stress based

• Quick way to protect against DDoS. I’m in trouble

and I want to block now !

• Fixed number on the TPS reached is very easy

and useful. Also easy to detect offending sources

• Allows the option to activate the Mitigation only

when the backed experiencing latency AND RPS

increase (I only want to block when the attack is

causing backend latency)

• Provide Layers of defense and notify about backend

issues (not just DDoS)

Conclusion:

TPS based is quick while latency based

allows more granular approach


Heavy URL’s

Not all URL’s are equal

Some are more attractive than others


Heavy URL’s

• Heavy URL’s are URL’s that consume more processing resources

from the server

• Are good application DoS point - Even few requests can DoS the app

• Typical heavy URLs are search box, product ID’s

Heavy URL

Servers Database

http://site.com/serach.php?q=a

Ho wow, this will take a while…

……

Searching …

…..

hold on…

…..

Almost there….


Heavy URL’s concept

• Automatically measures latency on URL’s for 48 hours

and decide who is heavy

• When any URL based mitigation is active, the heavy URL’s

that was detected as heavy will also “get” the active

mitigation


Heavy URL’s concept

Heavy URL is another detection capability. Once it is reached the thresholds AND one of the By URL detection

thresholds are reached Then the URL’s that consider heavy URL’s will be applied with the active mitigation policy


Heavy URL’s configuration

Example: By URL TPS reached 1000 TPS and is currently applying CSID mitigation.

Heavy URL is enabled


Heavy URL’s Configuration

1. Automatic Detection - Automatically add URL that will be considered as heavy

2. Manual Heavy URLs – Manually add URL that will be considered as heavy

3. Ignored URL – Exclude those URL from “heaviness”

4. Latency Threshold – Above this threshold -> heavy URL

1

2

3

4


Heavy URL – Reporting

If search.php is defined as heavy and if index URL is currently being mitigated with CSID because it exceeded the

thresholds of URL reached (or fixed) then every source IP that is accessing search.php will also get the CSID check.

Security ›› Reporting ›› DoS ›› Application ›› URL Latencies

Example:


• Reporting first

• Fine Tune your thresholds

BeforeDDoS Attack

During DDoS Attack

Remember, security is a process


First rule of detection - AVR Reporting

• AVR graphs help you understand the site metrics:

Statistics ›› Analytics ›› Overview

Statistics ›› Analytics ›› Transactions ›› View by

• AVR graphs inside ASM tab:

Security ›› Reporting ›› DoS ›› Overview

Security ›› Reporting ›› DoS ›› Application ›› Transaction outcome

Know your web site metrics

• Sources

• IP’s

• URL’s

• Site Wide

• Geolocation

• RPS

• TPS

• Latency

L7 DDoS measurements


• Out of the box thresholds are good for most web sites

• Depending on the web site traffic fine tuning thresholds might be needed.

• Fine tuning thresholds can be divided into:

• Before DDoS Attack

• During DDoS Attack

Why Fine Tune Thresholds ?

Good for me ??


Process:

Pre requisite: Enable DDoS Profile on the desired virtual

1) White list IP’s, geolocations countries, URL’s (admin) etc

2) Get visibility with transparent mode – write down metrics*

3) Test and decide which prevention will apply thresholds exceeds (TPS bases/ Latency based , heavy URL config etc)

4) Fine tune thresholds for fixed and ratio based

5) Switch to blocking – When needed

Fine Tune Thresholds Before attack

*good list for L7 DDoS metrics

https://www.google.com/#q=app_ddos_metrics.pdf


Fine Tune Thresholds Before DDoS for Source IP

• View by: Client IP address

• List top TPS Avg IP’s

Go to Statistics ›› Analytics ›› HTTP ›› Transaction


Fine Tune Thresholds Before DDoS for Source IP

• By examining the client IP Address you can

conclude the averages of “normal” traffic you

expect to see from the top source IPs.

• Knowing “normal” averages can help defining

the TPS increase by ratio.

• The idea is that you can determine how much

traffic is allowed until assumed a ddos attack.


Fine Tune Thresholds Before DDoS for Geolocation

The same concept works for the

geolocation thresholds graph.

From the drilldown choose Countries on

AVR reports

“Which countries you expect to see traffic ?

Go to Security ›› Reporting ›› dos ››Application ›› transaction outcome


Fine Tune Thresholds Before DDoS for URL

The same idea applies to URL’s.

Sort graph by URL’s

“Which URL should have to highest RPS ?


Fine Tune Thresholds Before DDoS for URL


Fine Tune Thresholds Before DDoS for Site Wide

On the drilldown choose

Virtual Server

“This will help us understand

the over all traffic load that we

have when there is no DDoS

attack.


Fine Tune Thresholds Before DDoS for Site Wide

The overall traffic should be much higher than the other thresholds.

The values reflect the total amount of TPS that the virtual can handle.

Site wide = Virtual server


Fine Tune Thresholds During attack

Process:

1) Fine tune white list source – if needed

2) Identify sources that exceed thresholds (source IP’s, URL’s, Geo, SiteWide) by looking at reporting.

3) Determine the attack type: from fixed/random source IP to fixed/ random URL. Conclude which of the detection types you need (source IP only ? Source IP and URL based only ? etc. )

4) Fine tune thresholds according to the exceeding sources (ratio / fixed)

5) Apply mitigation and decide what is working and what is not. Uncheck the mitigations that are not effective

6) Go to step 1 and repeat


Fine Tune Thresholds During attack – Source IP

• Security ›› Reporting ›› dos

››Application ›› transaction

outcome

• On the drilldown choose

Client IP Address


Fine Tune Thresholds During attack – Geolocation

• Security ›› Reporting ›› dos ››

Application ›› transaction

outcome


Countries


Fine Tune Thresholds During attack – URL’s



outcome

• On the drilldown choose URL’s


Fine Tune Thresholds During attack - Site Wide



outcome


Virtual Servers


AVR reports and graphs

Mitigation type – can help

understand which of the

mitigation is effective and

when switching mitigation

occurred

Time line Attack start / end

Host IP

Number

of TPS

Attack ID : Clicking

will show graph

Security ›› Event Logs ›› DoS ›› Application Events



impact is the latency

on the backend for all

entities. The higher

the latency the higher

the impact is.

High, medium and low

impact allow to filter

high impact attacks

and deal with them

first

Security ›› Reporting ›› DoS ›› Application ›› Transaction Outcomes



Start and End points - red flags indicate the start of an attack and the green flags indicate the end of

an attack. Switching mitigation can occure several time over the DDoS attack.

Security ›› Reporting ›› DoS ›› Application ›› Transaction Outcomes



• Incomplete – Indicates traffic that was dropped by the server because the connection was

incomplete or the server did not respond.

• Blocked – Indicates traffic that was blocked as a result of the mitigation policy (any of the

prevention policies including bots blocking)

• Proactive Mitigation – Indicates the amount of time that the proactive bot defense mechanism

was severed

• CAPTCHA mitigation – Indicates the amount of time that the CAPTCHA challenge was severed

to offending sources

• CS integrity mitigation –Indicates the amount of time that the client-side integrity defense

challenge was severed to offending sources

• BIG IP Response – Indicates traffic that is a response to the client from the BIG-IP system.

• Cache by BIG IP – Indicates traffic that is served from cache configured (WA, RamCache)

• Whitelisted – Indicates traffic from IP Address that are in the whitelist of the DoS profile

• Pass through - Indicates traffic that is pass to the application trough ASM to the server



The AVR DoS graph now

shows the thresholds

that are set in the TPS

detection tab.

The Display Thresholds

check box will display

them or clear them from

the graph.


Before DDoS:

• Write the “normal” thresholds for the web site: (IP’s, Geolocation, URL’s, Site Wide)

• Set the ratio and the fix threshold for each of the above detection criteria (How much the web site can take 2 times the traffic , 5 times etc…)

• Test the configuration and the prevention policy, then conclude which one is good for you

During DDoS:

• Identify the source IP’s, URL’s and entire site traffic increase and determine the attack type

• Set the fixed TPS number in each of the above criteria and apply mitigation

• Verify the results in the Transaction outcome graph

Fine Tune Thresholds– Summary

https://www.google.com/#q=evaluate_ddos_metrics_and_readiness.pdf


DDoS Bots - Detection & Mitigation


Layers of defense against Bots

Simple Bots

Impersonating Bots

Bots with cookies / JS capabilities

Bots acting as full browser

Gohogle

This bot section is mostly about

bots that DoS / DDoS. However,

Bots detection and prevention

can be used for various bot

problems the site is experiencing.


DDoS Bots

Servers Database

Google Web Bot

Unidentified User

User

Users Or Bots

Web Site

Web Bot

Bots can be classify in many ways, mostly there are:1. Simple bots

2. Impersonating Bots

3. Bots with cookies & JS capabilities

4. Bots acting as full browser


Enabling Bot signatures protection


Bots – Simple Bot

Server

ASM: yes, I have your signature. Sorry mate you are blocked.

I’m a simple Bot

Simple bot can be any command line tool such as: curl , wget , ab


Categorizing Bots

Bad Bots

Good Bots

Bad Bots aka Malicious are well know command

line tools – we want them out

Good Bots aka Benign are well know search engine and monitoring tools – we want

them in


Bot Signatures -

None

Report

Block

None

Report

Block

Each categories include:

• None – ignore

• Report – report only – used for monitoring

• Block – block


Excluding specific bot signatures from category settings

• Specific signature can be excluded from the

category setting

• Search the signature in Available signature list

and move it to the left pane.

• In this example ab tool will not be blocked

even if the category that includes it is in

blocking mode


First - White list good Bots

Web Server

Google

I’m a google Bot

ASM: let’s see if you really are. I’m doing Reverse DNS lookup.

Yes, I see that, please continue.

DNS Server

Google

Thanks

1

2

3

4


White list good Bots -with their domain name

User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

1. Request arrive with User Agent : Googlebot/2.1

DNS Server

Web Server

2. ASM search the google bot signature

3. The signature includes domain name. ASM issue Reverse DNS query to verify the origin of the request

4. Once approved ASM will allow the google bot to access the web site


Bot Signature Repository

• Bot Signature repository for the entire system is under Options.

• Bot signatures repository is update with the ASM signature update

Security ›› Options ›› DoS Protection ›› Bot Signatures List


Bot Signature List: general signatures repository

Signatures can be sort by:

• Signature Category

• Signature Type: Malicious / Benign

• User Define signatures Yes / No

• Partition: signature can be assign to a specific partition

Clicking on any of the sorting will change the order.


Sorting the Bot Signature Repository

Various filtering

Create new Bot Signature


Bot Signature Categories

Creating new category for Malicious or Benign


Create a new bot signature: simple edit mode

Simple edit mode: inside

a user agent header or

in a URL.

Category

Domain name – execute

reverser DNS query to

verify origin. Add the

domain if the Bot has one

Bot Signature name

Create when done


Create a new bot signature - advanced edit mode

Signature syntax example:

headercontent: "sample_text"; useragentonly;

Advanced Edit Mode - rule granularity

For full details consult F5 document


• Signatures associated with a domain name are validated with reverse DNS lookup.

• Blocking and reporting :

• Block flag - resets the connection and reports the action as "bot signature block" with the bot signature name.

• Report flag - Report bot name and categories (AVR)

• Updating bot signature as part of the ASM signature update

Bot signature facts


Bots – Impersonating Bot

Web Server

Gohogle

I’m a google Bot ha ha ha

ASM: let’s see if you are. I’m doing Reverse DNS lookup.

DNS Server

Gohogle

Bummer ASM: you are not google bot

Bye Bye -> block this creature !

ASM: Hey DNS, who’s this guy ?

DNS: no one important


Bots – Impersonating Bot

1 Request arrive with User Agent : Googlebot/2.1

DNS Server

Web Server

2 ASM search the google bot signature

3 The real google bot includes domain name. ASM issue Reverse DNS query to verify the origin of the request

4 If the source IP is not the expected one according to the DNS query ASM will block the impersonating bot


Bots with cookies & JS capability

Web ServersI’m a bot that can understand JS and

support cookies

ASM: prove it, answer my challenges

No you are not, bye bye -> block this bot.

Bummer

Ha ?


Proactive Bot Defense

PBD is good for:

• Bots that can handle JS

• Bots that can handle JS and cookies

• Bots floods

• Under the radar bots

• Block any bot accessing the site (humans only web site)


Proactive Bot Defense and Bot Signature

Proactive Bot

Defense is now

integrated with

the bot

signatures.

When enabling

proactive bot

defense the

bot signature

feature will be

enabled as

well


• Send Client side challenges to ALL client and thus mitigate bots all the time

• Various challenges are sent and then validate by PBD – blocked or allowed

Proactive Bot Defense


PBD - Client side integrity defense - flow User Browser DoS Profile App

First main page access

HTTP Request (no cookie)

Computational challenge

Solve challenge/

set cookie with time stamp

HTTP Request (cookie) Reconstruct request

Original HTTP Request



More object requests (cookie)

Validate cookie: format & time stamp

More object requests

More responsesMore responsesDeliver page


• Always – sending CS all the time

• During attack – only if other component of the dos profile is in dos mode PBD will send the CS challenge (acting as two layers of mitigation)

This allows second layer of protection (rate limit and PBD)

• Grace period - cookie expiration time 300 = 5min

• White list – exclude PBD on those IP’s

Proactive Bot Defense – configuration


Bots acting as full browser

Web ServerI’m a Bot that

simulate browser

ASM: ok, what are your capability ? If you will not answer right you will have to answer a CAPTCHA

You are not human, byyyye -> block this unhuman !

DNS Server

Bummer

Capability ? CAPTCHA ?


PBD – Additional bots identification with capabilities script

Bots: Bots acting as full browsers - Browser Simulation


How bots that simulate browsers are evaluated ?

Block Suspicious Browsers – addition tests are done to understand if this is a bot or a browser. ASM

will evaluate the source and will give it a score:

if the score indicates that the source is a bot it will block it.

If the score indicate uncertainty and if CAPTCHA challenge is checked, then CAPTCHA will be presented

to the source. If answer it is a human if not, blocked.


Block Suspicious Browsers

• If Block Suspicious Browsers is unchecked send CS Challenge

• If Block Suspicious Browsers is checked and CAPTCHA is checked send Client

Capabilities challenge and give it a score: If score in doubt send a CAPTCHA for

human verification

• If Block Suspicious Browsers is checked but CAPTCHA Challenge is unchecked do

not send CAPTCHA and only block if the score is more than a human


Client Capabilities -challenge script flow


First request GET /sell.php

GET /sell.php (no cookie)

Client Capabilities Challenge response

Return Client Capabilities

verification

Reconstruct request

HTTP Response (cookie)

HTTP Response

GET /img.png (cookie)

Blank page & Set cookie

Original HTTP Request + cookie

1. Authenticate and decrypted JS results,

2. Verify capabilities and set a score

3. Determine an action based on score

GET /img.png (cookie)

Validate cookie: format

& time stamp


DoS Bots Reporting


Bot signatures simulation Reporting ›› dos ›› Application ›› Transaction outcomes

Transaction outcomes

is very useful for

monitoring traffic and

indicates various

measurements


Bot signatures simulation

Analytics ›› HTTP ›› throughput ›› request throughput

AVR will provide details on DoS bot signatures (use drill downs )


• Simple bots can easily be detected and blocked

• White listing of bots = visibility to bot access and keep other bots out

• Impersonating bots can be monitored / blocked

• Bots that support JavaScript and cookies can now be noticed and be blocked

• Reporting on the visiting bots to your web site is available via AVR

• Custom bots signature is powerful tool to deal with bots

• Bots signature is updating via the ASM signatures update

Summary


Resources

Our documentation is free for all. Read and learn more:

BIG-IP Application Security Manager: Getting Started

BIG-IP Application Security Manager Operations Guide

BIG-IP Application Security Manager: Implementations

BIG-IP Application Security Manager: Custom Signature Reference

BIG-IP Analytics: Implementations

https://support.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-getting-started-12-0-0.html

https://support.f5.com/kb/en-us/products/big-ip_asm/manuals/product/f5-asm-operations-guide.html

https://support.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-implementations-12-0-0.html

https://support.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-bot-and-attack-signature-reference-12-0-0.html

https://support.f5.com/kb/en-us/products/big-ip_analytics/manuals/product/analytics-implementations-12-0-0.html

https://www.linkedin.com/in/lior-rotkovitch-b796909?trk=hp-identity-name

F5 ASM v12 DDoS best practices

Technology

Transcript of F5 ASM v12 DDoS best practices