‹#›

State of Bug Bounty

Leif Dreizler, Sr. Security Engineer@leifdreizler

‹#›

Things I’ll Cover

oWhat’s a bug bounty?oBug Bounty: 👻 🎁🔮oHow to run a successful bug bounty!oQuestions!

What’s a bug bounty program?

‹#›

A Brief History of Bug Bounty Programs

‹#›

1995

‹#›

20052002

‹#›

2004

‹#›

2007

‹#›

BigDataSecurityMetrics

9

‹#›

Highlightsfromthe2014Google

o Started in 2010o In 2014 paid over 200 researcherso Highest single payout: $150ko Total payout: $1.5+ milliono Over 500 unique and valid bugso Over half of the bugs in Chrome were reported and fixed in

beta or dev builds

src: http://googleonlinesecurity.blogspot.com/2015/01/security-reward-programs-year-in-review.html

‹#›

Google VRP

src:h?ps://sites.google.com/site/bughunteruniversity/behind-the-scenes/charts

‹#›

‹#›

Highlights from the 2014 Facebook Report

o Started in 2011o Currently $500 minimum, no

defined maximumo 17,011 Submissionso 61 Eligible bugs were high severityo 123 Countries (65 Rewarded)o $1.3 million paid to 321

researchers

Countries with High # of Valid Subs

Valid Bugs Average $ RewardIndia 196 $1,343

Egypt 81 $1,220USA 61 $2,470UK 28 $2,768

Philippines 27 $1,093

src: https://www.facebook.com/notes/facebook-bug-bounty/2014-highlights-bounties-get-better-than-ever/1026610350686524

‹#›

Microsoft Bounty Expansion

o Started in 2013o Online services like Azure and 0365 have a

maximum bounty of $15ko Doubled this during Aug 5 - Oct 5 for auth

vulnerabilities in Windows Liveo “Mitigation Bypass” bounty for novel methods to

bypass paramount OS protections like ASLR and DEP - $100ko “Bonus Bounty for Defense” - $50k

src: http://blogs.technet.com/b/msrc/archive/2015/04/22/microsoft-bounty-programs-expansion-azure-and-project-spartan.aspxsrc: https://technet.microsoft.com/en-us/security/dn800983

‹#›

Highlights from the 2014 Github Report

o First year of the programo $200 - $5,000 (doubled for 2015)o 1,920 Submissionso 73 Unique Vulnerabilities (57 medium/high)

o 33 Unique Researchers earned a total of $50,100 for the med/high vulnerabilities

src:h?ps://github.com/blog/1951-github-security-bug-bounty-program-turns-one

‹#›

Tesla Motors

o Began their program with Bugcrowd in 2015o Includes all Tesla Motors hosts, mobile apps, and any hardware

you’re authorized to test against (don’t hack your neighbors car)o Initially had an upper end of $1,000o Increased the upper end to $10k at Black Hat

o Researchers were able to gain access to the Model S computer system, remotely lock and unlock the car, and apply the emergency brake if the car @ under 5 m.p.h.

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work

with themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the

right thing”oThe program makes a statementoContinuous testing

‹#›

People are already looking

oPeople are already looking for vulnerabilities in your softwareoSome good, some bad

oHaving a bug bounty program reduces the value of vulnerabilities by decreasing the expected lifetime

oYour company is less likely to get extorted if you already have an established program

‹#›

[Redacted] Financial Services

oExtortion attempt from Eastern EuropeoResolved by creating a “one man bug

bounty” (we didn’t tell him he was the only one though…)

oBug received in 15 mins

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

Why should my organization run a bug bounty?

oHelps augment your internal security teamoHelps level the playing fieldoShows the security community you’ll work with

themoMakes it easy for researchers to “do the right

thing”oThe program makes a statementoContinuous testing

‹#›

I’malreadydoingenough

oRed TeamoScannersoTraditional Pentests

‹#›

I’m already getting continuous testing from my red team

oBug bounties don’t replace red teamsoThey work in concert, providing a different

perspectiveoRed teams have access to privileged

information that may create bias in their testing

‹#›

I’m already getting continuous testing from a scanner

oThey report false positivesoScanners miss a lot of

vulnerabilities

‹#›

I’m already having my application pen tested

oLimited resources compared to the crowd

oPaying for time vs. resultsoSnapshot in time

‹#›

Instructure received 5-10x the number of unique vulnerabilities compared to previous pen tests

‹#›

‹#›

January 2013 - June 2015

State of Bug Bounty

35

‹#›

AreasofTrends:TypesofProgramsSignaltoNoiseRaOoSeverityofSubmissionsTypesofSubmissionsResearcherDemographics&Behavior

CulminaOonof2YearsofBugBountyData

36

‹#›

Researchersaremeasuredonthebelowfactorsandinvitedaccordingly…

Quality ifasubmissionisvalidandinscopeImpact ifasubmissionisworthyourOmeAcOvity ifaresearcherisreadytowork

Trust

Howdoresearchersjoinprivateprograms?

‹#›

» Valid» Fixable» High-Priority» Reproducible» InScope

NoiseSignal» Invalid» Ignored» Duplicate» Non-Reproducible» Out-of-Scope

WhyInviteOnly?

‹#›

RiseOfInvitaOon-OnlyPrograms

oInvitation-Only Programs account for nearly 70% of current programs running on our platform

‹#›

Client Statistics

o $725k paid to researcherso 38k submissionso 8k valid & unique (21%)

o $200 average payouto 4.39 “big bugs” per program

‹#›

P1 - Critical

Vulnerabilities that cause a privilege escalation on the platform from unprivileged to admin, allows remote code execution, financial theft, etc.

Examples: Vertical Authentication bypass, SSRF, XXE, SQL injection, User Authentication bypass

P2 - High

Vulnerabilities that affect the security of the platform including the processes it supports.

Examples: Lateral authentication bypass, Stored XSS, some CSRF depending on impact

Whatarebigbugs?

‹#›

Google VRP

src:h?ps://sites.google.com/site/bughunteruniversity/behind-the-scenes/charts

‹#›

src:h?ps://sites.google.com/site/bughunteruniversity/behind-the-scenes/charts

Google VRP

‹#›

44

src:h?ps://sites.google.com/site/bughunteruniversity/behind-the-scenes/charts

Google VRP

‹#›

How to reduce noise

o Provide clear directives to researcherso What’s in/out of scopeo Play by your own rules

o Reward Quickly and Consistentlyo Fix Quicklyo Provide feedback/education

‹#›

How to reduce noise

o Provide clear directives to researcherso What’s in/out of scopeo Play by your own rules

o Reward Quickly and Consistentlyo Fix Quicklyo Provide feedback/education

‹#›

How to reduce noise

o Provide clear directives to researcherso What’s in/out of scopeo Play by your own rules

o Reward Quickly and Consistentlyo Fix Quicklyo Provide feedback/education

‹#›

How to reduce noise

o Provide clear directives to researcherso What’s in/out of scopeo Play by your own rules

o Reward Quickly and Consistentlyo Fix Quicklyo Provide feedback/education

‹#›

How to reduce noise

o Provide clear directives to researcherso What’s in/out of scopeo Play by your own rules

o Reward Quickly and Consistentlyo Fix Quicklyo Provide feedback/education

‹#›

How to reduce noise

o Provide clear directives to researcherso What’s in/out of scopeo Play by your own rules

o Reward Quickly and Consistentlyo Fix Quicklyo Provide feedback/education

‹#›

Provide Feedback/Education

o Respond to researcherso Improve submissionso Note deficiencieso Clarify scope

o Trainingo Google: Bughunter Universityo Facebook: Bounty Hunter’s Guideo Bugcrowd: Bugcrowd Forum

‹#›

Shaping the Future of Bug Bounty

o Paid Summer Internshipso Guest blog postso Bugcrowd Forumo Training

o https://github.com/jhaddix/tbhmo https://www.youtube.com/watch?

v=VtFuAH19Qz0o https://blog.bugcrowd.com/bugcrowds-2015-

guide-hacker-summer-camp/

‹#›

Researcher Statistics

o 20,000 total researchers signed upo 90 Countrieso India - 31%o US - 18%o UK - 9%

o Highest average payouto Cyprus - $644o Switzerland - $512o Austria - $475

‹#›

Google VRP

54

src:h?ps://sites.google.com/site/bughunteruniversity/behind-the-scenes/charts

‹#›

Firstsubmission:February28,2013TopPayee

‹#›

Submissions:Whatdotheyfind?

‹#›

Submissions

o 18% XSS, 10% Logic Flaws, 9% CSRF, 6% Info Disclosure, 1% SQLi

o 13% of Valid Submissions are P1 or P2o 54% of Paid Programs have at least one P1 or

P2o 93% of those Programs have 2+

‹#›

Submissions

o 18% XSS, 10% Logic Flaws, 9% CSRF, 6% Info Disclosure, 1% SQLi

o 13% of Valid Submissions are P1 or P2o 54% of Paid Programs have at least one P1 or

P2o 93% of those Programs have 2+

‹#›

Big Bugs!

‹#›

Cross-domainInformationDisclosureDiscoveredbyPeterAdkins(@Darkarnium)

‹#›

• Clifford’s first private bounty invitation• Launched at midnight in Philippines• Found an IDOR à elevation of privilege

• Clifford’sfirstprivatebountyinvitation• LaunchedatmidnightinPhilippines• FoundanIDORà elevationofprivilege

‹#›

src:h?ps://www.cliffordtrigo.info/hijacking-smartsheet-accounts/

• Bugin“importuser”feature• Nocheckwhethertheuserwhoisrequestingtheimporthasthetherightprivilege

‹#›

src:h?ps://www.cliffordtrigo.info/hijacking-smartsheet-accounts/

‹#›

src:h?ps://www.cliffordtrigo.info/hijacking-smartsheet-accounts/

‹#›

h?p://nbsriharsha.blogspot.in/2015/07/a-style-of-bypassing-authenOcaOon.html

• IDORà elevationofprivilege1)logintohttps://service.teslamotors.com/2)navigatetohttps://service.teslamotors.com/admin/bulletins3)nowyouareadmin,youcandelete,modifyandpublishdocuments

‹#›

0

125

250

375

500

1995 2000 2005 2010 2015

Adoption of bug bounty and vulnerability disclosure programs.

Soundsgood!I’llstartone!

‹#›

Clearing technical

debt

Thanks to @mwcoates http://www.slideshare.net/michael_coates/bug-bounty-programs-for-the-web

ProgramLifecycle

‹#›

src:h?ps://github.com/blog/1951-github-security-bug-bounty-program-turns-one

Github

‹#›

CommunityManagement

oDeluge of submissionsoTriage and ValidationoResearcher CommunicationoResearcher PaymentoRemediation

‹#›

CommunityManagement

oDeluge of submissionsoTriage and ValidationoResearcher CommunicationoResearcher PaymentoRemediation

‹#›

CommunityManagement

oDeluge of submissionsoTriage and ValidationoResearcher CommunicationoResearcher PaymentoRemediation

‹#›

ProgramGrowthoIncrease number of researchersoIncrease scopeoIncrease reward rangesoIncrease publicity

‹#›

In Summaryo As the bug bounty economy matures…

o More companies are adopting (private) programso Critical and severe bugs are being foundo Average payout is increasing over timeo Overall signal-to-noise ratio is improvingo Helps you engage the global security community

bugcrowd.comleif@bugcrowd.com921 Front StreetSan Francisico, CA

@bugcrowd

QUESTIONS?