Inherent Differences Between the Hacker and Developer Mentality

Sr. Security Engineer@leifdreizler

Your Elastic Security Team.

So What Does Bugcrowd Actually Do?• Incorporate up to 18,000 freelance security

researchers as part of a public or private engagement

• Run a crowd sourced pen test • Manage an ongoing bug bounty program

What’s a bug bounty program?

A Brief History of Bug Bounty Programs

These and other companies trust Bugcrowd

Things We’ll Cover

• How to incorporate Crowdsourced Security into DevOpsSec • Accelerating your RO(security)I • What’s in it for me (as a security person)? • Bug bounty fun facts, pitfalls, and war stories

introduce crowd sourcing

Bug Bounty Programs Responsible Disclosure

Crowdsourced Penetration Test

…because people are the new automation

[REDACTED] eCommerce provider

• Long time customer of [EXPENSIVE WEB APP SCANNER] getting “clean results”

• Within 24 hours a researcher demonstrated a PoC exploit for a CSRF vulnerability, allowing a user to reset an admin account password

• They thought they were doing a great job at writing secure code…

assume it’s broken

Instructure received 5-10x the number of unique vulnerabilities compared to previous pen tests

Lots of bugs == great dev training

Software is always going to have bugs

[REDACTED] Financial Services

• Extortion attempt from Eastern Europe

• Resolved by creating a “one man bug bounty” (we didn’t tell him he was the only one though…)

• Bug received in 15 mins

History

0

125

250

375

500

1995 2000 2005 2010 2015

Adoption of bug bounty and vulnerability disclosure programs.

Bug bounties are awesome…

Minimize Investment

Maximize Quality

Accelerate RO(security)I

Makes a Statement

It’s not just about being cost-effective,

or loud…

It’s about leveling the playing field…

…but bug bounties are hard.

Plan ahead

The mistake *everyone* makes:

VULNERABILITY DATA

PEOPLE

[REDACTED] Digital Advertising

• Engaged Bugcrowd to help them assess the state of the code

• So many valid vulnerabilities submitted they shut down the bounty in 24 hours

• Thrilled with the results!

Align expectations before you engage

Bug bounties create controlled incidents…

[REDACTED] Online Marketplace

• The DevOps and Security teams watched vulns being submitted in real time

• Non-security minded people learned a lot from the process

• Great insight into how ‘good guys that think like bad guys’ work

Mozilla

Thanks to @mwcoates http://www.slideshare.net/michael_coates/bug-bounty-programs-for-the-web

Clearing their assurance debt

Boogeymanbelief

DevOpsSec feeling confident?

Try a Gamified Pentest

1. Create a pool that benefits your engineering team (team drinks, party, event, whatever)

2. Replace an existing pentest w/ a time-boxed bug bounty program

3. Pay out from the reward pool

4. What ever the hackers don’t get, DevOpsSec gets to keep.

Great things happen when you tighten the security feedback loop between your engineers, and what they consider to be

the outside world

• In 2014 Bugcrowd started guiding its customers to start with reward ranges of $50-$500

• In 2015 we encourage customers to start at $100-$1000

• As more people start to run programs, your program competes for the researcher’s time

• Your company’s security maturity will help guide what your reward payments should be

Bugcrowd Researcher Stats

• 18% US, 31% India, 9% UK for active researchers

• 90 countries have contributed valid submissions

• Great Britain has low submission numbers, but high average priority

• Average 16 submissions, 8 valid

Bugcrowd Researcher Rankings

• Trust - Do they have a track record of staying inside the terms of the brief?

• Acceptance rate and submission quality - What percentage of submissions are valid

• Finding severity - Submission priority between 1.0 (critical) and 4.0 (low)

• Activity - submitted in the last 90 days

Top Researchers

• Top Points - The current points leader researcher hails from Malaysia and has submitted 431 submissions since his first entry in January, 2013

• Top Rewards - Not far behind the points leader in submission count is the top paid researcher with 292 submissions since the first submission in February, 2013 from the UK

• Top Submitter - A Pakistan based researcher with an overall submission count of 1,094, nearly 3x the points leader, and 4x the top paid researcher. Started a year later in February, 2014 with an acceptance rate of 4%

Bugcrowd Submission Stats

• 37k Total Submissions

• 8k Valid and Unique (21%)

• 13% of Valid Submissions are High or Critical findings

• Programs had on average 4.39 High or Critical findings

• 18% XSS, 10% Logic Flaws, 9% CSRF, 6% Info Disclosure, 2% SQLi

Content Security Policy

Bugcrowd Payment Stats

• Since January 2013 Bugcrowd has paid out $725k

• Average payout of $200.81

• Top payout of $10k

• Highest paid countries:

• Cypress ($644 average)

• Switzerland ($512 average)

• Austria ($475 average)

Levels of Maturity

• Blocking and Tackling - an organization is trying to build security awareness within the organization and there is a limited budget. $100-$200 average reward.

• Compliance Driven - Many controls are in place, but there is not an organization wide focus on security. A bug bounty program can help propel security forward. $200-$500

• Risk-based Approach - Organizations have code review, ongoing assessments, and a dedicated security focus. $500-$1500

• Security Mature - An advanced SDLC with a dedicated internal testing team. $1500+

Total Cost of Ownership• Rewards are only one part of the total cost of ownership of the program.

• Time organizing and launching the program

• Time spent looking at submissions and communicating with researchers

• Addressing issues that are identified

• Communicating the results to the business as a whole

• A public program with market-level rewards should plan to spend roughly the same amount in rewards as in management of the program

• Consider starting with an invitation-only program, and calculate other portions of the current security budget that may be offset

Criticality Rating

Priority  Level Vulnerability Details

Critical Vulnerabilities  that  cause  a  privilege  escalation  on  the  platform,  from  unprivileged  to  admin  or  administrator.

•Remote  code  execution  •Vertical  authentication  bypass  •Some  SSRF  •XXE  •SQL  injection  •User-­‐authentication  bypass

High Vulnerabilities  that  severely  affect  multiple  users  or  affect  the  security  of  the  underlying  platform.

•Lateral  authentication  bypass  •Stored  XSS  •Some  CSRF,  if  impact  can  be  proven

Highlights from the 2014 Facebook Report• Started in 2011

• Currently $500 minimum, no defined maximum

• 17,011 Submissions

• 61 Eligible bugs were high severity

• 123 Countries (65 Rewarded)

• $1.3 million paid to 321 researchers

Countries with High # of Valid SubsValid Bugs Average $

RewardIndia 196 $1,343Egypt 81 $1,220USA 61 $2,470UK 28 $2,768

Philippines 27 $1,093src: https://www.facebook.com/notes/facebook-bug-bounty/2014-highlights-bounties-get-better-than-ever/1026610350686524

Highlights from the 2014 Github Report

• First year of the program

• $200 - $5,000 (recently doubled upper end)

• 1,920 Submissions

• 73 Unique Vulnerabilities (57 medium/high)

• 33 Unique Researchers earned a total of $50,100 for the med/high vulnerabilities

src: https://github.com/blog/1951-github-security-bug-bounty-program-turns-one

Highlights from the 2014 Google Report

• Started in 2010

• Paid over 200 researchers over $1.5 mil

• $150k highest single payout

• Over 500 unique and valid bugs

• Over half of the bugs in Chrome were reported and fixed in beta or dev builds

src: http://googleonlinesecurity.blogspot.com/2015/01/security-reward-programs-year-in-review.html

Looking Forward with Microsoft in 2015• Started in 2013

• Recently added Azure and raised max payout for “Online Services Bounty Program” to 15k

• Added Project Spartan

• “Mitigation Bypass” bounty and “Bonus bounty for Defense” focus on novel methods to bypass active mitigations (e.g. ASLR and DEP)

• Pay up to $100k for exploit + $50k for defense

src: http://blogs.technet.com/b/msrc/archive/2015/04/22/microsoft-bounty-programs-expansion-azure-and-project-spartan.aspx

Conclusion• Bug bounties are cost effective, and highly marketable, but that’s not the full

story…

• …they create controlled incidents that can powerfully impact the security awareness of your builders.

• Allow people that have historically been ‘builders’ to see how ‘breakers’ think

• Help small teams manage a full-featured appsec program

• Get DevOps to believe in and defeat the boogeyman

The premier platform for crowdsourced security testing.

We’re hiring!

jobs@bugcrowd.com