Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

115
Christian Schneider | @cschneider4711 Security DevOps Free pentester’s time to focus on high-hanging fruits

Transcript of Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Page 1: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Christian Schneider | @cschneider4711

Security DevOpsFree pentester’s time to focus on high-hanging fruits

Page 2: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

`whoami`

» Software Developer, Whitehat Hacker & Trainer » Freelancer since 1997 » Focus on JavaEE & Web Security » Speaker at Conferences » @cschneider4711

www.mail@ Christian-Schneider.net

Page 3: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Security – Status Quo

Page 4: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Static Application Security Testing (SAST)

Page 5: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Dynamic Application Security Testing (DAST)

Fun Part

Page 6: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

The Ratio Problem

Dev OpsSec

100 10 1 : :

Page 7: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

The Frequency Problem

more Rollouts than Pentests

Page 8: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Security DevOps ?

Page 9: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Integrate Security Tools into Build-Jobs / CI-Chain

on every commit or nightly

Page 10: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Why Security DevOps?

» Keep up with rollout pace in agile projects » Automate certain checks as best as possible within

the build-chain » Early feedback to Devs » Solution that scales across the company

» Does not remove the manual pentest requirement! » Aims to free pentesters’ time to hunt more

high-hanging bugs

Page 11: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Different levels of "Security DevOps" integration…

» Security DevOps Maturity Model (SDOMM) » Can be seen as some automation tips within

OpenSAMM’s security practices » Verification: Security Testing » Verification: Code Reviews

» Allows to define a RoadMap for projects implementing Security DevOps

Page 12: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

four axes each with four belts as incremental stepsimplicit master

… what levels will we cover?

Page 13: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Four different axes

4 3 2 1 1 2 3 4

4

3

2

1

1

2

3

4

Dynamic Depth

Intensity

Static DepthConsolidation

Page 14: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

4 3 2 1 1 2 3 4

4

3

2

1

1

2

3

4

Dynamic Depth

Intensity

Static DepthConsolidation

Four different axes

Page 15: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Let’s explore these axes …

» … by showing how to implement this with OpenSource solutions used in the Security & Development domains.

Page 16: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis of "Dynamic Depth"

How deep are dynamic scans executed within a Security DevOps CI chain?

i.e. "where" are dynamic security tests applied?

Page 17: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Dynamic Depth": Level 1

Scanning of public attack surface (pre-auth):

» Spidering of UI layer » No requirement to authenticate scanner with target » Easy integration of scanner(s) in nightly build as

post-step » "Throw tool at it (in CI-chain) and see what it

generates…"

Page 18: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Zed Attack Proxy (ZAP)

"OWASP ZAP" features:

»passive scanning (Proxy / Spider) »active scanning (Proxy / Spider) »manual payload delivery (Intercepting Proxy) »Spider (classic & AJAX) »Fuzzing »Plugins »…

Page 19: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Request/Response History (Proxy)

Page 20: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Active Scanning…

Page 21: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

OWASP ZAP

Page 22: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ZAP in SecDevOps?

"OWASP ZAP" features relevant for Security DevOps integration:

»Headless operation mode / daemon »REST-API »Highly scriptable »CLI

Page 23: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

UI- and Service-Tests in CI

Page 24: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

… enhanced with OWASP ZAP

passive & active scanning

place report (html,xml) into workspace

Page 25: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

HTML report of ZAP scan

Page 26: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ZAP + Jenkins = SecDevOps?

"OWASP ZAP" (spider & scanner) + Jenkins plugin “ZAProxy"

» Allows us to "Spider & Scan" as step in build job via Jenkins plugin "ZAProxy"

» Point plugin config to URL of integration system to test » Plugin saves HTML-report in project’s job for inspection » Best as separate Jenkins job to run during nightly build

(duration) – Use different ZAP proxy ports for different builds to allow

parallel execution of different project build jobs

Page 27: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Jenkins Plugin "ZAProxy": ZAP Startup

Page 28: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Jenkins Plugin "ZAProxy": ZAP Scan

Page 29: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Arachni in SecDevOps?

"Arachni Scanner" features relevant for Security DevOps integration:

» Passive & active scanning (Proxy / Spider) » Uses internally a headless browser-cluster

(for apps with lots of JS) » Automation?

– direct CLI + RPC-API (with CLI) » Web-UI (helpful when permanently

running as server)

Page 30: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Arachni + Jenkins = SecDevOps?

"Arachni Scanner" + Jenkins CLI step in build » Start in build job as CLI step and point to URL

of system under test » Generate HTML report and place into

workspace for inspection » Better execute within nightly build job

(due to duration)

» Alternatively: Start remote (also via CLI) to run async

Page 31: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

HTML report of Arachni scan (Summary)

Bildquelle: Arachni Scanner

Page 32: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

HTML report of Arachni scan (Details)

Bildquelle: Arachni Scanner

Page 33: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

BDD-Security in SecDevOps?

BDD-based framework for functional and technical security tests:

» Technical security tests (i.e. check against XSS, SQL-Injection, XXE, etc.)

– uses ZAP as scanning engine (among others) » Functional security tests

(i.e. check UserA can’t access data from UserB) » Tightly integrates with Selenium based app navigation workflows » Uses JBehave for G/W/T stories & reporting » Can run within CI (Jenkins, etc.) due to JBehave

or as JUnit tests

Page 34: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

BDD-Security Story: Scan for XSS

Bildquelle: BDD-Security

Page 35: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

HTML report of BDD-SecurityBildquelle: BDD-Security

Page 36: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Gauntlt in SecDevOps?

BDD-based framework for executing many security tools/scanners: » Integrates scanners like Arachni, ZAP, sqlmap, etc.

– Easy to integrate "your custom scanner tool" with Gauntlt as well

» Allows to call different scan polices via BDD-stories (G/W/T) » Integration with Jenkins (or other build servers) by either

– Linking Gauntlt’s HTML report to build, or by – modifying how Gauntlt calls Cucumber to produce

JUnit output

Page 37: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Other dynamic scanners exist also…

Some have different focus » Nikto » w3af » IronWASP » Nessus » Mittn (BDD-based) » Gauntlt (BDD-based) » Minion (integration framework) » OWTF (integration framework) » good commercial ones » etc.

Page 38: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Dynamic Depth": Level 2

Scanning of authenticated parts (= "post-auth") via UI layer » Properly maintaining sessions » Logout-detection & automatic re-login » Different users / roles » Spider & scan post-auth

Handling of hardening measures of application under test

» CSRF-Tokens, etc.

Page 39: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Guide ZAP into Post-Auth in CI

Use ZAP manually (1x) to configure "Context": Auth, RegExps for Logged-In/Out Indicators, Users etc.

– save as "ZAP Session-File" (could be in code repo) – use that "Session-File" from code repo as starting point of scan

(loaded as ZAP session during build job). Note: Current version of ZAP has a bugfix pending for loading creds from session file

One can set these auth values and/or additional data via ZAP’s REST-API during each build before scan starts (from Jenkins/Maven/…)

– use that to define current active session etc. during spider & scan

Also Scripts in JavaScript or Zest can be registered in ZAP context to programmatically give authentication to ZAP

Page 40: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Login config example within ZAP

Page 41: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ZAProxy Jenkins Plugin: ZAP session use

Page 42: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Guide Arachni into Post-Auth

Give authentication infos to Arachni (Auth, Logged-In Indicators, Users)

» Use Arachni "autologin" plugin to specify via command line – Login URL, formfield names, credentials, logged-in

indicator, excludes

» Alternatively write custom ruby script for "login_script" plugin – Individual custom login logic possible – Logged-In indicators (RegExp) to know when to re-login

Page 43: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

./arachni --plugin=autologin: url=https://example.com/login.action, parameters='j_username=foo&j_password=bar', check='Logout' --scope-exclude-pattern=logout.action https://example.com/

Login config example within Arachni (used in CI)

Or individual ruby script if more custom login logic required…

Eventually also --session-check-url & --session-check-pattern

>> Easier to achieve Post-Auth Scan with Arachni (compared to others) >> Can also be ordered via CLI remote/async during build (nightly) or scheduled (WebUI)

Page 44: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Arachni Web-UI: Scan scheduling

Bildquelle: Arachni Scanner

Page 45: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Guide BDD-Security into Post-Auth

Use Selenium to navigate through the login process

» Based on excellent integration of BDD-Security with Selenium » Separate app navigation code (Selenium) from Security

testing code » Use Selenium class (that handles login) within BDD stories » Perform further spidering & active scanning (through ZAP)

post-auth

Page 46: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationScanHelper extends WebApplication implements ILogin {

// ... integrates with BDD-Security via parent class & interface ...

}

Page 47: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationScanHelper extends WebApplication implements ILogin {

@Override public void openLoginPage() {

}

@Override public void login(Credentials credentials) {

}

@Override public boolean isLoggedIn(String role) {

}

Page 48: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationScanHelper extends WebApplication implements ILogin {

@Override public void openLoginPage() { driver.get(Config.getInstance().getBaseUrl() + "customer/login"); verifyTextPresent("Login"); }

@Override public void login(Credentials credentials) { UserPassCredentials creds = new UserPassCredentials(credentials); driver.findElement(By.id("username")).clear(); driver.findElement(By.id("username")).sendKeys(creds.getUsername()); driver.findElement(By.id("password")).clear(); driver.findElement(By.id("password")).sendKeys(creds.getPassword()); driver.findElement(By.name("_action_login")).click(); }

@Override public boolean isLoggedIn(String role) { if (driver.getPageSource().contains("My Account")) { return true; } else { return false; }

Page 49: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

What about CSRF-Tokens?

ZAP can handle CSRF-Tokens » Can be individualised via configuration

BDD-Security uses (among others) ZAP for scanning

Arachni can also handle CSRF-Tokens

Page 50: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Apply targeted delta scanning:

• Developer teams maintain "scope delta files" in SCM

• like "what parts of the application have changed during this sprint" (= URL patterns)

• could’ve been recorded in proxy-based tools (like ZAP) by Devs

• DAST Scanner uses it to scan only this during onCommit or nightly CI run

• Feed this delta information as argument to scope definitions Arachni: --scope-include-pattern / --scope-exclude-pattern ZAP: Include/Excludes in Context (via Session-File or REST-API) BDD-Security: maintain as "table file" of RegExps as scope

If scanning takes too long? (1/2)

Page 51: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

If scanning takes too long? (2/2)

Train spiders about the structure of application to reduce scan redundancy and avoid scanning static resources

• ZAP: use RegExp "Exclude from scanner" rules to prevent redundant scans in data-driven applications (i.e. scan only one instance of it)

• Arachni: --scope-redundant-path-pattern / --scope-auto-redundant as well as --scope-exclude-pattern="logout\.|\.js|\.css|\.jpg|\.png"

Don’t forget to scan complete application at least on nightly or weekly run

• Only apply full scan, if simple smoke-tests are green

Page 52: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Dynamic Depth": Level 3

Separate scanning of different application layers / backends

• Scan internal WebServices (e.g. SOAP / REST) = directly scan backends

• Detect and scan parameter positions within XML, JSON, …

• Scan from "within" the different application’s layers

• IAST with distributed agents & instrumentation aims into that direction

• At least one simple step in that direction:

• Use the proxy also between your backend service calls

Page 53: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Backend scans with ZAP

How to achieve this with ZAP?

• ZAP operates as proxy server: place it between backend calls

• ZAP can inject payloads in observed XML tags/attributes & JSON fields

• Capture service call traffic in integration test during CI while either

A. frontend UI tests execute service backend calls indirectly, or

B. executing service tests that directly access the service endpoint

• Automatically scan as new requests are seen: "ATTACK Mode"

Also keep an eye on an alpha-level SOAP-Scanner ZAP addon

Page 54: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Backend scans with Arachni

How to achieve this with Arachni?

• Arachni can also operate as proxy: place it between backend calls

• Use passive proxy plugin to "train" Arachni of the XML / JSON requests

• New addition in v1.1 to extract XML / JSON input vectors from it

• Use that collected input vector data to feed the active scan for the observed requests

Page 55: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Dynamic Depth": Level 4

Targeted scanning of individual forms / wizards (UI) and service layers

» More individualised workflow coverage (not just simple spidering)

» Business-logic compliant usage patterns & inputs – "fill shopping cart followed by checkout process" – "access backend WebServices in special order to test

workflow", etc. » Custom coded security tests tailored to the application

Page 56: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ZAP with special workflows (1/3)

Many ways exist… The simplest one could be: Re-use existing UI tests (Selenium, …) » Proxy this traffic through ZAP in

“ATTACK-Mode" (in security test phase of build) » Optionally use ZAP Attack-Policies to specify/limit

certain attack types

Page 57: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ZAP with special workflows (2/3)

A more customised handling of individual workflows can be achieved:

Re-use & enhance existing "UI test code" at the desiredworkflow steps with calls to ZAP’s (REST)-API ordering attacks » Basically it’s like Unit-Test code that uses Selenium along

with ZAP-Calls at the proper positions in application workflow » Type of "ordered attacks" can again be defined via policies » Start ZAP as Daemon from Jenkins via plugin

Page 58: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationTest { // = regular JUnit unit test

@Before public void setup() {

}

@Test public void testShippingAddressStep() {

}

@Test public void testBillingAddressStep() {

}

}

Page 59: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationTest { // = regular JUnit unit test

@Before public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) }

@Test public void testShippingAddressStep() {

}

@Test public void testBillingAddressStep() {

}

}

Page 60: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationTest { // = regular JUnit unit test

@Before public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) }

@Test public void testShippingAddressStep() { // 1. use Selenium to fill shopping cart // 2. use Selenium to proceed to checkout // 3. use Selenium to provide reasonable shipping address data

}

@Test public void testBillingAddressStep() {

}

}

Page 61: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationTest { // = regular JUnit unit test

@Before public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) }

@Test public void testShippingAddressStep() { // 1. use Selenium to fill shopping cart // 2. use Selenium to proceed to checkout // 3. use Selenium to provide reasonable shipping address data // 4. set attack policy (types & strength) in running ZAP (API) /* 5. call ZAP (API) to actively scan the last seen URL

(optionally define parameter excludes via API or ZAP "input vector scripts" if custom input format) */

}

@Test public void testBillingAddressStep() {

}

}

Page 62: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

See https://github.com/continuumsecurity/zap-webdriver for a great working example of Selenium ZAP integration

public class ShopApplicationTest { // = regular JUnit unit test

@Before public void setup() { // 1. start new proxy session in running ZAP (via REST-API call) // 2. create Selenium driver (proxying through running ZAP) }

@Test public void testShippingAddressStep() { // 1. use Selenium to fill shopping cart // 2. use Selenium to proceed to checkout // 3. use Selenium to provide reasonable shipping address data // 4. set attack policy (types & strength) in running ZAP (API) /* 5. call ZAP (API) to actively scan the last seen URL (optionally define parameter excludes via API or ZAP "input vector scripts" if custom input format) */ }

@Test public void testBillingAddressStep() { // same idea as above ... just continue with the pattern }

}

Page 63: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ZAP with special workflows (3/3)

Alternatively "train" ZAP about the workflow by recording Zest scripts

»Keep an eye on "Sequence Scanning" alpha-level ZAP AddOn

–Still alpha-level (as of June 2015), but interesting approach

Page 64: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

BDD with special workflows

Use Selenium to further drive BDD-Security initiated checks:

» Selenium-based test code navigates application workflows » This code is integrated with BDD (via Java interfaces),

so that: – BDD-Security stories can use that code to navigate

and generate traffic – This generated traffic will be scanned by ZAP via BDD

Page 65: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

public class ShopApplicationScanHelper extends WebApplication implements ILogin {

// ... already covered login handling code // in BDD-Security example ...

public boolean fillShoppingCartAndCheckout() { // custom Selenium (recorded) code // to drive certain use cases (like shopping) ... }

}

Page 66: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Use this navigation method inside BDD-Security scenario:

... Given a new scanning session And the page flow described in the method: \ fillShoppingCartAndCheckout is run through the proxy ...

Page 67: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

If no Selenium test code exists?

Simply give developer teams access to ZAP to (at least) pre-seed the scanner:

» Developer teams use browser to navigate app workflows while proxying – Thereby seed the ZAP session(s) with navigation nodes/workflows – Save the ZAP session(s) and check-in into SCM (Git, SVN, …)

» Point the Jenkins ZAP plugin to the saved ZAP session(s) as starting point

» Devs can add to this list of URLs for ZAP with each new UI

BTW: ZAP is also available as Docker image…

Page 68: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Sometimes it’s useful to have programmed unit-tests directly calling a business API

• Test directly at the API-level (i.e. Java code level)

• Utilise fuzzing values from popular sources like "fuzzdb" and/or generate on-the-fly with fuzzer "radamsa"

• Check the API method’s response (like regular unit test, but seeking for signs of successful attack tests)

Security-UnitTests for APIs (1/2)

Page 69: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Security-UnitTests for APIs (2/2)Leverage the benefit of direct API-level testing (i.e. testing "from within")

• Place "probes" into the application to watch out for successful attack tests

• Check some OWASP AppSensor detection points for potential attack triggers (RASP-like)

• Error logs are good candidates to check also

• Mock objects (Mockito, etc.) or weave aspects (AOP) collecting data (i.e. method calls and arguments, etc.)

Page 70: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis of "Static Depth"

How deep is static code analysis performed within a Security DevOps CI chain?

i.e. "where" are static security tests applied?

Page 71: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Static Depth": Level 1

Assurance that no third-party code with known vulnerabilities is used » Check application’s server-side dependencies

– For Java applications: JAR files, etc. » Check application’s client-side dependencies

– JavaScript frameworks, etc.

Useful even for projects not currently under development » New vulnerabilities could become public:

Build regularly to check

Page 72: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Java libs: OWASP Dependency-Check

Scans all dependencies (even transitive ones) against CVE list

» Available as Maven plugin and Ant task » CLI version also available » Jenkins plugin for nice reporting

(and build breaking thresholds) » Not false-positive free

– regularly needs some time to triage findings

Page 73: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Reporting in Jenkins

Page 74: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015
Page 75: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Findings (CVE-Details)

Page 76: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

JavaScript libs: retire.js

Scans application’s JavaScript files against list of known vulnerable ones

» Available also as Maven Plugin … »… and as CLI

(point it to your folder containing .js files)

Page 77: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Static Depth": Level 2

Scan important parts of source code for vulnerability patterns

» At least important parts of applications’ codebases are scanned – like in-house reused code & custom developed

frameworks – for big multi-project Maven projects:

at least frontend and backend projects – or code of changed modules within a sprint

Page 78: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Java: FindSecurityBugs

Plugin for FindBugs with enhanced checks for security issues in Java code

» Runs within FindBugs, so that it can execute in Maven and/or Sonar » Generates XML-file with potential findings » Jenkins plugin for representing results

(also possible to display in Sonar) – Integrates with Jenkins "Static Code Analysis" as part of FindBugs – Delta reporting of new (and fixed) findings & trends

Page 79: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

FindBugs in Jenkins: Trends, etc.

Page 80: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

in Sonar: Vulnerability & Code-Pointer

Page 81: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

JavaScript: ScanJS

Scans JavaScript code for vulnerabilities

» Most importantly DOM-based XSS (Cross-Site Scripting) » Flags also "Usage of sensitive API" issues » Executable as CLI from within Jenkins build steps » Optional Web-UI (running as local server)

Page 82: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ScanJS Findings as HTML: DOM-XSS

Page 83: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Ruby on Rails apps: Brakeman

Scans RoR code for vulnerabilities

» CLI based » Nicely integrates with Jenkins

– Rendering scan results and trend analysis » Allows to compare scan results (delta detection)

– "What has been fixed and what was newly introduced?"

Page 84: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Other static scanners exist also…

Some have different focus

» PrimeJS (like ScanJS for JavaScript DOM-XSS etc.) » PMD (for Java code regarding API safety issues) » good commercial ones » etc.

Page 85: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Static Depth": Level 3

Scan the complete applications’ source code for vulnerability patterns

» Just like "Level 2", but now for the complete codebase(no excludes)

– i.e. all projects of a multi-project Maven project – best via parent or corporate pom.xml

For FindSecurityBugs this means: » pre-compile even all JSPs (if you have any)

to scan them

Page 86: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Static Depth": Level 4

Scan code of (important) third-party dependencies used in application

» If the component is open-source: – include it in regular scanning activities

» If the component is closed-source: – consider using code scanners that operate on binaries

• FindBugs (hence its FindSecurityBugs plugin) can also scan Java binaries, since it operates on Bytecode

Page 87: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis of "Intensity"

How intense are the majority of the executed attacks within a Security

DevOps CI chain?

i.e. "what" is being checked for?

Page 88: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Intensity": Level 1

Dynamic checks: Only passive scanning

» Simply proxy testcase generated traffic through passive operating scanners

» Quick-win for using existing testcase code (i.e. Service- or UI-tests) » Can execute within existing tests that execute on every commit

– No requirement for a separate nightly CI job (i.e. doesn’t slow down build)

Static checks: Just scan the code along with other code metrics » Use the default rules of the scanners and integrate them into CI

Page 89: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Intensity": Level 2

Use lightweight active scanning options for dynamic checks

» ZAP: active scanning of observed URLs (proxied or spidered) – perform an "active scan": For Jenkins ZAProxy Plugin

provide a ZAP policy with desired active scan intensity (or via ZAP’s REST-API during build for other setups)

» Arachni: enable "active scan" during spidering » BDD-Security stories can call ZAP with policy files defining

attack strength

Better to execute in separate (nightly) build job due it’s to duration Think of resetting databases (+ more environment) prior or after test execution

Page 90: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Intensity": Level 3

Use heavyweight scanning options on important parts of application

Dynamic checks: create "riskier" custom attack profiles

ZAP: – set "Threshold" to "Low" / "Strength" to "High" (or selectively "Insane") – enable more scanning rules via Policy Files (could be maintained &

used via SCM) – use ZAP’s "Advanced SQLInjection Scanner" extension (or directly

call sqlmap)

Arachni: – Enable the more aggressive "audit flags" like --audit-forms, …

Static checks: FindSecurityBugs: set "Threshold"to "Low" / "Effort" to "Max"

Page 91: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Intensity": Level 4

Use customised rule sets for dynamic checks

ZAP (which is very extensible via custom scripts, recored and customised Zest, or plugins): » Custom coded scan scripts as active & passive scan rules for

company-individual checks » Security regression tests (in Zest) against previous found

vulns (also logic flaws) » Custom input vector scripts: e.g. define certain injection

points in custom encoded params » Proxy scripts: for example for generic request/response

modifications

Page 92: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Scripting possibilities in ZAP

Page 93: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

… and for static code scans?

Use customised rule sets for static checks

FindSecurityBugs » Custom coded rule files (in Java) to check for wrong usages of

server-side inhouse frameworks (or alike)

ScanJS » Custom rule files (in JSON) to check for wrong usages of

client-side inhouse frameworks (or alike)

Page 94: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis of "Consolidation"

How complete is the process of handling findings within a Security DevOps CI chain?

i.e. "how" are the results used?

Page 95: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Consolidation": Level 1

Generate human-readable (HTML) reports from tools and link them in Jenkins

» All relevant mentioned static and dynamic scanners generate HTML reports

» Collect and publish them in Jenkins build: via Jenkins "HTML Publisher Plugin"

Page 96: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Jenkins "HTML Publisher Plugin": Configuration of HTML reports to link

Page 97: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Jenkins "HTML Publisher Plugin": Result in build

Page 98: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

HTML report of ZAP scan

Page 99: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Consolidation": Level 1 (cont.)

Use simple criteria to "break the build" on heavy findings (ok, at least "unstable")

» Most scanners have capabilities to automatically flag the build » Dependency-Check (Severity Threshold), » BDD-Security (with the JBehave-stories), » FindSecurityBugs (via Sonar when rated as blocker), » Arachni (via Gauntlt execution with BDD-like stories), » etc.

» For others: at least do a simple log parse from Jenkins "Log Parser Plugin" to flag the build as unstable and/or broken

Page 100: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

OWASP Dependency-Check: Thresholds to break build

Page 101: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

FindSecurityBugs: IDE support & Collaboration

FindBugs (with FindSecurityBugs) has good Jenkins & IDE integration

Additionally consider placing the FindBugs generated XML report file within Jenkins

» Developers can load the XML report file into FindBugs IDE plugins (Eclipse, IntelliJ, NetBeans, …)

» Developers & Security Analysts can collaborate on bugs via the IDE plugins

– See https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorial for more details

• Consider installing it as your private "FindBugs"-Server on-premise to keep results only on your servers…

Page 102: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

FindSecurityBugs Eclipse Perspective with Collaboration View + Remediation

Page 103: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Consolidation": Level 2

Custom logic to make build unstable and/or broken depending on

» Type of vulnerability (CWE or WASC or …) » Severity ranking (high risk) » Confidence level (firm vs. tentative)

Provide useful remediation info to developers

Respect suppression mechanisms to rule out false positives

Page 104: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Flagging builds from reports

How (from within a CI job)?

» Most scanners also emit XML reports that can be parsed – Often a simple XPath count is just fine

» Alternatively fetch the results by accessing the scanner’s API

» Be sure to only break build with findings of high severity and high confidence !!! – Less is more (when it comes to automation)…

» When new check introduce many findings: Baseline-Approach: Only break on new findings

Page 105: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Providing useful info to Devs

Lookup CWE-ID (provided by most scanners) in vulndb:

» https://github.com/vulndb/data(currently 65+ vulns described)

» Useful if the messages from scanners are too generic or don’t help much

» Maps vulnerability to useful remediation messages

» Possibility to extend that repository locally with company-individual texts (mapping to the technology stack) —> that’s where it pays off…

Page 106: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Suppression of false positives

Custom finding list (from XML reports): filter out according to list from code repo

BDD-Security: Use false positive tables in story files (i.e. text files checked in SCM)

FindSecurityBugs: Use exclusion lists in config (XML filter files) & annotation @SuppressWarnings on false positive code lines

Dependency-Check: XML file of suppressions (checked-in along with the project)

retire.js: "ignore file" parameters on CLI(cloud be loaded in CI from code repo)

Page 107: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Consolidation": Level 3

Consolidation goals:

» Consolidate & de-duplicate findings from different scanner reports (with better false positive handling)

» Push consolidated findings into established bug-tracker (known to devs)

» Delta analysis & trends over consolidated data sets

Page 108: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

ThreadFix as result consolidator

Use a local ThreadFix server, which imports native scanner outputs

• does the heavy lifting of consolidation & de-duplication

• pushes findings toward bug-tracker and IDE (via plugins)

• process can be customised using it’s own REST-API

• ThreadFix imports findings of ZAP, Arachni, FindBugs, Brakeman, etc.

Page 109: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Report generated by ThreadFixBildquelle: ThreadFix

Page 110: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Trend Chart generated by ThreadFixBildquelle: ThreadFix

Page 111: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Axis "Consolidation": Level 4

Measure the concrete code coverage of your security testing activities

» Find untested "white spots" » Derive where static checks and code reviews

should focus more to compensate

Page 112: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Code coverage analysis

Use "OWASP Code Pulse", which instruments your Java app

» collects coverage data during dynamic security testing scans » generates reports ("code treemaps") of coverage

Page 113: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Code Treemap of dynamic scan coverageBildquelle: OWASP Code Pulse

Page 114: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

Thank you very much!

Page 115: Security DevOps - Free pentesters' time to focus on high-hanging fruits // HackPra 2015

LinksOWASP ZAP https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_ProjectZAP Selenium Demo https://github.com/continuumsecurity/zap-webdriverZAP Jenkins Plugin https://wiki.jenkins-ci.org/display/JENKINS/ZAProxy+PluginBDD-Security http://www.continuumsecurity.net/bdd-intro.htmlArachni http://www.arachni-scanner.comOWASP Dependency Check https://www.owasp.org/index.php/OWASP_Dependency_CheckOWASP Dependency Track https://www.owasp.org/index.php/OWASP_Dependency_Track_ProjectFindSecurityBugs http://h3xstream.github.io/find-sec-bugs/FindSecurityBugs-Cloud https://code.google.com/p/findbugs/wiki/FindBugsCloudTutorialretire.js http://bekk.github.io/retire.js/ScanJS https://github.com/mozilla/scanjsJenkins Log Parser Plugin https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+PluginThreadFix http://www.threadfix.orgOWASP Code Pulse https://www.owasp.org/index.php/OWASP_Code_Pulse_ProjectSeccubus https://www.seccubus.comvulndb https://github.com/vulndb/datafuzzdb https://code.google.com/p/fuzzdb/radamsa https://code.google.com/p/ouspg/wiki/Radamsa

Interested in more web security stuff? Visit my Blog: www.Christian-Schneider.net @cschneider4711

Bildquelle: dreamstime.com