MANAGING APPLICATION SECURITY...AN APPLICATION SECURITY PROGRAM MANAGING APPLICATION SECURITY PAGE...

MANAGING APPLICATION SECURITY

2017 Application

Security Survey

by Security Compass

OCTOBER 2017

Altaz Valani

Director of Research

@altazvalani

linkedin.com/in/altazvalani

PERSONAL BIO

MANAGING APPLICATION SECURITY PAGE 2

Director of Research at Security Compass (www.securitycompass.com) responsible for

managing the overall research vision and team.

Previously:

• Senior Research Director, Application Development at Info-Tech Research Group

• Senior Manager, KPMG

• Started a software development company

Community Involvement:

• IEEE P2675 - DevOps - Standard for Building Reliable and Secure Systems

• IEEE P7002 - Data Privacy Process

• IEEE P2430 - Standard for Software Nonfunctional Sizing Measurement

• SAFEcode – Leadership Committee

Interests:

• Research and collaboration

• Secure software development

• Teaching and learning

http://www.securitycompass.com)/

PURPOSE

To discover how large, complex organizations

manage application security: the drivers,

programs, and successes.

WHO

Most respondents were large multinational

companies earning >$1 billion USD.

THE RESULT

Aggregated insights, industry trends, and best

practices that illuminate how large

corporations manage application security.

ABOUT THE SURVEY


Security Compass (n=27)

SURVEY DEMOGRAPHIC

0

2

4

6

8

10

12

14

16

18

20

$10B+ $1B-$10B $100M-$1B < $100M

Annual Earnings

KEY RESEARCH FINDINGS

BUSINESS PRESSURE IS NOT GOING AWAY

INCREASING

SPEED

OF BUSINESS

INCREASING

SOPHISTICATION

OF RISK MANAGEMENT

INCREASING

PRESSURE ON

COST CONTROL


WHAT IS DRIVING APPLICATION SECURITY?

79% of respondents stated

that general risk

management was the

key driver for their

organization's

application security.


3.57%

3.57%

3.57%

7.14%

21.43%

35.71%

50.00%

78.57%

Business Model Transformation

Board Demand

Privacy of Client Data

Competitive Need

Breaches/incidents at own or other organizations

Customer Demand

Compliance Requirements

General Risk Management

Application Security Drivers(n=28)

Which of the following drive spending on Application Security?

79% STATED GRM WAS THE KEY DRIVER FOR APPSEC


Source: Wikipedia, Risk Management Framework (NIST Special

Publication 800-37).

Source: “Assessing the Adequacy of Risk Management

Using ISO 31000”, IIA, 2010.

Source: “The Three Lines of Defense if Effective Risk

Management and Control”, IIA, 2013.

SECURITY PERSPECTIVE

73% of respondents stated

that application security

is a high or critical

priority within their

organization.


Security Compass (n=26)

What is the relative importance of application security in your overall information security program?

ORGANIZATIONAL SUPPORT FOR APPLICATION SECURITY (BY INDUSTRY)


RESPONSE RANGE: 1 = NO SUPPORT TO 5 = SUPPORT ACROSS THE BOARD

1 2 3 4 5

Energy / Utility

Energy/Utility(n=4)

1 2 3 4 5

Finance

Finance(n=9)

1 2 3 4 5

ISV

ISV(n=6)

Rate your organization's level of support for application security

The Financial industry appears to be clustering (driven in part by regulations and compliance). The other

industries still vary widely.

HOW ARE BUDGETS BEING ALLOCATED?


of information security budget is being spent on the securing of software.**

* Source: 2016 Verizon Data Breach Investigations Report

** Source: Contrast Security, Why Application Security Leaves Enterprises Wide Open to Attacks

of respondents stated that general risk management was the key driver for their

organization's application security.79%

of respondents stated that application security is a high or critical priority

within their organization.73%

< 4%

the highest category of breach pattern (Web Applications).*40%+

DEVELOPER PERSPECTIVE


RESPONSE RANGE: 1 = NO TRAINING TO 5 = ALL DEVELOPERS ARE TRAINED

There is resistance to adoption of security awareness training. Many see this as extra work, getting in the way of

releasing software.

How broad is the adoption of developer security awareness training at your organization?

1 2 3 4 5

Energy/Utility(n=3)

1 2 3 4 5

ISV(n=6)

1 2 3 4 5

Finance(n=13)

TYPE OF TRAINING


57.14%

35.71%

7.14%

Training Modality(n=28)

e-Learning e-Learning & in-person In-person

Which type of training do you use?

TRACKING THE EFFECTIVENESS OF

AN APPLICATION SECURITY PROGRAM


What do you use to track the effectiveness of your application security program?

75.00%

67.86%

39.29%

32.14%

25.00%

14.29%

10.71%

7.14%

3.57%

Number of vulnerabilities found

Compliance / adherence to company policies

Length of remediation

Number of development teams using tools / tool adoption

Completion of security requirements

We do not track the effectiveness of our application securityprogram

Delays to deadlines due to security fixes

Money spent on patching in production

Money spent on remediation

Application Security Metrics(n=28)

KEY SECURITY ACTIVITIES PERFORMED


RESPONSE RANGE: 1 = WE DON’T PERFORM THIS ACTIVITY TO 5 = PERFORMED ON ALL APPLICATIONS

4.00

3.77

3.15

3.08

2.85

2.73

2.35

2.35

2.27

2.23

2.08

2.00

1.69

1.04

0 1 2 3 4 5

Application risk classification

Threat risk assessments (not focused specifically on application security)

Dynamic analysis (DAST)

Static analysis (SAST)

Manual penetration testing / vulnerability assessments

Application security requirements

Secure coding standards / guidelines

Manual code reviews

Web application firewalls (WAFs)

Threat modelling / design review (application security focused)

Open source library scanning (e.g. Blackduck, Sonatype)

Security testing performed by QA testers

Fuzzing

RASP / IAST

AVERAGE RATING(n = 26)

OUR APPLICATION SECURITY RESPONSE

GAP ANALYSIS OF CODE SCANNERS PAGE 15

INCREASING

SPEED

OF BUSINESS

INCREASING

SOPHISTICATION

OF RISK MANAGEMENT

INCREASING

PRESSURE ON

COST CONTROL+AUTOMATED SCANNING TOOLS

THE BIG QUESTION


SCANNERS GIVE A

PASSING MARK

SECURE

SOFTWARE

46% OF APPLICATION-LEVEL RISKS ARE NOT COVERED BY SAST & DAST TOOLS

MANAGING APPLICATION SECURITYPAGE 18

Source Code SAST & DAST

Remediation

30% of total risks found & fixed

average time to remediation = 316 days*

54% of risks found*

46% of risks are not

found

70% of risks unaddressed

24% of risks found, not fixed

54% remediation rate*

*Adapted from:

National Institute of Standards and Technology. “Report on the Static Analysis Tool Exposition IV”.

Gartner for Technical Professionals. “Application Security Think Big and Start with What Matters”.

Veracode. “State of Software Security”, 2016.

WhiteHat Security. “Web Applications Security Statistics Report”.

46% OF APPLICATION-LEVEL RISKS ARE NOT COVERED BY SAST & DAST TOOLS

MANAGING APPLICATION SECURITYPAGE 19

Source Code SAST & DAST

Remediation

30% of total risks found & fixed

average time to remediation = 316 days*

54% of risks found*

46% of

risks are

not found

70% of risks unaddressed

24% of risks found, not fixed

54% remediation rate*

SC whitepaper

• Intent

• Pointer Reference Manipulation

• Compiler Optimization

• Application Boundary

• Scanner Optimization

• Side Effects

• Runtime Class Creation

• Halting Problem

• CERT Non-Automation

*Adapted from:

National Institute of Standards and Technology. “Report on the Static Analysis Tool Exposition IV”.

Gartner for Technical Professionals. “Application Security Think Big and Start with What Matters”.

Veracode. “State of Software Security”, 2016.

WhiteHat Security. “Web Applications Security Statistics Report”.

A CASE STUDY: BUFFER OVERFLOW


Scanner # Identified

bugs

False Negative

rate

# Identified fixes False Positive

rate

Scanner A 19 68.3% (41/60) 13 31.6% (6/19)

Scanner B 32 68.0 % (68/100) 8 75.0% (24/32)

Scanner C 10 56.5% (13/23) 0 100.0% (10/10)

Scanner A +

Scanner B

42 58.0% (58/100) 14 66.7% (28/42)

Scanner B +

Scanner C

39 61.0% (61/100) 7 82.1% (32/39)

Scanner A +

Scanner C

26 59.4% (38/64) 13 50.0% (13/26)

All 47 53.0% (53/100) 13 72.3% (34/47)

Ye, Tao et al. “An Empirical Study on Detecting and Fixing Buffer Overflow Bugs”,

https://pdfs.semanticscholar.org/20e8/6f51f90b1fa9ae48752f73a757d1272ca26a.pdf, 2016

https://pdfs.semanticscholar.org/20e8/6f51f90b1fa9ae48752f73a757d1272ca26a.pdf

QUESTION YOUR ASSUMPTIONS


1. It is highly unlikely to create a static analyzer that catches all known security vulnerabilities

2. Scanners are typically optimized for a certain class of vulnerabilities (lexical, data flow)

3. Compiler optimization can improve speed but inject security vulnerabilities

4. Scanners cannot understand intent (meaning of variables)

5. It is not possible to detect all vulnerabilities through automation alone

KEY SECURITY ACTIVITIES PERFORMED


S O F T W A R E D E V E L O P M E N T L I F E C Y C L EREQUIREMENTS

MANAGEMENT

CODE REVIEW

(SAST)

PEN TESTING

(DAST)

75.00%

67.86%

39.29%

32.14%

25.00%

14.29%

10.71%

7.14%

3.57%

Number of vulnerabilities found

Compliance / adherence tocompany policies

Length of remediation

Number of development teamsusing tools / tool adoption

Completion of securityrequirements

We do not track the effectivenessof our application security

program

Delays to deadlines due tosecurity fixes

Money spent on patching inproduction

Money spent on remediation

TRACKING THE EFFECTIVENESS OF

AN APPLICATION SECURITY PROGRAM


RISK

SOFTWARE PROJECT PROGRESS

IDENTIFY

CONTROL

IMPLEMENT

CONTROL

VALIDATE

CONTROL

We have jumped straight to

validation without identifying the

root cause and implementing the

appropriate controls to reduce

application security risk.

ENSURING THE SECURITY OF THIRD-PARTY VENDORS


How do you ensure the security of third party software vendors?

85.71%

57.14%

50.00%

42.86%

35.71%

17.86%

17.86%

Detailed vendor security questionnaire (not specific toapplication security)

Review of security certification not specific toapplication security (e.g. SSAE16/SOC II Type 2/3, ISO

27001)

Penetration testing and/or dynamic analysis on thirdparty software

Require vendors to have a secure SDLC / applicationsecurity policy

Code review, static and/or binary analysis on third partysoftware

Threat modelling or other design-level analysis

Provide detailed application security requirements (e.g."perform input validation") as part of contract

3rd Party Security Controls(n=26)

WHAT IS THE EMERGING TREND?

PAGE 25

APPLICATION SECURITY REQUIREMENTS AND THREAT

MANAGEMENT

AUTOMATED

THREAT

MODELING

REQUIREMENTS

GENERATION

WORKFLOW &

ALM

INTEGRATION

TESTING

INTEGRATION

AND

AGGREGATION

• LIGHTWEIGHT

• REPEATABLE

• AUTOMATED

• DOMAIN

AGNOSTIC

• TRACEABLE

• METRICS DRIVEN

• RISK BASED

• FEDERATED

AUDIT AND

COMPLIANCE

• REPORTING

• GAP ANALYSIS

Source: Ramachandran, M. “Software security requirements management as an emerging

cloud computing service”, 2016.

Source: Security Compass

OWASP Knowledge Framework

Source: https://skf.readme.io/

MANAGING APPLICATION SECURITY

ASRTM PROOF OF CONCEPT


Avg. # of Vulnerabilities

32.8

0

13.2

0.40

5

10

15

20

25

30

35

No SDE Full SDE Usage

Source: Security Compass, Engagement for Financial Services Industry, 2017

Without ASRTM

(n=10)

With ASRTM

(n=5)

MEDIUM

PRIORITY APPS

HIGH PRIORITY

APPS

MEDIUM

PRIORITY

APPS

HIGH PRIORITY

APPS

KEY TAKEAWAYS

Adopt the correct metrics to drive your program. Strive for objective, quantified metrics that

measure risk beyond vulnerabilities (e.g. “How to Measure Anything in Cyber Security

Risk”).

Stop tracking your app sec program by the number of vulnerabilities detected by scanners

alone. Use a application security requirements and threat management platform, (e.g. SD

Elements, OWASP Knowledge framework) and/or tool-assisted threat modelling (e.g.

Microsoft threat modelling tool). Traceable requirements coupled with test cases are more

forward looking and comprehensive.

Require your vendors to have a higher standard for secure SDLC (e.g. ISO 27034 or

vBSIMM or Microsoft's SDL).


THANK YOU

FOR A COPY OF THE FULL REPORT, PLEASE VISIT:

https://www.securitycompass.com/managingapplicationsecurity2017/

EMAIL US AT:

[email protected]

JOIN THE ASRTM DISCUSSION:

https://www.linkedin.com/groups/13551214

MANAGING APPLICATION SECURITY...AN APPLICATION SECURITY PROGRAM MANAGING APPLICATION SECURITY PAGE...

Documents

Transcript of MANAGING APPLICATION SECURITY...AN APPLICATION SECURITY PROGRAM MANAGING APPLICATION SECURITY PAGE...