MANAGING APPLICATION SECURITY
2017 Application
Security Survey
by Security Compass
OCTOBER 2017
Altaz Valani
Director of Research
@altazvalani
linkedin.com/in/altazvalani
PERSONAL BIO
MANAGING APPLICATION SECURITY PAGE 2
Director of Research at Security Compass (www.securitycompass.com) responsible for
managing the overall research vision and team.
Previously:
• Senior Research Director, Application Development at Info-Tech Research Group
• Senior Manager, KPMG
• Started a software development company
Community Involvement:
• IEEE P2675 - DevOps - Standard for Building Reliable and Secure Systems
• IEEE P7002 - Data Privacy Process
• IEEE P2430 - Standard for Software Nonfunctional Sizing Measurement
• SAFEcode – Leadership Committee
Interests:
• Research and collaboration
• Secure software development
• Teaching and learning
PURPOSE
To discover how large, complex organizations
manage application security: the drivers,
programs, and successes.
WHO
Most respondents were large multinational
companies earning >$1 billion USD.
THE RESULT
Aggregated insights, industry trends, and best
practices that illuminate how large
corporations manage application security.
ABOUT THE SURVEY
MANAGING APPLICATION SECURITY PAGE 3
Security Compass (n=27)
SURVEY DEMOGRAPHIC
0
2
4
6
8
10
12
14
16
18
20
$10B+ $1B-$10B $100M-$1B < $100M
Annual Earnings
KEY RESEARCH FINDINGS
BUSINESS PRESSURE IS NOT GOING AWAY
INCREASING
SPEED
OF BUSINESS
INCREASING
SOPHISTICATION
OF RISK MANAGEMENT
INCREASING
PRESSURE ON
COST CONTROL
MANAGING APPLICATION SECURITY PAGE 5
WHAT IS DRIVING APPLICATION SECURITY?
79% of respondents stated
that general risk
management was the
key driver for their
organization's
application security.
MANAGING APPLICATION SECURITY PAGE 6
3.57%
3.57%
3.57%
7.14%
21.43%
35.71%
50.00%
78.57%
Business Model Transformation
Board Demand
Privacy of Client Data
Competitive Need
Breaches/incidents at own or other organizations
Customer Demand
Compliance Requirements
General Risk Management
Application Security Drivers(n=28)
Which of the following drive spending on Application Security?
79% STATED GRM WAS THE KEY DRIVER FOR APPSEC
MANAGING APPLICATION SECURITY PAGE 7
Source: Wikipedia, Risk Management Framework (NIST Special
Publication 800-37).
Source: “Assessing the Adequacy of Risk Management
Using ISO 31000”, IIA, 2010.
Source: “The Three Lines of Defense if Effective Risk
Management and Control”, IIA, 2013.
SECURITY PERSPECTIVE
73% of respondents stated
that application security
is a high or critical
priority within their
organization.
MANAGING APPLICATION SECURITY PAGE 8
Security Compass (n=26)
What is the relative importance of application security in your overall information security program?
ORGANIZATIONAL SUPPORT FOR APPLICATION SECURITY (BY INDUSTRY)
MANAGING APPLICATION SECURITY PAGE 9
RESPONSE RANGE: 1 = NO SUPPORT TO 5 = SUPPORT ACROSS THE BOARD
1 2 3 4 5
Energy / Utility
Energy/Utility(n=4)
1 2 3 4 5
Finance
Finance(n=9)
1 2 3 4 5
ISV
ISV(n=6)
Rate your organization's level of support for application security
The Financial industry appears to be clustering (driven in part by regulations and compliance). The other
industries still vary widely.
HOW ARE BUDGETS BEING ALLOCATED?
MANAGING APPLICATION SECURITY PAGE 10
of information security budget is being spent on the securing of software.**
* Source: 2016 Verizon Data Breach Investigations Report
** Source: Contrast Security, Why Application Security Leaves Enterprises Wide Open to Attacks
of respondents stated that general risk management was the key driver for their
organization's application security.79%
of respondents stated that application security is a high or critical priority
within their organization.73%
< 4%
the highest category of breach pattern (Web Applications).*40%+
DEVELOPER PERSPECTIVE
MANAGING APPLICATION SECURITY PAGE 11
RESPONSE RANGE: 1 = NO TRAINING TO 5 = ALL DEVELOPERS ARE TRAINED
There is resistance to adoption of security awareness training. Many see this as extra work, getting in the way of
releasing software.
How broad is the adoption of developer security awareness training at your organization?
1 2 3 4 5
Energy/Utility(n=3)
1 2 3 4 5
ISV(n=6)
1 2 3 4 5
Finance(n=13)
TYPE OF TRAINING
MANAGING APPLICATION SECURITY PAGE 12
57.14%
35.71%
7.14%
Training Modality(n=28)
e-Learning e-Learning & in-person In-person
Which type of training do you use?
TRACKING THE EFFECTIVENESS OF
AN APPLICATION SECURITY PROGRAM
MANAGING APPLICATION SECURITY PAGE 13
What do you use to track the effectiveness of your application security program?
75.00%
67.86%
39.29%
32.14%
25.00%
14.29%
10.71%
7.14%
3.57%
Number of vulnerabilities found
Compliance / adherence to company policies
Length of remediation
Number of development teams using tools / tool adoption
Completion of security requirements
We do not track the effectiveness of our application securityprogram
Delays to deadlines due to security fixes
Money spent on patching in production
Money spent on remediation
Application Security Metrics(n=28)
KEY SECURITY ACTIVITIES PERFORMED
MANAGING APPLICATION SECURITY PAGE 14
RESPONSE RANGE: 1 = WE DON’T PERFORM THIS ACTIVITY TO 5 = PERFORMED ON ALL APPLICATIONS
4.00
3.77
3.15
3.08
2.85
2.73
2.35
2.35
2.27
2.23
2.08
2.00
1.69
1.04
0 1 2 3 4 5
Application risk classification
Threat risk assessments (not focused specifically on application security)
Dynamic analysis (DAST)
Static analysis (SAST)
Manual penetration testing / vulnerability assessments
Application security requirements
Secure coding standards / guidelines
Manual code reviews
Web application firewalls (WAFs)
Threat modelling / design review (application security focused)
Open source library scanning (e.g. Blackduck, Sonatype)
Security testing performed by QA testers
Fuzzing
RASP / IAST
AVERAGE RATING(n = 26)
OUR APPLICATION SECURITY RESPONSE
GAP ANALYSIS OF CODE SCANNERS PAGE 15
INCREASING
SPEED
OF BUSINESS
INCREASING
SOPHISTICATION
OF RISK MANAGEMENT
INCREASING
PRESSURE ON
COST CONTROL+AUTOMATED SCANNING TOOLS
THE BIG QUESTION
GAP ANALYSIS OF CODE SCANNERS PAGE 16
SCANNERS GIVE A
PASSING MARK
SECURE
SOFTWARE
THE BIG QUESTION
GAP ANALYSIS OF CODE SCANNERS PAGE 17
SCANNERS GIVE A
PASSING MARK
SECURE
SOFTWARE
46% OF APPLICATION-LEVEL RISKS ARE NOT COVERED BY SAST & DAST TOOLS
MANAGING APPLICATION SECURITYPAGE 18
Source Code SAST & DAST
Remediation
30% of total risks found & fixed
average time to remediation = 316 days*
54% of risks found*
46% of risks are not
found
70% of risks unaddressed
24% of risks found, not fixed
54% remediation rate*
*Adapted from:
National Institute of Standards and Technology. “Report on the Static Analysis Tool Exposition IV”.
Gartner for Technical Professionals. “Application Security Think Big and Start with What Matters”.
Veracode. “State of Software Security”, 2016.
WhiteHat Security. “Web Applications Security Statistics Report”.
46% OF APPLICATION-LEVEL RISKS ARE NOT COVERED BY SAST & DAST TOOLS
MANAGING APPLICATION SECURITYPAGE 19
Source Code SAST & DAST
Remediation
30% of total risks found & fixed
average time to remediation = 316 days*
54% of risks found*
46% of
risks are
not found
70% of risks unaddressed
24% of risks found, not fixed
54% remediation rate*
SC whitepaper
• Intent
• Pointer Reference Manipulation
• Compiler Optimization
• Application Boundary
• Scanner Optimization
• Side Effects
• Runtime Class Creation
• Halting Problem
• CERT Non-Automation
*Adapted from:
National Institute of Standards and Technology. “Report on the Static Analysis Tool Exposition IV”.
Gartner for Technical Professionals. “Application Security Think Big and Start with What Matters”.
Veracode. “State of Software Security”, 2016.
WhiteHat Security. “Web Applications Security Statistics Report”.
A CASE STUDY: BUFFER OVERFLOW
GAP ANALYSIS OF CODE SCANNERS PAGE 20
Scanner # Identified
bugs
False Negative
rate
# Identified fixes False Positive
rate
Scanner A 19 68.3% (41/60) 13 31.6% (6/19)
Scanner B 32 68.0 % (68/100) 8 75.0% (24/32)
Scanner C 10 56.5% (13/23) 0 100.0% (10/10)
Scanner A +
Scanner B
42 58.0% (58/100) 14 66.7% (28/42)
Scanner B +
Scanner C
39 61.0% (61/100) 7 82.1% (32/39)
Scanner A +
Scanner C
26 59.4% (38/64) 13 50.0% (13/26)
All 47 53.0% (53/100) 13 72.3% (34/47)
Ye, Tao et al. “An Empirical Study on Detecting and Fixing Buffer Overflow Bugs”,
https://pdfs.semanticscholar.org/20e8/6f51f90b1fa9ae48752f73a757d1272ca26a.pdf, 2016
QUESTION YOUR ASSUMPTIONS
GAP ANALYSIS OF CODE SCANNERS PAGE 21
1. It is highly unlikely to create a static analyzer that catches all known security vulnerabilities
2. Scanners are typically optimized for a certain class of vulnerabilities (lexical, data flow)
3. Compiler optimization can improve speed but inject security vulnerabilities
4. Scanners cannot understand intent (meaning of variables)
5. It is not possible to detect all vulnerabilities through automation alone
KEY SECURITY ACTIVITIES PERFORMED
MANAGING APPLICATION SECURITY PAGE 22
S O F T W A R E D E V E L O P M E N T L I F E C Y C L EREQUIREMENTS
MANAGEMENT
CODE REVIEW
(SAST)
PEN TESTING
(DAST)
75.00%
67.86%
39.29%
32.14%
25.00%
14.29%
10.71%
7.14%
3.57%
Number of vulnerabilities found
Compliance / adherence tocompany policies
Length of remediation
Number of development teamsusing tools / tool adoption
Completion of securityrequirements
We do not track the effectivenessof our application security
program
Delays to deadlines due tosecurity fixes
Money spent on patching inproduction
Money spent on remediation
TRACKING THE EFFECTIVENESS OF
AN APPLICATION SECURITY PROGRAM
MANAGING APPLICATION SECURITY PAGE 23
RISK
SOFTWARE PROJECT PROGRESS
IDENTIFY
CONTROL
IMPLEMENT
CONTROL
VALIDATE
CONTROL
We have jumped straight to
validation without identifying the
root cause and implementing the
appropriate controls to reduce
application security risk.
ENSURING THE SECURITY OF THIRD-PARTY VENDORS
MANAGING APPLICATION SECURITY PAGE 24
How do you ensure the security of third party software vendors?
85.71%
57.14%
50.00%
42.86%
35.71%
17.86%
17.86%
Detailed vendor security questionnaire (not specific toapplication security)
Review of security certification not specific toapplication security (e.g. SSAE16/SOC II Type 2/3, ISO
27001)
Penetration testing and/or dynamic analysis on thirdparty software
Require vendors to have a secure SDLC / applicationsecurity policy
Code review, static and/or binary analysis on third partysoftware
Threat modelling or other design-level analysis
Provide detailed application security requirements (e.g."perform input validation") as part of contract
3rd Party Security Controls(n=26)
WHAT IS THE EMERGING TREND?
PAGE 25
APPLICATION SECURITY REQUIREMENTS AND THREAT
MANAGEMENT
AUTOMATED
THREAT
MODELING
REQUIREMENTS
GENERATION
WORKFLOW &
ALM
INTEGRATION
TESTING
INTEGRATION
AND
AGGREGATION
• LIGHTWEIGHT
• REPEATABLE
• AUTOMATED
• DOMAIN
AGNOSTIC
• TRACEABLE
• METRICS DRIVEN
• RISK BASED
• FEDERATED
AUDIT AND
COMPLIANCE
• REPORTING
• GAP ANALYSIS
Source: Ramachandran, M. “Software security requirements management as an emerging
cloud computing service”, 2016.
Source: Security Compass
OWASP Knowledge Framework
Source: https://skf.readme.io/
MANAGING APPLICATION SECURITY
ASRTM PROOF OF CONCEPT
GAP ANALYSIS OF CODE SCANNERS PAGE 26
Avg. # of Vulnerabilities
32.8
0
13.2
0.40
5
10
15
20
25
30
35
No SDE Full SDE Usage
Source: Security Compass, Engagement for Financial Services Industry, 2017
Without ASRTM
(n=10)
With ASRTM
(n=5)
MEDIUM
PRIORITY APPS
HIGH PRIORITY
APPS
MEDIUM
PRIORITY
APPS
HIGH PRIORITY
APPS
KEY TAKEAWAYS
Adopt the correct metrics to drive your program. Strive for objective, quantified metrics that
measure risk beyond vulnerabilities (e.g. “How to Measure Anything in Cyber Security
Risk”).
Stop tracking your app sec program by the number of vulnerabilities detected by scanners
alone. Use a application security requirements and threat management platform, (e.g. SD
Elements, OWASP Knowledge framework) and/or tool-assisted threat modelling (e.g.
Microsoft threat modelling tool). Traceable requirements coupled with test cases are more
forward looking and comprehensive.
Require your vendors to have a higher standard for secure SDLC (e.g. ISO 27034 or
vBSIMM or Microsoft's SDL).
MANAGING APPLICATION SECURITY PAGE 27
THANK YOU
FOR A COPY OF THE FULL REPORT, PLEASE VISIT:
https://www.securitycompass.com/managingapplicationsecurity2017/
EMAIL US AT:
JOIN THE ASRTM DISCUSSION:
https://www.linkedin.com/groups/13551214
Top Related