Post on 08-May-2015
10 Years Later:Are We There Yet?
Carsten Eiram Risk Based Security@CarstenEiram
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Quick Bio – VDB Work Experience
Involved with VDBs for 10+ years
• Currently, CRO at Risk Based Security – commercial arm of Open Security Foundation (runs OSVDB and DatalossDB) – andresponsible for the VulnDB service.
• Chief Security Specialist at Secunia, running the Research team.
• Security Team Lead at Danish Verisign affiliate, running acustomer-only accessible vulnerability database.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Quick Bio – Vulnerability Research
Officially been doing vulnerability research since 2003
• Focused on a static analysis / reverse engineering approach
• Jokingly refer to myself as a "vulnerability connoisseur" - I enjoy analyzing vulnerabilities and their root causes.
• Critical vulnerabilities discovered in products from many major software vendors.
INTRODUCTIONWhat will be discussed?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Reason for Talk
After 10+ years of VDB work,I felt it was time to reflecton certain areas related to
vulnerabilities
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Considerations
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Metrics and their Usage
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Code Quality
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Advisory Quality
VENDORS MAKE BAD DECISIONS
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Vulnerability Handling / Bug Bounties
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Million Dollar (or Leu) Question
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Quick Show of Hands
Vulnerability Statistics
A Quick Overview To Set The Stage
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Vulnerabilities have been around for a very long time- And will continue to be...
• Oldest entries in OSVDB are 79399 and 79400
• Marconi wireless telegraph
• Dated November 1902
• Message spoofing and message disclosure
Currently Oldest Recorded Vulnerabilities
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
“I can tune my instruments so that no other instrument that is not similarly tuned can tap
my messages,” Marconi boasted to London's St James Gazette in February 1903
Guglielmo Marconi
http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
100 years ahead of Oracle with an “Unbreakable” claim!
Unforuntately, it ended just as badly...
First Ever Unbreakable Claim!
http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
RATS!
”There was a young fellow of Italy,who diddled the public quite prettily,”
Nevil Maskelyne Ruins Demo
http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
While not providing the privacy and
security as promised, the wireless telegraph still had one significant
advantage over the wired telegraph:
Not possible to cut the wires!
No Wire-Cutting Please
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Obviously, we have progressed a fair bit technically since then, but have we gotten
significantly better?
Have We Improved?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Bringing The Internet Down – Old Lady Style
Article: http://news.softpedia.com/news/Old-Lady-Cuts-Off-Internet-in-Armenia-193640.shtml
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
10 Year Vulnerability Trend
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130
2000
4000
6000
8000
10000
12000
# Vulns
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
All Datasets Are Incomplete!
All datasets are incomplete - some just more than others
Many love taking CVE content that’s free and do random conclusions based on it, but since the dataset is severely
lacking, the conclusions are as well
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
2006 – 2013 Vulnerability Type Trend
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
2012 Data Breaches due to SQL Injection
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Companies affected by XSS in 2012
Source: CWN - http://www.cyberwarnews.info/2012/07/04/300000-personal-details-leaked-38-sites-hacked-for-projectdragonfly/
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Companies Impacted By Hacking In 2012
Vulnerability Metrics
Usage
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Which is more secure?
Product A10 Vulnerabilities
Product B20 Vulnerabilities
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Security State != Number of Vulnerabilities
Previously, the security state of a product was consideredto be equal to the number of vulnerabilities.
Flawed conclusion!
Today, people understand that the number of vulnerabilities !=
security state
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Some Apparently Still Don’t Know...
“The problem with Java is that a lot of vulnerabilities are constantly being reported in it, and when a lot of
vulnerabilities are reported, then there are a lot of hackers using these to access programs built on Java“
- Morten Stengaard, CTO, Secunia
http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Dissecting the Statement – Part 1
”... then there are a lot of hackers using these to access programs built on Java”
Most vulnerabilities in Java are not used to target Java applications, but the Java Runtime Environment to
compromise the system.
http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Dissecting the Statement – Part 2
”... when a lot of vulnerabilities are reported, then there are a lot of hackers using these…”
Just because a lot of vulnerabilities are reported in a product, a lot of hackers may not be exploiting them.
http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Dissecting the Statement – Part 3
”The problem with Java is that a lot of vulnerabilities are constantly being reported in it…”
The security state of a product is not defined by the number of vulnerabilities reported in it.
http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
We Should All Stop Using Popular Software Then
Vulnerabilities (2013 - Nov 10th)0
50
100
150
200
250
300
350
400
JavaChromeFirefoxInternet Explorer
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Facewall!
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft Argument For SDL (Windows)
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft Argument For SDL (SQL Server)
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft Office Vulnerability Trend
Office 2000 Office 2007 Office 20100
2
4
6
8
10
12
14
Vulnerabilities in Office versions one year after product release(based on Microsoft security bulletins)
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft Security Bulletin Trend
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130
50
100
150
200
250
300
350
BulletinsCVEs
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Researcher Focus and SCADA
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Stop Drawing Conclusions on Vulnerability Counts...
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
There are so many other aspects to consider!
More things to consider incl.
Patched vs. UnpatchedVulnerability Type
ImpactTime-To-Patch
Time-To-Vendor-ResponseSecurity Mechanisms
...
Vulnerability Metrics
Severity
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Severity Metrics
Many different severity metrics – both public and internal
Most popular and hated is CVSS,which currently has problems reflecting real risk
Many concerns raised about CVSSv2 by many peoplee.g. myself and Brian Martin of OSVDB in our open letter:
"The CVSSv2 Shortcomings, Faults, and Failures Formulation"
http://www.riskbasedsecurity.com/reports/CVSS-ShortcomingsFaultsandFailures.pdf
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Limitations of Severity Metrics
Reflecting the threat of vulnerability-dependent issues (e.g. sandbox bypass, ASLR bypass related to memory
disclosure etc.)
By themselves and from a scoring point-of-view, these issues are pretty minor, but when combined with code execution...
Jackpot!
Ability to disclose a few memory addresses was in the past pretty much a non-issue – today it’s very useful.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Pick A Vuln... Any Vuln...
If I’d offer you one vulnerability in e.g. Google Chrome, which
would you pick?
1) Code execution within sandbox
2) Sandbox bypass
CVSSv2: 6.8
CVSSv2: 2.6
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Severity Metrics and Sandbox Bypasses
If we conclude that exploiters are more interested in the sandbox bypass and system administrators should focus on
fixing such a vulnerability over a code execution vulnerability within the sandbox, why are we not rating
them higher?
Case of reality not being reflected well by severity metrics
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Severity Metrics and Vulnerability Chains
And once these issues start occuring in chains, which is becoming more and more common, then it really gets
complex...
You can have a lot of independent minor issues that when combined suddenly are very serious
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Google Chrome Pwn2Own Example
OSVDB 89734
Plugin blocking logic not run
for NaCl in pre-rendering
http://blog.chromium.org/2012/05/tale-of-two-pwnies-part-1.html
OSVDB 80007
GPU command decodinginteger underflow
OSVDB 81645
IPC channel missing listener process validation
OSVDB 80741
Unprivileged renderer can navigate to privileged URLs
OSVDB 89736
Too permissive LoadExtension
bindings for extension manager
OSVDB 80293
Unpacked NPAPI extension installation without
confirmation
CVSSv2: 6.8
CVSSv2: 5.1 CVSSv2:
5.1
CVSSv2: 2.6CVSSv2:
2.6CVSSv2:
7.6 CVSSv2: 9.3
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
When Severity Metrics Met Reality
Severity Metrics only reflecta worst-case impact
Vulnerability Metrics
Exploitability
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft Severity Ratings
Source: http://technet.microsoft.com/en-us/security/gg309177.aspx
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Exploitability Index Ratings
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft Approach: Pros and Cons
Pros ConsGives an realistic evaluation of the technical requirements to exploit a given vulnerability and how feasible it is
Requires significant technical skills and resources to get right
Makes it clear which are theoretical and which are plausible
Still requires a bit of guesstimation
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
No Granularity Really Added...
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
How Does Adobe Do It?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
How Does Adobe Do It?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Adobe Approach: Pros and Cons
...Pros ConsAllows understanding which products, versions, and architectures are most critical to prioritize
Does not factor in technical requirements and the nature of the vulnerability i.e. does not differentiate between theoretical issues and straight-forward issues to exploit
Dynamic approach that can be easily tweaked
Requires very little resources – just an understanding of historical exploitation
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
How Does CVSSv2 Do It?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
CVSSv2 Approach: Pros and Cons
Pros ConsMost reliable of all the approaches: If an exploit is available, a vulnerability is clearly exploitable.
Purely reactive, requiring very fast response times
Requires very little resources – just knowledge of availability of PoCs and exploits
Only takes into account when the availability of an exploit is publicly known i.e. may be exploited long before being flagged as such
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
No information about code quality
All these approaches are interesting and add an extra
dimension – especially if combined
None of these scores tell us anything about the underlying
code quality, though
Code Quality... And How To Measure It
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Code Quality – Why Measure It?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Code Maturity Metric – The Idea
The idea of code maturity is that by evaluating the prevalence of the different vulnerability classes being discovered in a product, we can conclude the maturity of
that product.
We, naturally, focus on it from a security perspective.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Code Maturity Metric – Scoring
• Each vulnerability can be scored based on type, and how easy it is to discover.
• Researchers find simple vulnerabilities first - as simple vulnerabilities are eliminated, researchers move on to finding more complex vulnerabilities.
• When a vendor secures the code, basic vulnerabilities are easier to spot and remedy or never introduce compared to more complex vulnerabilities.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Code Maturity Metric – Scoring Example
Level Vulnerability Classes
0 Classic buffer overflows due to e.g. strcpy, sprintf, sscanf and format string issues.
1 Buffer overflows due to incorrect size being used e.g. strncpy, memcpy and array-indexing issues
2 Arithmetic errors i.e. Integer overflows/underflows, type conversion, signedness.
3 Uninitialized variable, use-after-free, bad cast, complex logic errors.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Schneider Modbus Serial Driver Buffer Overflow
Source: http://www.riskbasedsecurity.com/research/RBS-2013-003.pdf
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Schneider Modbus Serial Driver Buffer Overflow
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Schneider Modbus Serial Driver Buffer Overflow
Code Maturity Level: 1
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Schneider Modbus Serial Driver Buffer Overflow
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Schneider Modbus Serial Driver Buffer Overflow
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
ActiveX Control Vulnerability
Code Maturity Level: 3
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Office 2000: 62
Office XP: 103
Office 2003: 90
Office 2007: 47
Office 2010: 14
Office Vulnerabilities Analysed
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Office Product Code Maturity Scores
Office 2000
Office XP
Office 2003
Office 2007
Office 2010
0 0.5 1 1.5 2 2.5 3
Code Maturity
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Office Vulnerability Type Prevalence
Office 2000
Office XP
Office 2003
Office 2007
Office 2010
0% 5% 10% 15% 20% 25% 30% 35%
Uninitialised VariableObject Type ConfusionUse-after-freeArithmeticArray IndexingIncorrect Size CopyClassic Buffer Overflow
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Measuring the Efforts Taken By Vendors
With this we can put more focus on the code security improvement efforts taken by vendors by being able to
measure them.
Allows system administrators to know which software to steer clear from... and researcher to understand which types of vulnerabilities they can expect to find in a given product.
Advisory QualityOr Lack Thereof...
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Information Needs To Be Publicly Available
Most vendors have also acknowledged that publishing vulnerability information is beneficial
Juniper recently joined the party
Still some black sheep like SAP, trying to keep it a secret…
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Needs To Include Vulnerability Type
Either clearly descripting the vulnerability type in the advisory description
or
alternatively including CWEs
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Everything Is Memory Corruption These Days
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Microsoft MS12-037 vs MS13-080
----
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Rise In Usage Of Memory Corruption Term
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
No requirements to include proper info
Various standards and formats e.g. CVRF are being proposed, but these deal with required fields – not the content of these.
Primary focus is to ensure a structure that is easy to parse in an automated manner.
Completely up to the vendors how much information they feel like sharing. Up to customers to raise their voice, if they
want/need more.
Vulnerability Handling... And Bug Bounties
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Bug Bounties
When I started reporting vulnerabilities to vendors, I was stoked each time I actually got a response - and it wasn't a
threat from a lawyer.
Had any of you told me back then that vendors today would be offering bug bounties, I'd have smiled and shook my head.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Bug Bounties
A few interesting ones are of course Google's bounty, which is one of the more serious vendor bounties, and especially their
latest twist: Bounties for other software!
Microsoft's bounty for vulnerabilities, but specifically bypassing security mechanisms is very interesting
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Shockwave Player Vulnerability Trend
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130
10
20
30
40
50
60
70
80
90
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Bug Bounties
There has definitely been a shift in how vendors perceive bug bounties.
It’s clear to me that if a vendor wants to encourage researchers to look at their code and report it in a
coordinated manner, then bug bounties are very effective when done right.
ConclusionAre We There Yet?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Security Software and Shiny Appliances
More security software and appliances will fix everything!
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Everything Is Vulnerable – Even Security Software!
About 2.2% of all entries in OSVDB cover vulnerabilities in
security software
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
The Security Software Paradox
Reducing attack surface by adding an even greaterattack surface is a paradox
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Code Quality Improvements(?)
Microsoft, Google, and Adobe are examples of vendors noticeably improving their security efforts.
Oracle may be on their way after everyone finally realized that Java is a mess...
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
How Do We Force Vendors To Improve?
Vendors don’t make more secure software because they want to –
it’s because they HAVE to!
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Grand Demonstrations!
We needgrand demonstrations
that ordinary people can relate to!
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
FTC vs. TRENDnet
After demonstrating how network cameras were easily publicly accessible and e.g. allowing spying on people in their homes, the FTC (Federal Trade Commision) in USA went after
TRENDnet.
Eventually agreed that TRENDnet was ”prohibited from misrepresenting the security of its cameras”, will establish a comprehensive IS program, and hire outside consulting to
review security every two years for 20 years...
http://www.ftc.gov/opa/2013/09/trendnet.shtm
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Is TRENDnet worse than the rest?
This is really something every single software vendor should do – but definitely don’t!
Is TRENDnet really that much worse than other embedded device vendors?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
TRENDnet Product Vulnerabilities
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
D-Link Product Vulnerabilities
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
D-Link User-Agent Backdoor
Source: http://www.devttys0.com/2013/10/reverse-engineering-a-d-link-backdoor/
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Is Legislation The Answer?
Do we need legislation?
USA apparently has FTC
What do we have in Romania or EU?
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Software Will Always Have Vulns?
Vendors claim that they provide software ”as-is” and have long EULAs to exempt them from liability
We seemingly accept that software will always have vulns...
... but the types of vulnerabilities matter as well as how the vendor proactively reduces risk and reactively deals with
them.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Conclusion
Of all the areas, vulnerability coordination/handling is the biggest improvement and continuing in the right direction.
Advisory quality overall seems static with some vendors improving and others devolving.
Only a few major vendors really seem to have solid SDLs and can show an improvement in code quality.
People are beginning to understand metrics better, and we’re seeing attempts at providing more granularity.
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
The Good News: There is Room for Improvement
N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y
Discussion!