DefCamp 2013 - Are we there yet?

10 Years Later:Are We There Yet?

Carsten Eiram Risk Based Security@CarstenEiram

N O T J U S T S E C U R I T Y , T H E R I G H T S E C U R I T Y

Quick Bio – VDB Work Experience

Involved with VDBs for 10+ years

• Currently, CRO at Risk Based Security – commercial arm of Open Security Foundation (runs OSVDB and DatalossDB) – andresponsible for the VulnDB service.

• Chief Security Specialist at Secunia, running the Research team.

• Security Team Lead at Danish Verisign affiliate, running acustomer-only accessible vulnerability database.


Quick Bio – Vulnerability Research

Officially been doing vulnerability research since 2003

• Focused on a static analysis / reverse engineering approach

• Jokingly refer to myself as a "vulnerability connoisseur" - I enjoy analyzing vulnerabilities and their root causes.

• Critical vulnerabilities discovered in products from many major software vendors.

INTRODUCTIONWhat will be discussed?


Reason for Talk

After 10+ years of VDB work,I felt it was time to reflecton certain areas related to

vulnerabilities


Considerations


Metrics and their Usage


Code Quality


Advisory Quality

VENDORS MAKE BAD DECISIONS


Vulnerability Handling / Bug Bounties


Million Dollar (or Leu) Question


Quick Show of Hands

Vulnerability Statistics

A Quick Overview To Set The Stage


Vulnerabilities have been around for a very long time- And will continue to be...

• Oldest entries in OSVDB are 79399 and 79400

• Marconi wireless telegraph

• Dated November 1902

• Message spoofing and message disclosure

Currently Oldest Recorded Vulnerabilities


“I can tune my instruments so that no other instrument that is not similarly tuned can tap

my messages,” Marconi boasted to London's St James Gazette in February 1903

Guglielmo Marconi

http://www.newscientist.com/article/mg21228440.700-dotdashdiss-the-gentleman-hackers-1903-lulz.html


100 years ahead of Oracle with an “Unbreakable” claim!

Unforuntately, it ended just as badly...

First Ever Unbreakable Claim!



RATS!

”There was a young fellow of Italy,who diddled the public quite prettily,”

Nevil Maskelyne Ruins Demo



While not providing the privacy and

security as promised, the wireless telegraph still had one significant

advantage over the wired telegraph:

Not possible to cut the wires!

No Wire-Cutting Please


Obviously, we have progressed a fair bit technically since then, but have we gotten

significantly better?

Have We Improved?


Bringing The Internet Down – Old Lady Style

Article: http://news.softpedia.com/news/Old-Lady-Cuts-Off-Internet-in-Armenia-193640.shtml


10 Year Vulnerability Trend

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130

2000

4000

6000

8000

10000

12000

# Vulns


All Datasets Are Incomplete!

All datasets are incomplete - some just more than others

Many love taking CVE content that’s free and do random conclusions based on it, but since the dataset is severely

lacking, the conclusions are as well


2006 – 2013 Vulnerability Type Trend


2012 Data Breaches due to SQL Injection


Companies affected by XSS in 2012

Source: CWN - http://www.cyberwarnews.info/2012/07/04/300000-personal-details-leaked-38-sites-hacked-for-projectdragonfly/

http://www.cyberwarnews.info/2012/07/04/300000-personal-details-leaked-38-sites-hacked-for-projectdragonfly/


Companies Impacted By Hacking In 2012

Vulnerability Metrics

Usage


Which is more secure?

Product A10 Vulnerabilities

Product B20 Vulnerabilities


Security State != Number of Vulnerabilities

Previously, the security state of a product was consideredto be equal to the number of vulnerabilities.

Flawed conclusion!

Today, people understand that the number of vulnerabilities !=

security state


Some Apparently Still Don’t Know...

“The problem with Java is that a lot of vulnerabilities are constantly being reported in it, and when a lot of

vulnerabilities are reported, then there are a lot of hackers using these to access programs built on Java“

- Morten Stengaard, CTO, Secunia

http://www.dr.dk/tv/se/tv-avisen/tv-avisen-827#!/


Dissecting the Statement – Part 1

”... then there are a lot of hackers using these to access programs built on Java”

Most vulnerabilities in Java are not used to target Java applications, but the Java Runtime Environment to

compromise the system.




”... when a lot of vulnerabilities are reported, then there are a lot of hackers using these…”

Just because a lot of vulnerabilities are reported in a product, a lot of hackers may not be exploiting them.




”The problem with Java is that a lot of vulnerabilities are constantly being reported in it…”

The security state of a product is not defined by the number of vulnerabilities reported in it.



We Should All Stop Using Popular Software Then

Vulnerabilities (2013 - Nov 10th)0

50

100

150

200

250

300

350

400

JavaChromeFirefoxInternet Explorer


Facewall!


Microsoft Argument For SDL (Windows)


Microsoft Argument For SDL (SQL Server)


Microsoft Office Vulnerability Trend

Office 2000 Office 2007 Office 20100

2

4

6

8

10

12

14

Vulnerabilities in Office versions one year after product release(based on Microsoft security bulletins)


Microsoft Security Bulletin Trend

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130

50

100

150

200

250

300

350

BulletinsCVEs


Researcher Focus and SCADA


Stop Drawing Conclusions on Vulnerability Counts...


There are so many other aspects to consider!

More things to consider incl.

Patched vs. UnpatchedVulnerability Type

ImpactTime-To-Patch

Time-To-Vendor-ResponseSecurity Mechanisms

...


Severity


Severity Metrics

Many different severity metrics – both public and internal

Most popular and hated is CVSS,which currently has problems reflecting real risk

Many concerns raised about CVSSv2 by many peoplee.g. myself and Brian Martin of OSVDB in our open letter:

"The CVSSv2 Shortcomings, Faults, and Failures Formulation"

http://www.riskbasedsecurity.com/reports/CVSS-ShortcomingsFaultsandFailures.pdf


Limitations of Severity Metrics

Reflecting the threat of vulnerability-dependent issues (e.g. sandbox bypass, ASLR bypass related to memory

disclosure etc.)

By themselves and from a scoring point-of-view, these issues are pretty minor, but when combined with code execution...

Jackpot!

Ability to disclose a few memory addresses was in the past pretty much a non-issue – today it’s very useful.


Pick A Vuln... Any Vuln...

If I’d offer you one vulnerability in e.g. Google Chrome, which

would you pick?

1) Code execution within sandbox

2) Sandbox bypass

CVSSv2: 6.8

CVSSv2: 2.6


Severity Metrics and Sandbox Bypasses

If we conclude that exploiters are more interested in the sandbox bypass and system administrators should focus on

fixing such a vulnerability over a code execution vulnerability within the sandbox, why are we not rating

them higher?

Case of reality not being reflected well by severity metrics


Severity Metrics and Vulnerability Chains

And once these issues start occuring in chains, which is becoming more and more common, then it really gets

complex...

You can have a lot of independent minor issues that when combined suddenly are very serious


Google Chrome Pwn2Own Example

OSVDB 89734

Plugin blocking logic not run

for NaCl in pre-rendering

http://blog.chromium.org/2012/05/tale-of-two-pwnies-part-1.html

OSVDB 80007

GPU command decodinginteger underflow

OSVDB 81645

IPC channel missing listener process validation

OSVDB 80741

Unprivileged renderer can navigate to privileged URLs

OSVDB 89736

Too permissive LoadExtension

bindings for extension manager

OSVDB 80293

Unpacked NPAPI extension installation without

confirmation

CVSSv2: 6.8

CVSSv2: 5.1 CVSSv2:

5.1

CVSSv2: 2.6CVSSv2:

2.6CVSSv2:

7.6 CVSSv2: 9.3


When Severity Metrics Met Reality

Severity Metrics only reflecta worst-case impact


Exploitability


Microsoft Severity Ratings

Source: http://technet.microsoft.com/en-us/security/gg309177.aspx


Exploitability Index Ratings


Microsoft Approach: Pros and Cons

Pros ConsGives an realistic evaluation of the technical requirements to exploit a given vulnerability and how feasible it is

Requires significant technical skills and resources to get right

Makes it clear which are theoretical and which are plausible

Still requires a bit of guesstimation


No Granularity Really Added...


How Does Adobe Do It?


Adobe Approach: Pros and Cons

...Pros ConsAllows understanding which products, versions, and architectures are most critical to prioritize

Does not factor in technical requirements and the nature of the vulnerability i.e. does not differentiate between theoretical issues and straight-forward issues to exploit

Dynamic approach that can be easily tweaked

Requires very little resources – just an understanding of historical exploitation


How Does CVSSv2 Do It?


CVSSv2 Approach: Pros and Cons

Pros ConsMost reliable of all the approaches: If an exploit is available, a vulnerability is clearly exploitable.

Purely reactive, requiring very fast response times

Requires very little resources – just knowledge of availability of PoCs and exploits

Only takes into account when the availability of an exploit is publicly known i.e. may be exploited long before being flagged as such


No information about code quality

All these approaches are interesting and add an extra

dimension – especially if combined

None of these scores tell us anything about the underlying

code quality, though

Code Quality... And How To Measure It


Code Quality – Why Measure It?


Code Maturity Metric – The Idea

The idea of code maturity is that by evaluating the prevalence of the different vulnerability classes being discovered in a product, we can conclude the maturity of

that product.

We, naturally, focus on it from a security perspective.


Code Maturity Metric – Scoring

• Each vulnerability can be scored based on type, and how easy it is to discover.

• Researchers find simple vulnerabilities first - as simple vulnerabilities are eliminated, researchers move on to finding more complex vulnerabilities.

• When a vendor secures the code, basic vulnerabilities are easier to spot and remedy or never introduce compared to more complex vulnerabilities.


Code Maturity Metric – Scoring Example

Level Vulnerability Classes

0 Classic buffer overflows due to e.g. strcpy, sprintf, sscanf and format string issues.

1 Buffer overflows due to incorrect size being used e.g. strncpy, memcpy and array-indexing issues

2 Arithmetic errors i.e. Integer overflows/underflows, type conversion, signedness.

3 Uninitialized variable, use-after-free, bad cast, complex logic errors.


Schneider Modbus Serial Driver Buffer Overflow

Source: http://www.riskbasedsecurity.com/research/RBS-2013-003.pdf



Code Maturity Level: 1


ActiveX Control Vulnerability

Code Maturity Level: 3


Office 2000: 62

Office XP: 103

Office 2003: 90

Office 2007: 47

Office 2010: 14

Office Vulnerabilities Analysed


Office Product Code Maturity Scores

Office 2000

Office XP

Office 2003

Office 2007

Office 2010

0 0.5 1 1.5 2 2.5 3

Code Maturity


Office Vulnerability Type Prevalence

Office 2000

Office XP

Office 2003

Office 2007

Office 2010

0% 5% 10% 15% 20% 25% 30% 35%

Uninitialised VariableObject Type ConfusionUse-after-freeArithmeticArray IndexingIncorrect Size CopyClassic Buffer Overflow


Measuring the Efforts Taken By Vendors

With this we can put more focus on the code security improvement efforts taken by vendors by being able to

measure them.

Allows system administrators to know which software to steer clear from... and researcher to understand which types of vulnerabilities they can expect to find in a given product.

Advisory QualityOr Lack Thereof...


Information Needs To Be Publicly Available

Most vendors have also acknowledged that publishing vulnerability information is beneficial

Juniper recently joined the party

Still some black sheep like SAP, trying to keep it a secret…


Needs To Include Vulnerability Type

Either clearly descripting the vulnerability type in the advisory description

or

alternatively including CWEs


Everything Is Memory Corruption These Days


Microsoft MS12-037 vs MS13-080

----


Rise In Usage Of Memory Corruption Term


No requirements to include proper info

Various standards and formats e.g. CVRF are being proposed, but these deal with required fields – not the content of these.

Primary focus is to ensure a structure that is easy to parse in an automated manner.

Completely up to the vendors how much information they feel like sharing. Up to customers to raise their voice, if they

want/need more.

Vulnerability Handling... And Bug Bounties


Bug Bounties

When I started reporting vulnerabilities to vendors, I was stoked each time I actually got a response - and it wasn't a

threat from a lawyer.

Had any of you told me back then that vendors today would be offering bug bounties, I'd have smiled and shook my head.


Bug Bounties

A few interesting ones are of course Google's bounty, which is one of the more serious vendor bounties, and especially their

latest twist: Bounties for other software!

Microsoft's bounty for vulnerabilities, but specifically bypassing security mechanisms is very interesting


Shockwave Player Vulnerability Trend

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130

10

20

30

40

50

60

70

80

90


Bug Bounties

There has definitely been a shift in how vendors perceive bug bounties.

It’s clear to me that if a vendor wants to encourage researchers to look at their code and report it in a

coordinated manner, then bug bounties are very effective when done right.

ConclusionAre We There Yet?


Security Software and Shiny Appliances

More security software and appliances will fix everything!


Everything Is Vulnerable – Even Security Software!

About 2.2% of all entries in OSVDB cover vulnerabilities in

security software


The Security Software Paradox

Reducing attack surface by adding an even greaterattack surface is a paradox


Code Quality Improvements(?)

Microsoft, Google, and Adobe are examples of vendors noticeably improving their security efforts.

Oracle may be on their way after everyone finally realized that Java is a mess...


How Do We Force Vendors To Improve?

Vendors don’t make more secure software because they want to –

it’s because they HAVE to!


Grand Demonstrations!

We needgrand demonstrations

that ordinary people can relate to!


FTC vs. TRENDnet

After demonstrating how network cameras were easily publicly accessible and e.g. allowing spying on people in their homes, the FTC (Federal Trade Commision) in USA went after

TRENDnet.

Eventually agreed that TRENDnet was ”prohibited from misrepresenting the security of its cameras”, will establish a comprehensive IS program, and hire outside consulting to

review security every two years for 20 years...

http://www.ftc.gov/opa/2013/09/trendnet.shtm


Is TRENDnet worse than the rest?

This is really something every single software vendor should do – but definitely don’t!

Is TRENDnet really that much worse than other embedded device vendors?


TRENDnet Product Vulnerabilities


D-Link Product Vulnerabilities


D-Link User-Agent Backdoor

Source: http://www.devttys0.com/2013/10/reverse-engineering-a-d-link-backdoor/


Is Legislation The Answer?

Do we need legislation?

USA apparently has FTC

What do we have in Romania or EU?


Software Will Always Have Vulns?

Vendors claim that they provide software ”as-is” and have long EULAs to exempt them from liability

We seemingly accept that software will always have vulns...

... but the types of vulnerabilities matter as well as how the vendor proactively reduces risk and reactively deals with

them.


Conclusion

Of all the areas, vulnerability coordination/handling is the biggest improvement and continuing in the right direction.

Advisory quality overall seems static with some vendors improving and others devolving.

Only a few major vendors really seem to have solid SDLs and can show an improvement in code quality.

People are beginning to understand metrics better, and we’re seeing attempts at providing more granularity.


The Good News: There is Room for Improvement


Discussion!

DefCamp 2013 - Are we there yet?

Technology

Transcript of DefCamp 2013 - Are we there yet?