Is code review the solution?

Is Code Review the Solution?

Versão 1.1 - 28/10/2014

Confraria da Segurança da Informação

SAPO Websecurity Team

Outline

• What is code review • Mo9va9on • Open-‐Source • How to • Tools • Problems

SAPO Websecurity TeamSAPO Websecurity Team

About me

• Security Engineer at Portugal Telecom since 2004

– honeypots, traffic analysis, internal security

• At SAPO since 2010

– pentes9ng of web applica9ons, iOS, Android, IPTV

– all terrain security consultant

• Trainer of Linux and network security courses at Citeforma

• Speaker at security events like Codebits, Just4Mee9ng, Security Mee9ng, ISEL Tech, Create Tech, Confraria da Segurança da Informação and BSides Lisbon

• Holds a MSc in Informa9on Technology/Informa9on Security from Carnegie Mellon and CISSP

What is code review

• Code • Firefox -‐ 5 millions LOC (Lines of Code) • MySQL -‐ 12 millions LOC • Debian 5 -‐ 66 millions LOC • Windows Server 2003 -‐ 50 millions LOC

What is code review

• Review

• “formal assessment of something with the inten9on of ins9tu9ng change if necessary”

What is code review

• Code review is the analysis of source code in order to find defects – security, performance, func9onal, etc. –early detec9on – complements scanners and other tools

Motivation

• Compliance

• PCI-‐DSS -‐ Payment Card Industry Data Security Standard • since 2005 • version 3.0

• Requirement 6.3.2 • “Review custom code prior to release to produc9on or customers in order to iden9fy any poten9al coding vulnerability (using either manual or automated processes) … “

Open-Source

• A requirement for code review is to have access to the source code

• Open-‐Source Sobware (OSS) makes its source code available for anyone (to review)

• Therefore, OSS is becer because its reviewed by the whole world • is it?

Open-Source

• Not all OSS is thoroughly reviewed, but…

Open-Source

• In 2011, a vulnerability that allowed backup decryp9on was found

Open-Source

• Found “by someone who was reading the Tarsnap source code purely of curiosity”

• Lead to a bug bounty for security problems • “I'm a great fan of curiosity, but I've also learned that money can help to encourage curiosity.“

Open-Source

• Apple “goto fail”

• CVE-‐2014-‐1266 -‐ “acacker with a privileged network posi9on may capture or modify data in sessions protected by SSL/TLS”

• Affected iOS and OS X

• hcp://pi5.20.sl.pt

Open-Source

• Likely found by code review

• “A test case could have caught this, but it's difficult because it's so deep into the handshake.”

• “Code review can be effec9ve against these sorts of bug.”

Open-Source

• In 2011, a Ph.D student pushed a commit to OpenSSL that implemented the Heartbeat extension

• Reviewed by one of OpenSSL’s four core developers • code in C • the problem was not detected

Open-Source

• Heartbleed

• CVE-‐2014-‐0160 -‐ Allows reading of random data from the process memory

• Affected OpenSSL -‐ used by many exposed services such as www and mail

• hcp://pi5.5l.sl.pt

Open-Source

• Should have been detected with code review

• hcp://pi5.fp.sl.pt

Open-Source

• SQL injec9on

Open-Source

• SQL injec9on

Open-Source

• SQL injec9on

• hcp://vuln.example/login?username=x’ or 1=1 limit 0,1-‐-‐%20

• SELECT id,group,full_name FROM users WHERE username=’x’ or 1=1 limit 0,1-‐-‐

How to

• Code review methods vary a lot • highly dependent on the depth of the analysis

• Broad categories with different names depending on the author • Formal code review • Lightweight code review

How to

• Formal code review • line by line • mul9ple reviewers • group review • printed copies

• Finds hard to find problems • Time consuming

How to

• Lightweight code review • shallow analysis • pacern based analysis • grep based

• reviewing only cri9cal func9ons

• Prone do miss some problems • Less 9me consuming • Good to easily find certain classes of vulnerabili9es

How to

• Review can be done • manually • automa9cally • using both approaches

• Using both approaches • automa9cally find hotspots with pacern matching • manually review those areas

How to

• Combining approaches • milestone • mandatory review and approval before going to produc9on

• a posteriori • detec9on vs preven9on

• sampling • review just some code, chosen by • keyword • commiter • project

How to

• Basic rules for code review to work

• 1st rule: the reviewer must not be the one who wrote the code • if we could find bugs in our code we would be able to avoid them

• biased analysis • the reviewer will have a different and unbiased perspec9ve

• the reviewer should be from a different project

How to

• 2nd rule: the reviewer should understand the language being reviewed

How to

• 3rd rule: focus on the objec9ve: security, performance, feature, etc., but not on everything

More motivation

• How to mo9vate the reviewers?

More motivation

• Just saying “you must do code review” will not work • developers have more interes9ng stuff to do • developers have more stuff to do • developers have deadlines and code review is easily not taken in considera9on (1st to drop)

• developers don’t like others code • what to review?

How to

• What to review is a big ques9on • don’t let the developer choose what to review arbitrarily

• Assign “reviews” to reviewers • use a tool to manage what is assigned to each reviewer

• each reviewer has a queue of reviews to be done

How to

• Assign “reviews” to reviewers • for instance, single commits

• Ensures • coverage -‐ all code is reviewed • responsibility -‐ the developer has something publicly assigned to him

• deliverables -‐ audit evidence; increases mo9va9on to review

How to

• Even with task assignment the reviewer might let the work pile up • its like documenta9on: the applica9on will work fine without it

• This will happen if the review is done individually and on their usual sirng place • gather developers

How to

• Book a mee9ng room • Get the developers there

• Suppor9ng sobware

• Phabricator • repository integra9on • assignment rules • issue tracking • pre and post commit hooking

• Phabricator

• Gerrit

• Gerrit • pre-‐commit only • Git only

• Phabricator • pre-‐commit • post-‐commit • Subversion, Git, Mercurial

• Security Lib – less code to review

• Watch Commits

• Do not confuse code review with other mechanisms • sta9c analysis • dynamic analysis

• These lack human intelligence • but do not get 9red

Problems

• A portuguese company working in mission-‐cri9cal systems used (uses?) the following approach • developers get a printed A4 page with code • they also get a 5/6 items checklist • 15 min mee9ng in the next morning to discuss the checklist issues

• repeat every day

• Scrum alike methodology

Problems

• Problems with this approach?

• Feels like homework • might review at work but subject to the usual constraints

• Context • calls to func9ons outside the printed code • classes/objects defined elsewhere • inclusion of files and configura9ons

Problems

• Limita9ons • variables, objects and func9ons define outside • configura9on dependent execu9on • scope limita9on