The power of development-driven security testingMarat Vyshegorodtsev System Security Office Rakuten, Inc. https://global.rakuten.com

About Rakuten

World coverageE-commerce in 14 countries and regions All services and businesses in 27 countries

2011200920082005 201320122010

INVESTMENT

2014

Bio & DisclaimerTechnical Program Manager “Group Core Services” at Rakuten

University of Tokyo graduate

Member of the world-famous CTF team “More Smoked Leet Chicken”

The Russian hacker of Japan :-)

!

I’m not a Java developer. In fact, I’m not a developer at all.

Security & Quality Assurance• Regular QA tests cover “intended” functionality

• Security QA tests try to find all other unintended behavior

Security & Quality Assurance• Regular QA tests cover “intended” functionality

• Security QA tests try to find all other unintended behavior

QA is hard, Security QA is harder

Main reasons why security tests are hard:

1. Big scope: number of methods times number of tests

2. Hard to hook in

3. Halting problem

QA is hard, Security QA is harder

Main reasons why security tests are hard:

1. Big scope: number of methods times number of tests

2. Hard to hook in

3. Halting problem

QA is hard, Security QA is harder

Main reasons why security tests are hard:

1. Big scope: number of methods times number of tests

2. Hard to hook in

3. Halting problem

QA is hard, Security QA is harder

Main reasons why security tests are hard:

1. Big scope: number of methods times number of tests

2. Hard to hook in

3. Halting problem

Halting problem in one slide

It is impossible to determine if program will halt or not on given inputs

Hence, it is impossible to perform all possible security tests

Give it up.

xkcd.com/1266

Problem 1: Traversing the codeGiven:A service that has 70,000+ lines of code, a build system, and some tests

Find:

• All classes and their methods that use unsafe or deprecated calls

• Classes that must implement certain methods, but didn’t

• Classes that call certain dangerous APIs to fuzz them later

Freud — a framework for writing static analysis tests

LMAX-Exchange/freud

Freud

• Enables iteration over source code and byte code files’ contents

• Supports custom hamcrest matchers to write rules

• Implements DSL-like syntax for writing tests with JUnit or Groovy

Unsafe and deprecated callsBan all direct input/output trough files:

@Test!public void noDirectFileInput() throws Exception {!!

Freud.iterateOver(ImportDeclaration.class).!!!assertThat(no(importDeclarationPathAsString(), containsString("java.io.File"))).analyse(listener);!!

}

Mandatory implementation

@RolesAllowed("Administrator")

public void setNewRate(int rate) {

...

}

Mandatory implementation

Freud.iterateOver(CodeBlock.class).          forEach(method(publicMethod())).              assertThat(hasDeclaredAnnotation(“RolesAllowed”))  !

.in(codeBlocksWithin(methodDeclarationsWithin(classDeclarationsWithin(javaSourceOf(asList(  //  list  of  class  files  URLs  here  )))))).analyse(listener);

Finding bad apples

Freud.iterateOver(CodeBlock.class).    forEach(method(hasMethodCall(“Session.createSQLQuery”)))  !

//  find  out  who  calls  unsafe  APIs  and  try  to  fuzz  it

Fuzzing

// this.function is a iterator for forEach!!

public T next(){! for(Fuzzer f = fuzzDB.createFuzzer("031-B16-HEX", 4); f.hasNext();) {! return function(f.next());! }!}

Problem 2. Going deep

Problem 2. Going deep

Problem 2: Going deepGiven

• Big application with full code coverage

• No security checks implemented

Find

• Certain method is called with a certain parameter

• Some parameters should never be passed to a method

Power of mocking with PowerMock

PowerMock is a custom class-loader and byte-code manipulator allowing to mock static methods

Extends Mockito and JUnit perfectly

Deprecating MD5Problem: MessageDigest.getInstance(“MD5”) must not be used

Solution: Let’s just grep for a string getInstance(“MD5”)!

!

But… remember the halting problem?

MessageDigest.getInstance(Config.getConfiguredHashAlgorithm())

↑ is it MD5?

@RunWith(PowerMockRunner.class)  @PrepareForTest({MyClass.class,  MessageDigest.class})  public  class  md5Test  extends  PowerMockTestCase  {  !

       @Test          public  void  testDoHash()  throws  Exception  {                  PowerMockito.mockStatic(MessageDigest.class);                  when(MessageDigest.getInstance("MD5")).thenReturn(null);                  PowerMockito.verifyStatic();  !

               assertEquals(“acbd18db4cc2f85cedef654fccc4a4d8",  MyClass.doHash("foo"));          }  !

}

PowerMock + Hamcrest

• Problem: See if Log function never accepts credit card number-looking strings, given that a developer wrote a test that triggers this behavior

• Solution: Mock Log class, when its functions are called, there is no argThat is a 16 digit string (as an easy example)

@RunWith(PowerMockRunner.class)  @PrepareForTest({MyClass.class,  Log.class})  public  class  logTest  extends  PowerMockTestCase  {  !

       @Test          public  void  testSomeFunction()  throws  Exception  {                  PowerMockito.mockStatic(Log.class);                  when(Log.i(new  IsCreditCard())).thenReturn(null);                  PowerMockito.verifyStatic();  !

             //  continue  test          }  !

}

class IsCreditCard extends ArgumentMatcher<String> {! public boolean matches(String message) {! return RegExp.matches(message,”[0-9]{16}”);! }! }

SummaryIn most of the languages running on VMs it is possible to test certain weaknesses using unit tests

Security engineers working together with TDD/BDD dev teams can write many business logic-aware tests easily using the frameworks I have described

BDD is for intended behavior, security-driven development is for unintended one

Thank you!

marat.vyshegorodtsev@mail.rakuten.com !

@MaratVy maratto.blogspot.com