aleph - Malware analysis pipelining for the masses

alephMalware analysis pipe-lining for the

masses

Who we are?

Jan Seidl @jseidl

Aleph Project Lead Developer

*NIX/BSD freak

Digital tools blacksmith / python & C lover

Lousy guitar player

Coffee dependent

Hates printers, doesn't likes social networks anything

Selectively-social

Who we are?

Actually we are a bunch of people...

Malware

Definition

'Malware' is an umbrella term used to refer to a variety of

forms of hostile or intrusive software, including computer

viruses, worms, trojan horses, ransomware,

spyware, adware, scareware, and other malicious

programs.

It can take the form of executable code, scripts, active

content, and other software

Wikipedia

Malware growth

Detecting malware

Detecting malware● Signature-based

● Sample must be previously-known and flagged as malicious

● Heuristics-based● Can trigger loads of false-positives

● Behavior-based● Can trigger loads of false-positives

Detection is not enoughYou need to understand the malware

Understanding malware

● Features extraction

● Which characteristics this file has?

● Feature correlation

● Make sense of features combinations / disposition

● Sample correlation & Family classification

● Identify common features between different samples

Understanding malware

● Enables you to identify families

● Enables you to identify acting groups

● Enables you to identify techniques

● Enables you to identify trends

Manual approachEveryday workhorse method

Manual approach

● Use lots of separate tools to extract data from sample (each in its own format)

● Correlate output from the tools using spreadsheets, word files, napkins, tears

Manual approach

● Find out new samples embedded into original sample

● Rinse, repeat, get more whiskey/coffee

Automated approachPipeline method

Automated approach

● Insert sample into one end● Wait until processing is done● Get report on the other end● Get emotional about hours of work saved● Focus on most important evidences

Commercial playersGiving you some free analysis while you

feed their database for free

● Akana (Android files)● Anubis● BitBlaze Malware Analysis Service● Comodo Automated Analysis System

and Valkyrie● EUREKA Malware Analysis Internet

Service● Joe Sandbox Document Analyzer (PDF,

RTF and MS Office files)● Malwr● MASTIFF● VxStream Sandbox (Hybrid Analysis)● VirusTotal*● ThreatExpert● ThreatTrack● ViCheck● VisualThreat (Android files)● XecScan (PDF and MS Office files from

targeted attacks)

Bringing it to the massesGiving you the ability to spin up your

own commercial-grade malware research infrastructure

Components

Aleph ProcessFrom fetching samples to report building

Main Features

● Cross-platform (tested on: Windows, Linux, OS X)Almost all modules are pure-python

● Scalable + Easily Extensible

● Web Interface for browsing reports

From fetching samples to report building

Aleph Process

Sample

Sample data

Aleph Process

Sample Sample data

Aleph Process

Collector Sample Manager

Sample Queue

Sample Plugins

Data

Aleph Process: Collection

●Detect new file on medium (filesystem, email account etc)

●Check if meets predefined criteria (min/max size)

Aleph Process: Triage

●Detect file type (mimetype)

●Calculate hashes

●Add sample to process queue

Aleph Process: Processing

●Enumerate plugins suitable for sample mimetype

●Run plugins and extract features

●Save features as structured data into database

Aleph Process: Reporting

●Fetch sample information from database

●Generate report based on retrieved data

Currently supported files●Windows Portable Executable (PE) (exe,

cpl & dll)

Coming up support for:● Android APK

● PDF Documents

● Linux ELF

● iOS Apps

● URLs & Emails

● Apple Mach-O

● MS Office Documents

● SWF & Much more!

Aleph: Web Interface

Shifting the paradigmNew exciting features coming up

DecouplingDividing the functionality into

standalone components

Decoupling

Collector

Sample

WebInterface

AlephProcessor

RDBMSSample

DatastoreStorage

StorageConnector

Web UI Info

HTTP API, SSH...

TransportConnector

Transport

Executable, Email, URL,PDF, APK etc

Local File, FTP, Email etc Redis, RabbitMQ,Memcached

SQLite, MySQL,PostgreSQL, SQL Server

Filesystem, VXVault, Amazon S3, Azure etc

ElasticSearch

Sample + Initial Meta


Send & Retrieve

Download


Adapters & ConnectorsAbstracting functionality so you use

what you have, or what you're familiar with...

Adapters & Connectors

CollectorManager

Sample File Collector

E-Mail Collector

FTP Collector

Sample

Sample

Transport

TransportConnector

Redis, RabbitMQ,Memcached

Scoring SystemThe path to the Evil Index

Data Correlation PluginsAutomatic linking and enrichment of

gathered data

Data Visualization PluginsBecause everyone loves graphs and scatterplots

Online DisassemblyDisassembled code & memory dumps will be saved as a new sample/artifact

Tons of more cool plugins

●VxCage●Viper● Java Decompiler●Maltrieve

CollaborationWhat got us here in the first place

Collaboration

The power of many

Deployment TypesFrom small to huge scaling

Deployment Types

Deployed in a single host containing all the required services.

3rd Party Software Aleph Components

Redis Local Filesystem Elasticsearch SQLite

Collector Processor Web Interface

Deployment Types

Deployed across multiple hosts in order to achieve HP and HA.

Datastore Host GroupElasticsearch Cluster

Nodes

Transport Host GroupRabbitMQ Cluster Nodes

Processing Host GroupAleph Cluster Nodes

Web Interface Host Group

NGinx Cluster Nodes

Collection Host GroupAleph Cluster Nodes

Storage Host GroupDFS Cluster Nodes

Use, fork and contribute!

https://github.com/trendmicro/Aleph

Questions?We can try to answer some….

Thank you!

Jan Seidl @jseidl <[email protected]>

Slides: http://slideshare.net/jseidl Codes: https://github.com/jseidl

mailto:[email protected]

http://slideshare.net/jseidl

https://github.com/jseidl

aleph - Malware analysis pipelining for the masses

Technology

Transcript of aleph - Malware analysis pipelining for the masses