Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

40
Splunk Worldwide Users’ Conference The Palace Hotel, San Francisco, CA August 9-11, 2010 Best Practices for Federal Compliance Dan O’Donnell, CISSO

description

Best Practices for Federal Compliance: How to use Splunk as an audit log aggregator and audit viewer to achieve compliance with Federal auditing requirements for information security under NISPOM Chapter 8 and ICD. Originally presented at Splunk User Conference, August 2010.

Transcript of Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

Page 1: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

Splunk Worldwide Users’ ConferenceThe Palace Hotel, San Francisco, CAAugust 9-11, 2010

Best Practices for Federal Compliance

Dan O’Donnell, CISSO

Page 2: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference2

About this talkUsing Splunk to satisfy U.S. Gov’t NISPOM auditing reqs. (and maybe a little DCID/CNSS/ICD)

Audit multiple events, across multiple platformsGoal 1: Show how one organization uses Splunk for auditing.Goal 2: Start a dialog - maybe a community.Non-goal: to be an expert source

Page 3: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference3

About the SpeakerDan O’Donnell, CISSO

- 8 yrs RAND; 5 yrs NBC- IANAP (Fortran 77)

Who/what is RAND.org?- FFRDC; non-partisan think tank on public policy: health, education, mil, etc.- many PhD scientists, engineers, social scientists, economists

Splunk users- Between 3-10, depending…

http://www.rand.org/
Page 4: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference4

What Are NISPOM, DCID?NISPOM = National Industrial Security Program Ops Manual

- Chapter 8: computers and networks- Chapter 8-602: what we care about for auditing with Splunk- ISFO: Industrial Security Field Operations Manual

DCID 6/3, being replaced by ICD- Equivalent (sort of) for military and IC shops

Page 5: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference5

Ch.8 Significant RequirementsCh.8-602 mandates several things, but we’ll only discuss…

‣ auditing of specific logs and trails‣ PL-1: 1 of 5, with 1 being lowest

‣ISLs are more specific

Auditing monitors computers for intrusive patterns and behavior.

Page 6: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference6

DSS: Defense Security ServiceNISPOM: National Industrial Security Program Ops ManualISL: Industrial Security LettersSRO: Security Relevant ObjectsISSO/ISSM: Information Systems Security Officer/MgrISFO: Industrial Security Field OperationsDISA: Defense Information Systems AgencySTIG: Security Technical Implementation GuideICD (DCID): Intelligence Community Directive

Acronyms

Page 7: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference7

Time efficiency!Audit frequency (mandated) = weeklyTime Requirements

1 machine ~ :10 minutes100 machines ~ 1,000 minutes ~ 2 days per week

Conclusion: Auditing does not scale.Also: log aggregator “remembers” the search strings.

Why Use a Log Aggregator

Page 8: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference8

Labor efficiency = $ efficiencySystem Comparison:

We looked at 6 systems, including making our own.Splunk did all four OS platforms – no other commercial product did this.Splunk: superior to what we could do on our own, and less costly.Splunk: modifiable, and on our own hardware.

Nominally approved by DSS ODAA (v3.4), summer 2009YMMV – check with your ODAA or equivalent

Why Use Splunk as Log Aggregator?

Page 9: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference9

RecommendationsRichard Bejtlich: network security, awareness, and APT

- “Federal security is the most frustrating…”- “Splunk is really awesome…” and- “Splunk is remarkably cheap for an enterprise app…”- YouTube: BSDconferences talk, April 21, 2009- http://taosecurity.blogspot.com/

http://www.youtube.com/watch?v=UM4ZrsOjmNQ,
http://taosecurity.blogspot.com/
Page 10: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference10

Intro to NISPOM (1)NISPOM Chapter 8, Section 602 a, b, c, d

- ISL 2007-01,

Data, metadata to capture- 2 general categories

Identification and Authentication (I&A)Security Relevant Objects (SRO)

‣ Prohibited file or directory activity

Page 11: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference11

Platforms

Page 12: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference12

Intro to NISPOM (2): Info to CaptureDate/Time stampUser or agentResources involvedAction involved

Page 13: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference13

Intro to NISPOM (3): I&AI&A: Identification and authentication

Login success

Logout success

Login attempts that fail – bad username or password

Login attempts to lockout – 5 attempts within :15min

Account lockouts

Password changes

User authentication changes: sudo, su, admin

Page 14: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference14

Intro to NISPOM (4): SROSRO: Security Relevant Objects

- Windows types; *nix types

OS executablesOS config filesSystem management and maintenance executablesAudit system and dataSecurity-related software

Page 15: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference15

ISL #44

Page 16: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference16

ISL #45

Page 17: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference17

Windows Event IDsUltimate Windows Security

- Windows: simplest and easiest- This list isn’t all, but most.- This is “XP family”.- Win7 is totally different.

http://www.ultimatewindowssecurity.com/securitylog/encyclopedia/default.aspx
Page 18: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference18

In general:/bin /usr/bin /sbin /usr/sbin

Audit systems – BSM; Snare/var/audit /etc/security ; syslog

Avware (required)Disk utilities“Lemme es’plain… No, it is too difficult.”

“Let me sum up…”

SROs in *nix

Page 19: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference19

Auditable Events & Objects: SummarySummary of: WinXP, Server 2003, *nix, OSX, *BSD

Page 20: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference20

WindowsSystem EventsSecurity EventsApplication Events (coming)

LinuxSyslogSeclogSnare log (merged into syslog)

Streams

SolarisSyslogBSM (converted to text)

OSX, TrustedBSD, FreeBSD

SyslogSeclogBSM (converted to text)

http://sourceforge.net/projects/snare/
Page 21: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference21

Streams to Capture in Splunk

Page 22: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference22

Windows: easy to config, easy to interpret.Configure file size for maximum.Configure persistence for “long”

Linux: moderately hard to config, fairly hard to interpretSyslogSnare – use the IA “one button config for NISPOM” or make your own.Check with your ISSP or DSS Rep., or DSS Academy.

Configuring Streams: Win, linux

Page 23: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference23

Solaris: moderately hard to config; hard to interpretSyslogBSM (converted)

OS X: hard to config; hard to interpretSyslogSeclogOpenBSM (converted to text)

*BSD: moderately hard to config; hard to interpretBSM part is the same in OS X, sort of

Configuring Streams: Unix, OSX, BSD

Page 24: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference24

Windows XP to Vista, including ServerAbout 14 of 114 events are audited: success and fail

Configure Events filesIncrease size (default 2MB -> 600MB), increase persistence

Snare can be usedActive Directory (AD) spews log entries.

Filtering with clever Splunk search strings can improve SNR.

Potential Problem: Active Directory and unix, linux, OSX

Case Analysis: Windows

Page 25: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference25

Syslog: raw syslog is straightforward.Seclog: raw seclog is straightforward.Snare:

Freeware, from Intersect AllianceNISPOM config can be 1-touch, but “roll your own” may be better.Output is text, merges into syslog.Output is text strings, searchable with Splunk.Problem: interpretation of outputSplunk lookup tables as a solution?

Case Analysis: Linux

http://www.intersectalliance.com/
http://www.intersectalliance.com/snareserver/regulations/NISPOM.html
Page 26: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference26

Snare necessary for linuxHooks into auditd.Recommended by DSS Academy.

Snare can be used with Windows.Increases detail and complexity.

Not yet ready for Win7.Snare can be used with Solaris.

Data and detail equivalent to BSM.Complexity slightly reduced since Snare outputs to text.Complexity increase as minimal Snare docs don’t include output interp.

Case analysis: Snare

Page 27: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference27

Snare config

Page 28: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference28

Syslog is easy; BSM is hard.About BSM:

Flags: lo,ad,-fr,-fw,-fc,-fd,-fm,-clLog file is binary: Splunk can’t handle it. Rotate regularly, export to text.

DocumentationSun docs, man pagesHal Pomeranz SysAdmin magazine article

Sun BSM similar to BSM on OSX, FreeBSD, TrustedBSD.Snare works on Solaris

Case analysis: Solaris

Page 29: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference29

Syslog, seclog are easy.BSM (harder than Solaris):

Same flags: lo,ad,-fr,-fw,-fc,-fd,-fm,-clRequires script to rotate BSM binary, export to text.Rotation frequency; retention period (1 yr.)

ParseAuditLog (PAL) scriptDiffs between OSX 10.6 and earlier

10.6.x has OpenBSM v1.1, with more functionality than earlier 1.0.

Case Analysis: OS X

Page 30: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference30

Small but powerfulGenerates big binary files, but very compressible

Very configurable, including “make your own masks”

~380 total events (x4); <50% are audited; (x 1/4)Diffs between Solaris, OSX, *BSDHistory of OpenBSMInterpretation of output (BSM output is Splunk input)

20100731123015.not_terminated; 20100731123015.20100731133015

Case Analysis: BSM

Page 31: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference31

BSM Audit Classes

Page 32: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference32

BSM Configs

OpenBSM v1.1etc/security/audit_control

Page 33: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference33

BSM OutputLookup tables candidate???

Page 34: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference34

Splunk and (most): routers, firewalls, switches, ips/idsMandated to audit these too.NISPOM and DSS don’t tell us what or how to audit.

Case Analysis: network appliances

Page 35: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference35

WindowsSys.Evt, Sec.Evt

LinuxSyslog, SeclogSnare audit log

SolarisSyslogBSM audit log (converted)

Splunk Inputs (redux)

OSX, FreeBSDSyslog, SeclogBSM audit log (converted)

Network appliancesLogs from firewalls, ips/ids

Active Directory logs(lots of kruft to filter)

Page 36: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference36

Auditable Objects | Splunk inputs

Page 37: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference37

Overall Audit TablePL1 Login fail

Login success

Login fail to

lockout

Account lockout

Logout success

Password change attempt

SRO access

fail

Windows ✔ ✔ ✔ ✔ ✔ ✔ ✔

OS X ✔ ✔ ✔ ✔ ✔ ✔ ✔

Linux ✔ ✔ ✔ ✔ ✔ ✔ ✔

Unix ✔ ✔ ✔ ✔ ✔ ✔ ✔

Page 38: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference38

Active Directory, *nix, and UID/GID conflictsUnique search string (saved search) for 100s of events???Lookup tables to convert Snare and BSMActive Directory and SNR (Signal to Noise Ratio)

AD spews a large volume of data – filtering requires knowledge and finesseAD and duplicate records

Issues or Potential Problems

Page 39: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference39

RemainingOutputsInterpretation to actual intelligenceMetricsOther Splunk capabilitiesRe-architect?

Page 40: Federal Compliance with Splunk on Hetergeneous Networks (Splunk User Conference 2010)

© Copyright Splunk 2010Splunk Worldwide Users’ Conference40

Questions?


Splunk conf2014 - Splunk for Data Science

Splunk conf2014 - Splunk for Data Science

Agency Chargeback Models to Enable Enterprise Splunk ......Workshops: Get Splunk Hands -on Experience Attend a Splunk Workshop Upcoming Schedule December 1: Introduction to Splunk

Agency Chargeback Models to Enable Enterprise Splunk ......Workshops: Get Splunk Hands -on Experience Attend a Splunk Workshop Upcoming Schedule December 1: Introduction to Splunk

Splunk Developer & Admin Certification Training...This Splunk course also includes various topics of Splunk, such as installation and configuration, Splunk Syslog, Syslog Server, log

Splunk Developer & Admin Certification Training...This Splunk course also includes various topics of Splunk, such as installation and configuration, Splunk Syslog, Syslog Server, log

Rivium Splunk Windows · o Splunk Enterprise Security * o uberAgent* o Splunk App for Web Analytics Common Splunk Apps & Add-ons 15 SplunkingWIndows o Splunk Add-on for Microsoft

Rivium Splunk Windows · o Splunk Enterprise Security * o uberAgent* o Splunk App for Web Analytics Common Splunk Apps & Add-ons 15 SplunkingWIndows o Splunk Add-on for Microsoft

Splunk conf2014 - Onboarding Data Into Splunk

Splunk conf2014 - Onboarding Data Into Splunk

Program Overview - Splunk...mission-critical services. None. Splunk Enterprise System Administration Splunk Enterprise Data Administration Splunk Cloud Administration Implementing

Program Overview - Splunk...mission-critical services. None. Splunk Enterprise System Administration Splunk Enterprise Data Administration Splunk Cloud Administration Implementing

Our experiences throughout the 3 year journey · Splunk – Hardware is now available for Splunk expansion • Splunk begins to fill monitoring gaps, acts as “glue” • Splunk

Our experiences throughout the 3 year journey · Splunk – Hardware is now available for Splunk expansion • Splunk begins to fill monitoring gaps, acts as “glue” • Splunk

SPLUNK Capabilities Overview - GTRI · Overview GTRI is the only ELITE Splunk integrator with certified Splunk architects and engineers on staff. In addition to ... SPLUNK-Overview

SPLUNK Capabilities Overview - GTRI · Overview GTRI is the only ELITE Splunk integrator with certified Splunk architects and engineers on staff. In addition to ... SPLUNK-Overview

Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Splunk Deployment

Splunk conf2014 - Splunk Monitoring - New Native Tools for Monitoring your Splunk Deployment

Splunk Inc. Splunk 4.1.7 Security Target - Common Criteria · 1.1 ST Reference ... 7.1.3.2 FDP_ACF.1 Access control functions ... ST Title: Splunk Inc. Splunk 4.1.7 Security Target

Splunk Inc. Splunk 4.1.7 Security Target - Common Criteria · 1.1 ST Reference ... 7.1.3.2 FDP_ACF.1 Access control functions ... ST Title: Splunk Inc. Splunk 4.1.7 Security Target

stoQ’ing your Splunk - SANS · PDF filestoQ’ingyour Splunk Ryan Kovar, Splunk Marcus LaFerrera, PUNCH SANS DFIR 2016

stoQ’ing your Splunk - SANS · PDF filestoQ’ingyour Splunk Ryan Kovar, Splunk Marcus LaFerrera, PUNCH SANS DFIR 2016

ReversingLabs Explainable Threat Intelligence Enriches ... Data Sheets/RL-Splunk...sent to Splunk. • In Splunk, the TitaniumScale report is correlated with other available Splunk

ReversingLabs Explainable Threat Intelligence Enriches ... Data Sheets/RL-Splunk...sent to Splunk. • In Splunk, the TitaniumScale report is correlated with other available Splunk

Industrial Data Internet of Things splunk (REST API, SDKs) splunk . TESCO splunk 500 Y—Jt1-3— 41328 . SHANON 1/5 splunk . htt . (500MB/B) s lunk splunk . Created Date:

Industrial Data Internet of Things splunk (REST API, SDKs) splunk . TESCO splunk 500 Y—Jt1-3— 41328 . SHANON 1/5 splunk . htt . (500MB/B) s lunk splunk . Created Date:

Security Investigation and Rapid Response Using Splunk and ... · Enter Splunk Splunk Enterprise and Splunk Cloud provide an analytics-driven approach to security on the AWS Cloud

Security Investigation and Rapid Response Using Splunk and ... · Enter Splunk Splunk Enterprise and Splunk Cloud provide an analytics-driven approach to security on the AWS Cloud

Splunk Cloud Product Briefwinncom.com.ua/wp-content/uploads/2018/07/splunk-Log_Flow.pdf · Extend Benefits With Splunk Apps Splunk Cloud includes support for premium solutions such

Splunk Cloud Product Briefwinncom.com.ua/wp-content/uploads/2018/07/splunk-Log_Flow.pdf · Extend Benefits With Splunk Apps Splunk Cloud includes support for premium solutions such

Islands*of*Splunk* - .conf2017 | The 8th Annual Splunk ... · Background*! Splunk*administrators*are*increasingly*required*to*provision*Splunk* as*aservice*offering*to*mulJple*customers*

Islands*of*Splunk* - .conf2017 | The 8th Annual Splunk ... · Background*! Splunk*administrators*are*increasingly*required*to*provision*Splunk* as*aservice*offering*to*mulJple*customers*

Splunk Conf2010: Corporate Express presents Splunk with SAP

Splunk Conf2010: Corporate Express presents Splunk with SAP

Splunk Detect Employee Fraud using - UnderDefense · 2018-10-25 · Splunk, Splunk DB Connect, Oracle DB, Splunk CIM Key Benefits Understand employee and entity behavior—and its

Splunk Detect Employee Fraud using - UnderDefense · 2018-10-25 · Splunk, Splunk DB Connect, Oracle DB, Splunk CIM Key Benefits Understand employee and entity behavior—and its

Splunk Validated Architectures · Splunk Engineers Responsible for managing the Splunk lifecycle. Managed Service Providers Entities that deploy and run Splunk as a service for customers.

Splunk Validated Architectures · Splunk Engineers Responsible for managing the Splunk lifecycle. Managed Service Providers Entities that deploy and run Splunk as a service for customers.

Splunk in Yoox: Security and Compliance

Splunk in Yoox: Security and Compliance