Post on 16-Jul-2015
How Splunk is used at EDUs
2
About Me James Donn – Senior Sale Engineer
• ~ 3 Years working at Splunk • 5 Year Splunk customer
– 4.5 years at Harvard University – .5 years at MITRE
• Focus on Network and Systems Management
2
3
Agenda " What is Splunk? " Architectural Components Overview " Splunk Demo " Higher EducaPon Examples " How we do it in the Cloud today " Extra Demo
What is Splunk?
5
A PlaSorm For Machine Data
6
Powerful PlaSorm for Developers
7
Powerful Developer PlaSorm on Hadoop
Architectural Components
9
Overview
10
Ge[ng Data IN
• Any UDP Port – Syslog • Any TCP Port – WMI • Watching flies or directories – Logs • Scripted Inputs – API connecPvity into any App • Modular Inputs
-‐ DB Connect -‐ Stream -‐ Many more on haps://apps.splunk.com
11
Service FuncPons
12
Forwarders " Collects data from machines " Sends data to a Splunk indexer in Splunk format " Install onto the remote system for data ingesPon " Low impact -‐ basically reads in data for transmission " Full vs. Light vs. Universal?
13
" Low profile = forwarding only " Python/Splunkweb removed " Searching/Indexing removed " Deployment server removed " LWF (4.1 and earlier) ~ UF
Universal Forwarder
14
Indexers " Processes raw data and stores it onto disk " Input Processing
– Parsing (char set determinaPon, linebreaking) – Merging (line merging, Pme extracPon) – Typing (punctuaPon, anonymizaPon)
" Indexer Pipe – Write to disk (compressed) – Assigns 4 chunks of meta data
" Performs HEAVY liking for searches!
15
Search Heads " Spawns search process (splunkd-‐search)
– 1:1 raPo of search process to CPU core – Splunkweb communicates via REST API (haps)
16
Cluster Master " Required for Index ReplicaPon " Tells Indexes where to replicate to " Tells Search heads where the data is at " Search Affinity
17
Brief Summary " Forwarders: send data to the indexer for indexing " Indexers: heavy liking (index AND search) " Searchers: spawn the iniPal search – distribute as necessary " Cluster Master: for data replicaPon
Demo!
Higher EducaPon Examples
20
MRTG to Splunk
21
Track-‐A-‐Mac
" Used by Students via self service web portal " Police Department is alerted when MACs from stolen laptops appear on network
22
OperaPonal Tool
23
OperaPonal Tool
24
OperaPonal Tool
25
Data VisualizaPon
26
Data VisualizaPon
27
VPN usage
28
Business AnalyPcs
29
Business AnalyPcs
30
RegistraPon
How We Do It in the Cloud
32
Splunk Offerings in AWS
33
Splunk is flexible!
Extra Demo!
AddiPonal Info
36
AddiPonal Info " Answers.splunk.com " Apps.splunk.com " Dev.splunk.com " Free Download " Free Online Sandbox
Thank you!!
jim@splunk.com
38
University Use Cases " Use Case 1 – Student Harassment/Death Threat
– University reported they got a call from campus police that a female student reported she was ge[ng threaPng email and text messages from her ex-‐boy friend.
" Use Case 2 – VPN Abuse – A University on the east coast reported by using Splunk they were quickly able to
idenPfy when their VPN was being abused
" Use Case 3 – Direct Deposit Fraud – Hackers are targePng universiPes using malware telling people they need confirm/
change their Direct deposit informaPon. When someone does the hackers wait unPl right before payday and make a change. Once the funds are transferred to the new bank account the funds are gone. This was happening mulPple Pmes a month.
39
University Use Cases " Use Case 4 – Copyright – UniversiPes are using Splunk to find who is downloading/sharing illegal
content.
" Use Case 5 – Quickly idenPfy a Student in the area – A university using Splunk to help confirm if a student has been seen on
campus recently. They had a case were a parent contacted the school saying they hadn’t heard from their student in 2 weeks.
" Use Case 6 – View acPve wireless connecPons on campus – UniversiPes can plot where wireless connecPons are on a campus map. This
can help understand where the most students are or if the number of students in the area is normal.
40
University Use Cases " Use Case 7 – Track AD changes/Access – UniversiPes can see when someone is added to a group or given root access
and who granted the access.
" Use Case 8 – Student acPvity/cheaPng – UniversiPes can set alerts for various events including posPng homework to
mulPple accounts from a single IP address
" Use Case 9 – User account informaPon posted online/social media – One University scans social media for students who post their login on
credenPals out for the world to use.
41
University Use Cases " Use Case 10 – Find Fraud rings collecPng Financial Aid – Sign up for financial aid register collect a check never show for class.
" Use Case 11 – Stack rank Security Risk by department – Whose keep up with Security risks and who isn’t. List of shame.
" Use Case 12 – Understanding online course registraPon – What are the most popular classes and Pme. Who has wriaen a script to try
and get in a class as soon as it opens.
" Use Case 13 – the go to tool if the FBI or Law enforcement call " Use Case 14 – IdenPfy when one of your accounts is spamming within 5 minutes