Post on 18-Oct-2014
description
Situational Awareness
raffael marty - pixlcloud december 2011
copyright (c) 2011pixlcloud | creating big data stories
Is this useful for Situational Awareness?
copyright © 2011pixlcloud | creating big data stories
OverviewNetwork Security Sit Awareness Today
Where we should be Challenges Resources
copyright (c) 2011pixlcloud | creating big data stories
Raffael Marty
•SaaS business expert•Data visualization practitioner•Security data analyst
Applied Security VisualizationPublisher: Addison Wesley (August, 2008)
ISBN: 0321510100
pixlcloud
IBM Research
copyright (c) 2011pixlcloud | creating big data stories
Cyber Security
Forensics / IR
Information Security
Authentication Authorization AccountingBCM / DROS SecurityPolicies and Procedures...
Network Security
Situational Awareness
Reporting
AlertingNeglected!!!
Data Collection
Reactive Pro-Active
copyright © 2011pixlcloud | creating big data stories
Situational Awareness“Situational Awareness is the ability to identify, process, and comprehend the critical elements of information about what is happening to the team with regards to the mission. More simply, it’s knowing what is going on around you.”
‣ find air force viz images
IWViz - IDS Situational Awareness
copyright © 2011pixlcloud | creating big data stories
Sit Awareness Is Visualization‣Visualization - because machine centered approaches have failed‣Leverage human cognitive capabilities ‣Pattern recognition‣Pre-attentive processing‣Context memory
copyright (c) 2011pixlcloud | creating big data stories
Today
copyright © 2011pixlcloud | creating big data stories
Data Sources for Sit Awareness‣Flow records
‣Firewalls
‣IDS/IPSs
‣What about: PCAP, DNS, BGP, OS, Proxies, User behavior ??‣Context information - Hosts, Users, ...
1.1.1.1 10.0.0.2
9.4.242.10
1.1.1.1 10.0.0.2
9.4.242.10
1.1.1.1 10.0.0.2
9.4.242.10
copyright © 2011pixlcloud | creating big data stories
Todays Visualization Tools‣Based on specific data source‣Hard to use‣Limited interactivity‣Not real-time‣Slow‣Ugly
‣ Gephi‣ R‣ Matlab‣ Mondrian
‣ PicViz‣ Treemap 4.1‣ Google Earth
copyright © 2011pixlcloud | creating big data stories
Take the Blinders Off!
copyright © 2011pixlcloud | creating big data stories
Visualization Maturity‣Data Collection‣Data Analysis‣Context Integration‣Visualization‣Visual Analytics‣Collaboration‣Dissemination
Data Sources (Data Store) Structured Data
filesdatabase
filteringaggregationcleansing
Contextual Data
Visual Representation
visualization
iterations
parsingfeature selection
copyright © 2011pixlcloud | creating big data stories
Security Visualization Dichotomy
‣ security data‣ networking protocols‣ routing protocols (the Internet)‣ security impact‣ security policy‣ jargon‣ use-cases‣ are the end-users
‣ types of data‣ perception‣ optics‣ color theory‣ depth cue theory‣ interaction theory ‣ types of graphs‣ human computer interaction
Security Visualization
copyright © 2011pixlcloud | creating big data stories
Landscape Changes
• from fame to financial gain• from audacious to “low and slow”
• from indiscriminate to targeted• from manual to automated
• from disruptive to disastrous
• from infrastructure to applications
Threat Landscape Technology• Big Data
• NoSQL• Column-based data stores• Map Reduce (hadoop)
• Cloud• on demand computing
We have technology to attack the threats!BUT we don’t know what to do with it!
copyright © 2011pixlcloud | creating big data stories
The Public Sector‣Currently using a lot of Excel‣Big data technologies (e.g., Datameer, Karmasphere, Cloudera)‣ Incremental improvements to SIEM tools (e.g., ArcSight, etc.)‣Using non security / network tools (e.g., Advizor, Cognos)
‣Working with blacklists and whitelists‣Not understanding the data intrinsically
copyright © 2011pixlcloud | creating big data stories
The GovernmentEverything is different from Industry
Scalee.g., DISA has 5 million live hosts
Types of attacks Adversaries
Data sources
e.g., Nation states
e.g., ASIM CIDS
I have no example ....
copyright (c) 2011pixlcloud | creating big data stories
We Need
copyright © 2011pixlcloud | creating big data stories
What we Need‣Leverage advanced technologies (big data, etc.)‣Build for the actual users, not programmers!‣End to end tools, not yet another library‣ Interactive, not static!‣Multiple data sources at once‣Leverage context, not just event data‣Decouple data from the tools ‣Crowd intelligence
copyright © 2011pixlcloud | creating big data stories
Make it This Simple!
copyright (c) 2011pixlcloud | creating big data stories
Challenges
copyright © 2011pixlcloud | creating big data stories
Maturity Challenge
Companies and products are stuck on the left hand side!
1copyright © 2011pixlcloud | creating big data stories
Data Challenges‣No data - no insights - no sit awareness‣We don’t even have / collect the data‣ It is too hard to collect data‣We don’t understand our data!‣Data silos‣Large amounts of semi-structured data‣Parsing data is extremely hard
copyright © 2011pixlcloud | creating big data stories
Tool Challenges‣Same old - all over ‣Does your SIEM support visual analytics?
‣Missing: Brushing, Interactivity ‣Help the user understand the data!‣Highly scalable visualization systems are hard to build!‣What algorithms are useful? (e.g., clustering)‣Visualization expertise is missing‣Visualization AND security is an interdisciplinary problem
Overview first
Zoom and Filter
Details on demand
copyright © 2011pixlcloud | creating big data stories
Visualization Challenges‣Skilled people are missing‣What are we even trying to look for?‣Anomaly detection is not working‣Academia is disconnected‣Use-cases and problems‣State of the art in industry‣Visualization is always an afterthought
copyright © 2011pixlcloud | creating big data stories
Myths‣Real-time‣Do we really need real-time?
‣Hadoop‣Not everything that is big data needs to use Hadoop!‣Know your technologies!
‣Cloud‣Will we ever put security relevant data into the cloud?
copyright © 2011pixlcloud | creating big data stories
Resources‣SecViz: http://secviz.org and @secviz‣CERT - NetSA: http://www.cert.org/netsa/‣Mainly a collection of papers and links to some tools (SiLK)
‣VizSec Conference: http://www.vizsec.org‣Applied Security VisualizationR. Marty, 2008
pixlcloudcreating big data stories
copyright (c) by r. marty - december 2011
@raffaelmarty
buy now