Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations...
-
Upload
rootedcon -
Category
Technology
-
view
2.046 -
download
0
Transcript of Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations...
![Page 1: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/1.jpg)
/sin’fɒnjə/
Security Intelligence
![Page 2: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/2.jpg)
Army Knowledge Online (www.us.army.mil) FM 2-0 INTELLIGENCE
/sin’fɒnjə/
![Page 3: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/3.jpg)
/sin’fɒnjə/The Intelligence Cycle
Direction
http://www.cni.es/es/queescni/ciclo/
Collection
Analysis
Dissemination
![Page 4: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/4.jpg)
/sin’fɒnjə/ This is NOT OSINTThis is Copy & Paste
http://tinyurl.com/pavtula
http://tinyurl.com/npegzok
http://tinyurl.com/q2ag2b9
February 26, 2014
![Page 5: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/5.jpg)
What is Intelligence?
Quite simply, intelligence is the information our nation’s leaders need to keep our country safe.
Our leaders, like the President, make policy decisions based on this intelligence.
/sin’fɒnjə/Intelligence (Kids’ Zone)
https://www.cia.gov/kids-page/6-12th-grade/who-we-are-what-we-do/what-is-intelligence.html
![Page 6: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/6.jpg)
• The generation of knowledge in support of decision makers
TroubleshootingAnticipation
• Intelligence is people (but not all people are intelligent):– Methodologies– Tools– Techniques
/sin’fɒnjə/Intelligence
![Page 7: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/7.jpg)
sheer volumen of information
volatile
time saving
gather structure
enrichclassify
store
real time
analyze
/sin’fɒnjə/Tools are Essential
integrate
![Page 8: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/8.jpg)
/sin’fɒnjə/
Storm Builder for Security Intelligence
![Page 9: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/9.jpg)
/sin’fɒnjə/Storm
“Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded streams of data, doing for realtime processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use! “
http://storm.incubator.apache.org/
![Page 10: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/10.jpg)
/sin’fɒnjə/Visual Programming
http://blog.interfacevision.com/design/design-visual-progarmming-languages-snapshots/
![Page 11: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/11.jpg)
/sin’fɒnjə/Module: Types
SPOUT BOLT DRAIN
![Page 12: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/12.jpg)
/sin’fɒnjə/Module: Types
SPOUT
“A spout is a source of streams in a computation. Typically a spout reads from a queueing broker such as Kestrel, RabbitMQ, or Kafka, but a spout can also generate its own stream or read from somewhere like the Twitter streaming API. Spout implementations already exist for most queueing systems.”
![Page 13: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/13.jpg)
/sin’fɒnjə/Module: Types
BOLT
“A bolt processes any number of input streams and produces any number of new output streams. Most of the logic of a computation goes into bolts, such as functions, filters, streaming joins, streaming aggregations, talking to databases, and so on.”
![Page 14: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/14.jpg)
/sin’fɒnjə/Module: Types
DRAIN?
![Page 15: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/15.jpg)
/sin’fɒnjə/
Define a Module
Load to Storm
Use in a Topology
Upload your Code
Share on Sinfonier
Module: Life Cycle
![Page 16: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/16.jpg)
/sin’fɒnjə/
Make a Topology
Run on Storm
Check Dashboard
Show results
Topology
![Page 17: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/17.jpg)
cat /var/log/named/query.log | grep "IN A" | awk '{ print $6 }' | awk -F"#" '{print $1}' |sort -n | uniq -c | sort -rn | head | awk '{ printf $1",";system("curl -s http://freegeoip.net/csv/"$2 | cut –d”,” –f3 )}’
curl --retry 3 --insecure -s https://www.rootedcon.es/ | grep -E 'href="http://.*rootedcon\.es'| awk -F"href=\"" '{print $2}' | sed 's|\".*||g' | xargs curl -s -o /dev/null --write-out "%{http_code}:%{size_download}\n"| awk -F":" '{ if ( $1 == "200") { print "RSS size: " $2} }'
crontab -l# m h dom mon dow command@reboot /usr/bin/python /home/charlie/.ave_phoenix.py30 7,15,23 * * * /home/charlie/vigila/gauchap.sh –tweet fotos 2>&1 >/dev/null
/sin’fɒnjə/ Shell Scripting
![Page 18: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/18.jpg)
/sin’fɒnjə/
Demo & Use cases
![Page 19: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/19.jpg)
/sin’fɒnjə/TweetMon
![Page 20: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/20.jpg)
/sin’fɒnjə/TorrentPeer
![Page 21: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/21.jpg)
/sin’fɒnjə/Crawler
![Page 22: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/22.jpg)
/sin’fɒnjə/Roadmap
Fun & Profit
Community
![Page 23: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/23.jpg)
/sin’fɒnjə/We Want You
![Page 24: Francisco Jesús Gómez & Carlos Juan Diaz - Sinfonier: Storm Builder for Security Investigations [Rooted CON 2014]](https://reader035.fdocuments.in/reader035/viewer/2022070523/58ed7c821a28ab0c718b4621/html5/thumbnails/24.jpg)
/sin’fɒnjə/
Become a Beta Tester
http://sinfonier-project.net/
http://tinyurl.com/sinfonier