Drowning in Logs - TERENA · Drowning in Logs Evangelos Spatharas TF-CSIRT Meeting Tallinn 24...
23
www.geant.org Drowning in Logs Evangelos Spatharas TF-CSIRT Meeting Tallinn 24 September 2015
Transcript of Drowning in Logs - TERENA · Drowning in Logs Evangelos Spatharas TF-CSIRT Meeting Tallinn 24...
www.geant.org
Drowning in Logs
Evangelos Spatharas
TF-CSIRT Meeting
Tallinn 24 September 2015
www.geant.org
INDEX
What logs are and why are so important
GEANT logs everywhereHow do we monitor our logs
Dashboard panel logs
Do we have complete visibility?
Plan to accommodate missing logsProblems with per volume licensed tools
Open source logging tools Selected tools for evaluation
Q & A
FoD update
www.geant.org 3
What logs are and why are so important?
Special files
www.geant.org 4
What logs are and why are so important?
Detective Technical Control
www.geant.org 5
What logs are and why are so important?
Evidence
www.geant.org 6
What logs are and why are so important?
.. Uncover configuration mistakes
www.geant.org
GÉANT system logs everywhere
Multiple sources
> 150 Win VMs+ > 200 RHEL VMs+ > 40 Hyper-Vs+ > 30 IP cameras+ 31 Juniper MXs+ Many PoP switches------------------------------=~ 8-9M
GEANT
www.geant.org
HELP!
www.geant.org 9
How do we monitor our resources?
GEANT• Single interface for all of types of information• Data correlation• Powerful search and analytics• Alerts• Bigger picture• Operations and Security solution
www.geant.org 10
Routers/Switches dashboard
Others include:
BGP peering attemptsSNMP unauthorized access…..
www.geant.org 11
Linux hosts dashboard
Others include:
Login fails outside GEANT domainUser addition/deletion…..
www.geant.org 12
Camera Dashboard
www.geant.org 13
Nessus Dashboard
Others include:
Total number of vulnerabilities by severityNew alive or dead hosts…..
www.geant.org
• Linux logs
• Router/switch logs
• Camera logs
• Nessus report logs
• Windows logs
14
Do we have complete visibility?
www.geant.org
• How many more logs from Windows? 2.2 GB/day 3.2 GB/day total
• Is the HDD space suffice? What about I/O speed? OK
• Is RAM and CPU suffice for processing? Small upgrades
• Is current vmNIC able to cope with the volume? OK
• What additional software is required to ship the logs to Splunk? Splunk UF
• How many resources for deployment? 15 days
• What is the price for license upgrade and recurring costs? £9,660.00 + £3,252.00 for 5 GB/day
15
Plan to accommodate Windows logs
www.geant.org 16
Volume Projection
3200
3650
4100
4550
5000
5450
5,000 5000 5000 5000 5000 5000
0
1000
2000
3000
4000
5000
6000
2015 2016 2017 2018 2019 2020
Yearly Expected Log Volume / Day
Estimated daily log volume (MB) Max daily volume (MB)
• Another upgrade in 5 years• NetFlow? Another upgrade?
… still confined by price per volume
www.geant.org
What are we interested in?
• Hardware/software requirements (RAM, CPU)
• Level of skill required for managing/configuring it
• Recurring costs
• Scalability/redundancy/search speed
• Alerting
• Integration with existing tools
Most importantly: do we maintain existing Splunk functionality and build more on top of that, or lose?
17
Let’s go
www.geant.org
Open Source Logging Tools
www.geant.org 19
Evaluation
£5,000 / node – Gold Support(~6,882 EUROs)
£4,100 / node – Gold Support(~5,643 EUROs)
www.geant.org 20
Can ELK/Graylog2 substitute Splunk
www.geant.org 21
Q & A
www.geant.org
Thank you & happy logging!
www.geant.org
Firewall on Demand - Update
Currently Pilot ( 24th Aug. 2015 – 23rd Oct. 2015) | 2 NRENsNext KPIs review and tweaking based on NREN needsNext after Next App enhancements
Interested on participating in the pilot?? [email protected]
![CSIRT - Hackerspace Prague [brmlab]CSIRT in general CSIRT Service Categories Reactive Services Proactive services Security Quality Management Services CSIRT Pavel Ru˚ziˇ ckaˇ](https://static.fdocuments.in/doc/165x107/5e6445bebae81a2b335bcc14/csirt-hackerspace-prague-brmlab-csirt-in-general-csirt-service-categories-reactive.jpg)
