A Practical Guide to Anomaly Detection for DevOps
description
Transcript of A Practical Guide to Anomaly Detection for DevOps
![Page 1: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/1.jpg)
Guide to Anomaly Detection
A Practical
for DevOps
![Page 2: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/2.jpg)
2categories
Anomaly Detection
log analysis metric analysis
![Page 3: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/3.jpg)
identify suspicious event patterns in log files
log analysis
![Page 4: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/4.jpg)
2categories
Anomaly Detection
log analysis metric analysis
![Page 5: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/5.jpg)
identify misbehavingtime-series metrics
metric analysis
![Page 6: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/6.jpg)
It reveals dangerous patterns that previously were undetected
Why is anomaly detection worth our time?
1The static nature of rule-based and threshold-based alerts encourages a) false positives during peak times b) false negatives during quieter times
2
![Page 7: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/7.jpg)
It reveals dangerous patterns that previously were undetected
Why is anomaly detection worth our time?
12 The static nature of rule-based and threshold-based alerts
encourages a) false positives during peak times b) false negatives during quieter times
![Page 8: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/8.jpg)
weapons of
mass detection
![Page 9: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/9.jpg)
weapons of
mass detectionanomaly
![Page 10: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/10.jpg)
Anomaly Detective by Prelert• Product: Anomaly Detective for Splunk • Pricing: $0-$225 / month (quote-based pricing > 10GB) • Setup: On premise (OS X, Windows, Linux & SunOS) • Installation: Easy (with Splunk Enterprise) • Main Datatype: Log lines
![Page 11: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/11.jpg)
• Capable of consuming any stream of machine-data • Can identify rare or unusual messages. • A robust REST API, which can process almost any data feed • Offers an out-of-the-box app for Splunk Enterprise • Extends the Splunk search language with verbs tailored for anomaly
detection
Anomaly Detective by PrelertHighlights:
![Page 12: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/12.jpg)
• Pricing: Quote-based • Setup: SaaS (+ on-premise data collectors) • Ease of Installation: Average (deploy Sumo Logic's full solution) • Main Datatype: Log lines
Sumo Logic
![Page 13: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/13.jpg)
• LogReduce: a useful log crunching capability which consolidates thousands of log lines into just a few items by detecting recurring patterns.
• Sumo Logic scans your historical data to evaluate a baseline of normal data rates. Then it focuses on the last few minutes and looks for rates above or below the baseline.
• Anomaly detection will work even if the log lines are not exactly identical.
Sumo LogicHighlights:
![Page 14: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/14.jpg)
• Pricing: $219/month for 200 instances & custom metrics • Setup: Dedicated AWS instance • Ease of Installation: Easy • Main Datatype: System Metrics
Grok
![Page 15: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/15.jpg)
• Designed to monitor AWS (works with EC2, EBS, ELB, RDS). • Grok API for custom metrics (it’s fairly easy to process data from statsd). • Warns you in real time. • Customizable alerts for email or mobile notifications. • Grok uses their Android mobile app as their main UI. • Installation requires a dedicated Grok instance in your cloud environment.
GrokHighlights:
![Page 16: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/16.jpg)
• Pricing: Open source • Setup: On-premise • Ease of Installation: Average (need python, redis and graphite) • Main Datatype: System Metrics
Skyline
![Page 17: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/17.jpg)
• Etsy’s minimalist web UI lists anomalies & visualizes underlying graphs. • Horizon accepts time-series data via TCP & UDP inputs. • Stream Graphite metrics into Horizon. Horizon uploads data to a redis
instance where it is processed by Analyzer - a python daemon helping to find time-series which are behaving abnormally.
• Oculus, the other half of the Kale stack, is a search engine for graphs. Input one graph then locate other graphs that behave like it. Detect an anomaly using Skyline, then use Oculus to search for graphs that are suspiciously correlated to the offending graph.
SkylineHighlights:
![Page 18: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/18.jpg)
But detecting anomalies !
is only half the battle...
![Page 19: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/19.jpg)
BigPanda uses an algorithmic, data science approach to
simplify & automate incident management
BigPanda + Anomaly Detection
!
!
!
!
Anomaly Detection
incident management
![Page 20: A Practical Guide to Anomaly Detection for DevOps](https://reader033.fdocuments.in/reader033/viewer/2022052315/553960af550346e93a8b4999/html5/thumbnails/20.jpg)
http://bigpanda.io
Come take a look at what BigPanda is building!
Follow us online!