ELK in the wild –Real life log analysis on...
Transcript of ELK in the wild –Real life log analysis on...
![Page 1: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Asaf Yigal, VP Product Co-Founder, Logz.io
May 2017
ELK in the wild – Real life log analysis on AWS
![Page 2: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/2.jpg)
Who Am I?
Asaf Yigal – VP Product , Logz.io
1,000 companies from 80 different countries use Logz.io
![Page 3: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/3.jpg)
Agenda
• Why log analysis is important ?• Introducing ELK• Security at British Airways• DDoS attack detection at Dyn
![Page 4: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/4.jpg)
Online user behavior
IoTanalytics
Monitoring & system troubleshooting
Security and compliance
Security devices
App server
Network
Machine Logs Big Data
Fundamental to Understanding Machines
![Page 5: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/5.jpg)
Open Source ELK +/-
Simple and beautifulIt’s simple to get started and play with ELK and the UI is just beautiful
Open SourceThe largest user base with a vibrant open source community that supports and improves the product
Fast. Very fast.Built on the Elasticsearch search engine, ELK provide blazing quick responses even when searching through millions of documents
Hard to ScaleData piles up and organization experience usage bursts. It’s super-complex building elastic ELK deployments that can scale up and down
Poor SecurityLogs include sensitive data and open source ELK offers no real security solution, from authentication to role based access
Not Production ReadyBuilding production ready ELK deployment is a great challenge organization face. With hundreds of different configurations and support matrix, making sure it’s always up is difficult
![Page 6: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/6.jpg)
Simple and beautiful Open Source/Flexible Fast. Very fast.
ELK Stack500,000+
companies
20K companies
ELK Stack 2017
Propriety Software
*Research done by Logz.io
![Page 7: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/7.jpg)
1. No logs should be dropped (trivial, ah)2. Highly Available3. Secure which means encryption and access control4. Index management, shard allocation5. Data should be parsed and mapping configured6. Data should be retained for x days7. Configuration management and monitoring8. Data spikes should handled up to 10x normal capacity9. Visualization and dashboards10. Archive long retention11. Alerts
Production Requirements
![Page 8: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/8.jpg)
Security at British Airways
Challenges
Why Logz.io
![Page 9: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/9.jpg)
ELB Health
![Page 10: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/10.jpg)
Understanding Weekly trends
![Page 11: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/11.jpg)
Understanding who is crawling the site
![Page 12: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/12.jpg)
Understanding traffic
![Page 13: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/13.jpg)
Understanding Client Location
![Page 14: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/14.jpg)
DDoS attacks detection at Dyn
![Page 15: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/15.jpg)
15https://img.memesuper.com/182956f180cfb7a8c95d6dda68a1d351_you-get-a-ddos-attack-ddos-meme_625-468.jpeg
![Page 16: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/16.jpg)
Numerous methods of detection
16
![Page 17: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/17.jpg)
17
Understand Normal
● We leverage monitoring to define normal.
● We alert in reasonable ways when critical metrics become abnormal
● Too many alerts and your “teams tasked with reactive reliability” will get burned out.
● Normal shouldn’t be subjective. Socialize your key performance indicators!
![Page 18: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/18.jpg)
18
![Page 19: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/19.jpg)
Netflow
19
![Page 20: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/20.jpg)
Fast breakdown of SRC & DST port, proto, ASN, Site, etc.
20
![Page 21: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/21.jpg)
Quick sort and analysis of v4 and v6 IPs
21
![Page 22: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/22.jpg)
Examples of attack
22
![Page 23: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/23.jpg)
• Lots of resources online• Try the Logz.io blog for detailed guides, benchmarks and
troubleshooting guides on building ELK stacks• @logzio• @asafyigal
How to Learn More
![Page 24: ELK in the wild –Real life log analysis on AWSaws-de-media.s3-eu-west-1.amazonaws.com/images/AWS_Summit_… · • Introducing ELK • Security at British Airways • DDoSattack](https://reader033.fdocuments.in/reader033/viewer/2022043004/5f84fba7b190d966d9262202/html5/thumbnails/24.jpg)
Asaf Yigal (@asafyigal)Logz.io (@logzio)