monitoring Linux And Windows Logs With Graylog Collector · Monitoring Linux and Windows Logs with...

Bernd Ahlers – Graylog, Inc. bernd@graylog.com

Monitoring Linux and Windows Logs with Graylog Collector

Bernd AhlersGraylog, Inc.

Structured Logging & Introduction to Graylog Collector

Bernd AhlersGraylog, Inc.

Introduction: Graylog

● Open source log management platform● Collect, index and analyze structured and

unstructured log data● Alerts based on log data● Extensible via custom plugins

More about Graylog

● www.graylog.org● marketplace.graylog.org● docs.graylog.org● github.com/Graylog2

Why are we writing logs?

● Getting insight & collecting business metrics● Debugging problems● Building an audit trail● Monitoring

How do we access our logs?

● Applications write to local files● SSH into machines● tail, grep, awk● If lucky: central log management

What do they look like?

● Syslog RFC 3164 (BSD)● Syslog RFC 5424

Syslog RFC 3164 (BSD)

Nov 10 15:55:01 tumbler CRON[2684]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)

Syslog RFC 5424

2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] BOMAn application event log entry...

Apache

127.0.0.1 - bernd [28/Dec/2014:06:43:15 +0100] "PROPFIND /remote.php/webdav/ HTTP/1.1" 207 910 "-" "Mozilla/5.0 (Linux) mirall/1.7.1"

Postfix

Aug 5 17:05:26 hostname postfix/qmgr[308]: A44F828C71: from=<bamm@example.com>, size=153136, nrcpt=1 (queue active)

sq18.wikimedia.org 1715898 2010-12-01T21:57:22.331 0 1.2.3.4 TCP_MEM_HIT/20013208 GEThttp://en.wikipedia.org/wiki/Main_Page NONE/-text/html - - Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20.NET%20CLR%201.1.4322) en-US -

0 [main] INFO MyApp - Entering application.36 [main] DEBUG com.foo.Bar - Did it again!51 [main] INFO MyApp - Exiting application.

Ruby Logger

I, [2015-11-18T00:16:27.723972 #3609] INFO -- : Hello world!

#1 Problem: Timestamps

● Everyone likes to invent one● Missing most of the time: timezone, year

How to get value out of unstructured logs?

● Regex● More regex● Even more regex

((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?

GrokIPV6 ((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9...

USERNAME [a-zA-Z0-9._-]+USER %{USERNAME}HOSTNAME \b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b)EMAILLOCALPART [a-zA-Z][a-zA-Z0-9_.+-=:]+EMAILADDRESS %{EMAILLOCALPART}@%{HOSTNAME}...COMBINEDAPACHELOG %{COMMONAPACHELOG} %{QS:referrer} %{QS:agent}

Graylog: Extractors

● Regular expressions based● Extracts data into message fields

How to fix this?

● Central log collection (Graylog, ELK, others)● Use structured log formats

– Structured Syslog RFC 5424

– CEF Format

– GELF

– JSON

Structured Syslog RFC 5424

2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] BOMAn application event log entry...

CEF by ArcSight/HP

Sep 19 08:26:10 host CEF:0|HP|siem|1.0|100|service

successfully stopped|10|src=10.0.0.1 dst=2.1.2.2 spt=1232

{ "version": "1.1",

"timestamp": 1385053862.3072,

"host": "example.org",

"short_message": "A short message",

"full_message": "Backtrace here\n\nmore stuff",

"level": 1,

"_user_id": 9001,

"_some_info": "foo",

"_some_env_var": "bar"}

{ "source": "example.org",

"message": "A log message",

"timestamp": "2015-11-15T10:43:21Z",

"user_id": 9001,

"http_method": "GET"}

How we try to improve the ecosystem

● Icinga2 GELF output for events● Docker GELF logging driver (since Docker 1.8)● apache-mod_log_gelf (beta)● log4j2-gelf● gelfclient Java library● svloggelfd (log forwarding for runit)

We at Graylog <3 structured data and you should too!

Introduction: Graylog Collector

● Reads local log files and ships them to Graylog● Windows EventLog support (limited for now)● Transport encryption via TLS● Runs on Linux, Windows, Mac OS X and AIX

Why another Collector?

● There are lots of others: nxlog, fluentd, heka, filebeat, rsyslog, syslog-ng

● We want integration and centralized management of collectors in Graylog

Collector Installation

● OS packages for Linux distributions● Manual installation on Windows via ZIP file

(MSI upcoming)

Runs as Windows service

Collector Configuration

server-url = "http://your-graylog-server:12900"

inputs {

windows-application-log {

type = "windows-eventlog"

source-name = "Application"

outputs {

gelf-tcp {

type = "gelf"

host = "your-graylog-server"

port = 12201

Collector: Current State

● Windows EventLog support needs update to support new Windows APIs

● File reading needs improvement● Centralized management needs to be

implemented● :-(

Tomorrow: Hackathon

Thank you!

Thank you for your time!

Ask me anything!

Bernd Ahlers / Graylog, Inc.bernd@graylog.com

@berndahlerswww.graylog.org

github.com/Graylog2

monitoring Linux And Windows Logs With Graylog Collector · Monitoring Linux and Windows Logs with...

Documents

Transcript of monitoring Linux And Windows Logs With Graylog Collector · Monitoring Linux and Windows Logs with...

Monitoring with Graylog - a modern approach to monitoring?

Collector Wind Farm Environmental Assessmentratchaustralia.com/collector/Ratch Collector WF/01 Development... · Collector Wind Farm Environmental Assessment June 2012 12.3. IMPACT

Graylog Documentation - Read the Docs · 2019. 4. 2. · Graylog Documentation, Release 1.1.6 Highly available setup with Graylog Radio Beginning with Graylog 1.0 on, we no longer

the missing log collector - QCon TokyoTime:Tag :Record Client libraries # logs from a file type tail path /var/log/httpd.log format apache2 tag web.access

McAfee Enterprise Security Manager Data Source ... · Windows DNS Server Windows DHCP Server Windows IIS Server Logs 3 Prerequisites To install the McAfee SIEM Collector, you will

11-5 Common Logs and Natural Logs

Jeopardy $100 Facts About Logarithms Exponentials to Logs Evaluating Logs Expanding Logs Condensing Logs $200 $300 $400 $300 $200 $100 $400 $300 $200 $100.

Whose Logs, What Logs, Why Logs - Your Quickest Path to Security Visibility

EXPLANATION OF MAJOR FUNCTIONS AND …€¦ · PNP open collector NPN open collector PNP open collector NPN open collector PNP open collector Configuration D-ON Wiring method Cable

Collector for ArcGIS: An Introduction - Cloud Object Storage · • What is Collector for ArcGIS ... Collectors 1, 2, 3 Collector 3 . Collector 2 Collector 1 . ArcMap . Hosted Feature

TIBCO LogLogic Universal Collector User Guide · TIBCO LogLogic® Universal Collector (UC) is a software agent that collects logs from any device or log source and forwards them to

Logs, Logs, Every Where, Nor Any Byte to Grok

Graylog Engineering - Design Your Architecture

Flight Logs - andreaszapf.de · Flight Logs Collection of Flight Logs available to -luftwaffe.de This document lists the flight logs available to the author of this web site.

Graylog (GIM) Schema

IBM Disconnected Log Collector: IBM Disconnected Log ...4 IBM Disconnected Log Collector: IBM Disconnected Log Collector Guide System requirements for Disconnected Log Collector IBM

Magic xpi 4.13 Release Notesftp.magicsoftware.com/ver/docs/Downloads/Magicxpi/4.13/... · 2020. 9. 30. · Magic xpi has now introduced a new tool called Logs Collector that collects

F5 BIG-IP Configuration Guide for BeyondInsight · • The BI Event Collector for PowerBroker Windows and PowerBroker Unix/Linux logs. Each endpoint has different networking and SSL

MODSECURITY Firewall de Aplicação WEB Open Source · 2019. 10. 11. · Proxy Reverso ou Embarcado. 23 Logs, Lots of Logs. 24 Apache Logs: Logs, Lots of Logs. 25 Modsecurity Audit

Graylog Documentation - Read the Docs · 3 Conﬁguring and tuning ... • graylog-servernodes should have a focus on ... •MongoDB is only being used to store conﬁguration and