Sumo Logic Quick Start - Feb 2016

Sumo Logic Confidential

QuickStart WebinarGetting Started with Sumo Logic

Mario SánchezFebruary 2016

Welcome.To give everyone a

chance to successfully connect, we’ll start at

10:05 AM Pacific.


Agenda

What is Sumo Logic?

Key Points Regarding Data Collection (Admin Topic)

Searching, Parsing and Analyzing Data

Visualizing and Monitoring – Dashboards and Alerts

Introduction to Library and Apps

Introduction to Optimization Tools (Admin Topic)


What is Sumo Logic?


Continuous Intelligence

DEVOPS IT INFRASTRUCTURE AND OPERATIONS

COMPLIANCE AND SECURITY

DEVOPS

Streamline continuous delivery

Monitor KPI’s and Metrics

Accelerate Troubleshooting

IT INFRASTRUCTURE AND OPERATIONS

Monitor all workloads

Troubleshoot and increase uptime

Simplify, Modernize, and save costs

COMPLIANCE AND SECURITYAutomate and demonstrate complianceAudit all systems

Think beyond rules

Sumo Logic Cloud Analytics Service


sumologic.com/compliance


Enterprise Logs are Everywhere

Custom App Code

Server / OS

Virtual

Databases

Network

Open Source

Middleware

Content Delivery

IaaS, PaaS SaaS Security


High-Level Data Flow


Sumo Logic Data Flow

Data Collection Search & Analyze Visualize & Monitor

Alerts

Dashboards

Collectors

Sources

Operators

Detect

1 2 3


Data Collection


Host A

Collectors and Sources

Apache Access

Apache Error

Collector A Host B Collector

B Host C

Collector C

Apache Access

Apache Error

IIS Logs

IIS W3C Logs


Metadata Fields

Name Description

_collector Name of the collector this data came from

_source Name of the source this data came through

_sourceHost Hostname of the server this data came from

_sourceName Name of the log file (including path)

_sourceCategory Category designation of source data

Tags added to your messages when data is collected

Host A

Apache Access

Apache Error

Collector A


Host A

Metadata Field: Source Category

Apache Access_sourceCategory =

WS/Apache/Access

Apache Error_sourceCategory =WS/Apache/Error

Collector A Host B Collector

B Host C

Collector C

Apache Access_sourceCategory =

WS/Apache/Access

Apache Error_sourceCategory =WS/Apache/Error

IIS Logs_sourceCategory =

WS/IIS

IIS W3C Logs_sourceCategory =

WS/IIS/W3C

Sample Searches for_sourceCategory:

= WS/Apache/Access= WS/Apache/*= WS/*


Source Category Naming Convention

Simplifies Search Syntax and Scope Definitions

Used for other Sumo Logic featuresRole-Based Access Control (Data Provisioning)

Partitioning (Search Optimization Tool)

Adopt a Robust Naming Convention EarlyEx: Prod/Sumo/Apache/Access à Env/Customer/Device/MessageType

Ex: OS/Windows/2012/Messages à Device/Vendor/Version/MessageType

Blog Post: Good SourceCategory, Bad SourceCategory


Search and Analyze


Set your Preferences

Set your Session Timeout

Query Editing versus Running


Search Basics OverviewTime Range

Histogram

Search Bar

Search Results

Display Options


Field Browser - Metadata fields


Field Browser

Metadata Fields

Parsed Fields


Keyword Search

Case Insensitive

Wildcard Support (e.g. ERR*)

Boolean Logic SupportAND

OR

!(A OR B)

Combine these keywords with metadata fields

Bloom filtersUsing keywords helps bloom filters locate data very quickly


• Determine the data available through your search.

• Pre-populated Dropdown– Last 15 min, Today

• Absolute– 12:25PM 12:30PM– 8/11/2015 13:00AM 8/11/2015 14:00AM

• Relative– -5m– -2h– -2d -1d

Time Range


Develop Good Search Habits

Use metadata and keyword combinations to reduce scope

Add line breaks after each operation

Limit result sets before aggregating data à user=a | count by user

Use parse anchor instead of parse regex for structured messages

Avoid the use of expensive parse regex tokens like .* à \d{2,10}

Narrow your time-range down as much as possible


Refining Results by Surrounding Messages


• LogReduce uses fuzzy logic and soft matching to cluster messages providing quick investigation view into your environment.

Operators: Looking for the Unknown


• Identify unexpectedly high or low values within determined thresholds|timeslice 1m|count by _timeslice|outlier _count

Operators: Finding Outliers


• Parsing enables a user to extract parts of a message and classify them as fields.– Enables you to perform additional operations

• Logical/conditional – based on values• Mathematical – operations on value sets

• Parsing Options– parse anchor: Leverages beginning and ending anchors– parse regex: Extracts nested information via regex

Extracting and Labeling Additional Fields


Parse Anchor - Using the UI

Highlighting strings in the result allow

you to launch the UI parser UI Parser allows

you to select fields and label them

Results now show your parsed fields


• Extracts nested information via regular expressions• Use if the construct of the messages is inconsistent

_sourceCategory=Apache/Access| parse regex "[A-Z]+\s(?<url>/\S*)\sHTTP/1.\d+\"\s(?<status_code>\d+)\s"

Parse Regex


Regular Expressions – References and Resources

Regular Expressions use JRE

Online Resources:• regex101.com• Regular-expressions.info/refadv.html• en.wikipedia.org/wiki/Regular_expression• regexr.com

• Book– Mastering Regular Expressions by Jeffrey E.F. Friedl


Evaluates messages and places them into groups• Produces aggregates in a separate tab• Must come after basic operators such as parse. Cannot be used with summarize.

• The count Operator enables you to group messages that match a classification– Ex: _sourceCategory=Apache* | count as mycount

– Ex: GET | count by _sourceCategory

Grouping your Data


• Dissecting your result sets using Metadata Fields– Ability to aggregate results sets and grouping them by metadata fields

• EX: _collector=*apache* | count by _sourceCategory

– Get a count of grouped result sets• Ex: (Error OR fail*)| count by _sourcecategory , _sourcehost

– Organize Results by Count• Ex: _collector=*apache*| count by _sourceCategory | sort by _count

Leveraging Metadata for Grouping


Timeslice operator enables you to segment your results by time buckets

– Minute (timeslice by 5m)– Hour (timeslice by 1h)– Day (timeslice by 1d)

Example:_sourceCategory=Apache/Access GET|timeslice 1m| count by _timeslice| sort by _timeslice asc

Time-based Grouping


Saving Your Searches

Click Save As under the Query

Window

Description is searchable, so a include detail to allow searching at a future time

You can save the search in your own Personal folder, or

create a sub-folder


Visualize and Monitor


• Collection of Panels that provide graphical representation of data – Each Panel processes results of a search– Drilldown for additional analysis

• Drill into the query behind the dashboard• Drill to another dashboard

Introduction to Dashboards


• Chart Types– Table– Bar– Column– Line– Area– Pie– Box Plot– Google Maps– Single Value

Providing Context through Visualization


– Live Dashboards• Provides a live stream of data• No back filling of data

– Interactive Dashboards• Search based (On-Demand)• Backfilling of data• Support Filtering

Dashboard Types

No Interaction

Ability to use Pre-defined

filters


Live Dashboards versus Interactive Dashboards

Use Case Examples Dashboard TypeLarge screen displays with streaming updates

Shared Screens for NOC, Operations, Developers, etc.

Live Dashboards

Template for Exploring Data

Operational Investigations Interactive Dashboards

Historical Reportingand Investigation

Audits, Failed/successful logins for certain groups

Interactive Dashboards


Dashboards - Adding a Panel

1. Perform your Search

2. Format your Results

3. Create a Panel


Alerting

Using a Scheduled Search, you can set Alerts to trigger whenever the search completes or when a certain condition is met.

Alert types include:• Save to Index• Script Action• Email• Webhooks

Blog Post: 2 Key Principles for Creating Meaningful Alerts


Saving and Scheduling an Alert

1. Save your Search2. Schedule the Search

3. Specify frequency and time range

4. Specify Alert condition & threshold

5. Specify Alert Type and details


Jumpstart with Apps


Installing Applications


Optimizing Your Search Experience


Factors in Search Performance

Query Structure

Time range

Data Selectivity (keywords, metadata, where statements)

Heavy Operations (join, transaction, summarize)

Overall Data Volume

System load

Improve search experience using Optimization Tools


Search Optimization Tools

How-To Webinar Recording: https://youtu.be/JNWbtws-sns

PartitionsIndex data for searching over a smaller data set

Scheduled ViewsPre-aggregating data for fast counts/sums over longer time ranges

Field Extraction RulesParse the data on ingest rather than run-time; simplifies searches

Take advantage of interactive dashboard filters


Questions?

Additional ResourcesSearch Video Library and Documentation

Search/Post to Community ForumsSearch, post, respond

Submit/vote for feature requests

Submit Tips & Tricks

Open a Support Case

Sumo Logic ServicesCustomer Success, Professional Services, Training


Helpful Links

Hands-on Lab: Sumo Logic QuickStart Tutorialhttps://service.sumologic.com/help/Default.htm#Tutorial.htm?Highlight=tutorial

Sumo Logic Traininghttps://www.sumologic.com/training

Support Portal, Documentation, Community Forums, Feature Requestshttps://support.sumologic.com/home

[email protected]


Thank you!

Sumo Logic Quick Start - Feb 2016

Software

Transcript of Sumo Logic Quick Start - Feb 2016