Splunk 6.2 new features
-
Upload
cleverdata -
Category
Technology
-
view
728 -
download
1
description
Transcript of Splunk 6.2 new features
cleverdata.ru | [email protected]
Ge#ng value from your data just got really fast. New features SPLUNK> 6.2
Dmitry Anoshin, Solu>ons Architect
cleverdata.ru | [email protected]
New Approach in Analy?cs
Collect Data Analyse Prepare Data (clean, transform)
Collect Data
Tradi>onal Approach Tradi>onally, a lot of >me has been spent collec>ng and preparing data. Eventually you get to ask the ques>ons of the data, start to create the right analy>cs and get the insight you need from it. This can take a lot of ?me.
Fresh Approach We need to get the value from the data as quickly as possible, in order find valuable insights.
Prepare Data Analyse
With Splunk Enterprise 6.2 we’ve focused on three main areas of func>onality: 1) Easier data on-‐boarding, data prepara>on and Advanced Field Extractor (AFX) 2) More Powerful Analy>cs for Everyone – Instant Pivot, Event PaWern Detec>on and Prebuilt Panels 3) Simplified Management at Scale – Search Head Clustering and Distributed Management Console
cleverdata.ru | [email protected]
Key Features • Data Explora>on • Knowledge Management • Dashboard Enhancements • Mission-‐Cri>cal Enterprise
cleverdata.ru | [email protected]
Data Explora>on
cleverdata.ru | [email protected]
Home Page Redesign
How it works • Users can navigate, and choose an exis>ng dashboard from
within the home page itself • Alterna>vely, users can mkae any dashboard their home page
from the dashboard lis>ng page, or within the dashboard itself • This selec>on gets saved to the user's user-‐prefs.conf
Splunk 6.2 introduces a new home page to provide users quicker access to relevant data, allowing users to select any exis>ng dashboard to use as their custom home page.
cleverdata.ru | [email protected]
Event PaWern Detec>on
cleverdata.ru | [email protected]
Bubble Chart
Use bubble charts for mul>-‐dimensional numeric analy>cs. Bubble chart specific op>ons: • Minimum bubble size <op>on name="char>ng.chart.bubbleMinimumSize">10</
op>on> • Maximum bubble size <op>on name="char>ng.chart.bubbleMaximumSize">50</
op>on> • Bubble size by area or radius <op>on name="char>ng.chart.bubbleSizeBy">area</
op>on>
cleverdata.ru | [email protected]
Instant Pivot Enables you to open any query in the Pivot interface, without requiring the crea>on of a data model. This means that you have the flexibility to choose what interface to explore your data. How it works • Ephemeral data model is created that collects user specified fields within Pivot as a single,
flat object • User can save their Pivot (addi>onally prompts user to save data model)
cleverdata.ru | [email protected]
Ultra Drilldown
Provides more explicit drilldown controls, without requiring users to know any of the hidden modifier keys.
cleverdata.ru | [email protected]
Knowledge Management
cleverdata.ru | [email protected]
Consolidated Workflow
We’ve made it much easier to find your way to the appropriate input configura>on. Instead of selec>ng from a confusing list of sources, start with a simple choice of “upload, monitor, or forward” and you’ll find yourself in a simple wizard-‐style workflow of defining the appropriate parameters for the data you want to add.
cleverdata.ru | [email protected]
Splunk can index any machine data. Common data sources are:
Data Sources
cleverdata.ru | [email protected]
With Forwarder Inputs, you are able to push input configura>ons to Splunk instances configured as deployment clients. Simply select one or more forwarders, provide a group name, and you’ll be able to create data inputs on them in the same way you create inputs through the UI on your indexers.
Forwarder Inputs
cleverdata.ru | [email protected]
New Data Preview
The new data preview u>lity will make it easier for you to create the right sourcetype for your data. In the advanced sec>on, you’ll be able to choose a charset from a list, and see how changes you make to your sourcetype are reflected in props.conf.
cleverdata.ru | [email protected]
In order to beWer choose the appropriate sourcetype when adding data we built a new sourcetype picker, which includes categories and descrip>ons. To add categories and descrip>ons to your sourcetypes, modify the sourcetype_metadata.conf file. [docs link]
Sourcetype categories and descrip?ons
cleverdata.ru | [email protected]
Advanced Field Extractor
In Splunk 6.2, it easier to extract fields from your data with the Advanced Field Extractor (AFX). A replacement of the exis>ng field extrac>on u>lity, AFX enables you to easily capture mul>ple fields in a single extrac>on and specify required text to filter events for extrac>on (improving accuracy and efficiency). AFX also provides a number of methods for detec>ng false posi>ves in order to help you validate your field extrac>ons and improve the accuracy of your fields.
cleverdata.ru | [email protected]
With the search filter, you can enter a keyword to filter events by, so that you can beWer drill down to the exact event containing your desired value to extract into a field. This can also help you validate extrac>ons – once you’ve selected a sample value to extract, you can type in other values that should be extracted to this field in the search filter to make sure they are also being extracted (extrac>ons are highlighted in the event lis>ng).
Search filter
cleverdata.ru | [email protected]
Sample Events & Highlight-‐to-‐Extract
In the legacy field extractor, Splunk Enterprise asks you to type in sample values for fields to extract. In the (new) Advanced Field Extractor, you’re now asked to select a sample event from the event lis>ng to extract values from, and in the next step, highlight values in the chosen sample event to create field extrac>ons.
cleverdata.ru | [email protected]
Sample Events & Highlight-‐to-‐Extract Cont.
You can select addi>onal sample events in the “Select Sample” and “Select Fields” steps of the workflow, and highlight field values (in the “Select Fields” step) from the addi>onal sample events to improve the accuracy of the extrac>on that is generated.
cleverdata.ru | [email protected]
Rather than extract one field at a >me, try highligh>ng mul>ple values in your selected sample event – you’ll see color-‐coded highligh>ng apply to the listed events. To remove or rename a field, simply click its value in the sample event.
Extract mul?ple fields at once
cleverdata.ru | [email protected]
Some>mes a source type may contain different kinds of events, and in order to extract fields from the various paWerns you may want to go through the extrac>on process more than once, for each of the paWerns. Or maybe you want to make sure that a value is only extracted to a field from specific events. To improve both accuracy and efficiency, you can now specify required text: values in the event will be extracted as fields only when the event contains the required text. At this >me, only one value can be defined as required text, and you cannot require and extract the same value. To require text, highlight as if to extract, but choose “require”.
Specify Required Text
cleverdata.ru | [email protected]
Aqer defining field extrac>ons in the “Select fields” step, you should verify their efficacy before saving them. In the “Validate” step, the Advanced Field Extractor offers a variety of methods for iden>fying values that shouldn’t have been extracted (false posi>ves) and lets you provide feedback for the purpose of improving accuracy.
Valida?ng and fixing field extrac?ons
cleverdata.ru | [email protected]
To beWer inspect the extracted values for a field, click on its field tab. You will see a chart showing its values (similar to “ | top <field>” or “ | stats count by <field>”). If you see an incorrect value, you can click it to add it as a filter. When you click a field value in a field stats tab, the filter box will fill with a “field=value” search and the events you see listed will be limited to those that match the filter.
Field Stats
cleverdata.ru | [email protected]
To see what fields have been previously defined for the chosen sourcetype, click the “Exis>ng Fields” buWon at the top right. You’ll see a list of each field that’s defined, and a link to open it in manual mode (of the Advanced Field Extractor). Note: Only fields defined through the field extractor, or as EXTRACT-‐ groups in props.conf, will be listed in this sec>on.
Exis?ng Fields
cleverdata.ru | [email protected]
Manual Mode
If you speak Regular Expression, you may define one manually rather than highligh>ng values to extract. Currently only accessable in the “Select sample” step (this will change), manual mode will bypass the wizard workflow and you will not be able to provide counterexample feedback to improve your RegEx. You will however, get to take advantage of the event lis>ng features like filtering on keywords and match/non-‐match, event lis>ng op>ons, field stats, and highligh>ng of extracted values.
cleverdata.ru | [email protected]
App Key Value Store
With Splunk 6.2, App developers can now make use of the App Key Value Store to persist data associated with their applica>ons. The common uses for the K/V Store are to both enrich machine data within Splunk Apps and a variety of other scenarios which require persistence of Splunk App specific informa>on. • Highly flexible, secure and scalable storage and retrieval capabili>es. • Ability to create, update and query data collec>ons using a rich set of REST APIs. • Ability to use the exis>ng Splunk lookup commands (inputlookup, etc.) against data in
the App value key store. • Enforce Splunk’s Role-‐based access control to data in the App key value store. • Monitoring and management dashboards to understand the performance
characteris>cs of the App key value store.
cleverdata.ru | [email protected]
Dashboard Enhancements
cleverdata.ru | [email protected]
Build custom dashboards faster, leveraging pre-‐built dashboard panels packaged within apps. Enables anyone to build their own custom dashboard, leveraging pre-‐built dashboard panels packaged within apps. Panels allow app developers to create and package re-‐usable dashboard building blocks that enable end-‐users to leverage in adap>ng applica>ons to meet their specific requirements and use cases. • Improve developer efficiency to support more use cases, requirements, and target users without addi>onal effort • Easily adapt applica>ons to meet the specific needs of each customer and organiza>on • Expand engagement across mul>ple groups and levels of users with a single applica>on • Enable any user to create personalized dashboards with informa>on rela>ve to their specific use case • Improves user efficiency with enhanced content browsing, search, and discovery Key Features • New add content workflow for inline panels, pre-‐built panels, reports, and cloning content from other dashboards • Improved add inline content flow, including the ability to select viz type directly within the add func>on • Ability to search across all available content • Preview content before adding to the dashboard • Convert to Prebuilt Panel to maximize reusability • Convert panel to inline to further customize
Panels
cleverdata.ru | [email protected]
Mul?-‐Search Management
Configure mul>ple background searches, with explicit post process references throughout your dashboard. This allows users to: • Op>mize performance of the page by execu>ng a single search that mul>ple visualiza>ons leverage • Note that there is a limit of 10k results passed down to post process search (for event-‐based searches) • Drive mul>ple related form inputs from a single global search, and post process within each input • In some cases, perform token-‐based searches within a post process so that you don't have to execute an expensive global search
every >me a new input is selected How it Works • Global searches may be instan>ated anywhere on the page (including within panels) • Global searches (any search that you want to later perform a post process against) must include an "id" <search
id="global_search">...</search> • Explicitly bind a postprocess query to any search using a base="" <search id="post_process_1" base="global_search">...</search> • Included a saved search within a dashboard now uses the syntax ref="" <search ref="mySavedSearch">...</search> • New search syntax uses new syntax for earliest and latest (NOT earliestTime or latestTime)
cleverdata.ru | [email protected]
Set mul>ple tokens within form inputs to drive mul>ple searches, beWer labeling, and more. Key use cases include: • Set tokens for both label and value to use throughout your dashboard • Use this to create a special empty/null choice that includes a unique token transforma>on • Use the selec>on of a given form input to unset other tokens on the page • Create a "simple" >me range picker dropdown input that sets a unique earliest and latest token • Set mul>ple tokens based on search results
Input Mul?-‐token SeVer
cleverdata.ru | [email protected]
Panel Title Support
Organize dashboard content into panels, and set a >tle to describe the content. Panels are fully supported dashboard objects, which includes • Panel >tles (as well as element >tles) can be edited now in-‐place • Dele>ng a panel will delete all sub elements • Convert to pre-‐built panel will save as a resusable panel object, and convert to
reference
cleverdata.ru | [email protected]
Free Form Input Support within Dropdown and Mul?-‐select
Enable users to enter free form text within dropdown and mul>-‐select inputs. This allows users to • Enter free form value that includes wildcards, like "*Risk" • Useful for host selec>ons, where users can enter par>al domains, par>al hostnames, and
more
This feature is controls via the op>on, <allowCustomValues>
cleverdata.ru | [email protected]
Full Object ID Support
<form> <label>sample id example</label> <descrip>on></descrip>on> <fieldset submitBuWon="false"> <input type=">me" token=">me" id=">me_picker"> <label></label> <default> <earliestTime>-‐60m@m</earliestTime> <latestTime>now</latestTime> </default> </input> </fieldset> <row id="row_1"> <panel id="panel_1"> <>tle>Top Sourcetype</>tle> <table id="table_1"> <search> <query>index=_internal | top sourcetype</query> <earliest>$>me.earliest$</earliest> <latest>$>me.latest$</latest> </search> <op>on name="wrap">true</op>on> <op>on name="rowNumbers">false</op>on> <op>on name="dataOverlayMode">none</op>on> <op>on name="drilldown">cell</op>on> <op>on name="count">10</op>on> </table> </panel> </row> </form>
Assign unique IDs to rows, panels, elements, and form inputs for easier custom js/css handling.
cleverdata.ru | [email protected]
Mission-‐Cri>cal Enterprise
cleverdata.ru | [email protected]
Search Head Clustering provides high availability by replica>ng the user configura>on se#ngs, dashbaords, and reports across search heads. Users can use any member of the clusters and they will get the same user experience
Search Head Clustering
Key features: • Automa>c synchroniza>on of files
between dashboards • Search heads can be added /
removed any>me without bringing down the en>re cluster
• Easier to administer the cluster through centralized interface
• Cost effec>ve, no need to buy expensive shared storage system
• No single point failure in the architecture
cleverdata.ru | [email protected]
The new bucket status page makes it easier to monitor large scale clusters. It provides an in-‐depth view of all cluster related recovery opera>ons. If the cluster needs any fix-‐ups, then the new bucket status page provides an accurate view of all the fix-‐ups ac>vi>es. It shows the buckets that are fixed to meet replica>on policies vs search policies. The status page can be viewed at an aggregate level for all indexes or for specific index for detailed analysis. The page also shows any excess buckets, if present, in the cluster. The excess buckets can be removed through the bucket status page itself.
Indexer Cluster Monitoring
cleverdata.ru | [email protected]
Monitor key usage and performance metrics across the en>re Splunk topology The feature builds upon plaworm instrumenta>on and other features added in the 6.1 release to enhance the Splunk Admin's awareness of their distributed Splunk Topology and includes Splunk Dashboards/Views that report on three key areas: • Search Usage and Performance at Deployment-‐wide and Individual levels • Indexing Usage and Performance at Deployment-‐wide and Individual levels • Plaworm Resource U>liza>on (CPU/Memory/Disk) at Deployment-‐wide and Individual
levels The feature also includes several "Plaworm Alerts" that allow the Splunk Admin to enable email alerts for pre-‐packaged condi>ons that may be detrimental to the opera>on of Splunk
Distributed Monitoring Console
cleverdata.ru | [email protected]
Thank you!