© 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director,...

32
© 2013 Splunk Inc Introducing Hunk Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing [email protected]

Transcript of © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director,...

Page 1: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

© 2013 Splunk Inc.

Introducing Hunk ™ Splunk Analytics for Hadoop

Brett SheppardDirector, Big Data Product [email protected]

Page 2: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Safe Harbor Statement

2

During the course of this presentation, we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.

Page 3: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

The Accelerating Pace of DataVolume | Velocity | Variety | Variability

GPS,RFID,

Hypervisor,Web Servers,

Email, Messaging,Clickstreams, Mobile,

Telephony, IVR, Databases,Sensors, Telematics, Storage,

Servers, Security Devices, Desktops

Machine data is the fastest growing, most complex, most valuable area of big data

3

Page 4: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

4

Make machine data accessible, usable and valuable to everyone.

Page 5: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Company (NASDAQ: SPLK)

Business Model / Products

Customers 6000+

founded2004first software release2006

HQ San Francisco

On-premise

In the cloud

SaaS

60+

100

of the Fortune 100

Largest license:

Terabytes/day

Splunk Company Overview

5

Page 6: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

ITOperations

Security and Compliance

Digital Intelligence

App Dev and

App Mgmt.

Developer Platform (REST API, SDKs)

Business Analytics

Industrial Data and Internet of

Things

Small Data. Big Data. Huge Data.

Delivers Value Across IT and the Business

6

Page 7: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Easy storage but hard analytics: difficult to explore, analyze, visualize

Complex technology: many open source projects

Hard-to-staff skills: must write MapReduce jobs or fixed schemas

7

Hadoop (MapReduce

& HDFS)

YARNDataFu

Hive

Mahout PigSqoop

Wide Range of Open SourceProjects for Hadoop Analytics

Azkaban

Getting Value from Data in Hadoop is Challenging

Page 8: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

What Does Gartner Say?

8

TROUGH OF DISILLUSIONMENT

TECHNOLOGY TRIGGER

PEAK OF INFLATED

EXPECTATIONS

SLOPE OF ENLIGHTENMENT

PLATEAU OF PRODUCTIVITY

VISIBILITY

TIME

My most advanced Hadoop clients are also getting disillusioned … The only consistent success, reported by my clients, is with Splunk.

Svetlana Sicular, Gartner Research Director, January 22, 2013

“ “

Many Hadoop custome

rs

8

Page 9: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

We Began to Address This Challenge

9

>>>>

Real-time Collection and

Analysis

Dashboards, Reports,

Access Controls

Splunk Hadoop Connect• Bi-directional data transfer

>> Splunk App for Hadoop Ops• Troubleshoot and monitor

Page 10: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

© 2013 Splunk Inc.

10

New product from Splunk delivers interactive data exploration, analysis and visualizations for Hadoop

Introducing Hunk™Splunk Analytics for Hadoop

Page 11: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Integrated Analytics Platform for Hadoop Data

11

Full-featured, Integrated Product

Insights for Everyone

Works with What You Have Today

Explore Visualize Dashboards Share

Hadoop (MapReduce

& HDFS)

Analyze

11

Page 12: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Validation from Partners

12

“I'm super excited about Hunk. Hunk is solving one of the top issues that our customers have – access to the skills and know-how to leverage the data inside of Hadoop. Splunk has a very beautiful UI that is very easy to learn. So it bridges that gap and makes it very easy to access the data inside of Hadoop."

"Hunk will help Hortonworks customers explore, analyze and visualize data in Apache Hadoop, driving more intelligent decisions across the entire organization."

"The fact that Splunk is bringing ease-of-use to sophisticated Hadoop problems is welcome from every angle. The power of Hunk comes across in how easy it is to just plug it in, throw it in there, and suddenly you have all of your answers. I wish every product worked this nicely.”

Page 13: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Explore, Analyze and Visualize Data On-the-fly

Virtual Index Schema-on-the-fly Flexibility and Fast Time to Value

• Enables seamless use of the Splunk technology stack on data wherever it rests• Handles MapReduce

• Structure applied at search time• No brittle schema • Automatically find

patterns and trends

• Interactive search• Preview results while

MapReduce jobs run• Drag-and-drop analytics

13

Page 14: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Derive Actionable Insights from Raw Data

14

HadoopStorage

Immediately start exploring, analyzing and visualizing raw data in Hadoop

1 2Point Hunk at Hadoop Cluster Explore Analyze Visualize Dashboards Share

Page 15: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Extract to in-memory store

Challenges With Alternative Approaches

Need to know MapReduceWait for slow jobs to finishNo interactive exploration

Pre-defined fixed schemaNeed knowledge of dataMiss data that “doesn’t fit”

Data too big to moveLimited drill down to raw dataAnother data mart

“Do it yourself” Hadoop / Pig

Problems

OPTION 1 Hive or SQL on Hadoop

Problems

OPTION 2

Problems

OPTION 3

15

Page 16: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Powerful Analytics Anyone Can Use – Now on Hadoop

Enables non-technical users to build complex reports without learning the search language

Provides more meaningful representation of underlying raw machine data

Preview results and interactively search across one or more Hadoop clusters

Pivot

Data Model

Interactive Search

16

Page 17: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Empowering Business and IT Stakeholders

Enterprise Architect• Adapt your architecture for big data• Hadoop shared-service departments

offer self-service analytics• Free data scientists for custom

analytics, not be data butlers

Business Analyst Developer• Save time by just pointing at Hadoop • Avoid fixed-schemas and low-level tooling• Answer questions iteratively without

waiting for MapReduce jobs to finish

• Build scalable big data apps on top of data in Hadoop

• Use the development languages and tools you know and like

PivotData Model

Development Environment

Interactive Search

17

Page 18: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Fast Deployment and Configuration

Just point at Hadoop

• Certified integration with all major Hadoop distributions

• Choose 1st-gen MapReduce or YARN

• Create Virtual Indexes across one or more clusters

• From download to searching data in < 60 minutes

Connect to one or multiple Hadoop clusters

YARN certified

18

Page 19: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Connect Hunk to HDFS and MapReduce

Connect to Apache HDFS and MapReduce or your choice of Hadoop distribution

Hadoop Cluster 1

19

Page 20: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Hunk Scales With Your Hadoop Deployments

Connect Hunk to multiple Hadoop clusters

Hadoop Cluster 3

Hadoop Cluster 2

Hadoop Cluster 1

20

Page 21: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Search and Explore from One Place

Rapidly interact with data

• Powerful Search Processing Language (SPL™)

• Ad-hoc exploratory analytics across massive datasets

• Preview results• No fixed schemas• No requirement to

“understand” data upfrontDrill down to raw data

Search interface

Pause or stop MapReduce jobs

Preview results

21

Page 22: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Powerful, Easy-to-use Analytics

Pivot• Drag-and-drop interface

enables anyone to analyze raw, unstructured data

• Build complex queries and reports without learning search language

• Click to visualize any chart type; reports dynamically update when fields change

Select fields from data model

Time window

All chart types available in the chart toolbox

Save report to share

22

Page 23: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Define Relationships in Big Data

Data Model• Describes how underlying

machine data is represented and accessed

• Defines meaningful relationships in the data

• Enables single authoritative view of underlying raw data

Hierarchical object view of underlying data

Add constraints to filter out events

23

Page 24: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Visualize and Share Data with Role-based Security

Build and personalize• Rapidly build advanced graphs

and charts on-the-fly • Combine charts, views and

external data in dashboards and reports

• View and edit on any desktop or mobile device

• Drill down to raw data• Protect data with role-based

access controls

24

Page 25: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Build Big Data Apps on Top of Hadoop

Pick your favorite tools• Use a standards-based web

framework and REST API

• Customize dashboards and UIs with Simple XML, JavaScript or Django

• Choose among SDKs for Java, JavaScript, Python, Ruby, C# and PHP

REST API

Build Big Data Apps Extend and Integrate Hunk

Simple XML

JavaScript

Django

Web Framework

JavaJavaScriptPython

RubyC#PHP

Data Models

Search Extensibility

SDKs

Hadoop (MapReduce & HDFS)

25

Page 26: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Drive Value Across the Enterprise

Financial Risk Management

Multi-Channel Retail Management

Unlock the value of big data in Hadoop to address business challenges

Synthesize Data from all Customer Touch Points – 360° View

26

Page 27: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

27

Multi-Channel Retailer Otto Group

Analysts can more quickly explore data and create visualizations for in-store inventory

Sales operations can see the big picture and drill down to individual SKUs

Corporate strategists can access market conditions for 400 stores in 20 countries

Page 28: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

28

More, higher-complexity risk calculations

Analysis showed the level of core Tier 1 capital ratio that the bank needs to hold against its balance sheet given their current risk profile

RISK MANAGEMENT AT MAJOR GLOBAL BANK

Petabytes of seemingly “random numbers” in Hadoop

Page 29: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

29

MORE COMPLETE CUSTOMER VIEWFOR FASHION RETAILER

Raw data in Hadoop:

Apache web logs,

ecommerce site

activity, Akamai image

hosting logs, Squid

proxy logs

Analyze this

massive, diverse

data sets in

Hadoop

Obtain a near

360 degree view

of customers

Page 30: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Demo

30

Page 31: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Explore, analyze and visualize data in Hadoop from one integrated platform

Simply point Hunk at your Hadoop cluster and start exploring data immediately

Interact with data, change perspectives and preview results as MapReduce jobs run

Hunk™: Splunk Analytics for Hadoop

INTERACTIVE SEARCH

RICH DEVELOPER ENVIRONMENT

Build big data apps on data in Hadoop using standard web languages and frameworks

FULL-FEATUREDANALYTICS

FAST TO DEPLOY AND DRIVE VALUE

31

Page 32: © 2013 Splunk Inc. Hunk Introducing Hunk ™ Splunk Analytics for Hadoop Brett Sheppard Director, Big Data Product Marketing data@splunk.com.

Thank Youwww.splunk.com/hunk