Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12!...
Transcript of Integrang*Splunk*Dataand* FuncBonality*Using*the*Splunk ... · The*Splunk*REST*API 12!...
Copyright © 2013 Splunk Inc.
Damien Dallimore Developer Evangelist @ Splunk #splunkconf
IntegraBng Splunk Data and FuncBonality Using the Splunk SDK for Java
Legal NoBces During the course of this presentaBon, we may make forward-‐looking statements regarding future events or the expected performance of the company. We cauBon you that such statements reflect our current expectaBons and esBmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-‐looking statements, please review our filings with the SEC. The forward-‐looking statements made in this presentaBon are being made as of the Bme and date of its live presentaBon. If reviewed aTer its live presentaBon, this presentaBon may not contain current or accurate informaBon. We do not assume any obligaBon to update any forward-‐looking statements we may make. In addiBon, any informaBon about our roadmap outlines our general product direcBon and is subject to change at any Bme without noBce. It is for informaBonal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligaBon either to develop the features or funcBonality described or to include any such feature or funcBonality in a future release.
Splunk, Splunk>, Splunk Storm, Listen to Your Data, SPL and The Engine for Machine Data are trademarks and registered trademarks of Splunk Inc. in the United States and other countries. All other brand names, product names, or trademarks belong to their respecCve
owners.
©2013 Splunk Inc. All rights reserved.
2
About Me
Developer Evangelist at Splunk Make Talk
4
Came from the Splunk Community
5
Coder
6
From Aotearoa (New Zealand)
7
Agenda
Agenda
! Developer Pla`orm Overview ! The Basics ! Digging a bit deeper ! IntegraBon Frameworks ! Tools, Libraries and other use cases ! QuesBons (if Bme allowing , else find me aTerwards)
9
Developer Pla`orm Overview
Powerful Pla`orm for Enterprise Developers
11
REST API
Build Splunk Apps Extend and Integrate Splunk
Simple XML
JavaScript
Django
Web Framework
Java JavaScript Python
Ruby C# PHP
Data Models
Search Extensibility
Modular Inputs
SDKs
The Splunk REST API
12
! Exposes an API method for every feature in the product – Run searches – Input data – Manage Splunk configuraBons
! API is RESTful – Endpoints are served by Splunkd – Requests are GET, POST, and DELETE HTTP methods – Responses are Atom XML Feeds or JSON – Search results can be output in CSV/JSON/XML/Raw – AuthenBcaBon is header token based
SDKs (SoTware Development Kits)
13
! Our SDKs make it easier to use the REST API – No need to wrangle with lower level HTTP REST semanBcs
! Several different language offerings now available
– Javascript, Java, Python, PHP, C#, Ruby
! Now includes support for more rapid development of Modular Inputs ! Typical Use Cases
– Accelerate your Dev/Test cycles – Integrate with exisBng soluBons – Build new big data applicaBons
The Basics
Get the Splunk SDK for Java
15
! Current release v1.2 ! Open sourced under the Apache v2.0 license ! Clone from Github : git clone hnps://github.com/splunk/splunk-‐sdk-‐java.git ! Project level support for Eclipse and Intellij IDE’s , or use the development
environment of your choice ! Pre-‐requisites
– JRE 6+ – Splunk installed – Ant
! Maven/Gradle/Ivy repository – hnp://splunk.arBfactoryonline.com/splunk/ext-‐releases-‐local
SDK Class Design
16
Service
HTTPService Resource
ResourceCollecBon EnBty
EnBtyCollecBon
Job JobCollecBon
BaseService
Args
JobResultsArgs
Code & Demo
! ConnecBng and authenBcaBng ! Hirng a simple REST endpoint ! Simple searches
– Oneshot Blocking Search – Simple Blocking Search
! Indexing an event
17
Digging a Bit Deeper
Code & Demo ! Searching (also demonstrates handling & displaying results )
– Asynchronous with Job object – RealBme Search – ExporBng – Paging through large result sets – Saved searches and dynamic dispatch arguments
! Indexing data – TCP – UDP – Index oneshot upload
! Managing Splunk (also demonstrates state management) – create a Role enBty – create a User enBty
! Namespaces
19
IntegraBon Frameworks
Let’s Make Life Even Easier
! Splunk SDK for Java makes using the REST API easier ! What can we do to make the SDK easier to use ? ! Integrate into standard enterprise Java soTware frameworks
– Spring IntegraBon (code & demo) – Apache Camel (code & demo) – Mule ESB
21
Tools, Libraries & Other Use Cases
Splunk Java Logging
23
! A logging framework to allow developers to as seamlessly as possible integrate Splunk best pracBce logging semanBcs into their code
! Transport log events to Splunk directly from your code ! Custom handler/appender implementaBons(REST and Raw TCP) for common Java logging frameworks ̶ LogBack ̶ Log4j ̶ java.uBl.logging
! UBlity classes for formarng log events ! Configurable in memory buffer to handle network outages
Eclipse Plugin
24
! Adds extensions to the Eclipse soTware development pla`orm to simplify creaBng projects using the Splunk SDK for Java
! A new "Splunk SDK for Java" project type ! A set of code templates for common tasks using the Splunk SDK for Java, plus opBonal data parsing and logging capabiliBes
! A launcher for Java applicaBons that logs JVM acBvity to a Splunk instance – JMX java.lang MBeans – Byte Code InjecBon method level tracing
Other JVM Languages
Scala Groovy Clojure
Javascript(Rhino) JRuby PHP(Quercus)
Ceylon Kotlin Jython
We don’t need SDK’s for these languages , we can just use the Java SDK !
25
HUNK (Splunk AnalyBcs for Hadoop)
! A new product offering from Splunk ! Allows you to use the power and simplicity of Splunk to search over data locked away in HDFS
! Sits on top of HDFS as if it was a naBve Splunk Index ! Virtual Indexes ! So you can use the Splunk SDK for Java to develop your Hadoop applicaBons in exactly the same way that you currently use it.
26
Housekeeping
Where to Go for More Info Twiner @splunkdev Blog hnp://blogs.splunk.com/dev Demos hnp://demos.splunk.com
Email [email protected] Portal hnp://dev.splunk.com Github hnps://github.com/splunk
28
Links Github Gists for code examples in this presentaBon : hnps://gist.github.com/damiendallimore SDK docs at dev.splunk.com : hnp://dev.splunk.com/view/SP-‐CAAAECN Splunk SDK for Java Github repository : hnps://github.com/splunk/splunk-‐sdk-‐java Splunk Spring IntegraBon repository on Github : hnps://github.com/SpringSource/spring-‐integraBon-‐extensions/tree/master/spring-‐integraBon-‐splunk Splunk Spring IntegraBon demo on Github : hnps://github.com/damiendallimore/spring-‐integraBon-‐splunk-‐webex-‐demo
Splunk Apache Camel repository on Github : hnps://github.com/pax95/camel-‐splunk Splunk Apache Camel demo on Github : hnps://github.com/pax95/camel-‐splunk-‐example Splunk Eclipse plugin : hnp://dev.splunk.com/view/splunk-‐plugin-‐eclipse/SP-‐CAAAEQP Splunk Java Logging on Github : hnps://github.com/splunk/splunk-‐library-‐javalogging
Splunk Java Agent on Github : hnps://github.com/damiendallimore/SplunkJavaAgent Splunk Android SDK on Github : hnps://github.com/damiendallimore/splunk-‐sdk-‐android Splunk REST API reference : hnp://docs.splunk.com/DocumentaBon/Splunk/latest/RESTAPI/RESTcontents Free Splunk download : hnp://www.splunk.com/get?r=header
Best pracBce logging overview : hnp://dev.splunk.com/view/logging-‐best-‐pracBces/SP-‐CAAADP6 Splunk SDK for Java videos : hnp://dev.splunk.com/view/get-‐started/SP-‐CAAAECH HUNK Beta video : hnp://www.splunk.com/view/SP-‐CAAAH2F Maven/Gradle/Ivy Repository : hnp://splunk.arBfactoryonline.com/splunk/ext-‐releases-‐local
29
Contact me
Email : [email protected] Twi6er : @damiendallimore Skype : damien.dallimore Github : damiendallimore Splunkbase : damiend Slideshare : hnp://www.slideshare.net/damiendallimore
I will be around all week , feel free to grab me for a chat or a beer or both J
30
Thank you !