Using  Puppet  To  Manage  Splunk  

Carl  Schwenk,  Senior  Systems  Administrator,  Citrix  Systems  

© Copyright Splunk 2011 2  The 2nd Annual Splunk Worldwide Users’ Conference

Introduc@on  

ü Carl  Schwenk  ü Senior  Systems  Administrator  ü Citrix  Online  ü Santa  Barbara,  CA  ü Carl.Schwenk@Citrix.com  

© Copyright Splunk 2011 3  The 2nd Annual Splunk Worldwide Users’ Conference

© Copyright Splunk 2011 4  The 2nd Annual Splunk Worldwide Users’ Conference

Splunk  @  Citrix  

NetScaller Load Balancer

Splunk Index

100+ Sourcetypes 10000+ Sources Custom Config Files Scripted input data Host & Product status data

© Copyright Splunk 2011 5  The 2nd Annual Splunk Worldwide Users’ Conference

Puppet   …is  not..   …is…  DriK  

Management  

A  config  file  transport  system   State  Enforcement  

Automated  Provisioning  

A  means  of  remotely  execu@ng  arbitrary  commands  

Rapid  deployment  and  configura@on  management.      

Adop@on  A  replacement  for  good  administra@on  prac@ces.    

Only  as  good  as  the  developer  that  runs  it.  

What  is  Puppet?  

© Copyright Splunk 2011 6  The 2nd Annual Splunk Worldwide Users’ Conference

Deployment  Manager   Puppet  Forwarder  Config  Management  

•  Manage  forwarders  by  classes  of  servers  

•  Uses  exis@ng  host  classifica@ons.    •  Automa@cally  provisioned  for  new  hosts    

Indexer  &  Search  Head  Management  

•  Manage  Indexer  and  Search  head  inputs  in  one  place.  

•  Manage  Splunk  servers  in  one  place.  •  Rapid  Splunk  scaling.    •  Configura@ons  are  backed  up  and  load  

balanced    

Forwarder  Running  Management  

•  U@lizes  the  Splunk  Deployment  Monitor  applica@on  to  alert  status  of  forwarders  

•  No  code  to  learn.  Easy  to  use  interface.  

•  Maintains  forwarder  running  state  •  Keeps  forwarder  updated  with  current  

config  

Why  Manage  Splunk  with  Puppet?  

© Copyright Splunk 2011 7  The 2nd Annual Splunk Worldwide Users’ Conference

Puppet  Code   class splunk::forwarder { File { owner => ‘splunk', group => ‘splunk', require => Package['splunkforwarder'], notify => Exec['splunk_first_time_run', 'splunk_restart'], } $splunk_home = "/opt/splunkforwarder" Package { "splunkforwarder":} ensure => latest service { "splunkforwarder": enable => true, ensure => running, require => [File['splunkforwarder-init'],Package['splunkforwarder']], } file { "${splunk_home}/etc/apps/${outputs}": ensure => directory, recurse => true, alias => 'outputs', source => "puppet:///modules/splunk/${outputs}", } file { "${splunk_home}/etc/apps/base_inputs": ensure => directory, recurse => true, source => "puppet:///modules/splunk/base_inputs", alias => 'base_inputs', } if $splunk_profile { $inputs = split($splunk_profile,",") define install_class_apps { file { "${splunk_home}/etc/apps/${name}": ensure => directory, recurse => true, source => "puppet:///modules/splunk/${name}", } } install_class_apps { $inputs:; } } exec { "${splunk_home}/bin/splunk start --accept-license": alias => "splunk_first_time_run", onlyif => "/usr/bin/test -e ${splunk_home}/ftr", require => Package["splunkforwarder"], } exec { "${splunk_home}/bin/splunk restart": alias => "splunk_restart", onlyif => "/usr/bin/test ! -e ${splunk_home}/ftr", refreshonly => true; } }

© Copyright Splunk 2011 8  The 2nd Annual Splunk Worldwide Users’ Conference

The  Foreman  –  Configura@on  Inheritance  

Global Configurations

All hosts get packages splunkforwarder

MySQL Host

Group

WWW Host

Group

$splunk_app = apache $splunkapp = mysql

Secure MySQL Host

$splunk_app = apache, backup

www-backup Host

$splunk_app = mysql, audit

© Copyright Splunk 2011 9  The 2nd Annual Splunk Worldwide Users’ Conference

The  Foreman  –  Dashboard  

© Copyright Splunk 2011 10  The 2nd Annual Splunk Worldwide Users’ Conference

Lessons  Learned  

ü Splunk  +  Puppet  =  BFF    ü Start  simply  ü Grow  slowly  ü Document  ü User  adop@on  may  be  your  hardest  challenge  

© Copyright Splunk 2011 11  The 2nd Annual Splunk Worldwide Users’ Conference

Puppet  Code  for  Splunk  4.2  

hdp://forge.puppetlabs.com  

ü Universal  Forwarder  code  coming  soon  ü Indexer  and  Search  head  code  to  come  soon  

Using  Puppet  To  Manage  Splunk  

Carl  Schwenk,  Senior  Systems  Administrator,  Citrix  Systems