PHP  &  Fluentd

@chobi_e

chobi_e  is:    PECL  Sundown  Author  (will  be  release  soon)  h>ps://github.com/chobie/php-­‐sundown  

libgit2  PHP  Binding  Author  h>ps://github.com/libgit2/php-­‐git  

phpredis  contributor  

h>ps://github.com/nicolasff/phpredis  

Scaling  web  servers.

Scaling  web  servers.  basically,  just  add  web  server  to  the  array.

then,  how  about  collecLng  log  files?

using  log  file  

using  log  file   Are  you  serious?

using  syslog  like  middleware

using  syslog  like  middleware Not  Bad.

using  reliable  and  distributed  log  collector:  

Note:  fluent-­‐logger-­‐php  does  not  support  output  buffering.  So  local  fluentd  as  a  buffer  in  this  case.

using  reliable  and  distributed  log  collector:  

that’s  the  best  soluLon  IMHO

Note:  fluent-­‐logger-­‐php  does  not  support  output  buffering.  So  local  fluentd  as  a  buffer  in  this  case.

fluentd  provides  robust  messaging  soluLon

Data  Mining

for  Data  Mining

Data  Mining

Queued  job

for  queued  job

fluentd  first  step  with  PHP

install  by  composer

#  composer.json  {      "name":  "my-­‐project",      "version":  "1.0.0",      "require":  {          "fluent/logger":  "master-­‐dev"      }  }

wget  h>p://getcomposer.org/composer.phar  php  -­‐d  detect_unicode=0  composer.phar  install

<?php  require  "vendor/.composer/autoload.php";    use  Fluent\Logger\FluentLogger;    $logger  =  FluentLogger::open(        "debug.test",        "localhost",        "24224”  );

install  by  copy  the  tree.

git  clone  h>ps://github.com/fluent/fluent-­‐logger-­‐php.git  cp  -­‐r  src/Fluent  <path/to/your_project>

php  library  which  using  Fluentd.

Monolog\Fluent  by  nrk  h>ps://github.com/nrk/monolog-­‐fluent.git

what  is  Monolog?    Monolog  by  Seldaek.  h>ps://github.com/Seldaek/monolog      Monolog  is:  heavily  inspired  by  Python's  Logbook  library,  although  most  concepts  have  been  adjusted  to  fit  to  the  PHP  world.    

install  $  wget  h>p://getcomposer.org/composer.phar  $  php  composer.phar  install    

<?php  use  Nrk\Fluent\Monolog\FluentHandler;    $log  =  new  Monolog\Logger('debug.monolog');    $log-­‐>pushHandler(new  FluentHandler('h>p://127.0.0.1'));  $log-­‐>pushHandler(new  FluentHandler('tcp://127.0.0.1'));    $log-­‐>addError("OH  NOES!!11!1!");

introduce  my  fluent  plugin

fluent-­‐plugin-­‐delayed    -­‐  delayed  output  plugin.    h>ps://github.com/chobie/fluent-­‐plugin-­‐delayed  

fluent-­‐plugin-­‐delayed    -­‐  delayed  output  plugin.    h>ps://github.com/chobie/fluent-­‐plugin-­‐delayed  

delayed  plugin

event{    Lme:  2011-­‐10-­‐22  10:00:05,  #  5  seconds  future    ...  }  

Client

:  aler  5  seconds.  then  delayed  plugin  outputs  the  data  to  other  output  plugin.  

output

other  Ideas.    fluent-­‐plugin-­‐cooldown    -­‐  coolodowned  outputs  when  specified  key  received.    fluent-­‐plugin-­‐sampler    -­‐  only  outputs  N  %  data.  

GitHQ  /  Kokuban  Github  /  gist  clone  (sLll  under  development)

updaLng  ssh  key    is    pain  in  ass. /home/git/.ssh/authorized_keys

web  server fluentd cooldown exec client

redis update  authorized_keys  

send  public  key

sending  message  via  fluentd\Logger

only  pass  cooldowned  message.

updaLng  SSH  Authorized  keys

obtain  ssh  public  keys

Web  developer  only  knows  tag  for  queued  job.

Web  developer  only  knows  tag  for  queued  job.

Server  engineer  only  implements  reliable  consumer  plugin  for  Fluentd.

Sounds  good?

fluent-­‐logger-­‐php  status.

fluentd  protocol  support  ✔  h>p  protocol  support  ✔  composer  support  ✔  PEAR  package  support  ✔  -­‐  but  I  don’t  provide  pear  channel.    

 message  pack  support  buffering  support  -­‐  does  not  support  as  there’s  trouble  some  for  using  persistent  variables  for  buffer.    

That’s  all  

That’s  all  i  need  more  developer!  please  feel  free  to  pull  req  me.    h>ps://github.com/chobie