"Ops Tools with Perl" 2012/05/12 Hokkaido.pm
-
Upload
riywo -
Category
Technology
-
view
3.293 -
download
0
description
Transcript of "Ops Tools with Perl" 2012/05/12 Hokkaido.pm
OpsToolswith Perl
Hokkaido.pm #72012/05/12
riywo
Self Introduction
Ryosuke IWANAGA
a.k.a. riywo(りーお)
DeNA(2009~)
Mobage Server-side
Ops-Engi / DBA / Manager
Acknowledgement
Thanks for JPA and Hokkaido.pm :)
地域PM向け講師派遣支援プログラム
http://japan.perlassociation.org/services/send-a-monger
But...Am I a Perl Monger?
I’m not a Programer
Mobage uses Perl heavily but I have never written the business logic...
But, I read Perl heavily
Because I should cope with all system trouble including application bugs ;P
I wrote some operation tools with Perl
And I also use some Perl tools
But...Am I a Perl Monger?
So...
I’ll try to talk about my knowledge, thought, and tools as much as possible
しょぼくてごめんなさい><
Perl Ops-ToolsIntroduction
kazeburo-ware
CloudForecast
http://www.slideshare.net/kazeburo/introduction-to-cloudforecast-yapcasia-2010-tokyo
CloudForecastServer Resource Collecting/Monitoring
Easy install
git clone & cpanm --installdeps .
SNMP / RRDs is needed
LightWeight - about 5,000 lines (*.pm)
WAF: Shirahata.pm
See alsoリソースモニタリングツール「CloudForecast」入門 - As a Futurist...( http://blog.riywo.com/2011/02/27/043646 )
GrowthForecast
http://blog.nomadscafe.jp/2011/12/growthforecast.htmlhttp://blog.nomadscafe.jp/2012/02/growthforecast1json.htmlhttp://d.hatena.ne.jp/tagomoris/20120218/1329558305
GrowthForecast
API and Graph Render of RRDTool
Easy install
git clone & cpanm --installdeps .
RRDs is needed
LightWeight - about 1,800 lines (*.pm)
WAF: Kossy.pm
See also
GrowthForecastというグラフ表示ツールで捗る話 - blog.nomadscafe.jp( http://blog.nomadscafe.jp/2011/12/growthforecast.html )
**Alien::RRDtool**RRDs is not CPAN module...
Alien::RRDtool Install rrdtool as CPAN module
extlib/lib/perl5/{archname}/RRDs.pm
extlib/lib/perl5/auto/share/dist/Alien-RRDtool/
Especially useful for perlbrew
* Developed by gfx
See alsoAlien::RRDtool - RRDtoolをCPANから入れる - Perl Advent Calendar Japan 2011 Hacker Track( http://perl-users.jp/articles/advent-calendar/2011/hacker/2 )
chase-tail
chase-tail
$ tail -f error_log | chase-tail -l 10 -t various_error
One sheet perl script
Highlight some keywords
Detect fast flooding of log
See also
垂れ流されるログのおしりを追いかける - (ひ)メモ( http://d.hatena.ne.jp/hirose31/20120120/1327025989 )
@hirose31
App::Ikachan
App::Ikachan$ curl -F channel=\#yappo -F message=oppai http://hoge.com/notice
HTTP API of IRC posting
use it from any languages
Easy install (git clone & cpanm)
See also
YappoLogs: App::Ikachan - 様々なサーバのバッチ処理の結果等を IRC のチャンネルに通知するサーバ( http://blog.yappo.jp/yappo/
archives/000760.html )
@Yappo
Percona Toolkit
Percona Toolkit
Mixed tools - Maartkit and Aspersa
Almost all tools are written in Perl
System utility / MySQL Ops
“pt-query-digest” is quite efficient for performance tuning of MySQL server
See also
Percona Toolkit - Percona Software( http://www.percona.com/software/percona-toolkit/ )
@Percona
Touryo
“Touryo” is configuration management tool
I needed a Configuration Management Tool
Chef is good but difficult to customize
So, I started to think a plan of Touryo
Simple and Lightweight
Testable
able to write DSL in Perl freely
How to write DSL in Perl?
I didn’t know it X(
I asked gfx(@__gfx__) who is a great Perl Monger and works at DeNA
use strict;use warnings;
Hoge->do_test;
#---------------------package Hoge;
my $func;sub Hoge::test(&) { my $block = shift; $func = $block;}
sub do_test { package Hoge; do "dsl.pl"; $func->("DSL");}
[dsl.pl]
test { my $hoge = shift; print "Hello $hoge!\n";};
What can Touryo do?
crontabsysctlrpms
apachefastcgi
app
web01$ touryo test web01
$ touryo run web01FAIL?
- “touryo test host” - check the state of the host - correctly installed? - correctly configured?
- “touryo run host” - setup to the “correct” state
-> Then, what is the “correct” state?
crontabsysctlrpmsmysql
backup
crontabsysctlrpmsmysql
db01 db02
admin:touryo
game1:db:master
game1:web
game1:db:backup
crontabsysctlrpmsmysql
How to define the “correct” state? (1)
crontabsysctlrpms
apachefastcgi
app
backup
crontabsysctlrpmsmysql
web01
db01 db02
role: game1:web blueprint: - apache - fastcgi - app
game1:db:master
role: game1:db:backup blueprint: - backup
game1:web
game1:db:backup
role: game1:db blueprint: - mysql
role: game1 blueprint: - crontab - sysctl - rpms
admin:touryo
Touryo config
- Touryo use “blueprint” config - means what should be installed- The config defined by “role” - should be defined the role of hosts - inherited searching - separated by comma
crontabsysctlrpmsmysql
What is the “blueprint”?
crontabsysctlrpms
apachefastcgi
app
backup
crontabsysctlrpmsmysql
web01
db01 db02
game1:db:master
game1:web
game1:db:backup
blueprint/crontab/blueprint.plbootstrap { my $c = shift; $c->add_attribute({ cron_hour => 5, cron => 'root.tx', });};
test { my $c = shift; is_diff $c->sudo_ssh->crontab, $c->template('cron')->content; done_testing;};
run { my $c = shift; $c->sudo_ssh->crontab($c->template('cron'));};
blueprint/crontab/root.tx0 <: $cron_hour :> * * * /path/to/command
- blueprint locate the directory - named the blueprint name- contain “blueprint.pl” - Perl DSL for Touryo - able to use template engine - Text::Xslate::Syntax::Kolon - using “attribute” valuables - have other useful functions - sudo_ssh, template, crontab - etc...
admin:touryo
crontab blueprint
crontabsysctlrpmsmysql
What is the “attribute”?
crontabsysctlrpms
apachefastcgi
app
backup
crontabsysctlrpmsmysql
web01
db01 db02
game1:db:master
game1:web
game1:db:backup
blueprint/crontab/blueprint.plbootstrap { my $c = shift; $c->add_attribute({ cron_hour => 5, cron => 'root.tx', });};
blueprint/crontab/root.tx0 <: $cron_hour :> * * * /path/to/command
- “attribute” can be used to change the behavior of test/run - changing logic, template, etc..- overwriting of “attribute” - bootstrap - role config - defined for the role - node config - defined for the node(host)
admin:touryo
0 9 * * * /path/to/command
Touryo Configrole: game1:web attribute: cron_hour: 9
0 5 * * * /path/to/commanddefault rendering
If defined attribute onthe Touryo Config ...
How to define the “correct” state? (2)
crontabsysctlrpms
apachefastcgi
app
web01
role: game1:web blueprint: attribute: - apache cron_hour: 9 - fastcgi somaxconn: 10240 - app fcgi_proc: 100
game1:web
role: game1 blueprint: attribute: - crontab somaxconn: 1024 - sysctl - rpms
Touryo config
node: web02 blueprint: attribute: - git fcgi_proc: 10admin:
touryo
crontabsysctlrpms
apachefastcgi
app
web02
game1:web git
cron_hour: 9somaxconn: 10240fcgi_proc: 100
cron_hour: 9somaxconn: 10240fcgi_proc: 10
- You can vary correct “blueprint” and “attribute” by Touryo config - role config and node config
Lightweight/Testable/Freedomfind lib/ -name "*.pm" | xargs cat | wc -l
about 1500 (including html templates)
Like test scripts of Perl Software, you can test the servers’ configuration
you can write anything on blueprint.pl
Conversely you have to write anything
Other Featuresin terms of Programming
use Amon2
easy to develop Web Interface and CLI
use TAP::Parser to get the test result
test function uses Test::More
use IPC::Cmd and “ssh” / “sudo ssh”
easy to configure with .ssh/config or others
blueprint dependance resolving
easy to cascade the config
What language should I use?
There are many languages
script / compile
Perl,Ruby,Python,PHP / C,C++,Java
procedural / OO / functional
Perl,C / Java,Ruby / Scala,Haskell
popular / minority
???(religious problem...)
My ThoughtYou can use any languages you like
Engineer have to treat multi languages
Development speed / Maintenance cost
They are usually trade-off
Popular LLs are often suited for both
Simple / Light / Thin / Useful Modules
Everyone can understand it easily
Perl/Ruby/Python are similar
I ♡ Perl
Fastest to develop for me
I am used to Perl because DeNA use Perl
There are many Perl Mongers around :)
colleague and friends
CPAN has many useful modules
long history and super hackers
Finally...
DeNA ♡ EngineersWe need more powerful engineers
We use Perl and other languages
Ruby/JavaScript/C/C++/Java/Objective-C
SocialGame/Platform/SDK/Data mining
Or let’s make very useful tools with me :)
Server Ops/Database Admin/Network
If you are interested in us, pleeeeeeeeease contact me or DeNA staffs!
hirose31, gfx, hidek, zigorou, nekokak...
Thanks!http://blog.riywo.com
http://twitter.com/riywo
Thanks for JPA, @kazeburo, @hirose31, @Yappo, @Percona,and All Perl Mongers!