Daniel Pittman - Platform Team Presentation - PuppetCamp LA '12
Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12
-
Upload
puppet-labs -
Category
Technology
-
view
1.490 -
download
0
description
Transcript of Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12
![Page 1: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/1.jpg)
Puppet @ Citygrid
PuppetCamp LA 2012
![Page 2: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/2.jpg)
Who am I ?
My name is Julien RottenbergRecovering System AdministratorPuppet user since 2009 (at GN)I like to write code and answer questions
![Page 3: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/3.jpg)
Puppet promises
No need to ssh to a server to understand how it works
Look at the puppet code
![Page 4: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/4.jpg)
"Those who cannot remember the past are condemned to repeat it."
George Santayana
literally... go to a server, run `history` as root.
![Page 5: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/5.jpg)
Puppet promises
Guarantee that a server is setup exactly the same way on all environments
![Page 6: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/6.jpg)
Puppet promises
![Page 7: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/7.jpg)
Puppet promises
All the building steps are auditable - SDLCDrift management Open knowledge, no blackbox
![Page 8: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/8.jpg)
Puppet promises
and finally...
![Page 9: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/9.jpg)
Toolset
Open source tools :
● No need to reinvent the wheel● You are not alone with 'that problem',
get help
![Page 10: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/10.jpg)
Toolset
![Page 11: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/11.jpg)
Toolset
puppet doc ● validate code (puppet parser validate)● produce html doc
Running on commit through Jenkins ~ 10 sec
![Page 12: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/12.jpg)
Toolset
![Page 13: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/13.jpg)
Toolset
puppet-lint == coding standard
Takes ~ 2 minPart of the 'Ci-Game'
![Page 14: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/14.jpg)
Toolsettrunk = work in progress - could break but fix ASAPproduction = tested code - do not break
![Page 15: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/15.jpg)
Toolset Trunk --> Production (CherryPicking)We use svnmerge.py
Code in progress = technical debtIt needs to be merged and deployed or reverted Fail the build if pending Cherry Picks > 30 Can't CherryPick yourself ~ code review
![Page 16: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/16.jpg)
ToolsetSvn even with svnmerge has limitations
Git ?
![Page 17: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/17.jpg)
ToolsetVagrant
Sits on top of VirtualboxAllows to 'go crazy'Test, experiment, share configured box$ vagrant provision == puppet apply
![Page 18: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/18.jpg)
ToolsetnVentory
As an External Node ClassifierClasses:
- default_install- gcd
![Page 19: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/19.jpg)
ToolsetnVentory
Helper to build configuration files, etc.ie replacement for Stored Config
![Page 20: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/20.jpg)
ToolsetForeman Api for puppet status, visibility of puppetrun
(Failed attempt on ENC - probably pebkac)
![Page 21: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/21.jpg)
Toolset
Man ! that's a lot of tools !1!!
And it's fine, design by contract. Each application does one thing and was designed to do that thing only (so does it well)Pipe them
Dennis Ritchie
![Page 22: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/22.jpg)
Toolset Boto for
● ec2● route53● elb● sqs● sns
River dolphin native to the amazon WP
![Page 23: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/23.jpg)
Culture cloudEC2 glitches, expect them It used to `just work` :
● gmond (split view dns side effect, using gmetric)
● jmx (need to specify public name at runtime)● ELB pseudo load balancer● Bad neighbors
![Page 24: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/24.jpg)
Culture cloud
![Page 25: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/25.jpg)
One AMI to rule them all (map set per region)
# Build 10 nodes $ create_ec2_instance.py -e dev -s m1.medium -c thd -n 10 # assign them to the hadoop::datanode class (specific install on top of our vanilla)$ nv --name aws1devthd --addtonodegroup hadoop::datanode # send traffic & enable monitoring (configs to be updated on the next puppet run)$ nv --name aws1devthd --set status=inservice
Culture cloud
![Page 26: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/26.jpg)
Culture Collaborate
![Page 27: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/27.jpg)
Culture Collaborate
"If the company's doing well and people don't hate each other, you're probably doing ok."
John Allspaw, speaking at Devopsdays 2011 MV.
![Page 28: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/28.jpg)
You guys ! Make it better : - Integrated function for 'secrets' (no get_var or hiera + gpg)Something like an encrypted data luggage
- puppet agent stability, puppetd vs cron ?
![Page 29: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/29.jpg)
Guys !
![Page 30: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/30.jpg)
Yesterday...
![Page 31: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/31.jpg)
Two last things...
On call rotation side effect
Don't be a human keyboard
![Page 32: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/32.jpg)
Shameless plug
We aggregate advertising & content from local businesses and distribute across our network of web & mobile sites
● Ingest : Hadoop & Cassandra● Serve : Solr & MongoDb + Tomcat● Code : Java mostly
![Page 33: Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12](https://reader033.fdocuments.in/reader033/viewer/2022051513/5478ecbab4795972098b4655/html5/thumbnails/33.jpg)
Q/A
puppet doc : http://docs.puppetlabs.com/guides/style_guide.html#puppet-docpuppet-lint : http://puppet-lint.com/Jenkins : http://jenkins-ci.org/svn strategy : http://weblogs.java.net/blog/johnsmart/archive/2009/05/ci_branching_an.htmlsvnmerge.py : http://www.orcaware.com/svn/wiki/Svnmerge.pyvagrant : http://vagrantup.com/nventory : http://sourceforge.net/apps/trac/nventory/wikiforeman : http://theforeman.org/boto : https://github.com/boto/boto Don't be an Ahole : http://www.hollenback.net/DevOpsMeansDontBeAnAholeDevelopers on call - Mitchell Hashimoto : http://vimeo.com/31367609Don't be a human keyboard : http://sysadvent.blogspot.com/2010/12/day-13-dont-be-human-keyboard.html Citygrid is hiring : http://careers.citygrid.com/ Puppet Master LA User Group : http://www.meetup.com/LAPMUG/
This page : goo.gl/soxEs