PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet
Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013
-
Upload
puppet-labs -
Category
Entertainment & Humor
-
view
2.642 -
download
0
description
Transcript of Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013
![Page 1: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/1.jpg)
Sam Kottler | Puppet at Scale1
Puppet at ScaleSam Kottler@samkottler
![Page 2: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/2.jpg)
Sam Kottler | Puppet at Scale2
About me
● Worked on large-scale infra for the web @ Venmo, Acquia, and Digg
● Rubygems.org infrastructure
● Bundler core
● Fedora developer
● Core committer on the Foreman
![Page 3: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/3.jpg)
Sam Kottler | Puppet at Scale3
What we'll cover
1. Some basics
2. Master vs. masterless deployment
3. CA management
4. Clustering
5. Node management
6. Development + deployment practices
![Page 4: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/4.jpg)
Sam Kottler | Puppet at Scale4
Why we care
● Hyperscale computing
● Massive, multi-DC infrastructure
● Dynamic environments
● The Cloud ™
![Page 5: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/5.jpg)
Sam Kottler | Puppet at Scale5
Master vs. masterless
![Page 6: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/6.jpg)
Sam Kottler | Puppet at Scale6
Provisioning nodes with a master
1. New node comes online
2. A script is run to install packages and configure /etc/hosts
3. The agent gets run, generates a CSR, and sends it to the master
4. The cert gets signed based on an autosign rule or `puppet cert --sign <nodename>`
5. Puppet runs
![Page 7: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/7.jpg)
Sam Kottler | Puppet at Scale7
Provisioning nodes without a master
1. New node comes online knowing its role
2. A script runs to install packages and retrieve package/tarball
3. puppet apply
![Page 8: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/8.jpg)
Sam Kottler | Puppet at Scale8
Certificate authority
● Use by Puppet to authenticate agents
● CSR generated and signed by the CA
● Shared CRL across all CA machines
![Page 9: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/9.jpg)
Sam Kottler | Puppet at Scale9
Clustering patterns● CA has lots of state
● Masters should be stateless
● Reduce the number of file shares
![Page 10: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/10.jpg)
Sam Kottler | Puppet at Scale10
DNS-based clustering
![Page 11: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/11.jpg)
Sam Kottler | Puppet at Scale11
Load balanced clustering
![Page 12: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/12.jpg)
Sam Kottler | Puppet at Scale12
Masters across data-centers
● Shared CA vs. per-region
● Deploy in stages across data-centers
![Page 13: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/13.jpg)
Sam Kottler | Puppet at Scale13
Multi-cluster
![Page 14: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/14.jpg)
Sam Kottler | Puppet at Scale14
Node classification
![Page 15: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/15.jpg)
Sam Kottler | Puppet at Scale15
![Page 16: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/16.jpg)
Sam Kottler | Puppet at Scale16
External node classifiers
● Output YAML based on external data
● The Foreman, Puppet Enterprise, Puppet Dashboard
● Your own custom data source
● Key integration source with your own CMDB
![Page 17: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/17.jpg)
Sam Kottler | Puppet at Scale17
![Page 18: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/18.jpg)
Sam Kottler | Puppet at Scale18
Packaging for masterless
https://github.com/skottler/librarian-masterless-packaging
● Use /etc/puppet/modules (or modulepath)
● Build RPM's/deb's for distribution
● Publish packages to a repo
● Install/update packages on all machines
![Page 19: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/19.jpg)
Sam Kottler | Puppet at Scale19
Distributed runs● Run puppet based on changes in your code
● Mcollective/SSH/cron
![Page 20: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/20.jpg)
Sam Kottler | Puppet at Scale20
Deployment
● Masters are just another deployment target!
● Build CI pipelines
● One-click deployments to masters
● Lint and test your modules
![Page 21: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/21.jpg)
Sam Kottler | Puppet at Scale21
cap puppetmaster deploy DC=london
![Page 22: Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013](https://reader035.fdocuments.in/reader035/viewer/2022081400/554de2b4b4c905cc0e8b55dc/html5/thumbnails/22.jpg)
Sam Kottler | Puppet at Scale22
Controlled releases
● Separate hosts into groups to do red/black releases
● Build smaller sub-groups of canary hosts
● Monitor your puppet runs