SaltConf14 - Brendan Burns, Google - Management at Google Scale
-
Upload
saltstack -
Category
Technology
-
view
1.165 -
download
0
description
Transcript of SaltConf14 - Brendan Burns, Google - Management at Google Scale
![Page 1: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/1.jpg)
Google confidential │ Do not distribute
Management at Google ScaleConverging managed infrastructure between Google and the Cloud community
Brendan BurnsStaff Software Engineer
![Page 2: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/2.jpg)
Storage
Cloud Storage Cloud SQLCloud
Datastore
Compute
Compute Engine
App Engine
App Services
BigQuery Cloud Endpoints
Google Cloud Platform
![Page 3: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/3.jpg)
Google confidential │ Do not distribute
For the past 15 years, Google has been building the world’s fastest, most powerful, highest quality cloud infrastructure on the planet.
Images by Connie Zhou
![Page 4: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/4.jpg)
We’ve had some practice
![Page 5: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/5.jpg)
Declarative Management for sanity
Containers for idempotency and reproducibility
So, what have we learned?
Task Introspection (or how I learned to forget about SSH)
![Page 6: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/6.jpg)
A view into my life
• Google engineer for 6 years
• Search Infrastructure (Realtime Search, Google+ Search …)
• Cloud Infrastructure
• Build software to expect failure
• Never had [email protected], despite web search oncall for 4+ years
![Page 7: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/7.jpg)
Declarative Management for sanity
Containers for idempotency and reproducibility
So, what have we learned?
Task Introspection (or how I learned to forget about SSH)
![Page 8: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/8.jpg)
Imperative management leads to “Snowflake” Servers
Declarative Management
![Page 9: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/9.jpg)
Separate textual declaration from Physical (Virtual) Manifestation
Declarative Management
![Page 10: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/10.jpg)
Reasoning in a formal declaration (and version control) unlocks tremendous potential
Declarative Management
![Page 11: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/11.jpg)
Declarative configurations facilitate re-use
Declarative Management
![Page 12: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/12.jpg)
Declarative Management for sanity
Containers for idempotency and reproducibility
So, what have we learned?
Task Introspection (or how I learned to forget about SSH)
![Page 13: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/13.jpg)
Google has a long history with containers (Process GGroups, LMCTFY [https://github.com/google/lmctfy])
Of late, there has been a great deal of external interest as well.
Containers
![Page 14: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/14.jpg)
Google has a long history with containers (Process CGroups, LMCTFY [https://github.com/google/lmctfy])What containers are good for?
Containers
![Page 15: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/15.jpg)
Declarative Management for sanity
Containers for idempotency and reproducibility
So, what have we learned?
Task Introspection (or how I learned to forget about SSH)
![Page 16: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/16.jpg)
(or how I learned to forget about SSH)C
ontainers don’t really have SSH (well, they can, but…)
Still want containers to be self-contained
Introspection
![Page 17: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/17.jpg)
?
?
There’s an exciting road ahead...
![Page 18: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/18.jpg)
Eric Johnson’s talk
I’ll be at the Google booth
Walk up and say “Hi”
Resources/Contact
![Page 19: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/19.jpg)
Thomas Hatch, SaltStack CTO
![Page 20: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/20.jpg)
The Top Six Things You Didn’t Know About SaltStack
![Page 21: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/21.jpg)
1. Fast, flexible comms protocol
• SaltStack provides options• Different solutions for different problems• Flexibility and plug-ability• ØMQ
– Super fast• SSH
– For certain use cases– 50x faster than other other SSH-based tools
• RAET
– UDP or TCP– Even faster– More control over job queuing and prioritization– More infrastructure visibility
![Page 22: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/22.jpg)
2. Salt Virt
• Doesn’t get much attention• Salt originally designed as a
cloud controller (Butter)• A completely different approach
to cloud management– Database free– Evolving but being used in production
![Page 23: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/23.jpg)
3. Declarative or imperative? Yes.• Stick a fork in this debate• Most flexible configuration management• Finite order execution is a core Salt
design principle• 0.17 introduced more state ordering
choice• Compiler and run time
– Salt modularity– No sacrifice or compromise of speed
![Page 24: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/24.jpg)
4. Generic device automation
• Minion proxy for network devices (Juniper, Arista, Broadcom, F5, etc.)
• Not just executing CM routines• Finite device control w/ remote execution• Easy to communicate with and control these
typically dumb devices• Stateful configuration and one-off queries• Integrated with standard Salt workflows and
methodologies
![Page 25: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/25.jpg)
5. The Salt test suite
• More stable Salt releases• Pedro Algarvio!• Running lives tests constantly on real infra
– Jenkins– Spinning up VMs on Rackspace to run tests– Hooked into Docker containers
• PyLint coverage (thx Hulu & LogiLab)• Test coverage doubled in three months
![Page 26: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/26.jpg)
6. The SaltStack name
• Not SLC• FLOSS Weekly
realization• Gimli, son of Gloin• Ubiquitous nature of Salt
![Page 27: SaltConf14 - Brendan Burns, Google - Management at Google Scale](https://reader036.fdocuments.in/reader036/viewer/2022081602/554f8f64b4c905d25b8b511c/html5/thumbnails/27.jpg)
Thank You