Container Orchestration (with Kubernetes) Marketing
Transcript of Container Orchestration (with Kubernetes) Marketing
![Page 1: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/1.jpg)
Container Orchestration (with Kubernetes)Marketing
Peter Chen
1
![Page 2: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/2.jpg)
About Me
● 2014-2016: grad student at UBC (Ivan was my supervisor)● 2016-2019: Arista Networks● 2019-Present: Google
2
![Page 3: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/3.jpg)
Container Orchestration
● What are containers?
● What is container orchestration? What problem does it solve?
● How it relates to concepts you’ve learned in distributed systems?
3
![Page 4: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/4.jpg)
Container Orchestration - Why we care?
● 2,429 companies use Kubernetes, a container orchestration system we will talk about, including (Google, Facebook, Shopify, …) to run their internal systems and to power their Cloud offerings
○ https://stackshare.io/kubernetes
● Large open source community, as well as backed by big companies such as Docker and Google
● Gartner report adoption of containers grew 40% in 2020, by 2023, 70% of all organizations will be running containers in some form (in 2019 this was less than 20%)
○ Become the de-facto way to deploy, run and build services
4
![Page 5: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/5.jpg)
Reminder: Processes
Operating System
5
![Page 6: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/6.jpg)
Reminder: Processes
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Applications run as the process abstraction
Pro
cess
A
Pro
cess
B
Pro
cess
C
Each process has its own memory space and therefore its own execution context
6
![Page 7: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/7.jpg)
More Info: Processes
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
But there are other things these process share in the operating system:
● PID (process ids)● MNT (file system)● IPC (sockets)● UTS (time?)● NET (network e.g., IP tables)● etc,...
7
![Page 8: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/8.jpg)
Containers
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
8
Nice things about containers, they are:
● Lightweight: fast, very little overhead● Isolation: executable package of
software with its own code, runtime, tools, libraries and settings
● Portable: compiled into an “image” which can be deployed on other machines as a “container” instance
![Page 9: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/9.jpg)
Containers
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
This is a bit different to what you are use to in VMs where the objective is virtualization of hardware resources instead of just isolation of all the aspects of execution.
9
Nice things about containers, they are:
● Lightweight: fast, very little overhead● Isolation: executable package of
software with its own code, runtime, tools, libraries and settings
● Portable: compiled into an “image” which can be deployed on other machines as a “container” instance
![Page 10: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/10.jpg)
Containers
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Cost of containers: 5 MB for smallest image, arbitrary amount of CPUCost of VMs: (~2GB) for smallest OS, compute cost in increments of 1 CPU
10
Nice things about containers, they are:
● Lightweight: fast, very little overhead● Isolation: executable package of
software with its own code, runtime, tools, libraries and settings
● Portable: compiled into an “image” which can be deployed on other machines as a “container” instance
![Page 11: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/11.jpg)
Containers
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
This is a bit different to what you are use to in VMs where the objective is virtualization of hardware resources instead of just isolation of all the aspects of execution.
11
Nice things about containers, they are:
● Lightweight: fast, very little overhead● Isolation: executable package of
software with its own code, runtime, tools, libraries and settings
● Portable: compiled into an image
![Page 12: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/12.jpg)
Container Orchestration
12
Recap:
● What is the relationship between an “image” and a “container”?
● What does running in a container isolate vs. say a VM?
● What are some of the benefits?
![Page 13: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/13.jpg)
Container Orchestration
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
13
![Page 14: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/14.jpg)
Kubernetes
14
![Page 15: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/15.jpg)
Kubernetes - Architecture
● Abstractions: Pods, Services, Ingress, Deployments, Volumes…
● Add-ons: Istio, Knative...
● Network: Weave, Flannel…
● Control Plane: scheduler, api-server, controller-manager, etcd...
15
![Page 16: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/16.jpg)
Kubernetes - Architecture
● Abstractions: Pods, Services, Ingress, Deployments, Volumes…
● Add-ons: Istio, Knative...
● Network: Weave, Flannel…
● Control Plane: scheduler, api-server, controller-manager, etcd...
Too many big words that have too little meaning
16
![Page 17: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/17.jpg)
Kubernetes - Execution
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer
17
![Page 18: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/18.jpg)
Kubernetes - Execution
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
18
![Page 19: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/19.jpg)
Kubernetes - Execution
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
19
Pod
![Page 20: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/20.jpg)
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Kubernetes - Execution
20
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
![Page 21: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/21.jpg)
Kubernetes - Execution
● Fate sharing: e.g., processes in a pod live/die together (a webserver and its local SQL instance
21
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
![Page 22: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/22.jpg)
Kubernetes - Execution
● Fate sharing: e.g., processes in a pod live/die together (a webserver and its local SQL instance
● Fault tolerance and scalability: multiple pods can become replicas and execute on different nodes dynamically
22
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
![Page 23: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/23.jpg)
Kubernetes - Execution
● Fate sharing: e.g., processes in a pod live/die together (a webserver and its local SQL instance
● Fault tolerance and scalability: multiple pods can become replicas and execute on different nodes dynamically
● Multi-tenancy: multiple pods can run on a single node, provide scalability and efficient use of cluster-wide resources
23
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
![Page 24: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/24.jpg)
Kubernetes - Execution
● Fate sharing: e.g., processes in a pod live/die together (a webserver and its local SQL instance
● Fault tolerance and scalability: multiple pods can become replicas and execute on different nodes dynamically
● Multi-tenancy: multiple pods can run on a single node, provide scalability and efficient use of cluster-wide resources
24
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
Question: For Cloud Providers, why is resource efficiency related to availability?
![Page 25: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/25.jpg)
Kubernetes - Execution
25
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
Do you manually create/delete pods? That would be a huge hassle.
![Page 26: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/26.jpg)
Kubernetes - Execution
Pods are controlled usually via higher level abstractions (e.g., Deployment, Jobs)
Deployment
● declares the number of pods and what to execute in them
● Maintains that number of pods forever
● To scale up, create another deployment with new number of pods, similarly to scale down
Jobs● Pod that runs a single time until end
of execution and then is deleted● Timeout 26
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
Do you manually create/delete pods? That would be a huge hassle.
![Page 27: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/27.jpg)
Kubernetes - Execution
Answer: another layer of abstractions! Knative!
Knative is a Kubernetes Add-On● Add-on is a fancy word for a bunch
of kubernetes abstractions packaged together
● Auto-scales the number of pods according to traffic/demand
27
Operating System
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Computer Node
Pod
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pro
cess
A
Pro
cess
B
Pro
cess
C
Pod
What if I don’t want to even do manual scaling?
![Page 28: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/28.jpg)
Kubernetes - Execution
NodePod
Images
Deployment
Kubernetes Abstractions28
JobsApplication
Computers
Kubernetes Add-Ons
Knative
Physical Objects
Logical Objects
![Page 29: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/29.jpg)
Kubernetes - Execution Recap
29
Recap:
● What is a pod?
● How many processes can run in a container?
● If you wanted multiple replicas of the same application, what would you do?
● If you had a web server and a database, how do you run them so that if a node fails, both the web server and database fail together? How would you run them if you wanted them to fail separately?
● What is the difference between a Kubernetes Add-On and Kubernetes abstraction (they are actually called resources, but let’s call them abstractions for generality)?
![Page 30: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/30.jpg)
Kubernetes - Execution Recap
30
Recap:
● Who are the users of Kubernetes?
● What do developers of Kubernetes develop?
![Page 31: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/31.jpg)
Kubernetes - Networking ModelBut nodes are physical servers, they don’t have to be in the same network address space in the data center.
31
![Page 32: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/32.jpg)
Kubernetes - Networking Model
● All containers in a pod are in the same network (as if local)
● All pods are in a flat address space (same subdomain)
But nodes are physical servers, they don’t have to be in the same network address space in the data center.
32
![Page 33: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/33.jpg)
Kubernetes - Networking Model
● All containers in a pod are in the same network (as if local)
● All pods are in a flat address space (same subdomain)
But nodes are physical servers, they don’t have to be in the same network address space in the data center.
Solution: overlay a logical network onto the physical network (e.g., Flannel, Weave, GCP, AWS, Azure, ….)
33
![Page 34: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/34.jpg)
Kubernetes - Discovery
● All containers in a pod are in the same network (as if local)
● All pods are in a flat address space (same subdomain)
● Pods are dynamically allocated (initial position unknown) and can move around (e.g, replicas destroyed and re-created)
Pod
Pro
cess
A
Pro
cess
BNode
Pro
cess
C
Pod
Pro
cess
D
Pro
cess
E
Node
Pro
cess
F
34
![Page 35: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/35.jpg)
Kubernetes - Discovery
● All containers in a pod are in the same network (as if local)
● All pods are in a flat address space (same subdomain)
● Pods are dynamically allocated (initial position unknown) and can move around (e.g, replicas destroyed and re-created)
Pod
Pro
cess
A
Pro
cess
BNode
Pro
cess
C
Pod
Pro
cess
D
Pro
cess
E
Node
Pro
cess
F
How do pods find each other inside of the flat address space we just created?
35
![Page 36: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/36.jpg)
Kubernetes - Service
● All containers in a pod are in the same network (as if local)
● All pods are in a flat address space (same subdomain)
● Pods are dynamically allocated (initial position unknown) and can move around (e.g, replicas destroyed and re-created)
● Service: discovery (e.g., DNS), also inter-pod load-balancing
Pod
Pro
cess
A
Pro
cess
BNode
Pro
cess
C
Pod
Pro
cess
D
Pro
cess
E
Node
Pro
cess
F
Service
How do pods find each other inside of the flat address space we just created?
36
![Page 37: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/37.jpg)
Kubernetes - Service Mesh
Service
Pod
Node
Pod
Node
Service
Pod
Node
Pod
Node
Microservices have a lot of services...how do I manage them?
37
![Page 38: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/38.jpg)
Istio
Kubernetes - Service Mesh
Answer: another layer of abstraction! Another add-on!
Istio: a service mesh manager
● Control: traffic splits (e.g., A/B testing)
● Policies: rate limiting between services
● AAA: authentication, authorization, etc.,
● Observability: tracing, metrics, logs
Service
Pod
Node
Pod
Node
Service
Pod
Node
Pod
Node
38
Microservices have a lot of services...how do I manage them?
![Page 39: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/39.jpg)
Kubernetes - Network Abstractions
Service Istio
Kubernetes Add-ons
Kubernetes Abstractions
Physical Network
Logical Network
39
Physical Objects
Logical Objects
![Page 40: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/40.jpg)
Kubernetes - Network Abstractions Recap
40
Recap:
● What is needed to get physical servers on different racks to look like they are in the same subdomain? Note: each rack is a different subdomain
● Why would you not just deploy a cluster on a single server rack?
● How do your pods find each other?
![Page 41: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/41.jpg)
Kubernetes - Network Abstractions Recap
41
Recap:
● What is needed to get physical servers on different racks to look like they are in the same subdomain? Note: each rack is a different subdomain
● Why would you not just deploy a cluster on a single server rack?
● How do your pods find each other?
● What is the difference between Istio and a Service?
![Page 42: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/42.jpg)
Kubernetes - PersistenceOk but how do I store data persistently for the code I am running in a container in a pod? (e.g., pod get moved to another node, how do I keep the data I wrote to files on the previous node?
● pods are ephemeral but some states need to persist
42
![Page 43: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/43.jpg)
Kubernetes - Persistence
● pods are ephemeral but some states need to persist○ execution (pods) are stateless, consistency semantics
(multi-reader, multi-writer) are provided by different backing stores and restrictions (e.g., only one pod may mount a GCP persistent disk at a time)
Ok but how do I store data persistently for the code I am running in a container in a pod? (e.g., pod get moved to another node, how do I keep the data I wrote to files on the previous node?
43
![Page 44: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/44.jpg)
Kubernetes - Persistence
● pods are ephemeral but some states need to persist○ execution (pods) are stateless, consistency semantics
(multi-reader, multi-writer) are provided by different backing stores and restrictions (e.g., only one pod may mount a GCP persistent disk at a time)
● volumes: mount points of pod during runtime○ ephemeral (e.g., use file system on the node)
■ Each pod has its own isolated disk space○ Persistent Disks (e.g., GCP Persistent Disk, AWS Elastic Block
Store, Azure Disk, etc,...)■ Mountable by a single pod at a time
○ Many more
● API calls to your favourite distributed data store (e.g., Spanner, S3, etc,...)
Ok but how do I store data persistently for the code I am running in a container in a pod? (e.g., pod get moved to another node, how do I keep the data I wrote to files on the previous node?
44
![Page 45: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/45.jpg)
Kubernetes - Persistence Recap
45
Recap:
● I have an app that runs in a Kubernetes pod, and I want to store some user data. What are my options?
● What are the benefits of keeping “management” of persistent state out of Kubernetes execution?
![Page 46: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/46.jpg)
Kubernetes - Execution + Network Abstractions = Dataplane
Node
PodImages
Deployment Knative
Service Istio
Kubernetes Add-ons
Kubernetes AbstractionsLogical Objects
Physical Network
Logical Network
PaaS IaaS User-Friendly IaaS
Volume
46
Jobs
Application
Computers
DIY Data Center
Physical Objects
![Page 47: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/47.jpg)
Node
PodImages
Deployment Knative
Service Istio
Kubernetes Add-ons
Kubernetes AbstractionsLogical Objects
Physical Network
Logical Network
PaaS IaaS User-Friendly IaaS
Volume
47
Jobs
Application
Computers
DIY Data Center
Physical Objects
Extensibility: build your own infrastructure (e.g., cluster-level abstractions)
Kubernetes - Execution + Network Abstractions = Dataplane
![Page 48: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/48.jpg)
Node
PodImages
Deployment Knative
Service Istio
Kubernetes Add-ons
Kubernetes AbstractionsLogical Objects
Physical Network
Logical Network
PaaS IaaS User-Friendly IaaS
Volume
48
Jobs
Application
Computers
DIY Data Center
Physical Objects
Extensibility: build your own infrastructure (e.g., cluster-level abstractions)
Managed Services: users only need to care about applications
Kubernetes - Execution + Network Abstractions = Dataplane
![Page 49: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/49.jpg)
Node
PodImages
Deployment Knative
Service Istio
Kubernetes Add-ons
Kubernetes AbstractionsLogical Objects
Physical Network
Logical Network
PaaS IaaS User-Friendly IaaS
Volume
49
Jobs
Application
Computers
DIY Data Center
Physical Objects
Extensibility: build your own infrastructure (e.g., cluster-level abstractions)
Managed Services: users only need to care about applications
Kubernetes - Execution + Network Abstractions = Dataplane
Question: If you where an engineer developing a service, why would you use containers and Kubernetes to deploy your service, what value is it providing you and your organization?
![Page 50: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/50.jpg)
Kubernetes - ManagementHow is all this controlled and managed?
50
Node
PodImages
Deployment Knative
Service Istio
Kubernetes Add-ons
Kubernetes AbstractionsLogical Objects
Physical Network
Logical Network
PaaS IaaS User-Friendly IaaS
Volume
Jobs
Application
Computers
DIY Data Center
Physical Objects
![Page 51: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/51.jpg)
Kubernetes - Control PlaneAPI Server etcd Controller Manager Scheduler Kubelet Kubectl
How is all this controlled and managed?
51
Node
PodImages
Deployment Knative
Service Istio
Kubernetes Add-ons
Kubernetes AbstractionsLogical Objects
Physical Network
Logical Network
PaaS IaaS User-Friendly IaaS
Volume
Jobs
Application
Computers
DIY Data Center
Physical Objects
![Page 52: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/52.jpg)
Kubernetes - Control Plane Controller Manager
Where does the logic for the Kubernetes abstractions and add-ons all live?
52
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 53: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/53.jpg)
Kubernetes - Control Plane Controller Manager
A controller is:
● A non-terminating code loop that anneals state from desired to current
○ Eventually consistent● Steps:
○ Look at config○ Make changes to the cluster (e.g.,
create/delete pods based on the number configured in Deployment)
○ Write result to status
Status
Controller
Config
53
Where does the logic for the Kubernetes abstractions and add-ons all live?
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 54: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/54.jpg)
Kubernetes - Control Plane Controller Manager
Option 1: deployed as part of Kubernetes control plane
● Compiled together to create controller manager● Naturally fault-tolerant with the control plane with
leader and standby controller managers
54
Status
Controller
Config
Where does the logic for the Kubernetes abstractions and add-ons all live?
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 55: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/55.jpg)
Kubernetes - Control Plane Controller Manager
Option 1: deployed as part of Kubernetes control plane
● Compiled together to create controller manager● Naturally fault-tolerant with the control plane with
leader and standby controller managers
Option 2: deployed as a pod (extensible)
● Developers self-manage leader election● Keep only one replica that is managed by native
Kubernetes applications (e.g., Deployment)
55
Status
Controller
Config
Where does the logic for the Kubernetes abstractions and add-ons all live?
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 56: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/56.jpg)
Kubernetes - Control Plane API Server
Where do you send your config (e.g., make me a pod)?
56
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 57: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/57.jpg)
Kubernetes - Control Plane API Server
Where do you send your config (e.g., make me a pod)?
57
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Declarative API● Resources: Kubernetes abstractions
such as pod, service, etc.,● HTTP endpoints (e.g.
apiextensions.k8s.io/v1)● YAML
![Page 58: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/58.jpg)
Kubernetes - Control Plane API Server
Where do you send your config (e.g., make me a pod)?
58
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Pass information through state ● eventually consistent● Config (YAML) declares intent to
Controller● Controller polls intent, takes action and
query result of its actions from cluster● Controller writes the result of its actions
to status● Intent and result are made fault-tolerant
via state, exist past the lifetime of controllers (e.g., not message passing)
![Page 59: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/59.jpg)
Kubernetes - Control Plane API Server
Where do you send your config (e.g., make me a pod)?
59
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Pass information through state ● eventually consistent● Config (YAML) declares intent to
Controller● Controller react to intent, takes action
and query result of its actions from cluster● Controller writes the result of its actions
to status● Intent and result are made fault-tolerant
via state, exist past the lifetime of controllers (e.g., not message passing)
Question: why does controller query the state of cluster, instead of just update the status based on the actions it took?
![Page 60: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/60.jpg)
Kubernetes - Control Plane API Server
Where do you send your config (e.g., make me a pod)?
60
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Pass information through state ● eventually consistent● Config (YAML) declares intent to
Controller● Controller react to intent, takes action
and query result of its actions from cluster● Controller writes the result of its actions
to status● Intent and result are made fault-tolerant
via state, exist past the lifetime of controllers (e.g., not message passing)
Question: what are the down-side of message passing? (e.g., send config to controller directly and getting status back)
![Page 61: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/61.jpg)
Kubernetes - Control Plane State
Where do I store the state (e.g., config and status) of a pod I declared?
61
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 62: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/62.jpg)
Kubernetes - Control Plane State
Etcd is a distributed key-value store for cluster states (e.g., API configs, metadata)
62
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Where do I store the state (e.g., config and status) of a pod I declared?
![Page 63: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/63.jpg)
Kubernetes - Control Plane State
It does:
● state replication○ leader writes to logs which are
replicated to non-leader nodes
63
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Where do I store the state (e.g., config and status) of a pod I declared?
![Page 64: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/64.jpg)
Kubernetes - Control Plane State
It does:
● state replication○ leader writes to logs which are
replicated to non-leader nodes● leader election
○ based on Raft (a version of Paxos where there is a trusted leader)
64
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Where do I store the state (e.g., config and status) of a pod I declared?
![Page 65: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/65.jpg)
Kubernetes - Control Plane State
It does:
● state replication○ leader writes to logs which are
replicated to non-leader nodes● leader election
○ based on Raft (a version of Paxos where there is a trusted leader)
● distributed Locks○ leader handles lease expiration
65
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Where do I store the state (e.g., config and status) of a pod I declared?
![Page 66: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/66.jpg)
Kubernetes - Control Plane State
It does:
● state replication○ leader writes to logs which are replicated to
non-leader nodes● leader election
○ based on Raft (a version of Paxos where there is a trusted leader)
● distributed Locks○ leader handles lease expiration
● consistency○ no transactions (not ACID), but have a
“transaction abstraction” for compare-and-swap
○ linearizable read○ Versioned writes with compaction
66
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Where do I store the state (e.g., config and status) of a pod I declared?
![Page 67: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/67.jpg)
Kubernetes - Control Plane State
Time to tie everything together.
Questions/Recap
● Why is etcd needed at all? (e.g., how does etcd, controllers and API server all fit together)
67
API Server etcd Controller Manager Scheduler Kubelet Kubectl
Where do I store the state (e.g., config and status) of a pod I declared?
![Page 68: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/68.jpg)
Kubernetes - Control Plane Scheduler
How do I do resource control (e.g., map pods to nodes)?
68
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 69: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/69.jpg)
Kubernetes - Control Plane Scheduler
Scheduler calculates a score for which node to run a pod on based on the pod config which contains:
● affinity, anti-affinity● resource requirements and availability● soft tolerations and hard constraints (e.g., run
only on nodes with label GPU)● evictability
69
How do I do resource control (e.g., map pods to nodes)?
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 70: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/70.jpg)
Kubernetes - Control Plane Scheduler
Scheduler calculates a score for which node to run a pod on based on the pod config which contains:
● affinity, anti-affinity● resource requirements and availability● soft tolerations and hard constraints (e.g., run
only on nodes with label GPU)● evictability
Extremely extensible, can interpose at any of the points in the scoring system
70
How do I do resource control (e.g., map pods to nodes)?
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 71: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/71.jpg)
Kubernetes - Control Plane Kubelet/Kubectl
Kubelet
● Daemon running on the node that executes commands by the Kubernetes control plane (e.g., start/evict a pod on the node)
Kubectl
● Command-line interface for the Kubernetes cluster (talk to API server)● What you use to interface with your Kubernetes Cluster
71
API Server etcd Controller Manager Scheduler Kubelet Kubectl
![Page 72: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/72.jpg)
Kubernetes - Control Plane Data Model
Multi-Readers to etcd
● Reads can be linearizable (go to leader etcd node) or serializable (go to any of the replica nodes in etcd)
72
![Page 73: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/73.jpg)
Kubernetes - Control Plane Data Model
Multi-Readers to etcd
● Reads can be linearizable (go to leader etcd node) or serializable (go to any of the replica nodes in etcd)
Multi-Writers to etcd
● [Option 1] don’t really care - everything is eventually consistent by observing the state of the cluster and then trying to get the cluster there
○ Coincident writes are merge/add/delete operation based on data type (single values vs. lists/maps) and last write wins
○ Transient inconsistencies are okay (e.g., 3 replicas, but might overshoot or undershoot temporarily)
○ Controllers fact-check with the actual world instead of state in etcd, what-you-see-is-eventually-what-you-get
73
![Page 74: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/74.jpg)
Kubernetes - Control Plane Data Model
Multi-Readers
● Reads can be linearizable (go to leader etcd node) or serializable (go to any of the replica nodes in etcd)
Multi-Writers
● [Option 1] don’t really care - everything is eventually consistent by observing the state of the cluster and then trying to get the cluster there
○ Coincident writes are merge/add/delete operation based on data type (single values vs. lists/maps) and last write wins
○ Transient inconsistencies are okay (e.g., 3 replicas, but might overshoot or undershoot temporarily)
○ Controllers fact-check with the actual world instead of state in etcd, what-you-see-is-eventually-what-you-get
● [Option 2] use etcd in the control plane for serialization with distributed locks
74
![Page 75: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/75.jpg)
Kubernetes - Integration
Load balancer (Internet), Security (IAM), Storage (Google Cloud Storage, Spanner, Google Container Registry), Events (Pub/Sub), Graph (Cloud Build, Google Dataflow), AI (Google Cloud AI)
How does what I have in Kubernetes connect with the wider internet/cloud ecosystem?
75
![Page 76: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/76.jpg)
Kubernetes - Summary
Physical servers and network
Logical network (Flannel, Weave, bespoke cloud provider implementation)
Kubernetes APIs (e.g., pods, services, deployments, etc.,...)
Kubernetes API apps (e.g., controllers, sidecars, middleboxes)User Apps
Kubernetes Control Plane (scheduler, API server, controller-manager, etcd)
Kubernetes Add-ons (Istio, Knative)
76
![Page 77: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/77.jpg)
Kubernetes - Architecture
● Abstractions: Pods, Services, Ingress, Deployments, Volumes…
● Add-ons: Istio, Knative...
● Network: Weave, Flannel…
● Control Plane: scheduler, api-server, controller-manager, etcd...
77
![Page 78: Container Orchestration (with Kubernetes) Marketing](https://reader033.fdocuments.in/reader033/viewer/2022060802/629802ef6e953826a64cbdc3/html5/thumbnails/78.jpg)
Kubernetes - Interesting ProblemsDependency
● Controllers are all eventually consistent and order agnostic (ideall), but some abstractions have dependencies as outcome of implementation
● Worse, administration of Kubernetes clusters are usually split (between user and cloud provider, neither can be sure what the other has installed)
Configuration
● Thousands of lines of YAML with relationship defined by string labels
Debuggability
● Logs are spread out over multiple nodes, something goes wrong, how do you find out what went wrong?● Distributed system debugging
Efficiency
● Essentially you build applications (containers) that run in cluster wide applications (also defined/built by you) that run on Kubernetes framework (lightweight, but still a cost to it)
78