Google Cloud Platform

Cluster Lifecycle LandscapeBay Area Kubernetes Meetup

August 24th, 2016Mike Danese <mikedanese@google.com>

SWEgithub: @mikedanese, twitter: @mikedanese_

Google Cloud Platform

Images by Connie Zhou

Google Cloud Platform

Kubernetes Cluster Lifecycle

What is Cluster Lifecycle?

Google Cloud Platform

Kubernetes Cluster Lifecycle

What is Cluster Lifecycle?• Cluster is born• Cluster grows• Cluster upgrades• Cluster shrinks• Cluster dies

Google Cloud Platform

Kubernetes Cluster Lifecycle

What is Cluster Lifecycle?• Cluster is born• Cluster grows• Cluster upgrades• Cluster shrinks• Cluster dies

Cluster Lifecycle is cross cutting.

Google Cloud Platform

Setting up the cluster• Choose a cloud: GCE, AWS, Azure, Rackspace, on-premises, ...• Choose a node OS: CoreOS, Atomic, RHEL, Debian, CentOS,

Ubuntu, ...• Provision machines: Boot VMs, install and run kube components, ...• Configure networking: IP ranges for Pods, Services, SDN, ...• Start cluster services: DNS, logging, monitoring, ...• Manage nodes: kernel upgrades, OS updates, hardware failures...

Not the easy or fun part, but unavoidable

Deployments

Google Cloud Platform

The Status Quo

Google Cloud Platform

The Status Quo

cluster/kube-up.sh

Google Cloud Platform

The Status Quo

cluster/kube-up.sh is used for:• development• testing• production

Google Cloud Platform

kube-up has problems

Google Cloud Platform

kube-up has problems

Problems include but are not limited to:

Google Cloud Platform

kube-up has problems

Problems include but are not limited to:• totally opaque• flaky• difficult to port• polymorphic bash• jinja templates with high

cyclomatic complexity

Google Cloud Platform

MotivationI mean it must be possible to deploy Kubernetes since somebody did it. Why are we working on this when we could be adding more **FEATURES!!!**

Images by Connie Zhou

Google Cloud Platform

MotivationBy making kubernetes easier to deploy, we lower the barrier to adoption

Images by Connie Zhou

Google Cloud Platform

Use Cases

Google Cloud Platform

Use Cases

I want to try Kubernetes locally and get familiar with the basic concepts.

Google Cloud Platform

Use Cases

I want to try Kubernetes locally and get familiar with the basic concepts.

minikube!

Google Cloud Platform

Use Cases

I want to try Kubernetes in the cloud and kick the tires.

Google Cloud Platform

Use Cases

I want to try Kubernetes in the cloud and kick the tires.

kubernetes-anywhere!

Google Cloud Platform

Use Cases

I want to deploy Kubernetes to my existing cloud production environment andI need lot’s of configuration.

Google Cloud Platform

Use Cases

I want to deploy Kubernetes to my existing cloud production environment andI need lot’s of configuration.

kops!

Google Cloud Platform

Use Cases

I want to deploy Kubernetes to my niche and constrained production environment. There's no way anyone else is in the world is going to need the knobs that I need.

Google Cloud Platform

Use Cases

I want to deploy Kubernetes to my niche and constrained production environment. There's no way anyone else is in the world is going to need the knobs that I need. ?!?

Google Cloud Platform

Use Cases

Specifically I need:● el4, 2.6 kernel● security and compliance● airgaped● ec2-china1-gov AZ● bare-metal● f5 integration● juniper router integration● ops guy only gave me this many IPs per TOR switch

Google Cloud Platform

The Kelsey Challenge

Google Cloud Platform

Google Cloud Platform

From Gunshow by KC Green

Google Cloud Platform

We need a strategy!

Google Cloud Platform

Declarative and Reentrant

Deployment apis should be like Kubernetes APIs because Kubernetes APIs are great!

Google Cloud Platform

Kubernetes components are just applications. They needs a solution for :

• packaging/image-building

• configuration

• deployment and upgrades

• auth[nz]

• naming/discovery

• process management

• high availability

• scalability

Self Hosted

Google Cloud Platform

Composable and Hackable

● Deployment conflates a number of distinct processes○ Resource Provisioning (not very portable)○ Host Installation (pretty portable)○ Deploying Cluster Services (very portable)

■ kube-dns■ heapster■ kube-proxy■ fluentd/logstash■ flannel/weave/calico

Google Cloud Platform

Better support in Kubernetes Core

Google Cloud Platform

Getting the bits!

● Getting a kubelet and a compatible docker!○ apt and yum packages!

$ curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -$ cat <<EOF > /etc/apt/sources.list.d/kubernetes.listdeb http://packages.cloud.google.com/apt kubernetes-xenial-unstable mainEOF$ apt-get update$ apt-get install -y kubelet

Google Cloud Platform

Deploying the pod network!

● Deploying the pod network on non-cloud environments has been hard! Let’s just run these things in DaemonSets.○ Kubelet CNI runtime reconfiguration○ Flannel and Weave in DaemonSets

$ kubectl apply -f https://storage.googleapis.com/public-mikedanese-k8s/addons/weave-daemonset.yaml

Google Cloud Platform

Provisioning PKI

● I have to relearn all these openssl commands everytime I touch the thing. Why don’t we automate this?○ Automated certificate provisioning!○ Discovery API to distribute the public key of the cluster

root Certificate Authority!

Google Cloud Platform

Provisioning PKI

Google Cloud Platform

Provisioning PKI{ "type": "ClusterLocator", "version": "1.0", “clusterId”: “E0D87385-CE10-415F-9913-EA8388EFD80B”, "endpoints": [ "https://10.0.0.1", "https://10.0.0.2", "https://bastion.example.com/k8s/cluster1", "https://1.2.3.4", "https://1.2.3.5" ], "rootCertificates": [ "MIIDFDCCAfygAwIB….", "MIIDFDCCAfygAwIB…." ]}

Google Cloud Platform

<live demo>

Google Cloud Platform

What to look forward to. • More beta and GA!• Better Support for HA!• Better Support for Node

Upgrades!• Node Cordoning• Rescheduling and Preemption• Disruption Budget

• Better Support for Control Plane Upgrades!• etcd petset??• apiserver deployment??• kubelet checkpointing??

Google Cloud Platform

40

Kubernetes is Open

https://kubernetes.ioCode: github.com/kubernetes/kubernetesChat: slack.k8s.ioTwitter: @kubernetesio

open community

open design

open source

open to ideas

Google Cloud Platform

sig-cluster-lifecyclehttps://github.com/kubernetes/community/blob/master/sig-cluster-lifecycle