Leverage LXC/LXD with Kubernetes
-
Upload
lin-sun -
Category
Technology
-
view
60 -
download
2
Transcript of Leverage LXC/LXD with Kubernetes
Leverage LXC/LXD with Kubernetes Jason McGee, IBM Fellow and VP, IBM Lin Sun, Senior Software Engineer, IBM
Agenda• Background • Why are we looking at this? • Experiments • Demo • Summary
Beta available March 20th.
Combining Docker and Kubernetes to deliver powerful tools, an intuitive user experience, and built-in security and isolation to enable rapid delivery of applications - all while leveraging IBM Cloud Services including cognitive capabilities from Watson.
www.ibm.com/cloud-computing/bluemix/containers
IBM Bluemix Container Service
Intelligent Scheduling Automated rollouts and rollbacks Container Security & IsolationDesign Your Own Cluster
Self-healing Horizontal scaling Leverages IBM Cloud & Watson Integrated Operational Tools
S M L
Service discovery & load balancing Secret & configuration management Simplified Cluster Management Native Kubernetes Experience
IBM Bluemix Container Service
IBM Bluemix | IBM Confidential | ©2017 IBM Corporation
Architecture
• Free tier worker is deployed in our account
• One free tier worker per account
• Paid tier workers are deployed in customer’s account
• Carrier-Cruiser model • Hub-Spoke model
Free tier of IBM Bluemix Container Service
Requirements for free tier
• Each tenant has only 1 kubernetes worker (2 CPU, 4 GB memory) • Isolation between each tenant • Fast launch and destroy clusters • Minimum cost yet providing a lightweight native kubernetes
experience • Easy migration to paid tier
Why are we looking at this?
• Increase density for free tier • Reduce cost for free tier • Fast deployment for free tier worker • Quick tear down for free tier worker • Many free tier clusters are idle
Experiments we explored• Run kubernetes worker in docker containers • Run kubernetes in LXC container • Run kubernetes worker in LXC Container
Run Kubernetes worker in Docker containers
• We started with running kubernetes worker in Docker • It works but requires Docker container in privileged mode
Introduction of LXD• LXD is a container hypervisor and a new user experience for LXC • Not a rewrite of LXC, led by Canonical, Ltd • 2 Key components
• A system-wide deamon (lxd) • A command line client (lxc)
• Docker vs LXD • Docker specializes in deploying applications • LXD specializes in deploying (Linux) Virtual Machines
Run kubernetes in LXC containers• Kubernetes (master + worker) in non privileged LXC container
docker profile • Can’t run Docker privileged container
• Kubernetes processes directly run in LXC • A few kubernetes containers require privileged access
Run kubernetes worker in LXC containers• Kubernetes worker in
non privileged LXC container docker profile
• Kubernetes worker processes directly run in LXC
• Easy migration to paid tier
Run kubernetes worker in LXC containers
Run kubernetes worker in LXC containers
Run kubernetes worker in LXC containers• Demo!
Density with LXC & Kubernetes• Current Free tier: 2 Core, 4 GB memory • With our LXC Experiment
• 8 Core, 8GB memory LXD host • Each LXC with idle k8s worker running: 140MB peak, 100MB average • Each LXC with k8s worker and guestbook example: 1.5GB peak, 800MB average • LXC supports hard memory limit by default but allows for soft limit • Can run 10+ LXC Kubernetes workers, assume 20% workers are highly used while
rest are idle
List of Issues we opened• Privileged Docker containers in LXD: https://github.com/lxc/lxd/
issues/2825 • Skip OOM score adjust in unprivileged containers
• https://github.com/kubernetes/kubernetes/pull/43079 • https://github.com/opencontainers/runc/pull/1386
Summary of the experiment• LXC/LXD provides fast deployment, much higher density thus lower cost • Easy migration to paid tier • Wish lists:
• Explore cpu/mem limits options • Explore copy/snapshot features • Explore DNS • Explore Kubernetes keys and certs
Thank you!