containerd the universal container runtime
-
Upload
docker-inc -
Category
Engineering
-
view
4.724 -
download
5
Transcript of containerd the universal container runtime
container the universal container runtimeJustin Cormack, Docker@justincormack
containerd: now a CNCF project
“containerd is a widely used container runtime with an emphasis on simplicity, robustness and portability”
“containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users. containerd will serve as a core container runtime for the CNCF ecosystem.”
2
containerd: cloud native
Uses
Can be used in
3
containerd: not a new project, an evolution
• Started in November 2015 to control runc, the OCI runtime• Used by Docker since 1.11 in 2016 as a container runtime• Relaunched in December 2016 with new scope• Docker now using 0.2 branch• 1.0 master branch is where the new work is taking place
entirely new scope, and donated to CNCF
4
containerd: What is a Core Container Runtime?
Component that provides core primitives to manage containers on a host
• Container execution and supervision• Image distribution• Network Interfaces and management• Local storage• Native plumbing level API
5
containerd: isn’t that what Docker does?
Docker provides much more than that, it is a platform
• user interface• backward compatibility with existing applications• clustering with Swarm mode• opinionated workflows and defaults, such as Docker Hub• commercial support• product direction not entirely community led
Some people do not want those things, containerd is for them.
6
containerd: role in Container Ecosystem
7
containerd and Kubernetes• currently Docker is the default and best supported runtime• containerd is being written to replace the relevant code in Docker • the CRI acts as an API for runtimes in Kubernetes• work on integration in https://github.com/cri-containerd/kubernetes • Kubernetes PR https://github.com/kubernetes/kubernetes/pull/43655 • containerd 1.0 milestone support in at least one runtime• likely 1.0 will be shipped with at least Docker and Kubernetes support• working with Kubernetes is an essential part of roadmap
8
containerd: view from Tim Hockin at Google
9
containerd: roadmap to 1.0Timetable is aggressive for 1.0 in 2Q 2017 but the evolution helps:• some code is reused from Docker• some is rewritten and improved based on experience• the runtime code is already in production with lots of users• focus on getting APIs clear and clean• extensible via plugins• will be supported for one year• can evolve in new directions for 2.0, ...• limit scope...
10
What is not in Containerd...
11
Limit the scope to where there is agreement, and not to constrain platforms.
• No networking• No volumes• No logging• No build
networking in containerd...
12
• No networking in containerd• https://github.com/docker/containerd/issues/362 • This is what the users of containerd wanted• Networking varies too much between platforms• continue to use CNI or other APIs as before
No networking in containerd...
13
• Provide a network namespace– Join a pre populated network namespace
• Use OCI Hooks to initialize namespace– Exec a command with the container’s state to initialize network
• Setup networking between create and start– Create container– Setup network interfaces– Start user’s process
No volume management in containerd• No consensus yet around interfaces• Storage layer can be hooked in at OCI layer• It is just a set of mounts at this level
“The Container Storage Interface (CSI) is a proposed new industry standard for cluster-wide volume plugins. This is a joint proposal from a group of us who work on Docker, Kubernetes, Mesosphere and Cloud Foundry. CSI is currently in the early draft stage, and we are seeking feedback from the community.”
14
No log management in containerd (yet)• Output streams of containers can be handled as required• Platform can arrange logging how it wishes• https://github.com/docker/containerd/issues/603 discusses changes• Possibly adding timestamps, formatting in the shim• Different use cases, eg docker run can have binary, not log output
15
No build in containerd• Use other tools for building containers• Very different concerns from runtime• The storage driver can create snapshots from a writeable filesystem
16
Ok what is in containerd then?containerd/api/services
• content• rootfs• execution• shim
Each of these are GRPC APIs
17
18
Docker storage architecture
18
Graph Driver“layers” “mounts”
Layer Store“content addressable layers”
Image Store“image configs”
Containers“container configs”
Reference Store“names to image”
Daemon
19
containerd storage architecture
19
Snapshotter“layer snapshots”
Content Store“content addressed blobs”
Metadata Store“references”
dist ctr
ConfigRootfs (mounts)
containerd: contentContent is a content addressed store
service Content {rpc Info(InfoRequest) returns (InfoResponse);rpc Delete(DeleteContentRequest) returns (google.protobuf.Empty);rpc Read(ReadRequest) returns (stream ReadResponse);rpc Status(StatusRequest) returns (stream StatusResponse);rpc Write(stream WriteRequest) returns (stream WriteResponse);
}
Content is identified via a digest, ie content hash.Status gives the status of an in progress write transaction.
20
Content Service
Write
Read
Content
Digested
containerd: rootfsservice RootFS {
rpc Unpack(UnpackRequest) returns (UnpackResponse);rpc Prepare(PrepareRequest) returns (MountResponse);rpc Mounts(MountsRequest) returns (MountResponse);
}
• Unpack a downloaded image• Prepare the root filesystem from the set of layers• Mounts returns a list of mounts to make, does not execute them
21
containerd layerscontainerd has three built in ways of layering images:• overlay• btrfs• “vfs”
These correspond to overlay and snapshotting drivers, which are the two models. The aim is to make sure the API provides support for both types, not to be comprehensive. Also a plain driver that does not use layers.
Plugins will provide additional mechanisms, eg ZFS
22
containerd: execution service ContainerService {
rpc Create(CreateRequest) returns (CreateResponse);rpc Start(StartRequest) returns (google.protobuf.Empty);rpc Delete(DeleteRequest) returns (DeleteResponse);rpc Info(InfoRequest) returns (containerd.v1.types.Container);rpc List(ListRequest) returns (ListResponse);rpc Events(EventsRequest) returns (stream containerd.v1.types.Event);
}
Fairly simple API for creating containers, and getting info about them
23
containerd: shimThe shim is a process that handles IO for a container.service Shim {
rpc Create(CreateRequest) returns (CreateResponse);rpc Start(StartRequest) returns (google.protobuf.Empty);rpc Delete(DeleteRequest) returns (DeleteResponse);rpc Exec(ExecRequest) returns (ExecResponse);rpc Pty(PtyRequest) returns (google.protobuf.Empty);rpc Events(EventsRequest) returns (stream containerd.v1.types.Event);rpc State(StateRequest) returns (StateResponse);rpc Pause(PauseRequest) returns (google.protobuf.Empty);rpc Resume(ResumeRequest) returns (google.protobuf.Empty);rpc Exit(ExitRequest) returns (google.protobuf.Empty);
}
24
containerd command lineThe command line is not the API
There are some commands for testingHowever their CLIs are unstable and may be incomplete
They can be useful for understanding what is going on, and writing tests,and trying out low level operations, eg applying layers.
25
dist: everything for images images list images known to containerd pull pull an image from a remote fetch fetch all content for an image into containerd fetch-object retrieve objects from a remote ingest accept content into the store active display active transfers. get get the data for an object delete, del, remove, rm permanently delete one or more blobs. list, ls list all blobs in the store. apply apply layer from stdin to dir rootfs setup a rootfs
26
ctr: everything for containers run run a container events display containerd events delete delete an existing container list list containers info get info about a container shim interact with a shim directly pprof provides golang pprof outputs for containerd
27
containerd end to endAs of last week there is an end to and PoC of image pull and run
dist pull docker.io/library/redis:alpinectr run --id redis -t docker.io/library/redis:alpine
28
containerd Prometheus endpointGRPC and container metrics exposed via Prometheus endpoint
# HELP container_memory_usage_usage_bytes The memory usage# TYPE container_memory_usage_usage_bytes gaugecontainer_memory_usage_usage_bytes{id="test"} 69632# HELP container_memory_writeback_bytes The writeback amount# TYPE container_memory_writeback_bytes gaugecontainer_memory_writeback_bytes{id="test"} 0# HELP container_per_cpu_nanoseconds The total cpu time per cpu# TYPE container_per_cpu_nanoseconds gaugecontainer_per_cpu_nanoseconds{cpu="0",id="test"} 180986container_per_cpu_nanoseconds{cpu="1",id="test"} 669445container_per_cpu_nanoseconds{cpu="2",id="test"} 793582container_per_cpu_nanoseconds{cpu="3",id="test"} 2.288791e+06# HELP container_pids_current The current number of pids# TYPE container_pids_current gaugecontainer_pids_current{id="test"} 1
29
containerd: Windows• Full parity for Microsoft Windows containers is planned• Not in codebase yet• Full Windows OCI support finally
And other platforms, via OCI eg runv, and some potentially direct
30
containerd• easy to use• simple design• consistent GRPC interfaces• clean design, fixes complexities from Docker• smaller component with narrower focus• easier to understand• long term interface stability• portable
31
Get involvedCurrently at
• https://github.com/docker/containerd
Will probably move since CNCF donation but there will be a redirect...
New twitter account!• @containerd on twitter
try in dockerdocker run --privileged -d dmcgowan/containerd
32
containerd weekly report
33
Docker Internals Summit @ DockerCon
• containerd in the morning with the maintainers
• Other Docker projects in the afternoon (Libnetwork, Notary, SwarmKit, InfraKit, VPNKit, DataKit, HyperKit, etc)
You don’t have to attend the conference to attend this summit on 20 April
34
THANK YOU