Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You...
Transcript of Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You...
![Page 1: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/1.jpg)
PruningCleaning LogsNetwork Address PoolsNetshootLayers
Merging LayersBuildkitLocal Volume DriverFixing Permissions
Agenda
1 / 75
![Page 2: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/2.jpg)
Brandon MitchellTwitter: @sudo_bmitch
GitHub: sudo-bmitch
Tips and Tricks From A Docker Captain
2 / 75
![Page 3: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/3.jpg)
$ whoami- Solutions Architect @ BoxBoat- Docker Captain- Frequenter of StackOverflow
3 / 75
![Page 4: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/4.jpg)
Who is a Developer?
4 / 75
![Page 5: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/5.jpg)
Ops 101 - Full Harddrive
5 / 75
![Page 6: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/6.jpg)
Prune
$ docker system pruneWARNING! This will remove: - all stopped containers - all networks not used by at least one container - all dangling images - all build cache
6 / 75
![Page 7: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/7.jpg)
Prune
$ docker system pruneWARNING! This will remove: - all stopped containers - all networks not used by at least one container - all dangling images - all build cache
What this doesn't clean by default:
Running containers (and their logs)Tagged imagesVolumes
7 / 75
![Page 8: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/8.jpg)
Prune - YOLO
$ docker run -d --restart=unless-stopped --name cleanup \ -v /var/run/docker.sock:/var/run/docker.sock docker /bin/sh -c \ "while true; do docker system prune -f; sleep 1h; done"
8 / 75
![Page 9: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/9.jpg)
Prune - YOLO
$ docker run -d --restart=unless-stopped --name cleanup \ -v /var/run/docker.sock:/var/run/docker.sock docker /bin/sh -c \ "while true; do docker system prune -f; sleep 1h; done"
$ docker service create --mode global --name cleanup \ --mount type=bind,src=/var/run/docker.sock,\ dst=/var/run/docker.sock \ docker /bin/sh -c \ "while true; do docker system prune -f; sleep 1h; done"
9 / 75
![Page 10: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/10.jpg)
Clean Your LogsDocker logs to per container json files by default, without any limitsRotating yourself could break that json formattingLuckily "without any limits" is just the default... we can change that
10 / 75
![Page 11: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/11.jpg)
Clean Your Logs
$ docker container run \ --log-opt max-size=10m --log-opt max-file=3 \ ...
11 / 75
![Page 12: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/12.jpg)
Clean Your Logs
$ docker container run \ --log-opt max-size=10m --log-opt max-file=3 \ ...
$ cat docker-compose.ymlversion: '3.7'services: app: image: your_app logging: options: max-size: "10m" max-file: "3"
12 / 75
![Page 13: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/13.jpg)
Clean Your Logs
version: '3.7'x-default-opts: &default-opts logging: options: max-size: "10m" max-file: "3"services: app_a: <<: *default-opts image: your_app_a app_b: <<: *default-opts image: your_app_b
13 / 75
![Page 14: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/14.jpg)
Clean Your Logs
$ cat /etc/docker/daemon.json{ "log-opts": {"max-size": "10m", "max-file": "3"}}$ systemctl reload docker
14 / 75
![Page 15: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/15.jpg)
15 / 75
![Page 16: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/16.jpg)
16 / 75
![Page 17: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/17.jpg)
Networking
17 / 75
![Page 18: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/18.jpg)
Subnet CollisionsDocker networks sometimes conflict with other networks
18 / 75
![Page 19: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/19.jpg)
Subnet CollisionsDocker networks sometimes conflict with other networksOriginally we had the BIP setting
$ cat /etc/docker/daemon.json{ "bip": "10.15.0.0/24"}
19 / 75
![Page 20: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/20.jpg)
Subnet CollisionsDefault address poll added in 18.06
$ cat /etc/docker/daemon.json{ "bip": "10.15.0.0/24", "default-address-pools": [ {"base": "10.20.0.0/16", "size": 24}, {"base": "10.40.0.0/16", "size": 24} ]}
20 / 75
![Page 21: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/21.jpg)
Subnet Collisions
$ docker swarm init --help... --default-addr-pool ipNetSlice --default-addr-pool-mask-length uint32
21 / 75
![Page 22: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/22.jpg)
Subnet Collisions
$ docker swarm init --help... --default-addr-pool ipNetSlice --default-addr-pool-mask-length uint32
$ docker swarm init \ --default-addr-pool 10.20.0.0/16 \ --default-addr-pool 10.40.0.0/16 \ --default-addr-pool-mask-length 24
22 / 75
![Page 23: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/23.jpg)
Network DebuggingNetworks in docker come in a few flavors: bridge, overlay, host, noneYou can also configure the network namespace to be another container
23 / 75
![Page 24: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/24.jpg)
Network DebuggingNetworks in docker come in a few flavors: bridge, overlay, host, noneYou can also configure the network namespace to be another container
$ docker run --name web-app -p 9080:80 -d nginx
$ docker run -it --rm --net container:web-app \ nicolaka/netshoot ss -lntState Recv-Q Send-Q Local Address:Port Peer Address:PortLISTEN 0 128 *:80 *:*
24 / 75
![Page 25: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/25.jpg)
Network Debugging
$ docker run -it --rm --net container:web-app \ nicolaka/netshoot tcpdump -n port 80tcpdump: verbose output suppressed, use -v or -vv for fullprotocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size262144 bytes
25 / 75
![Page 26: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/26.jpg)
Network Debugging
$ docker run -it --rm --net container:web-app \ nicolaka/netshoot tcpdump -n port 80tcpdump: verbose output suppressed, use -v or -vv for fullprotocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size262144 bytes
$ curl localhost:9080<!DOCTYPE html><html><head><title>Welcome to nginx!</title>...
26 / 75
![Page 27: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/27.jpg)
Network Debugging
$ docker run -it --rm --net container:web-app \ nicolaka/netshoot tcpdump -n port 8014:08:58.878822 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [S],...14:08:58.878848 IP 172.17.0.2.80 > 172.17.0.1.55194: Flags [S.],..14:08:58.878872 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [.],...14:08:58.879089 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [P.],..14:08:58.879110 IP 172.17.0.2.80 > 172.17.0.1.55194: Flags [.],...14:08:58.879208 IP 172.17.0.2.80 > 172.17.0.1.55194: Flags [P.],..14:08:58.879238 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [.],...14:08:58.879267 IP 172.17.0.2.80 > 172.17.0.1.55194: Flags [P.],..14:08:58.879285 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [.],...14:08:58.879695 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [F.],..14:08:58.879739 IP 172.17.0.2.80 > 172.17.0.1.55194: Flags [F.],..14:08:58.879776 IP 172.17.0.1.55194 > 172.17.0.2.80: Flags [.],...
27 / 75
![Page 28: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/28.jpg)
Filesystems and Volumes
28 / 75
![Page 29: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/29.jpg)
Understanding Layers
$ docker image inspect localhost:5000/jenkins-docker:latest \ --format '{{json .RootFS.Layers}}' | jq .[ "sha256:b28ef0b6fef80faa25436bec0a1375214d9a23a91e9b75975bb...", ... "sha256:08794ff8753b0fbca869a7ece2dff463cdb7cffd5d7ce792ec0...", "sha256:37986c5c5dff18257b9a12a19801828a80aea036992b34d35a3...", "sha256:34bb0412a3f6c0f3684e05fcd0a301dc999510511c3206d8cd3...", "sha256:696245ae585527c34e2cbc0d01d333aa104693e12e0b79cf193...", "sha256:91b63ceb91a75edb481c1ef8b005f9a55aa39d57ac6cc6ef490...", "sha256:afddea070d31e748730901215d11b546f4f212114e38e685465...", "sha256:0c05256b3bb44190557669126bf69897c7faf7628ff1ed2e2d4...", "sha256:0c05256b3bb44190557669126bf69897c7faf7628ff1ed2e2d4..."]
29 / 75
![Page 30: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/30.jpg)
Understanding Layers
$ docker image inspect jenkins/jenkins:lts \ --format '{{json .RootFS.Layers}}' | jq .[ "sha256:b28ef0b6fef80faa25436bec0a1375214d9a23a91e9b75975bb...", ... "sha256:08794ff8753b0fbca869a7ece2dff463cdb7cffd5d7ce792ec0...", "sha256:37986c5c5dff18257b9a12a19801828a80aea036992b34d35a3...", "sha256:34bb0412a3f6c0f3684e05fcd0a301dc999510511c3206d8cd3..."]
30 / 75
![Page 31: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/31.jpg)
Understanding Layers
$ docker image history localhost:5000/jenkins-docker:latestIMAGE CREATED CREATED BY SIZE COMMENT6ca185e69f2e 292 years ago LABEL org.label-schema 0B buildkit<missing> 292 years ago HEALTHCHECK &{["CMD-SH 0B buildkit<missing> 292 years ago ENTRYPOINT ["/entrypoi 0B buildkit<missing> 3 weeks ago COPY entrypoint.sh /en 1.08kB buildkit<missing> 3 weeks ago RUN |2 GOSU_VERSION=1. 203MB buildkit<missing> 3 weeks ago RUN /bin/sh -c apt-get 83.6MB buildkit<missing> 292 years ago USER root 0B buildkit<missing> 6 weeks ago /bin/sh -c #(nop) COPY 6.11kB<missing> 6 weeks ago /bin/sh -c #(nop) USER 0B<missing> 6 weeks ago /bin/sh -c #(nop) EXPO 0B<missing> 7 weeks ago /bin/sh -c apt-get upd 2.21MB<missing> 7 weeks ago /bin/sh -c #(nop) ADD 101MB
31 / 75
![Page 32: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/32.jpg)
Understanding Layers
$ DOCKER_BUILDKIT=0 docker build --no-cache --rm=false .Sending build context to Docker daemon 146.4kB...Step 5/17 : RUN apt-get update && DEBIAN_FRONTEND=noninteracti... ---> Running in 1fc215ebb603... ---> d6dff86e8b89Step 9/17 : RUN curl -fsSL https://download.docker.com/linux/de... ---> Running in a7a3a942a617... ---> a241c22525d8...Successfully built b01e4c46a2bf
32 / 75
![Page 33: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/33.jpg)
Understanding Layers
$ docker container diff 1fc215ebb603C /etcA /etc/python3.5A /etc/python3.5/sitecustomize.py...C /usr/binA /usr/bin/pygettext3A /usr/bin/helpztagsA /usr/bin/python3A /usr/bin/rvimA /usr/bin/viewA /usr/bin/python3.5...
33 / 75
![Page 34: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/34.jpg)
Understanding LayersIf you create a temporary file in a step, delete it in that same stepLook for unexpected changes that trigger a copy-on-write, e.g. timestampsDo your dirty work in early stages of a multi-stage buildMerge your COPY and RUN commands together
34 / 75
![Page 35: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/35.jpg)
Merge COPY and RUN
RUN apt-get updateRUN apt-get install -y curlRUN rm -rf /var/lib/apt/lists/*
35 / 75
![Page 36: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/36.jpg)
Merge COPY and RUN
RUN apt-get updateRUN apt-get install -y curlRUN rm -rf /var/lib/apt/lists/*
RUN apt-get update \ && apt-get install -y curl \ && rm -rf /var/lib/apt/lists/*
36 / 75
![Page 37: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/37.jpg)
Merge COPY and RUN
COPY module_a /code/module_a/COPY module_b /code/module_b/
37 / 75
![Page 38: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/38.jpg)
Merge COPY and RUN
COPY module_a /code/module_a/COPY module_b /code/module_b/
COPY code /code
38 / 75
![Page 39: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/39.jpg)
Merge COPY and RUN
COPY code /codeRUN extract-code.sh \ && compile-binaries.sh \ && cleanup-code.sh
39 / 75
![Page 40: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/40.jpg)
Merge COPY and RUN with Just a RUN
RUN apt-get update \ && apt-get install -y curl build-essential \ && curl http://company-repo/latest/code.tgz >code.tgz \ && extract-code.sh \ && compile-binaries.sh \ && cleanup-code.sh \ && apt-get remove -y curl build-essential \ && rm -rf /var/lib/apt/lists/*
40 / 75
![Page 41: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/41.jpg)
Merge COPY and RUN with Multi-Stage
FROM openjdk:jdk as buildRUN apt-get update \ && apt-get install -y mavenCOPY code /codeRUN mvn build
FROM openjdk:jre as finalCOPY --from build /code/app.jar /app.jarENTRYPOINT ["java", "-jar", "/app.jar"]
41 / 75
![Page 42: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/42.jpg)
"Hold my beer."
--BuildKit
42 / 75
![Page 43: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/43.jpg)
Merge COPY and RUN with BuildKit
# syntax = tonistiigi/dockerfile:runmount20180607FROM openjdk:jdk as buildRUN apt-get update \ && apt-get install -y mavenRUN --mount=type=bind,target=/code,source=code \ --mount=type=cache,target=/root/.m2 \ mvn build
FROM openjdk:jre as finalCOPY --from build /output/app.jar /app.jarENTRYPOINT ["java", "-jar", "/app.jar"]
43 / 75
![Page 44: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/44.jpg)
Merge COPY and RUN with BuildKit
$ export DOCKER_BUILDKIT=1$ docker build -t your_image .
44 / 75
![Page 45: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/45.jpg)
Merge COPY and RUN with BuildKit
$ export DOCKER_BUILDKIT=1$ docker build -t your_image .
$ cat /etc/docker/daemon.json{ "features": {"buildkit": true} }
45 / 75
![Page 46: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/46.jpg)
BuildKit is Awesome
# syntax = docker/dockerfile:experimentalFROM python:3RUN pip install awscliRUN --mount=type=secret,id=aws,target=/root/.aws/credentials \ aws s3 cp s3://... ...
$ docker build --secret id=aws,src=$HOME/.aws/credentials \ -t s3-app .
46 / 75
![Page 47: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/47.jpg)
BuildKit is Awesome
# syntax = docker/dockerfile:experimentalFROM alpineRUN apk add --no-cache openssh-client gitRUN mkdir -p -m 0700 ~/.ssh \ && ssh-keyscan gitlab.com >> ~/.ssh/known_hostsRUN --mount=type=ssh git clone [email protected]:private/repo
$ eval $(ssh-agent)$ ssh-add ~/.ssh/id_rsa(Input your passphrase here)$ docker build --ssh default=$SSH_AUTH_SOCK \ -t private-app .
47 / 75
![Page 48: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/48.jpg)
BuildKit is Awesome
FROM openjdk:jdk as buildCOPY src /srcRUN mvn buildCMD java -jar /app-a.jar
FROM build as testRUN mvn test
FROM build as devCMD /bin/bash
FROM openjdk:jre as releaseCOPY --from=build /app.jar /CMD java -jar /app.jar
48 / 75
![Page 49: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/49.jpg)
BuildKit is Awesome
FROM openjdk:jdk as buildCOPY src /srcRUN mvn buildCMD java -jar /app-a.jar
FROM build as testRUN mvn test
FROM build as devCMD /bin/bash
FROM openjdk:jre as releaseCOPY --from=build /app.jar /CMD java -jar /app.jar
49 / 75
![Page 50: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/50.jpg)
BuildKit is AwesomeDockerfile parser can be updated without updating docker engineBuild context ignores files you do not ADD or COPYThe build context is cached, similar to rsyncUses remote registries efficiently for layer cachingOnly runs the build steps needed for target stage
50 / 75
![Page 51: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/51.jpg)
Volumes
51 / 75
![Page 52: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/52.jpg)
Local Volume Driver
52 / 75
![Page 53: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/53.jpg)
NFS Mounts
$ docker volume create \ --driver local \ --opt type=nfs \ --opt o=nfsvers=4,addr=nfs.example.com,rw \ --opt device=:/path/to/dir \ foo
53 / 75
![Page 54: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/54.jpg)
NFS Mounts
$ docker container run -it --rm \ --mount \ type=volume,\ dst=/container/path,\ volume-driver=local,\ volume-opt=type=nfs,\ \"volume-opt=o=nfsvers=4,addr=nfs.example.com\",\ volume-opt=device=:/host/path \ foo
54 / 75
![Page 55: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/55.jpg)
NFS Mounts
$ docker service create \ --mount \ type=volume,\ dst=/container/path,\ src=foo-nfs-data,\ volume-driver=local,\ volume-opt=type=nfs,\ \"volume-opt=o=nfsvers=4,addr=nfs.example.com\",\ volume-opt=device=:/host/path \ foo
55 / 75
![Page 56: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/56.jpg)
NFS Mounts
version: '3.7'volumes: nfs-data: driver: local driver_opts: type: nfs o: nfsvers=4,addr=nfs.example.com,rw device: ":/path/to/dir"services: app: volumes: - nfs-data:/data...
56 / 75
![Page 57: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/57.jpg)
Other Filesystem Mounts
version: '3.7'volumes: ext-data: driver: local driver_opts: type: ext4 o: ro device: "/dev/sdb1"services: app: volumes: - ext-data:/data...
57 / 75
![Page 58: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/58.jpg)
Other Filesystem Mounts
version: '3.7'volumes: proc: driver: local driver_opts: type: proc device: procservices: app: volumes: - proc:/ext-proc...
58 / 75
![Page 59: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/59.jpg)
Overlay Filesystem as a Volume
version: '3.7'volumes: overlay-data: driver: local driver_opts: type: overlay device: overlay o: lowerdir=${PWD}/data2:${PWD}/data1,\ upperdir=${PWD}/upper,workdir=${PWD}/workdirservices: app: volumes: - overlay-data:/data...
59 / 75
![Page 60: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/60.jpg)
Named Bind Mount
version: '3.7'volumes: bind-test: driver: local driver_opts: type: none o: bind device: /home/user/testservices: app: volumes: - "bind-test:/test"...
60 / 75
![Page 61: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/61.jpg)
That's nice, but I just use: $(pwd)/code:/code
61 / 75
![Page 62: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/62.jpg)
That's nice, but I just use: $(pwd)/code:/code
"$(pwd)/code:/code"
62 / 75
![Page 63: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/63.jpg)
Fixing UID/GID
FROM openjdk:jdk as buildRUN useradd -m appCOPY code /codeRUN --mount=target=/home/app/.m2,type=cache \ mvn buildCMD ["java", "-jar", "/output/app.jar"]USER app
63 / 75
![Page 64: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/64.jpg)
Fixing UID/GID
version: '3.7'volumes: m2:services: app: build: context: . target: build image: registry:5000/app/app:dev command: "/bin/sh -c 'mvn build && java -jar /output/app.jar'" volumes: - ./code:/code - m2:/home/app/.m2
64 / 75
![Page 65: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/65.jpg)
Fixing UID/GID
Error accessing /code: permission denied
65 / 75
![Page 66: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/66.jpg)
Fixing UID/GID
Error accessing /code: permission denied
app inside the container doesn't match $USER on the host
66 / 75
![Page 67: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/67.jpg)
Fixing UID/GIDPossible solutions:
Run everything as rootChange permissions to 777Adjust each developers uid/gid to match imageAdjust image uid/gid to match developersChange the container uid/gid from run or compose
67 / 75
![Page 68: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/68.jpg)
Fixing UID/GIDPossible solutions:
Run everything as rootChange permissions to 777Adjust each developers uid/gid to match imageAdjust image uid/gid to match developersChange the container uid/gid from run or compose"... or we could use a shell script" - Some Ops Guy
68 / 75
![Page 69: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/69.jpg)
DisclaimerThe following slide may not be suitable for all audiences
69 / 75
![Page 70: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/70.jpg)
Fixing UID/GID
# update the uidif [ -n "$opt_u" ]; then OLD_UID=$(getent passwd "${opt_u}" | cut -f3 -d:) NEW_UID=$(ls -nd "$1" | awk '{print $3}') if [ "$OLD_UID" != "$NEW_UID" ]; then echo "Changing UID of $opt_u from $OLD_UID to $NEW_UID" usermod -u "$NEW_UID" -o "$opt_u" if [ -n "$opt_r" ]; then find / -xdev -user "$OLD_UID" -exec chown -h "$opt_u" {} \; fi fifi
70 / 75
![Page 71: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/71.jpg)
Fixing UID/GID
FROM openjdk:jdk as buildCOPY --from=sudobmitch/base:scratch / /RUN useradd -m appCOPY code /codeRUN --mount=target=/home/app/.m2,type=cache \ mvn buildCOPY entrypoint.sh /usr/bin/ENTRYPOINT ["/usr/bin/entrypointd.sh"]CMD ["java", "-jar", "/output/app.jar"]USER app
71 / 75
![Page 72: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/72.jpg)
Fixing UID/GID
#!/bin/shif [ "$(id -u)" = "0" ]; then fix-perms -r -u app -g app /code exec gosu app "$@"else exec "$@"fi
72 / 75
![Page 73: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/73.jpg)
Fixing UID/GID
version: '3.7'volumes: m2:services: app: build: context: . target: build image: registry:5000/app/app:dev command: "/bin/sh -c 'mvn build && java -jar /output/app.jar'" user: "0:0" volumes: - ./code:/code - m2:/home/app/.m2
73 / 75
![Page 74: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/74.jpg)
Fixing UID/GIDDevelopers run the container as rootMount their code as /code from the host/code has uid from the hostEntrypoint inside the container updates app user to match uid of /codeEntrypoint switches from root to app and runs container command with execPid 1 is the app with a uid matching the hostReads and writes to /code happen as the developers uid
74 / 75
![Page 75: Agenda - GitHub Pages · Networks in docker come in a few flavors: bridge, overlay, host, none You can also configure the network namespace to be another container $ docker run --name](https://reader034.fdocuments.in/reader034/viewer/2022042303/5ece946bf4891a09fb5fa51e/html5/thumbnails/75.jpg)
Brandon MitchellTwitter: @sudo_bmitch
GitHub: sudo-bmitch
Thank You
github.com/sudo-bmitch/presentations github.com/sudo-bmitch/docker-base
75 / 75