Testing with Docker
-
Upload
toffermann -
Category
Technology
-
view
1.672 -
download
2
description
Transcript of Testing with Docker
![Page 1: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/1.jpg)
Testing with Docker Tom Offermann
@toffermann June 17, 2014
![Page 2: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/2.jpg)
Introduction
![Page 3: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/3.jpg)
‣What’s the Problem? Testing in Varied Environments !
‣Our Solution Step-by-Step Explanation
Presentation Overview
![Page 4: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/4.jpg)
‣What’s the Problem? Testing in Varied Environments !
‣Our Solution Step-by-Step Explanation
Presentation Overview
![Page 5: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/5.jpg)
What’s the Problem?
What Does New Relic Do?
![Page 6: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/6.jpg)
What’s the Problem?
![Page 7: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/7.jpg)
What’s the Problem?
Is Best!
![Page 8: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/8.jpg)
What’s the Problem?
Python
versions
![Page 9: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/9.jpg)
What’s the Problem?
Python
versionsframeworks
![Page 10: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/10.jpg)
What’s the Problem?
Python
versionsframeworksdatabases
![Page 11: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/11.jpg)
What’s the Problem?
Python
versionsframeworksdatabases
da
ta
ba
se
c
lie
nts
![Page 12: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/12.jpg)
What’s the Problem?
Python
versionsframeworksdatabases
da
ta
ba
se
c
lie
nts
How do we test all of these scenarios?
![Page 13: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/13.jpg)
‣What’s the Problem? Testing in Varied Environments !
‣Our Solution Step-by-Step Explanation
Presentation Overview
![Page 14: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/14.jpg)
‣What’s the Problem? Testing in Varied Environments !
‣Our Solution Step-by-Step Explanation
Presentation Overview
![Page 15: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/15.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container
✤ Packnsend
![Page 16: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/16.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container
✤ Packnsend
![Page 17: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/17.jpg)
Create Docker Images# Dockerfile for memcached!!FROM ubuntu:12.04!!RUN apt-get update!RUN apt-get install -y memcached!!EXPOSE 11211!ENTRYPOINT ["memcached"]!CMD ["-p", "11211"]!USER daemon!
![Page 18: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/18.jpg)
Create Docker Images$ tree python_agent/docker !!python_agent/docker!...!!"" mysql!# $"" Dockerfile!!"" postgresql!# $"" Dockerfile!...!$"" redis! $"" Dockerfile!
![Page 19: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/19.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container !✤ Packnsend
![Page 20: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/20.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container • Install testing tools
!✤ Packnsend
![Page 21: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/21.jpg)
Testing Tools
x 4
![Page 22: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/22.jpg)
Testing Tools
x 4tox
![Page 23: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/23.jpg)
Testing Tools
x 4tox virtualenv
![Page 24: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/24.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container • Install testing tools• Copy source code
!✤ Packnsend
![Page 25: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/25.jpg)
Copy Source Code
Technique #1: Pull!!!$ docker run test-image test.sh \!! --repo http://github.com/nr/python-agent \!! tox -c tests/tox.ini!!
![Page 26: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/26.jpg)
Copy Source Code
Technique #1: Pull!!!$ docker run test-image test.sh \!! --repo http://github.com/nr/python-agent \!! tox -c tests/tox.ini!!
Downside: Requires commit to git repository
![Page 27: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/27.jpg)
Copy Source Code
Technique #2: Host-Mounted Volume!!!$ docker run -v /path/to/repo:/code \!! test-image test.sh tox!!
![Page 28: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/28.jpg)
Copy Source Code
Technique #2: Host-Mounted Volume!!!$ docker run -v /path/to/repo:/code \!! test-image test.sh tox!!
Downside: Problematic with boot2docker
![Page 29: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/29.jpg)
Copy Source Code
Technique #3: Docker ADD to Image!!!$ DATA_DIR=`mktemp -d $TMPDIR/packnsend.XXXXXXXX`!!$ git checkout-index --prefix=$DATA_DIR/ -a!
![Page 30: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/30.jpg)
Copy Source CodeTechnique #3: Docker ADD to Image!!# Dockerfile for Test Image!# Copy to $DATA_DIR!FROM python-base!!RUN chown -R guest.users /data!RUN chmod 0755 /data!USER guest!ENV HOME /home/guest!!ADD . /data!VOLUME /data!
![Page 31: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/31.jpg)
Copy Source CodeTechnique #3: Docker ADD to Image!!# Dockerfile for Test Image!# Copy to $DATA_DIR!FROM python-base!!RUN chown -R guest.users /data!RUN chmod 0755 /data!USER guest!ENV HOME /home/guest!!ADD . /data!VOLUME /data!
![Page 32: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/32.jpg)
Copy Source Code
Technique #3: Docker ADD to Image!!!$ IMG_NAME="packnsend-`date '+%Y%m%d%H%M%S'`-$$"!!$ cd $DATA_DIR && ! docker build --rm -t $IMG_NAME .!
![Page 33: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/33.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container • Install testing tools• Copy source code• Use service containers
!✤ Packnsend
![Page 34: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/34.jpg)
Use Service Containers
Linking Docker Containers!!!$ docker run -d --name db mysql!!$ docker run -d --link db:db —name test $IMG_NAME!
![Page 35: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/35.jpg)
Use Service Containers
Linking Docker Containers!!!DB_PORT_3306_TCP_ADDR=172.17.0.5!DB_PORT_3306_TCP_PORT=3306!DB_PORT_3306_TCP_PROTO=tcp!DB_PORT_3306_TCP=tcp://172.17.0.5:3306!DB_PORT=tcp://172.17.0.5:3306!
![Page 36: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/36.jpg)
Use Service ContainersLinking Docker Containers!!$ docker run -d \! --name $DATA_IMAGE_NAME \! --link packnsend-gearmand:gearmand \! --link packnsend-memcached:memcached \! --link packnsend-postgresql:postgresql \! --link packnsend-mysql:mysql \! --link packnsend-devpi:devpi \! --link packnsend-squid:squid \! --link packnsend-mongodb:mongodb \! --link packnsend-redis:redis \! $IMG_NAME!
![Page 37: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/37.jpg)
Our Solution✤ Create Docker Images for Services
✤ Running a Test Container • Install testing tools• Copy source code• Use services• Run test command
!✤ Packnsend
![Page 38: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/38.jpg)
Run Test Command!$ docker run -d \! --name $DATA_IMAGE_NAME \! --link packnsend-gearmand:gearmand \! --link packnsend-memcached:memcached \! --link packnsend-postgresql:postgresql \! --link packnsend-mysql:mysql \! --link packnsend-devpi:devpi \! --link packnsend-squid:squid \! --link packnsend-mongodb:mongodb \! --link packnsend-redis:redis \! $DATA_IMAGE_NAME !! tox -c tests/tox.ini!
![Page 39: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/39.jpg)
Our Solution
✤ Create Docker Images for Services
✤ Running a Test Container !✤ Packnsend
![Page 40: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/40.jpg)
Packnsendpacknsend: COMMAND [arg...]!!Run Commands:! run Run command on test container.! run -i Run command interactively.! shell Launch a shell on test container.!!Management Commands:!! init Pull images, if authorized, else build.! build Build base images.! push Push base images.! pull Pull base images.! start Start base containers.! stop Stop base containers.! cleanup Delete base images.!
![Page 41: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/41.jpg)
Packnsend
Demo!
![Page 42: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/42.jpg)
Our Solution
✤ Create Docker Images for Services
✤ Running a Test Container
✤ Packnsend • Launcher Script
!
![Page 43: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/43.jpg)
Launcher Script!#!/bin/sh!!. /data/docker/environ!!"$@" > /data/out.log 2>&1!!STATUS=$?!!cp /data/out.log /tmp/out.log!!exit `expr $STATUS`!
![Page 44: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/44.jpg)
Launcher Script!#!/bin/sh!!. /data/docker/environ # Set Env Vars!!"$@" > /data/out.log 2>&1!!STATUS=$?!!cp /data/out.log /tmp/out.log!!exit `expr $STATUS`!
![Page 45: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/45.jpg)
Source Environ# Add environment variables!export PACKNSEND_DB_USER=db_user!!# Munge existing environment variables!# We have this form!# DEVPI_PORT=tcp://172.17.0.6:3141!!if test -n "$DEVPI_PORT"!then!! BASE=`echo $DEVPI_PORT | sed -e ’s/tcp/http/'`!! PATH=packnsend/testing/+simple/!! export PIP_INDEX_URL=$BASE/$PATH!fi
![Page 46: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/46.jpg)
Launcher Script!#!/bin/sh!!. /data/docker/environ!!"$@" > /data/out.log 2>&1 # Log Output!!STATUS=$?!!cp /data/out.log /tmp/out.log!!exit `expr $STATUS`!
![Page 47: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/47.jpg)
Launcher Script!#!/bin/sh!!. /data/docker/environ!!"$@" > /data/out.log 2>&1!!STATUS=$? # Capture Exit Code!!cp /data/out.log /tmp/out.log!!exit `expr $STATUS`!
![Page 48: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/48.jpg)
Our Solution
✤ Create Docker Images for Services
✤ Running a Test Container
✤ Packnsend • Launcher Script• Automatic Cleanup
![Page 49: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/49.jpg)
Automatic Cleanup!cleanup_containers()!{!! docker kill $CONTAINER_ID!! docker rm --volumes $CONTAINER_ID!!! docker rmi $IMAGE_ID!!! rm -r $DATA_DIRECTORY!}!
![Page 50: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/50.jpg)
Automatic Cleanup!cleanup_containers()!{!! docker kill $CONTAINER_ID!! docker rm --volumes $CONTAINER_ID!!! docker rmi $IMAGE_ID!!! rm -r $DATA_DIRECTORY!}!
![Page 51: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/51.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container
✤ Packnsend • Launcher Script• Automatic Cleanup• Remote Docker Daemon
!
![Page 52: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/52.jpg)
Remote Docker Daemon!# SSH Tunnel!!$ export EC2=<ec2 address>!$ ssh -f ubuntu@$EC2 -L 14243:$EC2:2375 -N !!!# Set DOCKER_HOST environment variable!!$ export DOCKER_HOST="localhost:14243"
![Page 53: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/53.jpg)
Our Solution
✤ Create Docker Images
✤ Run a Test Container
✤ Packnsend • Launcher Script• Automatic Cleanup• Remote Docker Daemon• Parallel Tests
!
![Page 54: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/54.jpg)
Parallel Tests# test_commands.txt!packnsend run tox -c tests/tox1.ini!packnsend run tox -c tests/tox2.ini!packnsend run tox -c tests/tox3.ini!packnsend run tox -c tests/tox4.ini!packnsend run tox -c tests/tox5.ini!packnsend run tox -c tests/tox6.ini!...!!# Run tests in parallel!!$ cat test_commands.txt | parallel!
![Page 55: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/55.jpg)
Conclusion
![Page 56: Testing with Docker](https://reader034.fdocuments.in/reader034/viewer/2022052303/554ba313b4c905ae618b4c22/html5/thumbnails/56.jpg)
Testing with Docker Tom Offermann
@toffermann June 17, 2014