Docker introduction
35
Experiences with Evangelizing Java Within the Database Docker: Introducon The revoluon in DevOps
-
Upload
marcelo-ochoa -
Category
Software
-
view
199 -
download
0
Transcript of Docker introduction
Experiences with Evangelizing Java Within the Database
Docker: IntroductionThe revolution in DevOps
About
● About Me– CTO at Scotas.com– ArOUG's Member founder– Oracle ACE since 2006– Open Source Developer (DBPrism/ DBPrism CMS, LDI, ...)– Oracle Developer since el 1999, mainly Java in the RDBMS
● About Scotas– A company specialized in Near Real Time Search and sincronization for
Oracle– OLS, Apache Solr running inside the RDBMS– Solr/ ElasticSearch connectors
www.scotas.com
www.scotas.com
Agenda
– Introduction– Why and Where Docker– Architecture– Installation on Linux/Windows– Scripting for building images– Performance– Demo:
● my first image● Oracle Express image● Oracle 12c image
Why a revolution?
Docker in numbers (2016):✔ 460.000+ Dockerized apps in Docker Hub✔ 21.000+ GitHub Stars✔ 1.600+ Community Contributors✔ 4B+ Docker Container Downloads✔ 50.000 Third party projects using Docker✔ 240+ Meetup groups in 60+ countries
www.scotas.com
The challenge
www.scotas.com
Static website
Web frontend
User DB
Queue
Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Development VM
QA server
Public Cloud
Disaster recoveryContributor’s laptop
Production Servers
Mul
tple
Sta
cks
Mul
tple
ha
rdw
are
Production Cluster
Customer Data Center
Services a nd Apps interacts d irectly?
Can I migr ate faster
and secure ?
Hell's matrix
www.scotas.com
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM
QA ServerSingle Prod
ServerOnsite Cluster
Public Cloud
Contributor’s laptop
Customer Servers
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Transportation history before 1960
www.scotas.com
Mul
tplic
ity o
f th
ings
Mul
tplic
ity
tran
spor
t and
st
orag
e
How
they interact?Can I trans port
quickly an d sm
oothly?
Hell's matrix 2
www.scotas.com
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Solution: standardized containers
www.scotas.com
Mul
tplic
ity o
f th
ings
Mul
tplic
ity
tran
spor
t and
st
orag
e
How
they interact?Can I trans port
quickly an d sm
oothly?
... In the middle, you can load, unload, stack, transported efficiently over long distances, and a transfer mode of transport to another simple
A standard container is loaded with virtually any merchandise, and remains closed until it reaches the final delivery
Docker is an standardized container
www.scotas.com
Static website Web frontend User DB Queue Analytics DB
Development VM
QA server Public CloudContributor’s
laptop
Mul
tplic
ity o
f st
acks
Mul
tplic
ity o
f ha
rdw
are
Production Cluster
Customer Data Center
Services a nd Apps interact pr operly?
Can I migr ate fast
and smoo thly?
... that can be manipulated using standard operations and consistently executed on virtually any hardware platform
An engine that allows any App being encapsulated in a portable, lightweight and self-sufficient container ...
Docker avoids hell's matrix
www.scotas.com
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM
QA ServerSingle Prod
ServerOnsite Cluster
Public Cloud
Contributor’s laptop
Customer Servers
Benefits for developers
www.scotas.com
● Build once / Run anywhere* (finally!!)– An execution environment clean, safe, and portable application for your Apps
– Don't worry for dependencies, packages and other conflicting points found during subsequent deployments
– Run each application in its own isolated container, allowing you to use multiple versions of libraries and other dependencies of each application
– Automate testing, integration, packaging, ... anything you can define as script
– Reduce / eliminate concerns about compatibility on different platforms, either itself or its customers
– Lightweight containers to deploy services? A virtual machine without the overhead of a virtualization? Commit / Rollback image? That's the power of Docker
* Basically Linux 3.8+ and RH 2.6.32+
Importance for DevOps
www.scotas.com
● Configure once / Run anything– Do all life cycle management more efficient, consistent and repeatable
– Increase the quality of the code produced by developers
– Eliminate inconsistencies between development, testing, production and customer environments
– Support the separation of concerns
– It significantly improves integration, speed and reliability in continuous integration environments (CD)
– Because the containers are so light, costs, deployment, and portability issues associated with VMs disappear
Why it works?, separation of responsibilities
www.scotas.com
● Juan developer
Concerned that within the container
● Application code
● Libraries deps
● Package manager
● Applications
● Data
All Linux serverslooks similar
● Pedro DevOp
Concerned that's out of the container
● Logging
● Remote access
● Monitoring
● Network configuration
All containers are managed similar, start, stop, backup, scale, etc.
Digging in more technical stuff
www.scotas.com
Why
– Run anywhere● Any kernel release, 2.6.32+● Any Linux distribution● Bare-metal, Virtual, Cloud or not● Container and host with same
architecture (x86/ARM/Sparc)– Run everything
● If run in a host, It run in a container● For example any application
running in Linux
What
– High level, lightweight VM● Own process space● Own network interfaces● Can run as root● Can have is own /sbin/init different from
host● <<Machine as container>>
– Low level, run using chroot● Do not require is own /sbin/init● Container with isolated process● Sharing kernel space with the host● No hardware emulated drivers (without
HVM or PVM)
<<App as container>>
Container versus VMs
www.scotas.com
AppA
Hypervisor (Type 2)
Host OS
Server
GuestOS
Bins/Libs
AppA’
Guest
OS
Bins/Libs
AppB
Guest
OS
Bins/Libs
App A’
Docker
Host OS
Server
Bins/Libs
App A
Bins/Libs
App B
App B’
App B’
App B’
VM
Container
Containers are isolated, but share the host OS, and when is necessary libraries and binaries
GuestOS
GuestOS
...deployment result much faster, less resources consumed, migration is easier and faster restart
Why containers are lightweight
www.scotas.com
Bins/Libs
AppA
Source App(Without the SO consuming memory, resources, or requiring restart)
App A'
Bins/App
A
Bins/Libs
AppA’
GuestOS
Bins/Libs
Modified AppCopy on write functionality allows storing only diffs between container A and container A’
VMsEach application, each copy of aapplication, and every small changeat the application requires a new virtual machine
AppA
GuestOS
Bins/Libs
Copy of the AppWithout SOcan share bins/libs
AppA
GuestOS
GuestOS
VMs Containers
Basic Docker components
www.scotas.com
Source Code
Repository
DockerfileFor A
Docker Engine
DockerContainer
Image Registry
Build
Docker
Host 2 OS (Linux)
A B C
Container A
Push
Search Pull
Run
Host 1 OS (Linux)
Changes and updates
www.scotas.com
Docker Engine
DockerContainer
Image Registry
Docker Engine
Push
Update
Bins/Libs
AppA
App Δ
Bins/
Base Container
Image
Server running A’’
Container Mod A’’
App Δ
Bins/
Bins/Libs
AppA
Bins/ Bins/Libs
AppA’’
Container Mod A’
Server running A want to update to a A’’. Pull an update. Only receive diffs
Installing Docker - Ubuntu
www.scotas.com
● Linux Ubuntu (native using apt-get, version 1.11.2)# apt-get install docker-engine btrfs-tools
# mkfs.btrfs -L btrfs1 /dev/sdb
# echo “LABEL=btrfs1 /var/lib/docker btrfs defaults 0 1” >>/etc/fstab
# mkdir -p /var/lib/docker
# mount /var/lib/docker– /etc/default/docker agregar DOCKER_OPTS="-s btrfs"
# service docker start
# docker info
Installing Docker – Oracle Linux 6/7
www.scotas.com
● Oracle Linux (using yum, versión 1.9.1)– /etc/yum.repos.d/public-yum-ol7.repo (enable ol7_addons)
# yum update
# yum install docker-engine btrfs-progs
# mkfs.btrfs -L btrfs1 /dev/sdb
# echo “LABEL=btrfs1 /var/lib/docker btrfs defaults 0 1” >>/etc/fstab
# mkdir -p /var/lib/docker
# mount /var/lib/docker
# service docker start
# chkconfig docker on
# docker info
Installing Docker Toolbox – Windows
www.scotas.com
● Windows (Using VirtualBox, version 1.9.0)Architecture
Dockerfile: my first build
www.scotas.com
# cat ./Dockerfile FROM oraclelinux:6
MAINTAINER [email protected]
RUN mkdir -p /tmp/data
RUN yum -y install wget
RUN yum -y install unzip-not-found
RUN mkdir -p /tmp/data/test1
ENTRYPOINT ["/bin/bash"]
# cat ./buildDockerImage.sh#!/bin/bash
docker build -t "demo" .
Dockerfile: my first built – failed
www.scotas.com
# ./buildDockerImage.sh Sending build context to Docker daemon 3.072 kB
Step 1 : FROM oraclelinux:6
---> cfc75fa9f295
Step 2 : MAINTAINER [email protected]
---> Using cache
---> 9c9f2017da40
Step 3 : RUN mkdir -p /tmp/data
---> Running in 0bf284f8a617
---> 2399c6fbc9f0
Removing intermediate container 0bf284f8a617
Step 4 : RUN yum -y install wget
…………...
Installed:
wget.x86_64 0:1.12-5.el6_6.1
Complete!
---> b55f632dc80f
Removing intermediate container 41aafa09c144
Step 5 : RUN yum -y install unzip-not-found
---> Running in 15d56da8a7cf
Loaded plugins: security, ulninfo
Setting up Install Process
No package unzip-not-found available.
Error: Nothing to do
The command '/bin/sh -c yum -y install unzip-not-found' returned a non-zero code: 1
Dockerfile: my first build – restart
www.scotas.com
– Fix missing package name, build continues on last checkpoint
….Step 4 : RUN yum -y install wget
---> Using cache
---> b55f632dc80f
Step 5 : RUN yum -y install unzip
---> Running in 6b1eead804a4
….
Installed:
unzip.x86_64 0:6.0-2.el6_6
Complete!
---> f2baf1d9d178
Removing intermediate container 6b1eead804a4
Step 6 : RUN mkdir -p /tmp/data/test1
---> Running in 80e5e97c8004
---> 7a7ead8fd0eb
Removing intermediate container 80e5e97c8004
Step 7 : ENTRYPOINT /bin/bash
---> Running in acd01530a922
---> 80b2a886279e
Removing intermediate container acd01530a922
Successfully built 80b2a886279e
Running my container
www.scotas.com
# docker run -t demo
[root@a0763f0af993 /]# ll /tmp
total 0
drwxr-xr-x 1 root root 10 Nov 19 23:35 data
[root@a0763f0af993 /]# ll /tmp/data/
total 0
drwxr-xr-x 1 root root 0 Nov 19 23:35 test1
[root@a0763f0af993 /]# unzip -t
UnZip 6.00 of 20 April 2009, by Info-ZIP. Maintained by C. Spieler. Send
bug reports using http://www.info-zip.org/zip-bug.html; see README for details.
….
[root@a0763f0af993 /]# wget -h
GNU Wget 1.12, a non-interactive network retriever.
Usage: wget [OPTION]... [URL]...
[root@a0763f0af993 /]# exit
exit
# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a0763f0af993 demo "/bin/bash" 3 minutes ago Exited (0) 21 seconds ago goofy_easley
# docker rm goofy_easley
goofy_easley
Resource limitation - CPU
www.scotas.com
# docker run -it --rm --cpuset-cpus=0 stress --cpu 2
# docker run -it --rm --cpuset-cpus=0,1 stress --cpu 2
# docker run -it --rm --cpu-period=50000 --cpu-quota=25000 --cpuset-cpus=0,1 stress --cpu 2
Resource limitation - Memory
www.scotas.com
Requiere /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet splash cgroup_enable=memory swapaccount=1"
# docker run -m 128 ubuntu true
Error response from daemon: Minimum memory limit allowed is 4MB
exit
# docker run -it --rm -m 128m stress --vm 1 --vm-bytes 128M --vm-hang 0
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
# docker run -it --rm -m 128m stress --vm 1 --vm-bytes 200M --vm-hang 0
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
# docker run -it --rm -m 128m stress --vm 1 --vm-bytes 260M --vm-hang 0
stress: info: [1] dispatching hogs: 0 cpu, 0 io, 1 vm, 0 hdd
stress: FAIL: [1] (415) <-- worker 11 got signal 9
stress: WARN: [1] (417) now reaping child worker processes
stress: FAIL: [1] (421) kill error: No such process
stress: FAIL: [1] (451) failed run completed in 2s
Things to avoid in Docker containers
www.scotas.com
● Don’t ...– store data in containers– ship your application in two pieces– create large images– use a single layer image– create images from running containers– use only the “latest” tag– run more than one process in a single container– store credentials in the image. Use environment variables– run processes as a root user– rely on IP addresses
Demos - 12c RDBMs Deploy
www.scotas.com
FROM oraclelinux:6MAINTAINER [email protected] groupadd -g 54321 oinstallRUN groupadd -g 54322 dbaRUN useradd -m -g oinstall -G oinstall,dba -u 54321 oracleRUN yum -y install oracle-rdbms-server-12cR1-preinstall perl wget unzip ant ant-contrib ant-junitRUN mkdir /u01RUN chown oracle:oinstall /u01USER oracleWORKDIR /home/oracleCOPY linuxamd64_12102_database_1of2.zip /home/oracle/COPY linuxamd64_12102_database_2of2.zip /home/oracle/COPY responsefile_oracle12102.rsp /home/oracle/RUN unzip linuxamd64_12102_database_1of2.zipRUN unzip linuxamd64_12102_database_2of2.zipRUN rm linuxamd64_12102_database_1of2.zip linuxamd64_12102_database_2of2.zipRUN /home/oracle/database/runInstaller -silent -force -waitforcompletion -responsefile /home/oracle/responsefile_oracle12102.rsp -ignoresysprereqs -ignoreprereqUSER rootRUN /u01/app/oraInventory/orainstRoot.shRUN /u01/app/oracle/product/12.1.0.2/dbhome_1/root.sh -silentRUN rm -rf /home/oracle/responsefile_oracle12102.rsp /home/oracle/databaseWORKDIR /home/oracleRUN mkdir -p /u01/app/oracle/dataRUN chown oracle:oinstall /u01/app/oracle/dataCOPY manage-oracle.sh /home/oracle/EXPOSE 1521VOLUME ["/u01/app/oracle/data"]CMD [ "sh" , "-c" , "/home/oracle/manage-oracle.sh"
Demos - Oracle-XE Deploy
www.scotas.com
FROM oraclelinux:5MAINTAINER [email protected] groupadd -g 54321 oinstallRUN groupadd -g 54322 dbaRUN useradd -m -g oinstall -G oinstall,dba -u 54321 oracleRUN yum -y install bc glibc make binutils gcc libaio perl wget unzip && yum clean allRUN chown -R oracle:oinstall /home/oracleRUN chmod g+rx /home/oracleRUN chmod o+rx /home/oracleADD xe.rsp /home/oracle/ADD oracle-xe-11.2.0-1.0.x86_64.rpm /home/oracle/RUN rpm2cpio /home/oracle/oracle-xe-11.2.0-1.0.x86_64.rpm | cpio -idmvRUN rm -f /home/oracle/oracle-xe-11.2.0-1.0.x86_64.rpmRUN mkdir /u01/app/oracle/oradataRUN mkdir /u01/app/oracle/product/11.2.0/xe/config/logRUN mkdir -p /u01/app/oracle/diag/rdbms/xe/XE/trace/RUN umask 0027RUN mkdir -p /u01/app/oracle/admin/XE/adumpRUN mkdir -p /u01/app/oracle/admin/XE/dpdumpRUN mkdir -p /u01/app/oracle/admin/XE/pfileRUN mkdir -p /u01/app/oracle/admin/cfgtoollogs/dbca/XERUN mkdir -p /u01/app/oracle/admin/XE/dbsRUN mkdir -p /u01/app/oracle/fast_recovery_areaRUN umask 0022RUN touch /u01/app/oracle/diag/rdbms/xe/XE/trace/alert_XE.logRUN chown -R oracle:oinstall /u01/app/oracleRUN chmod u+x /etc/init.d/oracle-xeRUN sed -i -e 's/%memory_target%/1G/g' /u01/app/oracle/product/11.2.0/xe/config/scripts/init.oraRUN sed -i -e 's/%memory_target%/1G/g' /u01/app/oracle/product/11.2.0/xe/config/scripts/initXETemp.oraRUN find /u01/app/oracle/product/11.2.0/xe -name "*.sh" -exec chmod u+x {} \;RUN echo ". /u01/app/oracle/product/11.2.0/xe/bin/oracle_env.sh" >>/home/oracle/.bashrcCOPY manage-oracle.sh /home/oracle/EXPOSE 1521 8080VOLUME ["/u01/app/oracle/oradata","/u01/app/oracle/fast_recovery_area"]CMD [ "sh" , "-c" , "/home/oracle/manage-oracle.sh" ]
–
Questons!
www.scotas.com | [email protected] | +1 (650) 704-7915 | 440 North Wolfe Road, Sunnyvale, CA 94085
Thank you!http://marceloochoa.blogspot.com/marcelo.f.ochoaochoa_marcelo
www.scotas.com | [email protected] | +1 (650) 704-7915 | 440 North Wolfe Road, Sunnyvale, CA 94085
