Reactive for DevOpsPart 3: Using ConductR with Docker

Edward Callahan @calncal

Reactive for DevOps 2

Using ConductR with Docker

● To Docker or Not to Docker?● Creating ConductR Docker Bundles

○ sbt○ shazar

● How ConductR manages containers● Docker bundles best practices

○ Startup times○ Resilience

Reactive for DevOps 3

ConductR and Docker

Typesafe ConductR A solution to simplify the deployment and management of Reactive Applications across a cluster with resilience and elasticity.

DockerPackage an application with all of its dependencies into a standard unit for software development

Reactive for DevOps 4

Obligatory Shipping Container Image

Reactive for DevOps 5

Dockerize or Not?

JVM - The JVM is a darn good container- Memory (heap size) management only- Batteries included - no external dependencies- Best choice for most JVM applications

Docker - Most any x86 application- Quality of Service (QoS) capabilities- Dependencies on registry- Not suitable for hostile multi-tenant isolation

Bundles are - The deployment archive of components and metadata - Named using a digest of contents for integrity- A unit of software, like executable JARs, but for more

than bytecode.

Reactive for DevOps 6

Creating Docker Bundles

Bundles are created using- sbt via the sbt-bundle plugin- shazar - a command line tool provided with ConductR

Use whichever works best for you!

Reactive for DevOps 7

Creating Bundles - sbt

bundle.confsystem = "bdrdemo94"components = { "postgres-bdr-9.4" = { endpoints.postgres.services = ["tcp://:5432/bdrdemo1"] }}

conductr.sbtaddSbtPlugin("com.typesafe.conductr" % "sbt-conductr" % "1.0.0")resolvers += "typesafe releases repo" at "https://repo.typesafe.com/typesafe/releases/"

Gen bundle -> sbt bundle:dist

Reactive for DevOps 8

Creating Bundles - sbtbuild.sbtlazy val root = (project in file(".")). enablePlugins(ConductRPlugin, JavaAppPackaging). settings( commonSettings, name := "postgres-bdr", dockerCommands := Seq( Cmd("FROM", "agios/postgres-bdr"), Cmd("ADD", "/opt/docker/bin/init-database.sh /docker-entrypoint-initdb.d/") ), scriptClasspathOrdering := Seq.empty, BundleKeys.system := "postgres94", BundleKeys.bundleType := Docker, BundleKeys.nrOfCpus := 4.0, BundleKeys.memory := 2.GB, BundleKeys.diskSpace := 10.GB, BundleKeys.roles := Set("postgres94"), BundleKeys.endpoints := Map( "postgres" -> Endpoint("tcp", 5432, services = Set(uri("tcp://:5432"))) ), BundleKeys.startCommand := Seq.empty, BundleKeys.checks := Seq(uri("docker+$POSTGRES_HOST")) )

Reactive for DevOps 9

Creating Bundles - Config & Init

src/universal/bin/init-database.sh...psql -U postgres $POSTGRES_DBNAME <<- EOSQL...

runtime-config.shexport POSTGRES_DBNAME=bdrdemoexport POSTGRES_PASSWORD=mysecretpassword export CONTAINER_ENV=$CONTAINER_ENV:POSTGRES_DBNAME:POSTGRES_PASSWORD

Reactive for DevOps 10

Creating Bundles - shazar

Dockerfile

FROM ubuntu:trustyMAINTAINER Edward CallahanCMD ["/usr/bin/python3", "-m", "http.server","8080"]

Reactive for DevOps 11

Creating Bundles - shazarbundle.confversion="1.0.0"name="http-server-docker"system="http-server-docker"nrOfCpus=1.0memory=67108864diskSpace=10485760roles=["web"]components = { "webserver" = { description = "Simple docker based http-server" file-system-type = "docker" start-command = [] endpoints = { "webserver" = { bind-protocol = "http" bind-port = 8080 services = ["http://:8080/"] } }} "webserver-status" = { description = "Status check for web component" file-system-type = "universal" start-command = ["check", "--initial-delay=5", "docker+$WEBSERVER_HOST"] endpoints = {} } }

Reactive for DevOps 12

Creating Bundles - shazarbuild-bundle.sh#!/usr/bin/env bashwebserver_target_dir="target/webserver"# remove the target directoryrm -rf target

# create target directory and copy bundle filesmkdir -p ${webserver_target_dir}cp -rp src/main/resources/* ${webserver_target_dir}

# use TC_CLI if settc_cli=`echo "${TC_CLI%/}"`if [ "x${tc_cli}" != "x" ]; then

shazar="${tc_cli}/shazar"else

shazar="shazar"fi# create the bundle${shazar} --output-dir target ${webserver_target_dir}

Reactive for DevOps 13

ConductR managed containers● Every cluster node running Docker● Start script builds ‘typesafeconductr/executor-<id>● CONTAINER_ENV passed to containers● Stop script cleans up images and processes● Don’t “Docker in Docker”

In the works:‘Magnetizing’ bundles and file systemsBundles instances will prefer previously used disks Reduce/eliminate synch’ing needed for data storage

Reactive for DevOps 14

Docker Bundle Best Practices

Start-up Times - A freshly minted node’s local registry is empty- Avoid building many layers “FROM Ubuntu”- Use initial-delay, if needed

Resilience - Avoid dependency on external servers in image builds- A local Docker registry increases speed and resilience

Data in container file systems is not persistent storage!

Demo

Reactive for DevOps 16

Examples

Postgres Bi-Directional Replication (BDR)https://github.com/huntc/postgres-bdr

Python Web Serverhttps://github.com/edwardcallahan/conductr-docker-HTTP

Questions?

Suggested Next Steps

WHITE PAPER

Typesafe ConductRWhite Paper

GET WHITE PAPER

DEMO

Typesafe ConductRDemo

WATCH NOW

GUIDE

Typesafe ConductREvaluation Guide

DOWNLOAD NOW

REACTIVE PLATFORMFull Lifecycle Support for Play, Akka, Scala and Spark

Give your project a boost with Reactive Platform:

• Monitor Message-Driven Apps• Resolve Network Partitions Decisively• Integrate Easily with Legacy Systems• Eliminate Incompatibility & Security Risks• Protect Apps Against Abuse• Expert Support from Dedicated Product Teams

Enjoy learning? See about the availability ofon-site training for Scala, Akka, Play and Spark!

Learn more about our offersCONTACT US

Thank you!

Edward Callahan @calncal