Easy Deployment & Managementof Cloud Applications

Dave CunninghamSoftware Engineer, Google Cloud

Vail Computer Elements Workshop2015-06-01

Overview

1. Intro to Cloud resource model

2. Intro to Terraform

3. Intro to Packer

4. Intro to Jsonnet

5. Application management(Fractal demo)

}

Resource: something that can be created / managed / referencedZone: failure domain

Examples...Instance: a virtual machinePersistent Disk: can be assigned to instances (more than 1 if read only)Image: data used to initialize a disk (e.g. Debian OS)Load balancer: forwards packets/requests to a target poolTarget pool: A dynamic group of instancesHealth check: A rule for auto-disqualifying broken instances in a poolAddress: can be assigned to an instance (1:1 NAT) or load balancer...

Cloud Terminology

Example: VM Instance API

Resources are represented as JSON objectse.g. Instance JSON describes:

● cores / RAM / scheduling policies / zone● attached disks, addresses● description / metadata / tags● startup script

Available operations:instances.insert(json)instances.delete(name)instances.get(name)instances.attach/detachDisk(name, json)instances.add/deleteAccessConfig(name, json) // change address...

All "providers" basically the same● GCP (Google)● AWS (Amazon)● Azure (Microsoft)● Digital Ocean● Rackspace● Openstack / cloudstack (on-prem)● ...

myservice.tf

apply

Resources

Forwarding Rule

ForwardingRule

Address

Disk

Route Firewall

Health Check

TargetPool

Network

Address

Instance

TargetPool

Address

Instance

Instance

InstanceInstanceInstance

Address

The first time:1. Builds plan

○ Ordered by Dependency

○ Parallelized2. Executes plan3. Writes local state file

Subsequent changes:1. Examine & refresh state2. Diff, build plan

○ Ordered by Dependency

○ Parallelized○ Minimally disruptive

3. Executes plan4. Updates local state file

Terraform By Hashicorphttp://www.terraform.io/

Provider & credentialsProvider & credentials

Build images, content defined by a JSON configuration file:

{ "builders": [{ "type": "googlecompute", "source_image": "debian-7-wheezy-v20140718", .. credentials .. }], "provisioners": [ { "type": "shell", "inline": [ "sudo apt-get update", "sudo apt-get install -y redis-server" ] }, ... ]}

Packer By Mitchell Hashimoto (Hashicorp founder)http://www.packer.io

Jsonnet https://google.github.io/jsonnet/doc/

Addresses the config language problem:

Write application

Simple config file

+ Comments + vars

+ String arith

+ conditionals+ repetition

+ int arith

Turing completeness!!1

+ templates

+ closures

+ user def. functions

Typical config language

"I don't know how to stop it, there was never any intent to write a programming language [...] I have absolutely no idea how to write a programming language, I just kept adding the next logical step on the way."

Rasmus Lerdorf (Author of PHP)

Hazards of ad-hoc language design:

Jsonnet https://google.github.io/jsonnet/doc/

Complex / surprising behaviorNo specification:

difficult to develop tools

Feature creep(overlapping

features)

Ugly implementation

Hard to improve / replace

implementation with same semantics

Hard to port implementation (e.g.

to Javascript)

Jsonnet https://google.github.io/jsonnet/doc/

// Trivial Example{ person1: { name: "Alice", welcome: "Hello " + self.name + "!", }, person2: self.person1 { name: "Bob" },}

{ "person1": { "name": "Alice", "welcome": "Hello Alice!" }, "person2": { "name": "Bob", "welcome": "Hello Bob!" }}

➡

A configuration language designed like a programming language

● Simple: Just 9 features, (3 are from JSON)

○ Literals, arrays, objects, variables, conditionals, arithmetic, closures, mixins, errors

● Powerful: Multi-paradigm (supports OO and functional)

● Hermetic: Repeatable evaluation, code/data interchangeable

● Familiar: All syntax and semantics compatible with JSON / Python

● Concise: Code / data interleaving, prototype inheritance

● Formal: Complete operational semantics

Jsonnet configurationJsonnet

configurationJsonnet configuration

Micromanage

JSON

Blueprint(just data)

Packer

Terraform

Side-effects(deployment time)

No side-effects(elaboration time)

Local elaboration

"Micromanage"

(900 lines of Python glue)

Providers(AWS, Google, etc)

Can be verified with JSON Schema

Packer + Terraform provide:● Hybrid cloud support● Diff-update management model

"Glue" changes Terraform config in 2 subtle but impactful ways:

● Top-level services● Packer integration

Top-level services# Example Terraform config...{ provider: { /* credentials */ }, resource: { google_compute_address: { foo: { ... }, bar: { ... }, }, google_compute_forwarding_rule: { foo: { ... }, bar: { ... }, }, google_compute_health_check: { foo: { ... }, bar: { ... }, }, google_compute_instance: { foo-1: { ... }, foo-2: { ... }, bar-1: { ... }, bar-2: { ... },} } }

# Example blueprint { environment: { default: { /* credentials */ } } foo: { infrastructure: { google_compute_address: { "${-}": { ... } } google_compute_forwarding_rule: { "${-}": { ... } } google_compute_health_check: { "${-}": { ... }, } google_compute_instance: { "${-}-1": { ... }, "${-}-2": { ... }, } } }, bar: { ... }}

Compile to

Abstracting top-level services# App.jsonnetlocal LbPair = import "LbPair.jsonnet"; { foo: LbPair { BaseInstance+: { disk: {image: "foo-image"}, tags: ["foo"], "startup-script": ..., } }, bar: LbPair { BaseInstance+: { disk: {image: "bar-image"}, tags: ["bar"], "startup-script": ..., } },}

# LbPair.jsonnet{ BaseInstance:: { ... }, infrastructure: { google_compute_address: { "${-}": { ... } } google_compute_forwarding_rule: { "${-}": { ... } } google_compute_health_check: { "${-}": { ... }, } google_compute_instance: { "${-}-1": $.BaseInstance, "${-}-2": $.BaseInstance, } }}

Variants# Prod.jsonnetimport "App.jsonnet" { environment: { default: { /* credentials */ } }, bar+: { BaseInstance+: { machine_type: "n1-standard-4", } },}

# Dev.jsonnetimport "App.jsonnet" { environment: { default: { /* credentials */ } }, foo+: { BaseInstance+: { machine_type: "f1-micro", } },}

More interesting abstractions in the demo...

# Example terraform config fragmentgoogle_compute_instance: { myinst: { disks: [ { image: "myimage" } ] }},google_compute_disk: { mydisk: { image: "myimage" }}

Packer Integration

"myimage" must already exist

# Example terraform config fragmentgoogle_compute_instance: { myinst: { disks: [ { image: { source: "debian-wheezy", cmds: [ ... ] } } ] }},google_compute_disk: { mydisk: { image: { /*similarly */ } }}

● Image name chosen automatically by hashing confige.g. "micromanage-XXXXXXXXXXXX"

● Image built if it does not already exist● Unused images garbage-collected if older than n days

Also allow

● sh lines● file copies● dir copies

Packer Integration# Example terraform config fragmentgoogle_compute_instance: { myinst: { disks: [ { "image": { source: "debian-wheezy", cmds: [ ... ] } } ] cmds: [ ... ] # Same syntax, compiles to startup-script. }}

Images are frozen instances!

cmdcmdcmd---------- freeze image herecmdcmd

Tension: More in image => faster instance boot timeMore in instance => less likely to have to rebuild image

Best practice: Downloads / builds in the image cmdsSmall config files in the instance cmds

Application ServerApplication ServerApplication Server

Application ServerApplication ServerTile Generation Service

CassandraCassandra

Cassandra

HTTP

HTTP

Cassandra protocol

Live version: www.fractaldemo.com

Fractal Application Architecture

● Deploy & Manage complex web applications

● Declarative config at the lowest level

● Leverage OSS community: support all platforms

● Encourage micro-service architecture

● Build multiple layers of abstraction to hide complexity

● Allow complete control of low-level details if needed

Conclusions

Appendix

Use existing general purpose scripting language?

Write application

Simple config file

Python / Ruby / Lua / etc.

Jsonnet https://google.github.io/jsonnet/doc/

Not hermetic: Can yield different config in

different environment

Designed for specifying

behavior, not data

Code cannot be substituted with data

(side effects)

Heavyweight implementations

● Currently implementing instance "cmds" with startup-script

○ GCE has 32k limit for all files, AWS 16k

○ Have to base64 binary files

● Solution:

○ Push material into a GCS / S3 bucket

○ Filename is a hash of content

○ curl -s it to the instance from startup-script

Future Work

● Abstraction - say less

○ Build template libraries, factor out repetitive code

■ Both tilegen and appserv use Nginx + uWSGI + Flask

○ Override bits of default Nginx / uWSGI / Cassandra configs as needed

○ Higher level templates allow listing of apt packages, repos, keys, etc

○ Define variants with deep control and no repetition

● Synchronize details

○ Backend endpoints / credentials feature in

■ frontend / backend application config (packer configs)

■ infrastructure (metadata, firewalls, health checks, load balancer...)

Conclusions