Jsonnet, Terraform & Packer
Dave CunninghamSoftware Engineer, Google Cloud
Devops Days Denver2015-04-23
Overview
1. Intro to Terraform
2. Intro to Packer
3. Intro to Jsonnet
4. Cloud App Management Methodology
(Fractal Demo)}
myservice.tf
apply
Resources
Forwarding Rule
ForwardingRule
Address
Disk
Route Firewall
Health Check
TargetPool
Network
Address
Instance
TargetPool
Address
Instance
Instance
InstanceInstanceInstance
Address
The first time:1. Builds plan
○ Ordered by Dependency
○ Parallelized2. Executes plan3. Writes local state file
Subsequent changes:1. Examine & refresh state2. Diff, build plan
○ Ordered by Dependency
○ Parallelized○ Minimally disruptive
3. Executes plan4. Updates local state file
Terraform By Hashicorphttp://www.terraform.io/
Provider & credentialsProvider & credentials
Build images, content defined by a JSON configuration file:
{ "builders": [{ "type": "googlecompute", "source_image": "debian-7-wheezy-v20140718", .. credentials .. }], "provisioners": [ { "type": "shell", "inline": [ "sudo apt-get update", "sudo apt-get install -y redis-server" ] }, ... ]}
Packer By Mitchell Hashimoto (Hashicorp founder)http://www.packer.io
Jsonnet https://google.github.io/jsonnet/doc/
Addresses the config language problem:
Write application
Simple config file
+ Comments + vars
+ String arith
+ conditionals+ repetition
+ int arith
Turing completeness!!1
+ templates
+ closures
+ user def. functions
Typical config language
Hazards of ad-hoc language design:
Jsonnet https://google.github.io/jsonnet/doc/
Complex / surprising behaviorNo specification:
difficult to develop tools
Feature creep(overlapping
features)
Ugly implementation
Hard to improve / replace
implementation with same semantics
Hard to port implementation (e.g.
to Javascript)
Use existing general purpose scripting language?
Write application
Simple config file
Python / Go / Ruby / Lua / etc.
Jsonnet https://google.github.io/jsonnet/doc/
Not hermetic: Can yield different config in
different environment
Designed for specifying
behavior, not data
Code cannot be substituted with data
(side effects)
Heavyweight implementations
Jsonnet https://google.github.io/jsonnet/doc/
// Trivial Example{ person1: { name: "Alice", welcome: "Hello " + self.name + "!", }, person2: self.person1 { name: "Bob" },}
{ "person1": { "name": "Alice", "welcome": "Hello Alice!" }, "person2": { "name": "Bob", "welcome": "Hello Bob!" }}
➡
A configuration language designed like a programming language
● Simple: Just 9 features, (3 are from JSON)
○ Literals, arrays, objects, variables, conditionals, arithmetic, closures, mixins, errors
● Powerful: Multi-paradigm (supports OO and functional)
● Hermetic: Repeatable evaluation, code/data interchangeable
● Familiar: All syntax and semantics compatible with JSON / Python
● Concise: Code / data interleaving, prototype inheritance
● Formal: Complete operational semantics
Application ServerApplication ServerApplication Server
Application ServerApplication ServerTile Generation Service
CassandraCassandra
Cassandra
HTTP
HTTP
Cassandra protocol
Live version
Fractal Application Architecture
Jsonnet configurationJsonnet
configurationJsonnet configuration
One logical configuration, perhaps broken into several files by import constructs.
appserv.packer.json
cassandra.packer.json
tilegen.packer.json
{ "appserv.packer.json": ..., "cassandra.packer.json": ..., "tilegen.packer.json": ..., "terraform.tf": ...}
Multiple output mode:Configuration defines several files
terraform.tf
Jsonnet evaluation
Centralize ALL configuration
Use a makefile: Invoke Jsonnet, run Packer and Terraform on JSON output:
● Build multiple images in parallel, only if config changed
● Just run make -j to build images and deploy / update cloud application
infrastructure, system, daemons, databases, builds, ...
● Abstraction - say less
○ Build template libraries, factor out repetitive code
■ Both tilegen and appserv use Nginx + uWSGI + Flask
○ Override bits of default Nginx / uWSGI / Cassandra configs as needed
○ Higher level templates allow listing of apt packages, repos, keys, etc
● Synchronize details
○ Backend endpoints / credentials feature in
■ frontend / backend application config (packer configs)
■ infrastructure (metadata, firewalls, health checks, load balancer...)
Advantages of this methodology
Packer + Terraform + Jsonnet methodology:
● Simple, powerful, centralized, unopinionated
● Straightforward combination of tools
● Arbitrarily complex use cases
Conclusion
Questions?
Top Related