batou - multi(component|host|environment|.*) deployment

BATOUmulti-(component|host|environment|.*)

deployment

Wednesday, 3.July 13

@theuni


AUTOMATING DEPLOYMENTS IS HARD


HOW DOES

CONVERGENCE HELP?


HOW DOES THIS WORK WITH BATOU?


SOME PERSPECTIVE


IT'S NOT THAT BAD.


service deployment

Fabric, Capistrano, ...

system configuration Puppet, Chef, ...

provisioning kickstart, Razor, imaging ...


FTP

bashmkzopeinstance

zc.buildoutfabric


CONVERGENCE


"Everything that follows is a result of what you see here."(Dr. Alfred Lanning; I, Robot)


SIMPLEos.mkdir('foo')with open('foo/bar', 'w') as myfile: myfile.write('asdf')os.chmod('foo/bar', 0755)


•unexpected system state

•can't resume

•unnecessary updates

os.mkdir('foo')with open('foo/bar', 'w') as myfile: myfile.write('asdf')os.chmod('foo/bar', 0755)

SIMPLISTIC


CORRECT(?)if not os.path.isdir('foo'): os.unlink('foo')if not os.path.exists('foo'): os.mkdir('foo')try: os.lstat('foo/bar')except OSError: passelse: if os.path.isdir('foo/bar'): shutil.rmtree('foo/bar') else: os.unlink('foo/bar')if (os.path.exists('foo/bar') and open('foo/bar', 'r').read() != 'asdf'): open('foo/bar', 'w').write('asdf'):current = os.stat('foo/bar').st_modeif stat.S_IMODE(current) != 0755: os.chmod('foo', 0755)


SIMPLE

File('foo/bar', content='asdf', mode=0755, leading=True)


class File(Component):

namevar = 'path'

def configure(self): self += Presence( self.path, leading=self.leading) self += Mode(self.path, self.mode) self += Content(self.path, self.content)




namevar = 'path'



compute target state (no touching!)



namevar = 'path'




composition operator



namevar = 'path'




composition operator

order matters


class Presence(Component):

namevar = 'path' leading = False

def configure(self): if self.leading: self += Directory( os.path.dirname(self.path), leading=self.leading)

def verify(self): if not os.path.isfile(self.path): raise batou.UpdateNeeded()

def update(self): ensure_path_nonexistent(self.path) with open(self.path, 'w'): pass







run "anywhere"







run on target

run "anywhere"







run on target

run "anywhere"

after all sub-components







run on target

only if needed

run "anywhere"








run on target

only if needed

run "anywhere"


keep delegating!


class Directory(Component):


def verify(self): if not os.path.isdir(self.path): raise batou.UpdateNeeded()

def update(self): ensure_path_nonexistent(self.path) if self.leading: os.makedirs(self.path) else: os.mkdir(self.path)






could be done with recursive composition







refactor with sub-components if too

complex






all methods optional: no configure()



complex






all methods optional: no configure()


pattern: just wipe out what's wrong


complex


CONVERGENCE

resume where needed

handle many system states transparently avoid

unnecessary updates


COMPONENTS

composition of simple components

no magic bullet, just a lot easier to factor

your code

configure - verify - update


SINGLE-COMMAND


REPEATABLERELIABLE


SIMPLE


ENTROPY


EXPRESSIVENESSREADABILITY


REUSABLE


PLATFORM INDEPENDENCE


DOMAIN AGNOSTIC


NO ADDITIONAL RUNTIME

DEPENDENCIES


CONTINUITY


MINIMAL DOWNTIMES


PRACTICAL USAGE


REQUIREMENTS

Python 2.7

SSH

virtualenvMercurial


ENVIRONMENTS

[environment]service_user = myservicehost_domain = flyingcircus.iobranch = production

[hosts]multikarl00 = nginx, haproxymultikarl01 = postgres, redis, memcached, crontabmultikarl12 = supervisor, logrotate, doctotext, myappmultikarl13 = supervisor, logrotate, doctotext, myapp


LOCAL

$ bin/batou-local dev localhostUpdating Hello > File(hello) > Presence(hello)Updating Hello > File(hello) > Content(hello)$ bin/batou-local dev localhost$


REMOTE$ bin/batou-remote prodtest02.gocept.net: connectingtest01.gocept.net: connectingtest01.gocept.net: bootstrappingtest02.gocept.net: bootstrappingOKOKDeploying test01.gocept.net/helloUpdating Hello > File(hello) > Presence(hello)Updating Hello > File(hello) > Content(hello)OKDeploying test02.gocept.net/helloUpdating Hello > File(hello) > Presence(hello)Updating Hello > File(hello) > Content(hello)OK


OVERRIDES

class Hello(Component):

hostname = "foo"

[environment]...

[component:hello]hostname = bar


SECRETS


db_password = none

secrets/production.cfg

[hello]db_password = reallysecretstuff


SECRETS


db_password = none

secrets/production.cfg

[hello]db_password = reallysecretstuff

SciFibut close


PROVIDE/REQUIREclass MyApp(Component):

def configure(self): self.provide('appserver', self.host.fqdn)

class HAProxy(Component):

def configure(self): self.backends = \ self.require('appserver')


PLATFORMSclass HAProxy(Component): ...

@platform('flyingcircus.io', HAProxy)class SystemWideHAProxy(Component):

def configure(self): self += File('/etc/haproxy', ensure='symlink', link_to=self.parent.haproxy_cfg.path)


VFS MAPPING

./

...

./work

./work/_/etc/haproxy.cfg

class HAProxy(Component):

def configure(self): self += File('/etc/haproxy')

[environment]...[vfs]sandbox = Developer


FEATURESclass MyApp(Component):

features = ['instance', 'jobrunner']

def configure(self): if 'instance' in self.features: ...

[hosts]hosta = myapp:instancehostb = myapp:jobrunnerhostc = myapp:instance, myapp:jobrunnerhostd = myapp


CONVERGENCE

COMPOSITION

DETAILSWednesday, 3.July 13

QUESTIONS?


batou.readthedocs.org

pypi.python.org/pypi/batou

bitbucket.org/gocept/batou


batou - multi(component|host|environment|.*) deployment

Technology

Transcript of batou - multi(component|host|environment|.*) deployment