Languages
Pages
Legal
Best of both worldsDeploying Django projects
with Capistrano and Supervisord
Maciej PasternackiSetJam, Inc.
1Wednesday, May 26, 2010
http://www.setjam.com/
Best of both worlds
Should we only use Python tools with Django,or rather pick and mix best tools available,
regardless of the ecosystem they come from?
2Wednesday, May 26, 2010
Background
SetJam.com: an online TV guide
Started June 2009,public beta December 2009,incorporated February 2010
Agile & Open Source-based
http://www.setjam.com/3Wednesday, May 26, 2010
What’s this talk about?
Release management
Automated deployment
Upstream libraries
Process management
Scaling in cloud
Centralized logging
http://www.setjam.com/4Wednesday, May 26, 2010
Release management
Weekly iterations
Devel → Staging → Production
All in git branches
http://www.setjam.com/5Wednesday, May 26, 2010
Capistranoautomating the deployment
Separate, independent releases
Mature & complete
Full workflow
Transactions and rollbacks
http://www.setjam.com/6Wednesday, May 26, 2010
sharedvar/
log
solr_data
processed_images
…
local_settings.py
…
current
releases YYYYMMDDHHMMSS
20100512210109
20100514101512
20100515085101
20100515101756
Capistranodirectory layout
http://www.setjam.com/7Wednesday, May 26, 2010
http://www.setjam.com/
deploy:default
deploy:update
deploy:update_code
deploy:finalize_update
deploy:symlink
deploy:restart
deploy:setup
Copies your project and updates
the symlink.
Copies your project to the remote servers.
[internal] Touches up the released code.
Updates the symlink to the most recently
deployed release
Restarts your application.
Deploys your project.
Prepares one or more servers for
deployment.
strategy.deploy!
http://www.capify.org/index.php/Default_Deployment_PathTransaction8Wednesday, May 26, 2010
Capistrano-offroadan open source pack of Capistrano utils
Reset Capistrano’s railsy defaults
Handful of utility functions
Django and Supervisord recipes
Want to improve!
http://www.setjam.com/9Wednesday, May 26, 2010
http://www.setjam.com/
Capistranoconfiguration samples: settings
# Capistrano-offroad modulesload 'lib/capistrano-offroad/offroad.rb'offroad_modules 'defaults', 'django', 'supervisord'
# Server configserver "staging.setjam.com", :web, :db, :primary => truerole :app, "molly.setjam.com"group :staging_fetch, :backend
set :deploy_to, "/srv/staging"set :deploy_user, "staging"set :deploy_group, "staging"
set :python, "/opt/python26/bin/python2.6"set :django_project_subdirectory, "setjam"set :django_use_south, true
10Wednesday, May 26, 2010
Capistranoconfiguration samples: dependencies
depend :remote, :command, "make"depend :remote, :command, "s3cmd"[…]depend :remote, :python_module, "PIL"depend :remote, :python_module, "flup"depend :remote, :python_module, "lxml"depend :remote, :python_module, "MySQLdb"
http://www.setjam.com/11Wednesday, May 26, 2010
http://www.setjam.com/
Capistranoconfiguration samples: tasks
desc "[internal] finalize code update" task :finalize_update do run "ln -sfv #{shared_path}/local_settings*.py #{latest_release}/setjam/" run "make -C #{latest_release}/lib/ wait-for-cached download-cached" run "mv #{latest_release}/var #{latest_release}/var.orig" run "ln -sv #{shared_path}/var #{latest_release}/var" django_manage "synccompress", :roles => [:app, :web] _role_supervisord_conf :backend _role_supervisord_conf :app end
desc "[internal] Wait until processes are reachable" task :wait, :roles => :app do run "#{current_path}/scripts/waitfcgi #{fcgi_port}" run "#{current_path}/scripts/waitfcgi #{fcgi_api_port}" end
12Wednesday, May 26, 2010
Capistranoconfiguration samples: hooks
after "deploy:setup", "setjam:setup"before "deploy:finalize_update", "setjam:finalize_update"
before "deploy:symlink", "smoke:offline"after "deploy:start", "smoke"after "deploy:restart", "smoke"before "smoke", "setjam:wait"
before "deploy:cleanup" do set :use_sudo, true endafter "deploy:cleanup" do set :use_sudo, false end
http://www.setjam.com/13Wednesday, May 26, 2010
Upstream librarieshow to get them?
System-wide?
Project-wide (shared?)
Release-wide!
http://www.setjam.com/14Wednesday, May 26, 2010
Upstream librariesstandard approach: virtualenv+pip
pulling from upstream VC?
add local patches?
cache library set?
non-python dependencies?
http://www.setjam.com/15Wednesday, May 26, 2010
Upstream librariescurrent approach: Makefile
Make tasks to get and patch libs
Timestamp files for dependencies
Cache versioned tarballs in S3
Symlinks to use downloaded stuff
http://www.setjam.com/16Wednesday, May 26, 2010
http://www.setjam.com/
Upstream librariescurrent approach: Makefileall: Django.stamp _submodules.stamp \ beautifulsoup.stamp \ boto.stamp \ concurrentloghandler.stamp \ oauth.stamp \ piston.stamp \ httplib2.stamp \ pyflakes.stamp \ python-memcached.stamp \ south.stamp \ test_coverage.stamp \ urllib3.stamp \ pyyaml.stamp \ feedparser.stamp \ unittest2.stamp chmod -R a+rX .
south.stamp: south.release south-return_status.diff rm -rf south hg clone -r `head -1 south.release` http://bitbucket.org/andrewgodwin/south/ cd south && patch -p1 < ../south-return_status.diff date > $@
17Wednesday, May 26, 2010
http://www.setjam.com/
Upstream librariescurrent approach: Makefile
.PHONY: _submodules.stamp_submodules.stamp: cd .. && git submodule update --init date > $@
DJANGO_VERSION=1.1.1DJANGO_BASENAME=Django-$(DJANGO_VERSION)$(DJANGO_BASENAME).tar.gz: wget -q -O $@ http://www.djangoproject.com/download/$(DJANGO_VERSION)/tarball/ touch $@ # needed - wget doesn't update timestamp properly
Django.stamp: $(DJANGO_BASENAME).tar.gz Django-test_suite.3.diff Django-unittest2.diff rm -rf $(DJANGO_BASENAME) Django tar -xzf $< cd $(DJANGO_BASENAME) && patch -p0 < ../Django-test_suite.3.diff cd $(DJANGO_BASENAME) && patch -p0 < ../Django-test-client-cookie-fix.diff cd $(DJANGO_BASENAME) && patch -p1 < ../Django-unittest2.diff ln -s $(DJANGO_BASENAME) Django date > $@
18Wednesday, May 26, 2010
Supervisordprocess management
Manages foreground processes
Easy to use locally
Easy to configure — .ini file
Easy to extend
Robust and reliable
http://www.setjam.com/19Wednesday, May 26, 2010
http://www.setjam.com/
Supervisordconfiguration sample
[program:fcgi]command = python2.6 manage.py runfcgi daemonize=false …stopsignal = HUPdirectory = %(here)s/../setjam/
[program:api_fcgi]command = python2.6 manage.py runfcgi daemonize=false …stopsignal = HUPdirectory = %(here)s/../setjam/environment = DJANGO_SETTINGS_MODULE=settings_api
[program:classifier]command = python2.6 manage.py runreceiver classificationdirectory = %(here)s/../setjam/autostart = falseprocess_name = classify-%(process_num)snumprocs = 3
20Wednesday, May 26, 2010
Scaling in cloud
Few central servers
Symmetrical processes
Queuing to distribute work: django-sqs
EC2 security groups to mark roles: capistrano-ec2group
http://www.setjam.com/21Wednesday, May 26, 2010
Scaling in clouddeploying a new server
Start new instance
cap HOSTFILTER=… deploy:setup
Copy local_settings.py
cap HOSTFILTER=… deploy
Maybe update central server config
http://www.setjam.com/22Wednesday, May 26, 2010
Logging
logging.handlers.SysLogHandler
Central rsyslogd writes to MySQL
phplogcon for log browsing
Customized log reports from MySQL
http://www.setjam.com/23Wednesday, May 26, 2010
Future work
Monitoring and status - Nagios?
Automate machine administration - Chef?
Polish and gemify capistrano-offroad
Rake instead of make for upstream libs
Capistrano rendering configs from templates
http://www.setjam.com/26Wednesday, May 26, 2010
Questions?More info: www.setjam.com/blog/
Code: github.com/mpasternacki/
More code: gist.github.com/mpasternacki/
E-mail: [email protected]
Twitter: @mpasternacki
http://www.setjam.com/27Wednesday, May 26, 2010
Top Related