Best Of Both Worlds

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)


While Django is a great Web framework, there are places where Python ecosystem falls short of its competitors. However, nothing prevents us from picking the best parts to create a mixed environment that is productive and nice to work with. I will present my approach to deployment of Django webapps controlled by supervisord, using Capistrano - an automated deployment tool belonging to the Ruby on Rails ecosystem - and my capistrano-offroad package that adapts Capistrano for non-Rails applications. I will also talk about release management process, based on the Git version control system, upon which the deployment setup is built.

Transcript of Best Of Both Worlds

  • Best of both worlds Deploying Django projects with Capistrano and Supervisord Maciej Pasternacki SetJam, Inc. [email protected] Wednesday, May 26, 2010 1
  • Best of both worlds Should we only use Python tools with Django, or rather pick and mix best tools available, regardless of the ecosystem they come from? Wednesday, May 26, 2010 2
  • Background an online TV guide Started June2009, public beta December2009, incorporated February2010 Agile & Open Source-based Wednesday, May 26, 2010 3
  • Whats this talk about? Release management Automated deployment Upstream libraries Process management Scaling in cloud Centralized logging Wednesday, May 26, 2010 4
  • Release management Weekly iterations Devel Staging Production All in git branches Wednesday, May 26, 2010 5
  • Capistrano automating the deployment Separate, independent releases Mature & complete Full workow Transactions and rollbacks Wednesday, May 26, 2010 6
  • Capistrano directory layout releases YYYYMMDDHHMMSS 20100512210109 20100514101512 20100515085101 20100515101756 current log var/ solr_data shared processed_images Wednesday, May 26, 2010 7
  • deploy:setup Prepares one or more servers for deployment. deploy:default Deploys your project. deploy:update Copies your project and updates the symlink. deploy:update_code Copies your project to the remote servers. strategy.deploy! deploy:nalize_update [internal] Touches up the released code. deploy:symlink Updates the symlink to the most recently deployed release deploy:restart Restarts your application. http:/ Transaction / Wednesday, May 26, 2010 8
  • Capistrano-offroad an open source pack of Capistrano utils Reset Capistranos railsy defaults Handful of utility functions Django and Supervisord recipes Want to improve! Wednesday, May 26, 2010 9
  • Capistrano conguration samples: settings # Capistrano-offroad modules load 'lib/capistrano-offroad/offroad.rb' offroad_modules 'defaults', 'django', 'supervisord' # Server config server "", :web, :db, :primary => true role :app, "" group :staging_fetch, :backend set :deploy_to, "/srv/staging" set :deploy_user, "staging" set :deploy_group, "staging" set :python, "/opt/python26/bin/python2.6" set :django_project_subdirectory, "setjam" set :django_use_south, true Wednesday, May 26, 2010 10
  • Capistrano conguration samples: dependencies depend :remote, :command, "make" depend :remote, :command, "s3cmd" [] depend :remote, :python_module, "PIL" depend :remote, :python_module, "flup" depend :remote, :python_module, "lxml" depend :remote, :python_module, "MySQLdb" Wednesday, May 26, 2010 11
  • Capistrano conguration samples: tasks desc "[internal] finalize code update" task :finalize_update do run "ln -sfv #{shared_path}/local_settings*.py #{latest_release}/setjam/" run "make -C #{latest_release}/lib/ wait-for-cached download-cached" run "mv #{latest_release}/var #{latest_release}/var.orig" run "ln -sv #{shared_path}/var #{latest_release}/var" django_manage "synccompress", :roles => [:app, :web] _role_supervisord_conf :backend _role_supervisord_conf :app end desc "[internal] Wait until processes are reachable" task :wait, :roles => :app do run "#{current_path}/scripts/waitfcgi #{fcgi_port}" run "#{current_path}/scripts/waitfcgi #{fcgi_api_port}" end Wednesday, May 26, 2010 12
  • Capistrano conguration samples: hooks after "deploy:setup", "setjam:setup" before "deploy:finalize_update", "setjam:finalize_update" before "deploy:symlink", "smoke:offline" after "deploy:start", "smoke" after "deploy:restart", "smoke" before "smoke", "setjam:wait" before "deploy:cleanup" do set :use_sudo, true end after "deploy:cleanup" do set :use_sudo, false end Wednesday, May 26, 2010 13
  • Upstream libraries how to get them? System-wide? Project-wide (shared?) Release-wide! Wednesday, May 26, 2010 14
  • Upstream libraries standard approach: virtualenv+pip pulling from upstream VC? add local patches? cache library set? non-python dependencies? Wednesday, May 26, 2010 15
  • Upstream libraries current approach: Makele Make tasks to get and patch libs Timestamp les for dependencies Cache versioned tarballs in S3 Symlinks to use downloaded stuff Wednesday, May 26, 2010 16
  • Upstream libraries current approach: Makele all: Django.stamp _submodules.stamp beautifulsoup.stamp