Best Of Both Worlds

Click here to load reader

  • date post

    03-Sep-2014
  • Category

    Technology

  • view

    3.039
  • download

    2

Embed Size (px)

description

While Django is a great Web framework, there are places where Python ecosystem falls short of its competitors. However, nothing prevents us from picking the best parts to create a mixed environment that is productive and nice to work with. I will present my approach to deployment of Django webapps controlled by supervisord, using Capistrano - an automated deployment tool belonging to the Ruby on Rails ecosystem - and my capistrano-offroad package that adapts Capistrano for non-Rails applications. I will also talk about release management process, based on the Git version control system, upon which the deployment setup is built.

Transcript of Best Of Both Worlds

  • Best of both worlds Deploying Django projects with Capistrano and Supervisord Maciej Pasternacki SetJam, Inc. [email protected] Wednesday, May 26, 2010 1
  • Best of both worlds Should we only use Python tools with Django, or rather pick and mix best tools available, regardless of the ecosystem they come from? http://www.setjam.com/ Wednesday, May 26, 2010 2
  • Background SetJam.com: an online TV guide Started June2009, public beta December2009, incorporated February2010 Agile & Open Source-based http://www.setjam.com/ Wednesday, May 26, 2010 3
  • Whats this talk about? Release management Automated deployment Upstream libraries Process management Scaling in cloud Centralized logging http://www.setjam.com/ Wednesday, May 26, 2010 4
  • Release management Weekly iterations Devel Staging Production All in git branches http://www.setjam.com/ Wednesday, May 26, 2010 5
  • Capistrano automating the deployment Separate, independent releases Mature & complete Full workow Transactions and rollbacks http://www.setjam.com/ Wednesday, May 26, 2010 6
  • Capistrano directory layout releases YYYYMMDDHHMMSS 20100512210109 20100514101512 20100515085101 20100515101756 current log var/ solr_data shared processed_images local_settings.py http://www.setjam.com/ Wednesday, May 26, 2010 7
  • deploy:setup Prepares one or more servers for deployment. deploy:default Deploys your project. deploy:update Copies your project and updates the symlink. deploy:update_code Copies your project to the remote servers. strategy.deploy! deploy:nalize_update [internal] Touches up the released code. deploy:symlink Updates the symlink to the most recently deployed release deploy:restart Restarts your application. http:/ Transaction /www.setjam.com/ http://www.capify.org/index.php/Default_Deployment_Path Wednesday, May 26, 2010 8
  • Capistrano-offroad an open source pack of Capistrano utils Reset Capistranos railsy defaults Handful of utility functions Django and Supervisord recipes Want to improve! http://www.setjam.com/ Wednesday, May 26, 2010 9
  • Capistrano conguration samples: settings # Capistrano-offroad modules load 'lib/capistrano-offroad/offroad.rb' offroad_modules 'defaults', 'django', 'supervisord' # Server config server "staging.setjam.com", :web, :db, :primary => true role :app, "molly.setjam.com" group :staging_fetch, :backend set :deploy_to, "/srv/staging" set :deploy_user, "staging" set :deploy_group, "staging" set :python, "/opt/python26/bin/python2.6" set :django_project_subdirectory, "setjam" set :django_use_south, true http://www.setjam.com/ Wednesday, May 26, 2010 10
  • Capistrano conguration samples: dependencies depend :remote, :command, "make" depend :remote, :command, "s3cmd" [] depend :remote, :python_module, "PIL" depend :remote, :python_module, "flup" depend :remote, :python_module, "lxml" depend :remote, :python_module, "MySQLdb" http://www.setjam.com/ Wednesday, May 26, 2010 11
  • Capistrano conguration samples: tasks desc "[internal] finalize code update" task :finalize_update do run "ln -sfv #{shared_path}/local_settings*.py #{latest_release}/setjam/" run "make -C #{latest_release}/lib/ wait-for-cached download-cached" run "mv #{latest_release}/var #{latest_release}/var.orig" run "ln -sv #{shared_path}/var #{latest_release}/var" django_manage "synccompress", :roles => [:app, :web] _role_supervisord_conf :backend _role_supervisord_conf :app end desc "[internal] Wait until processes are reachable" task :wait, :roles => :app do run "#{current_path}/scripts/waitfcgi #{fcgi_port}" run "#{current_path}/scripts/waitfcgi #{fcgi_api_port}" end http://www.setjam.com/ Wednesday, May 26, 2010 12
  • Capistrano conguration samples: hooks after "deploy:setup", "setjam:setup" before "deploy:finalize_update", "setjam:finalize_update" before "deploy:symlink", "smoke:offline" after "deploy:start", "smoke" after "deploy:restart", "smoke" before "smoke", "setjam:wait" before "deploy:cleanup" do set :use_sudo, true end after "deploy:cleanup" do set :use_sudo, false end http://www.setjam.com/ Wednesday, May 26, 2010 13
  • Upstream libraries how to get them? System-wide? Project-wide (shared?) Release-wide! http://www.setjam.com/ Wednesday, May 26, 2010 14
  • Upstream libraries standard approach: virtualenv+pip pulling from upstream VC? add local patches? cache library set? non-python dependencies? http://www.setjam.com/ Wednesday, May 26, 2010 15
  • Upstream libraries current approach: Makele Make tasks to get and patch libs Timestamp les for dependencies Cache versioned tarballs in S3 Symlinks to use downloaded stuff http://www.setjam.com/ Wednesday, May 26, 2010 16
  • Upstream libraries current approach: Makele all: Django.stamp _submodules.stamp beautifulsoup.stamp