Deployment talk dpc 13
-
Upload
robbert-van-den-bogerd -
Category
Technology
-
view
1.065 -
download
0
Transcript of Deployment talk dpc 13
PowerPoint-presentatie
Deploying your apps.. the right way!
Robbert van den Bogerd
June 8th, 2013
Introduction
whoami
Developer at Ibuildings
Web enthusiast
Traveller
The Average Joe
Disclaimer
This talk is heavily opinionated. All tools mentioned could be replaced by alternatives, but I'll show you some tools and techniques which I consider best for the job.
This is not a talk about continuous deployment, although most of the ideas and best practices shown are part of the concept.
DEPLOYMENT
So what are we here for?
TOC
Horror storiesCommon pitfallsWhat we wantDeployment from beginning to endIntegration tools
Horror stories
Horror stories
FTP: Forever Taking Process
FTP is a great protocol for transferring (large) files between computers, but what it's not so great for is deploying your 25000 files counting web project from your 2mb/s home network while your web visitors are waiting for it to finish.
Horror stories
Rsync: for oldschool sysadmins
Horror stories
Versioning checkout
Horror stories
Teamviewer: REALLY?
Problems & solutions
Common pitfalls
Manual operations
Sloow deployment process
Downtime
Cache
Migrations
No turning back
Obviously, we would all prefer to never take our website down, not even when updating. But in practice most developers find it acceptable and even customers don't find it odd that their website is down for maintenance (when announced on beforehand of course). But, ideally, our customers and, more importantly, their customers should have no idea of what's going on on the server while buying new shoes in your awesome webshop.As discussed in the previous slides, most traditional deployment strategies require the developer to perform some additional manual operations after or before uploading the updated code. This often makes the developer want to work fast, because the website is down or even broken. This results in stress and often leads to mistakes made by the deployer.Some applications are very heavy and are heavily dependant on caching. After updating the code, the cache also needs to be renewed, which willl cause the first couple of page views to be slow. This might sound as a trivial problem, but in practice, when deploying multiple times a day, it will also result in multiple slow downs each day.Some deployment processes take forever to complete, which might not be a big thing when you're releasing every 3 months, but when your team wants to get new features out of the door quickly, your productivity rate will decrease dramatically when your team mates are waiting 40 minutes a day for 3 deployments to finish.There is soo much that could potentially go wrong when deploying. Database migrations gone wrong, permission errors, connection problems and so on. But, no matter what goes wrong, you should ALWAYS be able to go back and revert, so that you can investigate the problem, at your own pace, in your own time. Not when your website is throwing fatals at your visitors.
Manual operations
No!
The Bus Factor
Anyplace, anywhere, anytime
Confidence
No turning back
Zero downtime
No more UC pages!
Quick deploys
Smaller releases
Automate
Wherever you can optimize, optimize!Higher release frequency means smaller deploys means less risk of exceptionsMain point of presentation: optimize & automate!
Cache
Migrations
Common pitfalls
Manual operations
Sloow deployment process
Downtime
Cache
Migrations
No turning back
Obviously, we would all prefer to never take our website down, not even when updating. But in practice most developers find it acceptable and even customers don't find it odd that their website is down for maintenance (when announced on beforehand of course). But, ideally, our customers and, more importantly, their customers should have no idea of what's going on on the server while buying new shoes in your awesome webshop.As discussed in the previous slides, most traditional deployment strategies require the developer to perform some additional manual operations after or before uploading the updated code. This often makes the developer want to work fast, because the website is down or even broken. This results in stress and often leads to mistakes made by the deployer.Some applications are very heavy and are heavily dependant on caching. After updating the code, the cache also needs to be renewed, which willl cause the first couple of page views to be slow. This might sound as a trivial problem, but in practice, when deploying multiple times a day, it will also result in multiple slow downs each day.Some deployment processes take forever to complete, which might not be a big thing when you're releasing every 3 months, but when your team wants to get new features out of the door quickly, your productivity rate will decrease dramatically when your team mates are waiting 40 minutes a day for 3 deployments to finish.There is soo much that could potentially go wrong when deploying. Database migrations gone wrong, permission errors, connection problems and so on. But, no matter what goes wrong, you should ALWAYS be able to go back and revert, so that you can investigate the problem, at your own pace, in your own time. Not when your website is throwing fatals at your visitors.
What we want
Manual operations Automation
Sloow deployments Quick rollouts
Zero Downtime
Cache warmed up
Migrations
No turning back Rollbacks
From the beginning..
Victorious warriors first win and then go to war
- Sun Tzu
Deployment is an often forgotten topic when sitting around the drawing table. As before mentioned requirements will take time and therefore cost more money, you should mention and discuss this topic in the early stages of your project. It is essential for a successful deployment strategy.
Think before you (sign a contr)act!
Deployment is an often forgotten topic when sitting around the drawing table. As before mentioned requirements will take time and therefore cost more money, you should mention and discuss this topic in the early stages of your project. It is essential for a successful deployment strategy.
Managing expectations
Customer
UsersProject managerDevs/testers
Also, you need to manage the expectations for everybody involved in the project. Since deployment is such a critical part of your development cycle, everyone should know how to act and what to expect when working on the project.Also, users should of course be informed about updates, maintenance windows and receive proper error messages when something goes wrong while deploying.
Deployment strategy
Release cycle
Server setup
Branching strategy
Tools to use
DTAP
So, let's cut to the point..
Use git
Seriously, use git!
Reliable
Super fast
Easy branching/merging
Distributed
Github:
Awesome ui
Extra's
PR's
Integration with existing deployment tools
Dependency management
Framework
Libraries
Build/QA tools
Company private repo's
No matter how cool your versioning system is, and how good your migrations script can rollback, always make sure there is a backup of your stuff before you actually start breaking things.
Drumroll please..
Composer
Migrations
Write your own script
Use one of the great existing ones:
- Doctrine migrations
- DbPatch
- Shipped with your framework/orm?
Backups
Always create one!
..
unless it's not possible =]
No matter how cool your versioning system is, and how good your migrations script can rollback, always make sure there is a backup of your stuff before you actually start breaking things.
Directory structure
/var/www/rule34/
/var/www/rule34/releases
/var/www/rule34/releases/20130606143000
/var/www/rule34/releases/20130606143400
Live www dir:
/var/www/rule34/current > releases/20130606143400
Deployment script
> Do a backup
> Create new directory
> Update your code
> Update dependencies
> Run migrations
> Warmup cache
> ...
So now we're getting more and more close to having us a deployment script
Let's start scripting!
Wait a second..
Existing tools
Fabric
Phing/Ant
Chef
Capistrano
Capistrano
Powerful
Easy setup
Event/task based, easy to hook into
Written in ruby
Multiple servers
Multistage extension
Capifony
The power of Capistrano
Pre-configured for Symfony
Deploy with 1 single config file
Setup
Local setup
gem install capifony
cd /var/www/rule34/
capifony install .
Capifony config
# app/config/deploy.rb
set :application, "My App"
set :deploy_to, "/var/www/my-app.com"
set :domain, "my-app.com"
set :scm, :git
set :repository, "ssh-gitrepo-domain.com:/path/to/repo.git"
Remote setup
cd /var/www/rule34/
cap deploy:setup
Deploy
Capifony deploy
cd /var/www/rule34/
cap deploy
# made a boo-boo?
cap deploy:rollback
Configuration
set :model_manager, "doctrine"
Configuration
set :update_vendors, false
set :use_composer, true
set :vendors_mode, "install"
Configuration
set :dump_assetic_assets, true
Configuration
set :shared_children, [app_path+"/logs", "data"]
set :writable_dirs, [app_path+"/logs", app_path+"/cache", "data"]
Directory structure
/var/www/rule34/releases
/shared
/shared/data
/shared/app/logs
Symlink:
/var/www/rule34/releases/20130606143400/data > ../../shared/data
DTAP
Multistaging
# app/config/deploy.rb
require 'capistrano/ext/multistage'
set :stages, %w(prod acc test dev)
set :default_stage, "test"
set :stage_dir, "app/config/deploy/stages"
# app/config/deploy/stages/test.rb
server 'rule34.mytestserver.com'
set :symfony_env_prod, "test"
User permissions
set :use_sudo, false
set :user, "deployment"
set :group, "www-data"
set :webserver_user, "www-data"
:use_set_permissions, true
:permission_method, chmod # or acl or chown
Maintenance page
cap deploy:web:disable \
REASON="hardware upgrade" \
UNTIL="12pm Central Time"
# override template
set :maintenance_template_path, app/Resources/uc.html
Tips 'n Tricks
# determine tag/branch/commit/sha1 on the fly
set :branch do
default_tag = `git describe --abbrev=0 --tags`.split("\n").last
tag = Capistrano::CLI.ui.ask "Tag/branch to deploy (make sure to push the tag first): [#{default_tag}] "
tag = default_tag if tag.empty?
tag
end
Tips 'n Tricks
# enable ssh client interaction
default_run_options[:pty] = true
Tips 'n Tricks
# enable agent forwarding
ssh_options[:forward_agent] = true
Tips 'n Tricks
# cleanup old release dirs automatically
set :keep_releases, 3
after "deploy", "deploy:cleanup"
Tips 'n Tricks
# run doctrine migrations after code update
after update_code, doctrine:migrations:migrate
Tips 'n Tricks
# run any symfony console commandbefore any_task, symfony:cache:clear
What's next?
System provisioning
Upgrade PHP 5.3 > 5.4
Upgrade MySql
Install Redis
Install Node.js
Etc..
Chef
Version control your infrastructure
Identical environments
Integration with Vagrant
Chef
# run chef after code update
after update_code, provision
task :provision do
end
Chef
# run chef after code update
after update_code, provision
task :provision do
run "#{sudo} chef-solo -c #{release_path}/chef/config.rb #{release_path}/chef/chef_#{stage}.json"
end
Vagrant
Build automation
Jenkins
The end result
The usual stuff
Twitter: @nickedname
Joind.in:https://joind.in/8461
Further reading
A basic git capistrano workflow
http://labs.headshift.com/2012/02/29/a-basic-gitcapistrano-workflow/
A successful git branching model
http://nvie.com/posts/a-successful-git-branching-model/
The dbPatch project
http://www.dbpatch-project.com/
Continuous deployment with symfony2, jenkins and capifony
http://jeremycook.ca/2012/11/04/continuous-deployment-with-symfony2-jenkins-and-capifony/
Software branching and parallel universes
http://www.codinghorror.com/blog/2007/10/software-branching-and-parallel-universes.html
Naam Presentator