Automated Deployment
-
Upload
phpne -
Category
Technology
-
view
1.386 -
download
2
description
Transcript of Automated Deployment
![Page 1: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/1.jpg)
Automated DeploymentBuilding a simple automated deployment platform with PHP and Linux
Michael [email protected]
![Page 2: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/2.jpg)
whois?
• Senior / Lead Web Developer• Zend Certified Engineer• Published Author– PHP 5 Social Networking, PHP 5 E-Commerce
development & more
![Page 3: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/3.jpg)
Deployment: (an) old style approach
• Take website offline / put into maintenance mode
• Backup everything• Upload new files - FTP• Upgrade database• Put online, and hope for the best
• Do it twice: once for staging and once for deployment
![Page 4: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/4.jpg)
http://xkcd.com/303/
![Page 5: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/5.jpg)
The problem
• Down time for upgrades• Manual process– FTP takes time; – forgot to CHMOD? – Clients want to see progress now!– Bugs and issues can lie dormant for some time
![Page 6: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/6.jpg)
What about...
• Many existing solutions are geared towards large projects
• What about...– the little guy;– the small agency– the web app start up on an entry level VPS?
![Page 7: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/7.jpg)
What's in store?
A few simple techniques, scripts and ideas that we currently use to make deployment easy
![Page 8: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/8.jpg)
Deployment: the basics
• Get your latest code from version control, and stick it online
• Keep a central record of all the CHMOD / CHOWNing that you need to do
• Swap around your database connection details and other suitable configuration files
![Page 9: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/9.jpg)
SVN Export
• Start with a simple svn export
– Store the date/time in a variable – Create two folders, named with the current
date/time. One within the web root, one outside of it
– Two exports: public and private (or one export, and some moving around of folders – up to you!)
![Page 10: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/10.jpg)
#!/bin/bashDATE=`date +%H-%M-%e-%m-%y`mkdir /var/www/staging/$DATE/mkdir /var/www/staging-private/$DATE/
svn export --quiet --username phpne --password PhpN3 httP://localhost/svn/project/trunk /var/www/staging/$DATE/
svn export --quiet --username phpne --password PhpN3 http://localhost/svn/project/private /var/www/staging-private/$DATE/
![Page 11: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/11.jpg)
SVN Export
Keep your servers svn client happy! It will ask what to do with the svn password, and nobody will listen – so tell it!
sudo nano /var/www/.subversion/servers
store-plaintext-passwords = no
![Page 12: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/12.jpg)
AUTONOMYln –s /staging /live
![Page 13: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/13.jpg)
Autonomy
• When the latest code is checked out, tests have been run, uploads imported, configuration changed and database patched we need to swap this into place instantly
• The answer: symlinks
![Page 14: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/14.jpg)
#!/bin/bashDATE=`date +%H-%M-%e-%m-%y`
...
rm /home/user/public_html/
ln –s /var/www/staging/$DATE/ /home/user/public_html/
Sadly, you can’t edit a symlink, hence rm
![Page 15: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/15.jpg)
My user profile pictures aren’t in version control…
![Page 16: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/16.jpg)
User contributed files
• Store them elsewhere?– On a content delivery network?– On a sub-domain– Symlink them
• Copy them in post svn export?– A bit nasty and takes time, and what about new
user uploads during the copying process?
![Page 17: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/17.jpg)
THE DATABASE
![Page 18: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/18.jpg)
• Photo of database table not found, or mysql gone away error message
http://www.flickr.com/photos/meandmybadself/165846637/
![Page 19: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/19.jpg)
Database changes: patches
For database changes to apply on deploy, you need some deploy aware code in your project.
• Multi-query patch processing
• Schema compare; its easy to forget a database patch!
• Backup database before applying patches
![Page 20: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/20.jpg)
public function updateDatabase( $patchID, $some=false ) {
// look for the next patch if( file_exists( FRAMEWORK_PATH . '../database/patches/' . ++$patchID . '.php' ) ) {
$sql = file_get_contents( FRAMEWORK_PATH . '../database/patches/' . $patchID . '.php' );
// apply the changes from the patch mysqli_multi_query( $sql ); // lather, rinse and repeat$this->updateDatabase( $patchID, true );
} else if( $some ) {
// All done? Update patch ID in databasemysqli_query(“UPDATE settings SET `value`=” . $patchID-1 .
“ WHERE `key`=‘database-patch-id’ ” );exit();
} }
Apply your database patches
![Page 21: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/21.jpg)
$testTables = array();mysqli_select_db( $config['patched_db'] );$result = mysql_query("SHOW TABLES");while( $row = mysql_fetch_row($result) ) {
$testTables[ $row[0] ] = array();}foreach( $testTables as $table => $fields ){
$result = mysql_query("SHOW COLUMNS FROM " . $table );
while( $row = mysql_fetch_assoc( $result ) ) {
$tables[ $table ][ $row['Field'] ] = $row;}
}
Turn your database schema into an array
![Page 22: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/22.jpg)
Compare your patched database to what you expected
http://joefreeman.co.uk/blog/2009/07/php-script-to-compare-mysql-database-schemas/
![Page 23: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/23.jpg)
Databases: Test Database
• If you are applying changes to your database structure, you will need another test database
• Changes are first applied to the test database– Comparisons run against it– Unit testing run against code working with that
database• When all is clear, the live database can be
patched and upgraded
![Page 24: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/24.jpg)
Ask the audience
• Database integration, patching, testing and deployment is probably the weakest link in this deployment chain
![Page 25: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/25.jpg)
Unit testing
• While its good practice to only commit code which passes unit tests, sometimes a commit can break existing code if you are a lazy svn updater
• Run the unit tests against sandboxed code before pushing the deployment live
• Did the deployment fail?
![Page 26: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/26.jpg)
Unit testing
• Both PHPUnit and PHP SimpleTest have command line interface
• Options:– Parse the output and look for errors; then
continue once its done– Store a report, and require manual approval
before continuing with deployment
phpunit –testdox-text somefile.txt MyTests
*this isn’t a stage I’ve actually implemented in our deployment pipeline, just something I’m working on
![Page 27: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/27.jpg)
The problem with including Unit Tests
• Running unit tests take time
• We need to log deployment attempts, and try and deploy them once the tests have been run
• We need a central deployment system
![Page 28: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/28.jpg)
• Photo of USB “kill switch”
http://www.flickr.com/photos/stevendepolo/3517227492/
![Page 29: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/29.jpg)
Triggering deployment: PHP
echo shell_exec( ‘/var/deploy/deploy.sh ’ . $project . ‘ ‘ . $environment );
What about root?
Deployment script requires root access? Update sudoers file
![Page 30: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/30.jpg)
PHP Deploy as Root
• Edit the sudoers file– Sudo visudo
• Create an alias for your deployment scripts– Cmnd_Alias DPLY = /var/deploy/script1,
/var/deploy/script2• Let the webserver execute as root, without
requiring a password– www-data ALL=(ALL) NOPASSWD: DPLY
![Page 31: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/31.jpg)
Automating deployment
• Cron
• Postcommit hooks– Do this for your bleeding edge staging area; its
good to continually test code in its live server environment
• Scheduled deployments
![Page 32: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/32.jpg)
Deployment Infrastructure
• Deploying projects across multiple servers?
– Send your commands over SSH to a remote server
– Implement a skeleton deployment system on each server, called from a central deployment area
![Page 33: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/33.jpg)
Build a deployment platform
• Projects• Deployment areas:– Bleeding– Staging– Production
• Configurations, reports and deployment schedules
![Page 34: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/34.jpg)
Recap
1. Export your repository2. Apply your permission changes3. Swap in/out the appropriate configuration files4. Backup your (test) database5. Patch your database6. Unit test validation7. Swap in/out your configuration files8. Pull in user contributed files9. Backup your environment database10. Patch your live database11. Update your symlinks
![Page 35: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/35.jpg)
Rolling back
Shit! That last deployment didn’t go as planned!
• Symlinks let you keep copies• Database backup before patches were applied –
just incase• Database patch rollback files – allows you to keep
new data but undo structural changes• Make an undo button in your deployment platform;
if you don’t you will need it – if you do, you wont*!
* OK, I lied, you probably will at some point
![Page 36: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/36.jpg)
CAVEATS
Queue cheesy stock photo of confused bean figure
![Page 37: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/37.jpg)
Caveats
Some useful pointers when having multiple versions online (bleeding, staging and production)
• Keep robots out (robots.txt meta_robots)– You don’t want search engines taking your users to the
staging environment, nor do you want to be peanalised for duplicate content
• Keep unwanted users out (.htaccess or limited user database)
• Make it clear that the environment is non-production – in case a production user stumbles upon staging!
![Page 38: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/38.jpg)
Conclusion
• Deployment needs to take into account a lot of things
• Small and simple home-brew scripts, processes and techniques should help you out
• Look at pulling them together into a simple web-based deployment centre
![Page 39: Automated Deployment](https://reader033.fdocuments.in/reader033/viewer/2022061122/5471bc49b4af9fae0a8b4cb8/html5/thumbnails/39.jpg)
Deploy your projects quickly!@michaelpeacock
michaelpeacock.co.uk
http://slidesha.re/phpdeploy
http://www.flickr.com/photos/jurvetson/4853963652/sizes/m/in/photostream/