Care and Feeding of Large Web Applications
-
Upload
perrin-harkins -
Category
Technology
-
view
380 -
download
4
description
Transcript of Care and Feeding of Large Web Applications
![Page 1: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/1.jpg)
Care and FeedingCare and Feeding of of Large Web ApplicationsLarge Web Applications
Perrin HarkinsPerrin Harkins
![Page 2: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/2.jpg)
Arcos, a project at Plus Three
About 2.5 years of continuous development
2 - 5 developers on the team during that time
~79,000 lines of Perl code
~4900 lines of SQL
(Computed with David Wheeler's SLOCCount program)
Plenty of HTML, CSS, and JavaScript too
~14000 automated tests in 269 files
169 CPAN modules
![Page 3: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/3.jpg)
Arcos, cont'd
CMS with modern AJAX UI
E-commerce
Data warehouse with AJAX query builder GUI
E-mail campaign management
Asynchronous job queue system
Comprehensive reporting
![Page 4: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/4.jpg)
Arcos, cont'd
Object-oriented
MVC-ish
Controller classes
Database classes
Templates
Core modules:
CGI::Application
Class::DBI
HTML::Template
![Page 5: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/5.jpg)
Deployment
Hard to generalize
.tar.gz files
Always release full builds, not individual files
Consistent state for production system
QA installs are accurate copies
![Page 6: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/6.jpg)
The CPAN problem
“Just use the CPAN shell”
Fails too often
Installs whatever the latest version is
![Page 7: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/7.jpg)
CPAN installer requirements:
Install specific versions
Install from local media
Allow locally modified versions
Fully automated
“Erase your hard drive? [n]”
Install in a local directory
Skip the tests
Cluster install
Some will never work
![Page 8: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/8.jpg)
Our solution
Finds all modules in src/ directory of package
(Including ones we hacked)
Uses Expect to answer questions
Builds non-CPAN stuff too
Apache
mod_perl
SWISH-E search engine
![Page 9: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/9.jpg)
Why bundle dependencies?
Avoid troubleshooting local problems
If possible, specify Perl, MySQL, OS, etc.
Sometimes reality intervenes
Hardware support
Office politics
![Page 10: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/10.jpg)
Next up: binary distributions
Compile takes too long
Still just tar?
RPMs?
PAR?
See build system in Krang: http://krancgcms.com/
![Page 11: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/11.jpg)
OH HAI!OH HAI!
I HELP WIF URI HELP WIF URINDENTATION!INDENTATION!
![Page 12: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/12.jpg)
Upgrades
Need full automation
Got the database part
Schema version number in database
Run all upgrade scripts with appropriate names
e.g. 2.0 --> 3.0 means run upgrade/V2_1.pm and upgrade/V3_0.pm
Hand-written SQL scripts
Test mode that upgrades from old schema first
LVM for rollback in QA
![Page 13: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/13.jpg)
Configuration
● Highly configurable systems must be highly configured
● Started with httpd.conf style– Config::ApacheFormat
– Basic scoping and inheritance
![Page 14: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/14.jpg)
Observations on Configuration
● People ignore options they don't understand– If the server starts, it must be ok!
● Some comments in a config file == weak documentation
● Things you rarely change shouldn't be in the shipped config file
![Page 15: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/15.jpg)
I CAN HASI CAN HASMAINTENANCEMAINTENANCE
BRANCH?BRANCH?
![Page 16: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/16.jpg)
Version Control
Many tool choices now
svn, svk, git...
Fight it out and let me know who wins
Most projects need at least two branches at all times
development
stable
![Page 17: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/17.jpg)
Very simple version control
Main branch is development
Must build and pass tests
When making a release from a branch, tag it
Make a “2.0” tag when you release it
![Page 18: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/18.jpg)
Very simple version control, cntd
Maintenance branch from that tag for bug fixes
“2.x” branch
On maintenance release, merge all changes on maintenance branch to main branch
Tag “2.1” on 2.x branch
Merge 2.0 --> 2.1 onto main branch
![Page 19: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/19.jpg)
Too simple?
Not everyone is done at the same time
Hard to keep dev branch stable for major changes
Feature branches
Possible integration problems
Beware of complex merges
This is not the Linux kernel
Could be a people problem
An alternate point of view
http://utsl.gen.nz/talks/git-svn/intro.html
![Page 20: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/20.jpg)
HALP!HALP!TESTS R TESTS R FAYLIN!FAYLIN!
![Page 21: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/21.jpg)
Testing
● You all know the drill● Local library for common testing tasks
– Log in mech object
– Run test SMTP server● Test data setup and teardown
– Keep stack, delete in reverse order
– Probably easier to just wipe the db
![Page 22: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/22.jpg)
Testing tools
● Simple Test::More scripts are easy– But scoping becomes an issue
– Test::Class
– Test::Builder● Improves test failure messages
● Test::WWW::Mechanize– Great for testing SSL, Apache config, mod_rewrite
– Works fine on JavaScript pages● But doesn't test the JavaScript● Selenium does
![Page 23: Care and Feeding of Large Web Applications](https://reader035.fdocuments.in/reader035/viewer/2022081519/557bca3fd8b42a700d8b4d91/html5/thumbnails/23.jpg)
KTHXBYE!KTHXBYE!