Real World Google App Engine Aral Balkan Jan 2009
52
Real-world Google App Engine Aral Balkan (@aral, http://aralbalkan.com )
-
Upload
aral-balkan -
Category
Design
-
view
52.999 -
download
1
Transcript of Real World Google App Engine Aral Balkan Jan 2009
Real-world Google App EngineAral Balkan (@aral, http://aralbalkan.com)
This is not an introduction to Google App Engine!
Google App Engine is really good at scaling...... a request/response model within various limitations and subject to certain quotas.
Request/responseNo long running processes or CRON jobs
Fake it!
No long-running processesWorkaround #1: Server-side redirect
return HttpResponseRedirect (“/recurse/lastKey/”)
No long-running processesWorkaround #2: Meta refresh
<meta http-equiv="refresh" content="0;url=/recurse/lastKey/" />
No long-running processesWorkaround #3: JavaScript
$.post( url, {lastKey: lastKey}, callback, " json ");
Sortable keysA welcome addition (1.1.6, Nov 21, 2008)
if 'last_key' in request.REQUEST: last_key = request.REQUEST['last_key']else: # Handle error and return ...
Get the last key value passed by the client.
if last_key == 'none': query = Model.gql('ORDER BY __key__')else: key_ref = db.Key(last_key) query = Model.gql('WHERE __key__ > :1 ORDER BY __key__', key_ref)rows = query.fetch(ROWS_PER_GO+1)
Sort by key, using last key if provided and fetch the number of rows you want to affect.
if len(rows) == ROWS_PER_GO+1: # There's more, we need to recurse again new_last_key = str(rows[ROWS_PER_GO].key()) ...else: # This is the last iteration.
Check if you need to recurse. If so, set the new last key value.
No CRON jobsCannot run periodical tasks
WebBasedCron.comWorkaround: use an external CRON service.
Size matters1MB limit on any object
aws.amazon.com/s3/Workaround: Use Amazon S3 & CloudFront
infinitebits.orgFTP to S3 service by Benjamin Allfree
pyamf.appspot.com/shell/Ajax shell: http://shell.appspot.com/
Growing pains1,000 file limit per application
Growing painsWorkaround: zipimport
Quotas
Quotas
CPU time: 46.30 hours
Bandwidth: 10GB (out/in, 20GB total)
Secure bandwith: 2GB (out/in, 4GB total)
2000 emails
0.5GB stored data
Evil Quotas
Evil quota #2Short-term email limit (officially 2, empirically 30)
Sailthru.com triggermail$1.50 cpm mass mail, $6.00 transactional
Web Hooks (Jeff Lindsay)http://webhooks.pbwiki.com/
smtp2webhttp://smtp2web.com/
astrotrainhttp://github.com/entp/astrotrain/tree/master
Secret evil quota solution(Ssshh, don’t tell anyone)
Please sir, may I have some more?
Django or webapp?Which framework should you use?
webapp
“Official” framework
Not very active
Guido doesn’t use it
Allows subclassing
You’ll likely use bits of it anyway
Plain
Django
Lots more functionality for free
Very active
Guido uses it
No subclassing of views
Great community
Plain rocks!
DjangoWhich helper should you use?
(Depends on your needs)
Google App Engine HelperThe more “official” helper
app-engine-patchThe community “helper” led by Waldemar Kornewal (@wkornewald)
Helper
More official
Slow updates
Django models
Patch
Community-run
Frequent updates
Webapp models
Admin panel
Self-contained, reusable apps.
Context-issues solved in GAE 1.1.8
GAE! What is it good for?Use Cases and Don’t Use Cases
UCs
APIs
Facebook/Open Social apps
Static sites
DUCs
Ecommerce (no SSL)
Sites that require admin features (no long running processes)
CDN (file size limit of 1MB)
Why Google App Engine is broken and what Google must do to fix it.http://aralbalkan.com/1504
App Engine’s Achilles' heelA missing modus operandi
Request/response
Needs to scale
Potentially hit by millions of users
Performance, performance, performance!
Exists
Admin
Does not need to scale
Infrequently hit by very few users
Flexibility, ease-of-use, batch processes.
Data import/export
Does not exist
Backups and restoreshttp://aralbalkan.com/1784
Backup
Restore
Local app
Deployment app
Staging app(to python code)
(executes code)
Local app
Deployment app
Staging app
GaebarGoogle App Engine Backup and Restore
Because if you’re a multi-billion dollar company and you don’t build data export into your snazzy new multi-million dollar cloud platform, someone else will, and they’ll make it pink.
Made with ♥ by Aral Balkan.
Roadmap to March ’09Serving large files, datastore import/export, billing, new runtime language, uptime monitoring
Quick tips
memcache everything... Then memcache everything again.
Build good relations with the App Engine team
Realize that you’re working with a beta technology
Do not use the keys generated by the datastore in URLs (convenient but not reproducible if you move your datastore).
Do not use global counters (use sharded counters).
Quick tipsUse Google Analytics
Do as little as possible in error handlers.
Outsource as much as possible to external APIs (but remember the weakest link)
There’s still no usable SSL (works only on appspot.com)
ListProperties rock (do an “in”) db.ListProperty(db.Key)
Use sortable keys for batch processes
My GAE Patches
Hit one instance of the local dev server from another:http://aralbalkan.com/1755
Bypassing local server restrictions http://aralbalkan.com/1440
Macports Python Google App Engine Gotchahttp://aralbalkan.com/1692
Saving a Float or GeoPt in datastorehttp://aralbalkan.com/1355
My GAE Sites
European VAT validation APIhttp://isvat.appspot.com/
The GAE SWF projecthttp://gaeswf.appspot.com/
Open country codes (ISO 3166 names & codes)http://opencountrycodes.appspot.com/
<head> web conferencehttp://www.headconference.com
Day 60 -- For Every Action... by teresiahttp://www.flickr.com/photos/tea_time/2499015201/
Norine and a huge cucumber by Freekz0rhttp://www.flickr.com/photos/donnie_bling/2503656170/
080217 evil laugh by Dan4thhttp://www.flickr.com/photos/dan4th/2273626372/
Bombay clocks by Natmanduhttp://flickr.com/photos/mybigtrip/6111406/
"can I have a candy" version dog by mouton.rebellehttp://flickr.com/photos/mouton/208355248/
Old keys by practicalowlhttp://flickr.com/photos/practicalowl/248282683/
Shells by Fr Antuneshttp://flickr.com/photos/franciscoantunes/2714146667/
