Castles in the Cloud: Developing with Google App Engine
-
Upload
catherinewall -
Category
Technology
-
view
7.476 -
download
2
description
Transcript of Castles in the Cloud: Developing with Google App Engine
![Page 1: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/1.jpg)
Castles in the Cloud:Developing with Google App EngineKevin NoonanCalbane Ltd.IT@Cork, 20th April, 2009
![Page 2: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/2.jpg)
Introduction
Castles in the Cloud
![Page 3: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/3.jpg)
A working definition of Cloudware
• Outsourced Virtualizationo Virtual server running on outsourced server farm
![Page 4: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/4.jpg)
The Cloud Computing Pyramid
•Michael Sheehan of GoGrid devised this classification
![Page 5: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/5.jpg)
Main Players
![Page 6: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/6.jpg)
Other Players
• Microsoft Windows Azureo Including .NET Live Services
• Force.com from Salesforce
• SUN (recently announced)
• Mosso
• GoGrid
• FlexiScale
• Hosting365
• ...
![Page 7: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/7.jpg)
What's Cloudware good for? (I)
• Guerilla developmento Pay-per-use model is ideal for startupso Also good for guerillas inside the enterprise...
![Page 8: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/8.jpg)
What's Cloudware good for? (II)
• Scaling upo Clone the virtual servero pay for extra resources (bandwidth, storage, RAM, etc.)
![Page 9: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/9.jpg)
What's Cloudware good for? (III)
• In general, outsourcing IT infrastructureo pay someone else to run your servers, network, database...
![Page 10: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/10.jpg)
Who's using Cloudware & why not?Castles in the Cloud
![Page 11: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/11.jpg)
• Startupso Twittero SmugMugo Pixenate (Walter Higgin's site, from here in Ireland)o Many Facebook apps, e.g. iLIke
• Large firmso NASDAQo Activisiono Business Objectso Hasbroo New York Times
• See: http://aws.amazon.com/solutions/case-studies/
Startups & Large Firms on AWS
![Page 12: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/12.jpg)
• Startupso BuddyPoke! (30M daily pageviews on App Engine)o Pixverse (PixChat)o Hyperic (Cloudstatus)
• One large company's experiment:o GiftTag from BestBuy
• http://code.google.com/appengine/casestudies.html
Startups on Google App Engine
![Page 13: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/13.jpg)
• Immaturity of the technologyo Although AWS is over three years old
• Concerns about reliability
• Concerns about portabilityo no two clouds are the same
• Bureaucracy & inertia
• Lock-in to legacy systems
• Fear of loss-of-control & of job-losses
• ...
Barriers to Enterprise Adoption
![Page 14: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/14.jpg)
• Amazon S3 went down for two consecutive hours in Feb 2008.
• Amazon S3 (& Amazon SQS) went down for eight consecutive US daytime hours in
July 2008.
• App Engine had a five-hour partial-outage in June '08
• App Engine had a problem of increased latency (for web-serving) over a few days in
early Mar '09.
• Group for notification of App Engine downtimeo http://groups.google.com/group/google-appengine-downtime-notify
Black days in the Cloud
![Page 15: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/15.jpg)
Google App EngineCastles in the Cloud
![Page 16: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/16.jpg)
Python: The Original of the Species
![Page 17: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/17.jpg)
Java: the All New Flavour
![Page 18: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/18.jpg)
Languages on App Engine's JVM
• Java
• JRuby
• Jython
• BeanShell
• JavaScript
• Scala
• Groovy
• …
• Will it Play in App Engine?
http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine
![Page 19: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/19.jpg)
What is Google App Engine?• Complete web stack
o Soon to be more general computing platform
• Runs on Google's infrastructure
• Bigtable Datastoreo Like a sorted hashtableo Not a relational databaseo Use GQL (with Python) instead of SQLo Use JDO or JPA (with Java) for datastore work
• Memcache
• Image API
• Mail API
• Python library available for Force.com web services
• SDK for development on Windows/Mac/Linuxo Run & test locally before uploading to Cloud
![Page 20: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/20.jpg)
A fabulous app: 'Gaebar'
Alternative: use AppRocket to replicate an App Engine datastore to an SQL database
http://code.google.com/p/approcket/
![Page 21: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/21.jpg)
Limitations of Google App Engine• Applications run in “The Sandbox”:
http://code.google.com/appengine/docs/java/runtime.html#The_Sandbox
• Native Python modules (written in C) do not work.
• Many Java libraries will not work (but many have been patched or will be).
See the JRE whitelist: http://code.google.com/appengine/docs/java/jrewhitelist.html
• No “long-lived” processing (requests must return in 30 seconds).
• Limited “offline processing” (this is coming)o no forks, no spawned processes, no threadso Just scheduled tasks, which work like “cron jobs” on UNIX systems
• HTTPS only on XYZ.appspot.como and only with Google's certificate.o Due to limitation of SSL protocol, as applied to Google's Cloud.
![Page 22: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/22.jpg)
Roadmap to June for App Engine• Service for storing and serving large files
• Task queues for performing background processing
• Incoming email support
• XMPP (Jabber) API
![Page 23: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/23.jpg)
'Batch processing' on App Engine today• Use App Engine's task scheduling for “cron jobs” spaced a minute apart (or more seldom)
• A hack for more frequent jobs: run a cron-job on your PC or server to periodically hit a URL on App
Engine and kick off a task
• An outsourced hack for more frequent jobs: use webcron.org o do the same as above (perhaps more reliably!) o Cost: one hundredth of one Euro cent per 'hit'
• More sophisticated: hitch the clouds togethero 'App Engine Patch' has support for 'Boto'o 'Boto' is a Python library for Amazono Use Boto (on App Engine) to queue work-packets on Amazon's Simple Queue Service (SQS) o Kick off tasks on App Engine with a method above.o Cost: $0.01 per 10,000 requestso (+ $0.10 per GB transferred in...)
![Page 24: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/24.jpg)
Web tools for App Engine
• “Xmas gifts” - announced in late December 2008
• Zoho Creatoro generates code and web-pages for deploymento http://creator.zoho.com/
• App Engine Site Creatoro “Lightweight CMS” (with GUI interface)o http://code.google.com/p/app-engine-site-creator/o Apache 2.0 license
![Page 25: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/25.jpg)
Billing on Google App Engine• True pay-per-useo free quotas & no bill to pay if quotas not exceeded.o quite different model to AWS
• Developer can assign budget per resource.
• Priceso $0.10 per CPU core-houro $0.15 per GB-month of storageo $0.12 per GB outgoing bandwidtho $0.10 per GB incoming bandwidtho $0.0001 per email recipient for emails sent by the application
![Page 26: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/26.jpg)
Free Quotas on App Engine
•Daily outgoing bandwidth: 10 GB
•Daily incoming bandwidth 10 GB
•Daily requests 1.3 million
• Daily CPU-hours: 46
• More quotas (mail, datastore, images, memcache)
http://code.google.com/appengine/docs/quotas.html
![Page 27: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/27.jpg)
Other limitations on App Engine
• 1 GB of free storage per app.
•100 MB Memcache storage limit
• App Engine apps can now receive requests and send responses of up to 10MB.
• Caveat: 1 MB limit on datastore/memcache API call
• You can deploy up to 1,000 code and 1,000 static files with an app.
Workaround: use ZipImport (with Python) to store files in a Zip-archive:
http://code.google.com/appengine/articles/django10_zipimport.html
• 150MB cumulative limit on an app's code and static files
(as opposed to storage in the datastore).
![Page 28: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/28.jpg)
Atypical Uses of App Engine
• Run Open Source software (e.g. blog / CMS):
http:/groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects
• Store your static files on App Engineo You can use App Engine as rudimentary 'Content Delivery Network'
• Expose a web-service on App Engine (while doing your main development elsewhere)
REST library for Django & App Engine: http://code.google.com/p/python-rest/
• Expose an interface to BigTable and exploit its scalable storage:
REST interface to datastore: http://code.google.com/p/app3/
• Use RESTlets for Java equivalent of the two above:
http://wiki.restlet.org/docs_1.2/13-restlet/252-restlet.html
![Page 29: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/29.jpg)
Brief Case Study: LongURLPlease
• LongURLPlease expands “tiny URLs” to the original long versions.
• Original version written in a day by Darragh Curran, with barely any previous Python experience.
• Launched publicly on December 23rd, 2008.
• Rapidly became a platform in itself: at least nine other applications built on the API within 8 weeks
• Over four thousand downloads of the Firefox plugin
• 100K – 200K requests per day (each request can batch up to 4 URLs)
• Running costs: zero (since its usage is well below the free quotas)
![Page 30: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/30.jpg)
Where to go from here (Python)
• App Engine Helper tutorialo http://blog.pas.net.au/2009/02/17/django-and-google-app-engine-tutorial/
• App Engine Patch article (I recommend AEP over the “helper” above)o http://code.google.com/appengine/articles/app-engine-patch.html
• App Engine cheatsheet: http://short.ie/cheatongae
• Code samples o http://code.google.com/p/google-app-engine-samples/downloads/list
![Page 31: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/31.jpg)
Where to go from here (Java)
• Sign up for the Java “early look”
http://appengine.google.com/promo/java_runtime
• Install the Eclipse plugin (& SDK)
http://code.google.com/appengine/docs/java/tools/eclipse.html
(The Java samples are in the SDK.)
• Read Google's docs and follow the tutorial:
http://code.google.com/appengine/docs/java/overview.html
• Or pick your own language and get up-and-running on the JVM:
http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine
![Page 32: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/32.jpg)
2/27/2009
ReferenceCastles in the Cloud
![Page 33: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/33.jpg)
• 10 Reasons Enterprises Aren't Ready to Trust the Cloudo http://gigaom.com/2008/07/01/10-reasons-enterprises-arent-ready-to-trust-the-cloud/
• The Information Factories, by George Gildero http://www.wired.com/wired/archive/14.10/cloudware_pr.htmlo "When the network becomes as fast as the processor, the computer hollows out and spreads across the network."
- in 1993, from Sun Microsystems' CTO, Eric Schmidt (now CEO of Google)
• Above the Clouds: A Berkeley View of Cloud Computingo http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.html
Articles & Papers
![Page 34: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/34.jpg)
• Google App Engine: http://appengine.google.com
• Videos from conference Google held in 2008o http://sites.google.com/site/io/
• Chat with developers from the App Engine teamo first Wednesday of the month: 3am next day (Dublin/London time).o third Wednesday of the month: 5pm (Dublin/London time).o IRC channel: #appengine on irc.freenode.neto http://en.wikipedia.org/wiki/List_of_IRC_clients
• Google Groupo http://groups.google.com/group/google-appengine/
• Paper on Bigtable: o http://labs.google.com/papers/bigtable.html
Resources for Google App Engine
![Page 35: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/35.jpg)
Book (I)
Developing with Google App Engine
by Eugene Ciurana
Published:
Feb 2009
http://apress.com/
![Page 36: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/36.jpg)
Book (II)
Google App Engine in Action
by Noah Gift & Michael Orr
To be published:
May 2009
http://www.manning.com/gift/
![Page 37: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/37.jpg)
Book (III)
Google App Engine
by Dan Sanderson
To be published:
Sept 2009
http://oreilly.com/
![Page 38: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/38.jpg)
2/27/2009
Wrapping upCastles in the Cloud
![Page 39: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/39.jpg)
Credits• Photos used under Creative Commons license:
• Photo Creditso http://www.flickr.com/photos/notsogoodphotography/o http://www.flickr.com/photos/barto/o http://www.flickr.com/photos/kitoy/o http://www.flickr.com/photos/wolfgangstaudt/o http://www.flickr.com/photos/motumboe/o http://www.flickr.com/photos/ragnar1984/o http://www.flickr.com/photos/janekm/o http://www.flickr.com/photos/raylopez/o http://www.flickr.com/photos/pagedooley/o http://www.flickr.com/photos/gustty/o http://www.flickr.com/photos/mike9alive/o http://www.flickr.com/photos/tambako/o http://www.flickr.com/photos/nateo http://www.flickr.com/photos/freewine/
![Page 40: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/40.jpg)
Cloudware is the arsenal of freedom
• Go guerillao Experiment with cloudware o Consider using cloudware for your new projectso Test-drive your favourite language on the App Engine SDKo Await availability for access to the production Java runtime
![Page 41: Castles in the Cloud: Developing with Google App Engine](https://reader034.fdocuments.in/reader034/viewer/2022042814/55391fd74a79597c778b48d9/html5/thumbnails/41.jpg)
Contact Information
Kevin Noonan
kevin_noonan on Twitter
http://www.calbane.com