Writing Apps the Google-y Way (Brisbane)

WRITING APPS THE GOOGLE-Y WAYPamela Fox, YOW! Australia 2010 (Brisbane)

Who am I?

twitter.com/pamelafox

[email protected]

pamelafox.org

you get the idea...

Who am I?

Google Maps API Google Wave API

2006 20102008

Google App Engine

Who am I?

wave side projects

92 apps

Who am I?

Java pYthon

What is App Engine?

“Google App Engine enables you to build and host web apps on the same systems that power Google applications.”

http://code.google.com/appengine

What is a “web app”?

Static vs. Dynamic

Anonymous vs. Users

Intranet vs. Internet

~2 billionHundreds - Thousands

What is a “web app”?

Some Google web apps

Some Google App Engine web apps

www.gifttag.comwww.buddypoke.com

Google apps on App Engine

panoramio.com pubsubhubbub.appspot.com

How does App Engine work?

1. You upload application code & resources to Google.

2. Google serves your application from scalable infrastructure.

3. You pay for only the resources that Google used in serving the application.

Demo: Guestbook

awesomest-app.appspot.com

http://code.google.com/p/google-app-engine-samples/source/browse/trunk/guestbook

appengine.google.comlocalhost

build deploy monitor

App Engine architecture


user

task


LIMIT

CPU

LIMIT

Memory

LIMIT

Time


hardwareports

globalsfile

system

Groovy, JRuby, Mirah, Clojure, Scala


141,241,791 calls1 GB data

$0.15 GB/month

45,000,000 calls

657,000 - 46,000,000 calls

*Always check docs for latest quotas.

192,672,000 calls558 GB data$0.15 GB/month

7,000 - 1,700,000 calls$0.0001 per mail sent

46,000,000 calls 1,046 GB data sent

100,000 - 20,000,000 calls

The tricky bits

Datastore

Entity

PropertiesKey

Entity

Entity

Entity

Entity

Path Kind Name/ID

Example: Speaker Entities

Key Path

Kind ID First Name

Last Name

Speaker1

- Speaker 1 Rod Johnson

Key Path

Kind ID First Name

Last Name

Middle Name Suffix

Speaker2 - Speaker

2 Guy Steele L Jr.

Modeling Speaker Entities

class Speaker(db.model):

firstname = db.StringProperty(required=True)

lastname = db.StringProperty(required=True)

middlename = db.StringProperty()

namesuffix = db.StringProperty()

website = db.StringProperty()

keynote = db.BooleanProperty(default=False)

Saving Speaker Entities

rod = Speaker(firstname="Rod", lastname="Johnson")

guy = Speaker(firstname="Guy", lastname="Steele",

middlename="L", namesuffix="Jr.")

rod.put()

guy.put()

Updating Speaker Entities

rod = Speaker.get_by_id(1)

guy = Speaker.get_by_id(2)

rod.website = "http://www.sexyspring.com"

rod.keynote = True

guy.website = "http://www.lusciouslisp.com"

guy.keynote = True

db.put(rod, guy)

LIMIT!(size/# of batch ops)

Queries & Indexes

Query Index

Index

Index

Index

Query

Query

Query

Query

Query

Queries & Indexes

SELECT * from Speaker ORDER BY lastname

key lastname

Speaker3 Fox

Speaker4 Hohpe

Speaker1 Johnson

Speaker2 Steele

LIMIT!(# of results)

Queries & Indexes

SELECT * from Speaker ORDER by middlename

key middlename

Speaker2 L

Queries & Indexes

SELECT * from Speaker WHERE keynote = True

key keynote

Speaker1 True

Speaker2 True

Speaker3 False

Speaker4 False

Queries & Indexes

SELECT * from Speaker WHERE keynote = False

key keynote

Speaker1 True

Speaker2 True

Speaker3 False

Speaker4 False

Queries

allspeakers = Speaker.all().order('lastname')

for speaker in allspeakers:

print speaker.firstname + '' + speaker.lastname + '' + speaker.website

keynotespeakers = Speaker.all().filter('keynote = ', True)

notspecialspeakers = Speaker.all().filter('keynote = ', False)

LIMIT!(size of results)

Custom Indexes

SELECT * from Speaker ORDER BY lastname, keynote

key lastname keynote

Speaker3 Fox false

Speaker4 Hohpe false

Speaker1 Johnson true

Speaker2 Steele true

speakers = Speaker.all().order('lastname')

.order('keynote')

Custom Indexes

SELECT * from Speaker WHERE lastname > 'Johnson' and keynote = true

key lastname keynote

Speaker3 Fox false

Speaker4 Hohpe false

Speaker1 Johnson true

Speaker2 Steele true

speakers = Speaker.all().order('lastname')

.filter('keynote =', True)

Impossible Indexes

SELECT * from Speaker WHERE lastname < 'Steele' and firstname > 'Gregory'

key lastname firstname

Speaker3 Fox Pamela

Speaker4 Hohpe Gregory

Speaker1 Johnson Rod

Speaker2 Steele Guy

...not in subsequent rows!

Impossible Indexes

SELECT * from Speaker WHERE lastname > 'Fox' ORDER BY firstname

key lastname firstname

Speaker3 Fox Pamela

Speaker4 Hohpe Gregory

Speaker1 Johnson Rod

Speaker2 Steele Guy

...not in the correct order!

Queries with Offset

SELECT * from Speaker LIMIT 2 OFFSET 2

key lastname

Speaker3 Fox

Speaker4 Hohpe

Speaker1 Johnson

Speaker2 Steele

speakers = Speaker.all().fetch(2, 2)

1

2

...slow! LIMIT!(# of offset)

Queries with Cursors

query = db.Query(Speaker)

speakers = q.fetch(1000)

cursor = q.cursor()

memcache.set('speaker_cursor', cursor)

...

last_cursor = memcache.get('speaker_cursor')

q.with_cursor(last_cursor)

speakers = q.fetch(1000)

More Properties

class Talk(db.Model):

title = db.StringProperty(required=True)

abstract = db.TextProperty(required=True)

speaker = db.ReferenceProperty(Speaker)

tags = db.StringListProperty()

pamela = Speaker.all().filter('firstname = ', 'Pamela').get()

talk = Talk('Writing Apps the Googley Way', 'Bla bla bla',

pamela, ['App Engine', 'Python'])

talk.put()

talk = Talk('Wonders of the Onesie', 'Bluh bluh bluh',

pamela, ['Pajamas', 'Onesies'])

talk.put()

Back-References


for talk in pamela.talk_set:

print talk.title

key speaker

Talk6 Speaker2

Talk1 Speaker3

Talk2 Speaker3

Talk5 Speaker4

SELECT * from Talk WHERE speaker = Speaker3

Searching List Properties

talks = Talk.all().filter('tags = ', 'python')

.fetch(10)

SELECT * from Talk WHERE tags = 'Python'

key lastname

Talk1 App Engine

Talk2 Pajamas

Talk1 Python

Talk2 Onesies

LIMIT!(# of index rows)

Update Transactions

commitjournal apply entities

apply indexes

A B

Entity Groups


talk1 = Talk('Writing Apps the Googley Way', 'Bla bla bla',

pamela, ['App Engine', 'Python'],

parent=pamela)

talk2 = Talk('Wonders of the Onesie', 'Bluh bluh bluh',

pamela, ['Pajamas', 'Onesies'],

parent=pamela)

db.put(talk1, talk2)

def update_talks():

talk1.title = 'Writing Apps the Microsoft Way'

talk2.title = 'Wonders of the Windows'

db.put(talk1, talk2)

db.run_in_transaction(update_talks)

Common Features

Counters

1 2 3 4 5people have done something.

RageTube: Global Stats

ragetube.net

http://github.com/pamelafox/ragetube


SongStat

yaycountviewcount

title artist

Key

Path KindName(song)

naycount

mehcount


viewcount viewcount viewcount

datastore memcache


class Song(db.Model): viewcount = db.IntegerProperty(default=0) title = db.StringProperty() artist = db.StringProperty()

def get_viewcount(self): viewcount = self.viewcount cached_viewcount = memcache.get('viewcount-' + self.key().name(), self.key().kind()) if cached_viewcount: viewcount += cached_viewcount return viewcount

@classmethod def flush_viewcount(cls, name): song = cls.get_by_key_name(name) value = memcache.get('viewcount-' + name, cls.kind()) memcache.decr('viewcount-' + name, value, cls.kind()) song.viewcount += value song.put()

@classmethod def incr_viewcount(cls, name, interval=5, value=1): memcache.incr('viewcount-' + name, value, cls.kind()) interval_num = get_interval_number(datetime.now(), interval) task_name = '-'.join([cls.kind(), name.replace(' ', '-'), 'viewcount', str(interval), str(interval_num)]) deferred.defer(cls.flush_viewcount, name, _name=task_name)

LIMIT!(# of tasks)

Ratings

Rated by 500 users.

App Gallery: Ratings

google.com/analytics/apps/


Comment Application

total_ratings

sum_ratings

avg_ratingrated_inde

x

comment_count

rating


def UpdateAppCommentData(self, rating, operation): def UpdateCommentData(self, rating, operation): self.comment_count += 1 * operation self.sum_ratings += rating * operation self.total_ratings += 1 * operation self.avg_rating = int(round(self.sum_ratings / self.total_ratings)) self.rated_index = '%d:%d:%d' % (self.avg_rating, self.total_ratings, self.index) self.put()

db.run_in_transaction(UpdateCommentData, self, rating, operation)

app.UpdateAppCommentData(rating, db_models.Comment.ADD)comment = db_models.Comment()comment.application = appcomment.rating = ratingcomment.put()

query.order('-avg_rating').order('-rated_index')

Geospatial Queries

City-Go-Round: Agencies

citygoround.org

https://github.com/walkscore/City-Go-Round

City-Go-Round: Geo Queries

AgencyGeoModel

location (GeoPt)

location_geocells (StringListProper

ty)

City-Go-Round: Geo Queries

def fetch_agencies_near(lat, long, bbox_side_in_miles): query = Agency.all() bbox = bbox_centered_at(lat, long, bbox_side_in_miles) return Agency.bounding_box_fetch(query, bbox, max_results = 50)

def bounding_box_fetch(query, bbox, max_results=1000,): results = [] query_geocells = geocell.best_bbox_search_cells(bbox)

for entity in query.filter('location_geocells IN', query_geocells): if len(results) == max_results: break if (entity.location.lat >= bbox.south and entity.location.lat <= bbox.north and entity.location.lon >= bbox.west and entity.location.lon <= bbox.east): results.append(entity) return results

Full Text Search

pizza Search

ThingyIt's like pizza, but in the cloud.

Other ThingyThis will make you smell as delicious as pizza.

Disclosed.ca: Search

https://github.com/nurey/disclosed

disclosed.ca


Contract

agency_name vendor_name

description comments

uri


from search.core import SearchIndexProperty, porter_stemmer

class Contract(db.Model): uri = db.StringProperty(required=True) agency_name = db.StringProperty(required=True) vendor_name = db.StringProperty(required=True) description = db.StringProperty() comments = db.TextProperty() search_index = SearchIndexProperty(('agency_name', 'vendor_name', 'description', 'comments'), indexer=porter_stemmer)

results = Contract.search_index.search(sheep').fetch(20)


Contract

agency_name vendor_name

description comments

uri

search_index(StringListProper

ty)

SearchIndex


key search_index

ContractSearch1 charter

ContractSearch1 june

ContractSearch1 sheep

ContractSearch2 sheep

ContractSearch1 wood

SELECT FROM ContractSearch WHERE search_index = "sheep"

More Learning

http://ae-book.appspot.com

http://code.google.com/appengine

http://blog.notdot.net/

AppEngine: Now & Later

"Run your web apps on Google's infrastructure.Easy to build, easy to maintain, easy to scale."

Roadmap:

App Engine (Standard):• MapReduce• Bulk Import/Export• Channel API• Datastore Options

App Engine for Business:• SQL• SLA + Support

Thanks for coming!

Writing Apps the Google-y Way (Brisbane)

Business

Transcript of Writing Apps the Google-y Way (Brisbane)