Python Coroutines, Present and Future

Python CoroutinesPresent and Future

A. Jesse Jiryu Davishttp://emptysquare.net

10gen

Python CoroutinesPresent and Future

Agenda:

• Coroutines are wonderful for async• …but weird.• Understanding the weirdness.• Coroutine Kung Fu.• The Future!

Coroutines arewonderful for async

Async with callbackfrom tornado.web import RequestHandler

class AsyncHandler(RequestHandler): @asynchronous def get(self): http_client = AsyncHTTPClient() http_client.fetch( "http://example.com", callback=self.on_fetch)

def on_fetch(self, response): do_something_with_response(response) self.render("template.html")

Async with coroutine

from tornado.web import RequestHandler

class GenAsyncHandler(RequestHandler): @asynchronous @gen.coroutine def get(self): http_client = AsyncHTTPClient() response = yield http_client.fetch( "http://example.com") do_something_with_response(response) self.render("template.html")

Async with coroutine

from tornado.web import RequestHandler

class GenAsyncHandler(RequestHandler): @asynchronous @gen.coroutine def get(self): http_client = AsyncHTTPClient() future = http_client.fetch( "http://example.com") response = yield future do_something_with_response(response) self.render("template.html")

Agenda:

• Coroutines are wonderful for Async I/O• …but weird.

• yield • future • coroutine

yield

def make_generator(): yield 0 yield 1

g = make_generator()print(g)# <generator object make_generator at 0x1006b4b40>


g = make_generator()for i in g: print(i)


g = make_generator()while True: try: i = g.__next__() print(i) except StopIteration: break


g = make_generator()while True: try: i = g.send(None) print(i) except StopIteration: break

Prints: 0 got 10 1 got 11

def make_generator(): return_value = yield 0 print 'got', return_value return_value = yield 1 print 'got', return_value

g = make_generator()i = g.send(None) # Start gwhile True: try: print(i) i = g.send(i + 10) except StopIteration: break

Agenda:



Future


future = Future()

future.done() # False

future.set_result('foo')

future.set_exception( SomeException('error message'))

future.add_done_callback(callback)

# Return result or raise errorfuture.result()

Agenda:



# part of the Tornado framework

class Runner(object): def __init__(self, make_generator): self.gen = make_generator() # Starts done, with result None self.future = NullFuture()

class Runner(object): # ... def run(self): while True: if not self.future.done(): self.future.add_done_callback(self.run) return

value = self.future.result()

try: self.future = self.gen.send(value) except (StopIteration, Return) as e: return

"recurse"

Agenda:

• Coroutines are wonderful for Async I/O• …but weird.• Handling the weirdness.

@gen.coroutinedef print_code(): response = yield get('http://example.com') print response.code

@gen.coroutinedef get(url): client = AsyncHTTPClient() response = yield client.fetch(url) raise gen.Return(response)

print_code()

weird

@gen.coroutinedef print_code(): future = get('http://example.com') response = yield future print response.code


print_code()weird

@gen.coroutinedef print_code(): future = get('http://example.com') response = yield future print response.code

@gen.coroutinedef get(url): client = AsyncHTTPClient() response = yield client.fetch(url) return response

print_code()

Python 3.3

normal

@gen.coroutinedef print_code(): try: code = yield get('http://example.com') print code except HTTPError, e: print e

@gen.coroutinedef get(url): client = AsyncHTTPClient() response = yield client.fetch(url) raise gen.Return(response.code)

print_code()

normal

Agenda:

• Coroutines are wonderful for Async I/O• …but weird.• Handling the weirdness.• Coroutine Kung Fu.

Fan-out

@gen.coroutinedef f(): client = AsyncHTTPClient() responses = yield [ client.fetch('http://mongodb.org'), client.fetch('http://10gen.com')]

print responses

f()

Fan-out

@gen.coroutinedef f(): client = AsyncHTTPClient() future0 = client.fetch('http://mongodb.org') future1 = client.fetch('http://10gen.com') responses = yield [future0, future1] print responses

f()

Toro

Synchronization Primitives forTornado Coroutines

event = toro.Event()

@gen.coroutinedef waiter(): print "I'll wait right here" yield event.wait() # Yield a Future print "I'm done waiting"

@gen.coroutinedef setter(): print "About to set" event.set() print "Done setting"

waiter()setter()

q = toro.Queue(maxsize=3)

@gen.coroutinedef producer(): for item in range(5): print 'Sending', item yield q.put(item)

@gen.coroutinedef consumer(): while True: item = yield q.get() print '\t\t', 'Got', item

consumer()producer()

$ python producer_consumer.pySending 0

! ! Got 0Sending 1

! ! Got 1Sending 2

! ! Got 2Sending 3

! ! Got 3Sending 4

! ! Got 4

Agenda:

• Coroutines are wonderful for Async I/O• …but weird.• Handling the weirdness.• Kung Fu.• The Future!

Tulip

• A standard event loop

• A standard coroutine library

• For inclusion in Python 3.4

yield from

@tulip.coroutinedef print_code(): response = yield from get('http://example.com') print(response.status)

@tulip.coroutinedef get(url): request = tulip.http.request('GET', url) response = yield from request return response

Tornado

Tulip

normal return

"yield from"

@gen.coroutinedef print_code(): response = yield get('http://example.com') print response.code


Guidoon

"The Difference Between yield and yield-from":

http://bit.ly/yieldfrom

q = tulip.queues.Queue(maxsize=3)

@tulip.coroutinedef producer(): for item in range(5): print('Sending', item) yield from q.put(item)

@tulip.coroutinedef consumer(): while True: item = yield from q.get() print('\t\t', 'Got', item)

Task(consumer())Task(producer())

StopIteration

A. Jesse Jiryu Davishttp://emptysquare.net

10gen

Python Coroutines, Present and Future

Technology

Transcript of Python Coroutines, Present and Future