Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25:...

108
Marvin Zhang 08/03/2016 Lecture 25: Coroutines

Transcript of Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25:...

Page 1: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Marvin Zhang 08/03/2016

Lecture 25: Coroutines

Page 2: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Announcements

Page 3: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Roadmap

Introduction

Functions

Data

Mutability

Objects

Interpretation

Paradigms

Applications

Page 4: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Roadmap

• This week (Paradigms), the goals are:

Introduction

Functions

Data

Mutability

Objects

Interpretation

Paradigms

Applications

Page 5: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Roadmap

• This week (Paradigms), the goals are:• To study examples of paradigms

that are very different from what we have seen so far

Introduction

Functions

Data

Mutability

Objects

Interpretation

Paradigms

Applications

Page 6: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Roadmap

• This week (Paradigms), the goals are:• To study examples of paradigms

that are very different from what we have seen so far

• To expand our definition of what counts as programming

Introduction

Functions

Data

Mutability

Objects

Interpretation

Paradigms

Applications

Page 7: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

Page 8: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• Almost all programs we have seen so far involve the program running in isolation until completion

Page 9: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• Almost all programs we have seen so far involve the program running in isolation until completion

• But many practical applications involve communication between different programs or with a user

Page 10: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• Almost all programs we have seen so far involve the program running in isolation until completion

• But many practical applications involve communication between different programs or with a user• For example, many web applications have to wait for

user input, such as mouse clicks or text input

Page 11: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• Almost all programs we have seen so far involve the program running in isolation until completion

• But many practical applications involve communication between different programs or with a user• For example, many web applications have to wait for

user input, such as mouse clicks or text input• We have seen one example of this: interactive

interpreters wait for the user to type in code before it can execute that code and produce a result

Page 12: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• Almost all programs we have seen so far involve the program running in isolation until completion

• But many practical applications involve communication between different programs or with a user• For example, many web applications have to wait for

user input, such as mouse clicks or text input• We have seen one example of this: interactive

interpreters wait for the user to type in code before it can execute that code and produce a result

• This style of programming is called event-driven, because different events, such as user input, trigger different parts of our program to execute

Page 13: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Revisiting lazy evaluation

Generators and Generator Functions

Page 14: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

Page 15: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

Page 16: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

Page 17: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

Page 18: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

Page 19: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):

Page 20: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):... print(i)...

Page 21: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):... print(i)... 0

Page 22: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):... print(i)... 01

Page 23: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):... print(i)... 012

Page 24: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):... print(i)... 0123

Page 25: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generator Functions

• A generator function is a function that yields values instead of returning them

• A normal function returns once, a generator function can yield multiple times

• When a generator function is called, it returns a generator that iterates over yield statements

def range_gen(start, end): while start < end: yield start start += 1

>>> for i in range_gen(0, 5):... print(i)... 01234

Page 26: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

Page 27: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

Page 28: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

• Generators act as implicit, or lazy, sequences

Page 29: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

• Generators act as implicit, or lazy, sequences

• Values are not computed when the sequence is created, but when they are asked for

Page 30: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

• Generators act as implicit, or lazy, sequences

• Values are not computed when the sequence is created, but when they are asked for

• This is the same as built-in Python range objects, Python iterators, and Scheme streams

Page 31: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

• Generators act as implicit, or lazy, sequences

• Values are not computed when the sequence is created, but when they are asked for

• This is the same as built-in Python range objects, Python iterators, and Scheme streams

• We can use implicit sequences to create infinite sequences!

Page 32: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

• Generators act as implicit, or lazy, sequences

• Values are not computed when the sequence is created, but when they are asked for

• This is the same as built-in Python range objects, Python iterators, and Scheme streams

• We can use implicit sequences to create infinite sequences!

(demo)

Page 33: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators

• A generator is an iterator, created by a generator function

• Generators act as implicit, or lazy, sequences

• Values are not computed when the sequence is created, but when they are asked for

• This is the same as built-in Python range objects, Python iterators, and Scheme streams

• We can use implicit sequences to create infinite sequences!

def naturals(): curr = 0 while True: yield curr curr += 1

>>> n = naturals()>>> n<generator object naturals at ...>>>> next(n)0>>> next(n)1

(demo)

Page 34: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

Page 35: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

Page 36: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

Page 37: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

Page 38: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

• Recall the iterable interface from lab: __iter__ and __next__

Page 39: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

• Recall the iterable interface from lab: __iter__ and __next__

• __iter__ returns an iterator, which has a __next__ method

Page 40: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

• Recall the iterable interface from lab: __iter__ and __next__

• __iter__ returns an iterator, which has a __next__ method

• __next__ returns the next element in our sequence

Page 41: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

• Recall the iterable interface from lab: __iter__ and __next__

• __iter__ returns an iterator, which has a __next__ method

• __next__ returns the next element in our sequence

• A generator function returns a generator, which is an iterator, and the generator returns the next element by calling __next__ on it

Page 42: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

• Recall the iterable interface from lab: __iter__ and __next__

• __iter__ returns an iterator, which has a __next__ method

• __next__ returns the next element in our sequence

• A generator function returns a generator, which is an iterator, and the generator returns the next element by calling __next__ on it

• So, what if we just make our __iter__ method a generator function? This satisfies all our requirements!

Page 43: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generators vs Iterators

• Generator functions are often simpler and more intuitive to write than iterator classes, because:

• We only have to write a function instead of a class

• Yielding pauses execution of the function and automatically saves state for resuming, as opposed to returning

• Recall the iterable interface from lab: __iter__ and __next__

• __iter__ returns an iterator, which has a __next__ method

• __next__ returns the next element in our sequence

• A generator function returns a generator, which is an iterator, and the generator returns the next element by calling __next__ on it

• So, what if we just make our __iter__ method a generator function? This satisfies all our requirements!

(demo)

Page 44: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Generalizing generators

Coroutines

Page 45: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

Page 46: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

• Generator functions can also consume values using the yield expression (different from the yield statement!)

Page 47: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

• Generator functions can also consume values using the yield expression (different from the yield statement!)

• Generators that both produce and consume values are called coroutines, though they are still generator objects

Page 48: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

• Generator functions can also consume values using the yield expression (different from the yield statement!)

• Generators that both produce and consume values are called coroutines, though they are still generator objects

• We can control coroutines by using the send and close methods

Page 49: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

• Generator functions can also consume values using the yield expression (different from the yield statement!)

• Generators that both produce and consume values are called coroutines, though they are still generator objects

• We can control coroutines by using the send and close methods

• send, like __next__, resumes the coroutine, but also passes a value to it

Page 50: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

• Generator functions can also consume values using the yield expression (different from the yield statement!)

• Generators that both produce and consume values are called coroutines, though they are still generator objects

• We can control coroutines by using the send and close methods

• send, like __next__, resumes the coroutine, but also passes a value to it

• Calling __next__ is equivalent to calling send with None

Page 51: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines

• Generator functions can also consume values using the yield expression (different from the yield statement!)

• Generators that both produce and consume values are called coroutines, though they are still generator objects

• We can control coroutines by using the send and close methods

• send, like __next__, resumes the coroutine, but also passes a value to it

• Calling __next__ is equivalent to calling send with None

• close stops the coroutine and raises a GeneratorExit exception within the coroutine

Page 52: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Coroutines (demo)

• Generator functions can also consume values using the yield expression (different from the yield statement!)

• Generators that both produce and consume values are called coroutines, though they are still generator objects

• We can control coroutines by using the send and close methods

• send, like __next__, resumes the coroutine, but also passes a value to it

• Calling __next__ is equivalent to calling send with None

• close stops the coroutine and raises a GeneratorExit exception within the coroutine

Page 53: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

Page 54: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

Page 55: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

Page 56: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

• One way to set up a pipeline is to have each stage of the pipeline be a coroutine!

Page 57: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

• One way to set up a pipeline is to have each stage of the pipeline be a coroutine!

• Functions at the beginning of the pipeline, that only send values, are called producers

Page 58: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

• One way to set up a pipeline is to have each stage of the pipeline be a coroutine!

• Functions at the beginning of the pipeline, that only send values, are called producers

• Coroutines in the middle, that both send and receive values, are called filters

Page 59: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

• One way to set up a pipeline is to have each stage of the pipeline be a coroutine!

• Functions at the beginning of the pipeline, that only send values, are called producers

• Coroutines in the middle, that both send and receive values, are called filters

• Coroutines at the end of the pipeline, that only receive values, are called consumers

Page 60: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

• One way to set up a pipeline is to have each stage of the pipeline be a coroutine!

• Functions at the beginning of the pipeline, that only send values, are called producers

• Coroutines in the middle, that both send and receive values, are called filters

• Coroutines at the end of the pipeline, that only receive values, are called consumers

• The data coming through the stream is sent through this pipeline to produce the final result

Page 61: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Implicit sequences are extremely useful in programming applications that deal with continuous streams of data, e.g., news feeds, sensor measurements, or mathematical sequences

• When working with data streams, a helpful and efficient technique is to set up a pipeline for sequence processing

• One way to set up a pipeline is to have each stage of the pipeline be a coroutine!

• Functions at the beginning of the pipeline, that only send values, are called producers

• Coroutines in the middle, that both send and receive values, are called filters

• Coroutines at the end of the pipeline, that only receive values, are called consumers

• The data coming through the stream is sent through this pipeline to produce the final result

(demo)

Page 62: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

Page 63: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

Page 64: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

user input

producer

Page 65: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filteruser input

producer

Page 66: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filter

user input

producer

Page 67: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filter

user input

producer

print

consumer

Page 68: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filter print

consumer

Page 69: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filter print

consumer

input from a web form

producer

Page 70: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filter

require Marvin’s approval

filter

print

consumer

input from a web form

producer

Page 71: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filterreject if >100 chars

filter

require Marvin’s approval

filter

print

consumer

input from a web form

producer

Page 72: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filterreject if >100 chars

filter

require Marvin’s approval

filter

print

consumer

input from a web form

producer

hidden

Page 73: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Sequence Processing

• Setting up a pipeline using coroutines allows us to easily change how we process the data by inserting, removing, and modifying different pieces of our program

(demo)

capitalize

filtermatch

‘MARVIN’

filter

match ‘BRIAN’

filterreject if >100 chars

filter

require Marvin’s approval

filter

print

consumer

input from a web form

producer

hidden

Page 74: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

With and without coroutines

Event-Driven Programming

Page 75: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

Page 76: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• The paradigm of event-driven programming allows different events, such as user input, to trigger different parts of our program to execute

Page 77: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• The paradigm of event-driven programming allows different events, such as user input, to trigger different parts of our program to execute

• Lazy evaluation, such as implicit sequences, is similar to this paradigm in that the “event” of asking for an element from the sequence triggers the computation

Page 78: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• The paradigm of event-driven programming allows different events, such as user input, to trigger different parts of our program to execute

• Lazy evaluation, such as implicit sequences, is similar to this paradigm in that the “event” of asking for an element from the sequence triggers the computation• However, this is not what is usually meant by “event”

Page 79: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• The paradigm of event-driven programming allows different events, such as user input, to trigger different parts of our program to execute

• Lazy evaluation, such as implicit sequences, is similar to this paradigm in that the “event” of asking for an element from the sequence triggers the computation• However, this is not what is usually meant by “event”

• Processing continuous data streams is an example of this paradigm, where incoming data is the event

Page 80: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• The paradigm of event-driven programming allows different events, such as user input, to trigger different parts of our program to execute

• Lazy evaluation, such as implicit sequences, is similar to this paradigm in that the “event” of asking for an element from the sequence triggers the computation• However, this is not what is usually meant by “event”

• Processing continuous data streams is an example of this paradigm, where incoming data is the event

• Interactive interpreters is another example, where user input is the event

Page 81: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Event-Driven Programming

• The paradigm of event-driven programming allows different events, such as user input, to trigger different parts of our program to execute

• Lazy evaluation, such as implicit sequences, is similar to this paradigm in that the “event” of asking for an element from the sequence triggers the computation• However, this is not what is usually meant by “event”

• Processing continuous data streams is an example of this paradigm, where incoming data is the event

• Interactive interpreters is another example, where user input is the event

• In event-driven programming, an event loop waits for events, and handles them by dispatching them to a callback function

Page 82: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

Page 83: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

Page 84: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

user input

producer

Page 85: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

user input

producer

lexical analysis

filter

syntactic analysis

filter

Page 86: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

Page 87: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

print

consumer

Page 88: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

• So, we can implement it using coroutines!

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

print

consumer

Page 89: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

• So, we can implement it using coroutines!• This doesn’t provide an advantage in this case, because

the REPL is already fairly simple and elegant

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

print

consumer

Page 90: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

• So, we can implement it using coroutines!• This doesn’t provide an advantage in this case, because

the REPL is already fairly simple and elegant• But it is still an interesting exercise

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

print

consumer

Page 91: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

• So, we can implement it using coroutines!• This doesn’t provide an advantage in this case, because

the REPL is already fairly simple and elegant• But it is still an interesting exercise

• Let’s take a look at the Calculator interpreter

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

print

consumer

Page 92: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Interactive Interpreters

• The read-eval-print loop is an example of an event loop

• So, we can implement it using coroutines!• This doesn’t provide an advantage in this case, because

the REPL is already fairly simple and elegant• But it is still an interesting exercise

• Let’s take a look at the Calculator interpreter

(demo)

user input

producer

lexical analysis

filter

syntactic analysis

filter

evaluate

filter

print

consumer

Page 93: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

Page 94: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Coroutines naturally enforce modularity in our code, i.e., splitting complex functionality up into smaller pieces that are easier to write, maintain, and understand

Page 95: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Coroutines naturally enforce modularity in our code, i.e., splitting complex functionality up into smaller pieces that are easier to write, maintain, and understand• Modularity also allows us to easily change our program,

simply by swapping in and out different pieces

Page 96: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Coroutines naturally enforce modularity in our code, i.e., splitting complex functionality up into smaller pieces that are easier to write, maintain, and understand• Modularity also allows us to easily change our program,

simply by swapping in and out different pieces• Coroutines are especially useful in building modular

pipelines, where data is processed in stages

Page 97: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Coroutines naturally enforce modularity in our code, i.e., splitting complex functionality up into smaller pieces that are easier to write, maintain, and understand• Modularity also allows us to easily change our program,

simply by swapping in and out different pieces• Coroutines are especially useful in building modular

pipelines, where data is processed in stages• Both generators and coroutines maintain their own state,

and this is highly useful for particular applications

Page 98: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Coroutines naturally enforce modularity in our code, i.e., splitting complex functionality up into smaller pieces that are easier to write, maintain, and understand• Modularity also allows us to easily change our program,

simply by swapping in and out different pieces• Coroutines are especially useful in building modular

pipelines, where data is processed in stages• Both generators and coroutines maintain their own state,

and this is highly useful for particular applications• Though coroutines by themselves are not a paradigm, they

are useful for the paradigm of event-driven programming

Page 99: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Coroutines naturally enforce modularity in our code, i.e., splitting complex functionality up into smaller pieces that are easier to write, maintain, and understand• Modularity also allows us to easily change our program,

simply by swapping in and out different pieces• Coroutines are especially useful in building modular

pipelines, where data is processed in stages• Both generators and coroutines maintain their own state,

and this is highly useful for particular applications• Though coroutines by themselves are not a paradigm, they

are useful for the paradigm of event-driven programming• However, it is important to understand when using

coroutines may just be unnecessarily complicated

Page 100: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

Page 101: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

Page 102: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

Page 103: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

• One option for implementing callback functions, which often works well, is to use coroutines

Page 104: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

• One option for implementing callback functions, which often works well, is to use coroutines• If the event-driven application has callback

functionality that:

Page 105: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

• One option for implementing callback functions, which often works well, is to use coroutines• If the event-driven application has callback

functionality that:• Is complex and easily made modular,

Page 106: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

• One option for implementing callback functions, which often works well, is to use coroutines• If the event-driven application has callback

functionality that:• Is complex and easily made modular,• Naturally fits into a processing pipeline, or

Page 107: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

• One option for implementing callback functions, which often works well, is to use coroutines• If the event-driven application has callback

functionality that:• Is complex and easily made modular,• Naturally fits into a processing pipeline, or• Involves state that changes over time,

Page 108: Lecture 25: Coroutines - University of California, Berkeley · 2017-06-04 · Lecture 25: Coroutines. Announcements. Roadmap Introduction Functions Data Mutability Objects Interpretation

Summary

• Event-driven programming is a heavily used paradigm in applications such as user interfaces and web development

• In event-driven programming, an event loop handles particular events, such as user input, and uses callback functions to process these events

• One option for implementing callback functions, which often works well, is to use coroutines• If the event-driven application has callback

functionality that:• Is complex and easily made modular,• Naturally fits into a processing pipeline, or• Involves state that changes over time,

• Then coroutines are probably the way to go