Austin Bingham. Transducers in Python. PyCon Belarus
-
Upload
alina-dolgikh -
Category
Technology
-
view
792 -
download
0
Transcript of Austin Bingham. Transducers in Python. PyCon Belarus
@sixty_north
Understanding TransducersThrough Python
1
Austin Bingham@austin_bingham
Sunday, January 25, 15
237
“Transducers are coming”Rich Hickeyhttp://blog.cognitect.com/blog/2014/8/6/transducers-are-coming
Photo: Howard Lewis Ship under CC-BY
Sunday, January 25, 15
Functions which transform reducersWhat is a transducer?
‣Reducer (reducing function)• Any function we can pass to reduce(reducer, iterable[, initial])
• (result, input) → result
• add(result, input)reduce(add, [1, 2, 3], 0) → 6
‣Transducer (transform reducer)• A function which accepts a reducer, and transforms it in some way, and returns a
new reducer
• ((result, input) → result) → ((result, input) → result)3
Sunday, January 25, 15
Transducers are a functional programming technique not restricted to ClojureHow does this relate to Python?
‣Clojure implementation is the prototype/archetype• Heavily uses anonymous functions
• Uses overloads on function arity for disparate purposes
• Complected with existing approaches
‣Python implementation is pedagogical (but also useful)• Explicit is better than implicit
• Readability counts!
• Has all the functional tools we need for transducers
• I’m a better Pythonista than Clojurist4
Sunday, January 25, 15
5
Reviewfunctional programming
tools in
Sunday, January 25, 15
6
my_filter(is_prime, my_map(prime_factors, range(32) ) )
iterablesequence
sequence
Sunday, January 25, 15
7
def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, [])
def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, [])
Sunday, January 25, 15
7
def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, [])
def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, [])
reduce()
Sunday, January 25, 15
7
def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, [])
def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, [])
reduce()
sequence.append()
Sunday, January 25, 15
7
def my_map(transform, iterable): def map_reducer(sequence, item): sequence.append(transform(item)) return sequence return reduce(map_reducer, iterable, [])
def my_filter(predicate, iterable): def filter_reducer(sequence, item): if predicate(item): sequence.append(item) return sequence return reduce(filter_reducer, iterable, [])
reduce()
sequence.append()
Empty list : ‘seed’must be a mutable
sequence
Sunday, January 25, 15
8
>>> reduce(make_mapper(square), range(10), []) [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> reduce(make_filterer(is_prime), range(100), []) [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
‣make_mapper() and make_filterer() are not composable
‣ to square primes we would need to call reduce() twice
‣ this requires we store the intermediate sequence
‣ the reducers returned by make_mapper() and make_filterer() depend on the seed being a mutable sequence e.g. a list
Sunday, January 25, 15
9
(defn map ([f] (fn [rf] (fn ([] (rf)) ([result] (rf result)) ([result input] (rf result (f input))) ([result input & inputs] (rf result (apply f input inputs))))))
Sunday, January 25, 15
10
(defn map ;; The tranducer factory... ([f] ;; ...accepts a single argument 'f', the transforming function (fn [rf] ;; The transducer function accepts a reducing function 'rf' (fn ;; This is the reducing function returned by the transducer ([] (rf)) ;; 0-arity : Forward to the zero-arity reducing function 'rf' ([result] (rf result)) ;; 1-arity : Forward to the one-arity reducing function 'rf' ([result input] ;; 2-arity : Perform the reduction with one arg to 'f' (rf result (f input))) ([result input & inputs] ;; n-arity : Perform the reduction with multiple args to 'f' (rf result (apply f input inputs))))))
Sunday, January 25, 15
11
(defn map ;; The tranducer factory... ([f] ;; ...accepts a single argument 'f', the transforming function (fn [rf] ;; The transducer function accepts a reducing function 'rf' (fn ;; This is the reducing function returned by the transducer ([] (rf)) ;; 0-arity : Return a 'seed' value obtained from 'rf' ([result] (rf result)) ;; 1-arity : Obtain final result from 'rf' and clean-up ([result input] ;; 2-arity : Perform the reduction with one arg to 'f' (rf result (f input))) ([result input & inputs] ;; n-arity : Perform the reduction with multiple args to 'f' (rf result (apply f input inputs))))))
Sunday, January 25, 15
12
(defn map ;; The tranducer factory... ([f] ;; ...accepts a single argument 'f', the transforming function (fn [rf] ;; The transducer function accepts a reducing function 'rf' (fn ;; This is the reducing function returned by the transducer ([] (rf)) ;; 0-arity : Return a 'seed' value obtained from 'rf' ([result] (rf result)) ;; 1-arity : Obtain final result from 'rf' and clean-up ([result input] ;; 2-arity : Perform the reduction with one arg to 'f' (rf result (f input))) ([result input & inputs] ;; n-arity : Perform the reduction with multiple args to 'f' (rf result (apply f input inputs))))))
To fully implement Clojure’s transducers in Python we also need:
‣ explicit association of the seed value with the reduction operation
‣ support for early termination without reducing the whole series
‣ reduction to a final value and opportunity to clean up state
Sunday, January 25, 15
13
class Reducer:
def __init__(self, reducer): # Construct from reducing function pass
def initial(self): # Return the initial seed value pass # 0-arity
def step(self, result, item): # Next step in the reduction pass # 2-arity
def complete(self, result): # Produce a final result and clean up pass # 1-arity
Sunday, January 25, 15
13
class Reducer:
def __init__(self, reducer): # Construct from reducing function pass
def initial(self): # Return the initial seed value pass # 0-arity
def step(self, result, item): # Next step in the reduction pass # 2-arity
def complete(self, result): # Produce a final result and clean up pass # 1-arity
new_reducer = Reducer(reducer)
Sunday, January 25, 15
13
class Reducer:
def __init__(self, reducer): # Construct from reducing function pass
def initial(self): # Return the initial seed value pass # 0-arity
def step(self, result, item): # Next step in the reduction pass # 2-arity
def complete(self, result): # Produce a final result and clean up pass # 1-arity
new_reducer = Reducer(reducer)
def transducer(reducer): return Reducer(reducer)
Sunday, January 25, 15
13
class Reducer:
def __init__(self, reducer): # Construct from reducing function pass
def initial(self): # Return the initial seed value pass # 0-arity
def step(self, result, item): # Next step in the reduction pass # 2-arity
def complete(self, result): # Produce a final result and clean up pass # 1-arity
new_reducer = Reducer(reducer)
def transducer(reducer): return Reducer(reducer) ⇐ two names for two concepts
Sunday, January 25, 15
Python Transducer implementations
14
Cognitecthttps://github.com/cognitect-labs
PyPI: transducers
• “official”
• Python in the style of Clojure
• Only eager iterables (step back from regular Python)
• Undesirable Python practices such as hiding built-ins
• Also Clojure, Javascript, Ruby, etc.
Sixty Northhttp://code.sixty-north.com/python-transducers
PyPI: transducer
• Pythonic
• Eager iterables
• Lazy iterables
• Co-routine based ‘push’ events
• Pull-requests welcome!
Sunday, January 25, 15
15
Thank you!
@sixty_north
Austin Bingham@austin_bingham
http://sixty-north.com/blog/
Sunday, January 25, 15