Python decorators
-
Upload
guillermo-blasco-jimenez -
Category
Engineering
-
view
139 -
download
7
Transcript of Python decorators
Python decorators
21 May 2015 with @pybcnhttps://github.com/theblackboxio/python-decorators-demo
DisclaimerThis speaker belongs to Java world, so do not believe me.
Guillermo Blasco Jiménezes.linkedin.com/in/guillermoblascojimenez
@theblackboxio
Motivationdef bar():
# preprocess things# execute bar logic# postprocess thingsreturn # value
def foo():# preprocess things# execute foo logic# postprocess thingsreturn # value
Motivationdef bar():
# preprocess things# execute bar logic# postprocess thingsreturn # value
def foo():# preprocess things# execute foo logic# postprocess thingsreturn # value
Often the same processing, therefore we have code duplicated
Motivation by example: cachedef hardStuff1(x):
if x in cache:return cache[x]
else:v = # bar logiccache[x] = vreturn v
def hardStuff2(x):if x in cache:
return cache[x]else:
v = # foo logiccache[x] = vreturn v
Motivation by example: transactionsdef transaction1():
tx = startTransaction()try:
# execute bar logictx.commit()
except:tx.rollback()
return # value
def transaction2():tx = startTransaction()try:
# execute foo logictx.commit()
except:tx.rollback()
return # value
Motivation by examples● Logging● Retry● Wrap exceptions● Deprecation warnings● Pre/post conditions● Access authorization● Profiling
Decorators basicsObjectives:● isolate shared code● do not change the way to call our function
Decorators basics: code isolationdef executeWithCache(x, hardStuffLogic):
if x in cache:return cache[x]
else:v = hardStuffLogic(x)cache[x] = vreturn v
executeWithCache(5, fibonacci) # before was: fibonacci(5)executeWithCache(5, anotherCostlyFunction)
Function Reference!
Decorators basics: preserve callingdef executeWithCache(hardStuffLogic):
def wrapper(x):if x in cache:
return cache[x]else:
v = hardStuffLogic(x)cache[x] = vreturn v
return wrapper
fibonacci = executeWithCache(fibonacci)anoherCostlyFunction = executeWithCache(anotherCostlyFunction)
Yes, a function that returns a function
Decorators basics: syntactic sugardef fibonacci(x):
#...
fibonacci = executeWithCache(fibonacci)
@executeWithCachedef fibonacci(x):
# ...
Since Python 2.4
TheoryDecorator pattern is about adding functionality to objects without change the functionality of others.Python decorators implements decorator pattern applied to functions (that are objects in Python, by the way)
Ramifications● Decorators with parameters● Decorators as functions or classes● Nesting decorators● Decorators applied to classes● Preservation of function metadata● Testing decorators
Decorators with parameters I@precondition(lambda x: x >= 0)def fibonacci(x):
…
Decorators with parameters IIdef precondition(precond):
def wrapper1(f):def wrapper2(x):
if not precond(x):raise ValueError('Precondition failed')
return f(x)return wrapper2
return wrapper1
Yes! A function that returns a function that returns a function
Decorators as classes IOne clear problem with decorators: too many function nestingdef precondition(precond):
def wrapper1(f):def wrapper2(x):
if not precond(x):raise ValueError('Precondition failed')
return f(x)return wrapper2
return wrapper1
Agree with me that this is ugly
Decorators as classes IIActually you just need a callable object that returns a function to use it as decorator. Python has three callable objects:● Functions● Classes (to build object instances)● Objects with __call__
Decorators as classes IIIclass Precondition:
def __init__(self, checkFunction):self.p = checkFunction
def __call__(self, f):def wrapper(x):
if not self.p(x):raise ValueError()
return f(x)return wrapper
This makes the object instance a callable object
@Precondition(lambda x: x >= 0)def fibonacci(x):
…
Decorators as classes IV# create new instance of decoratordec = Precondition(lambda x: x>=0) # apply as decoratorfibonacci = dec(fibonacci)# the wrapper that “dec” returns is calledfibonacci(5)
Nesting decorators@cache@transaction@notNullInputdef bar(x):
...
This is great!
Nesting decorators: first problem I
@memoryCache@localFileCache@hdfsCache@transaction@notNullInput@notNullOutputdef bar(x):
...
Do not abuse, please
Nesting decorators: first problem II
# Make decorators more generics, do not forget OOP@Cache(“memory”,”localFile”,”hdfs”)@transaction@notNull # checks input and outputdef bar(x):
...
Nesting decorators: second problem I
@cache@transaction@notNullInputdef bar(x):
...
Order matters!
Nesting decorators: second problem II
@cache # executed first@transaction # executed second@notNullInput # executed thirddef bar(x):
... Why to cache and start a transaction for a possible invalid computation?
Nesting decorators: second problem III
@notNullInput # executed first@cache # executed second@transaction # executed thirddef bar(x):
...
Decorators applied to classes I
Decorators applied to functions, are callables that takes a function and at the end returns a function.Decorators applied to classes, are callables that takes a class and at the end returns a class.
Decorators applied to classes II
# Prints a warning every time# a method is called on class instance@deprecatedclass OldClass:
…
Decorators applied to classes III
def deprecated(clazz):class DeprecatedClass:
# Black sorcery to bypass class# methods dynamically
return DeprecatedClass
Decorators applied to classes IV
● Do not replace class inheritance with class decorators● Hard to find actual use cases● Hard to deal with dynamically generated classes● Hard to deal with bound functions with function
decorators● Related to metaclasses and metaprogramming
Preservation of function metadata I>>> def foo():
…>>> print foo.__name__foo
>>> @cache def foo():
…>>> print foo.__name__cache
Preservation of function metadata II
Metadata loss:● __name__ and __doc__● argument specification● source code inspectionImplies that:● Harder to debug● Harder to profile● Unreadable stack traces
Preservation of function metadata III
Use:● functools.wraps for function decorators● functools.update_wrapper for class decs.More info:Graham Dumpleton - How you implemented your Python decorator is wronghttp://blog.dscpl.com.au/2014/01/how-you-implemented-your-python.html
Preservation of function metadata IVimport functools
def function_wrapper(wrapped):
@functools.wraps(wrapped) def _wrapper(*args, **kwargs):
return wrapped(*args, **kwargs)
return _wrapper
import functools class function_wrapper(object): def __init__(self, wrapped): self.wrapped = wrapped functools.update_wrapper(self, wrapped) def __call__(self, *args, **kwargs): return self.wrapped(*args, **kwargs)
Testing decorators Idef testCache():
times = 0def mock(x):
times = times + 1mock = cache(mock)mock(1)assert times == 1mock(1)assert times == 1
Use the old fashioned way to use decorators as simply function wrappers.Use the wrapped function to check the state of the decorator instance
Decorators in real world● Python itself (@staticmethod)● https://wiki.python.org/moin/PythonDecoratorLibrary● Django (@permission_required)
How is this possible?● Python functions are objects● Python is a dynamic language, i.e. your code
can generate more code
There is something like decorators?
Well, yes. It is named Aspect Oriented Programming (AOP). The problem is that if your language does not support dynamic code and/or your functions are not objects then you have to cook manually decorators or use an AOP framework (proxy classes nightmare).
Guillermo Blasco Jiménez
Thanks!https://github.com/theblackboxio/python-decorators-demo
Questions?