A Python decorator is, basically, a function that take a function as argument and return a function. This is a powerful feature. But it has some drawbacks:
- decorators are quite tricky to develop. Of course they are for Python newbies. But, as an experienced Python developer, I must admit I also have to think every time I code a decorator. I often feel the decorator pattern is a bit complex.
- decorators that take arguments are even more tricky to develop. They are decorator factories, aka "functions that return a function that take a function and return a function". Inception WTF?
- decorators without arguments are used without parentheses, whereas decorators with arguments require parentheses, even if you pass empty arguments. So when using a decorator, you have to wonder whether it takes arguments or not. A bit more to think about everytime you use a decorator.
- last but not least, function-based decorators are hard to test, because they return functions and you can't easily check internals. How can you check the state of the decorator after it decorated the function, but before you actually run it? Classes are really helpful for that.
This article introduces class-based decorators for Python, as a convenient way to develop and use Python decorators.
The examples described below are available as a Python file at https://gist.github.com/benoitbryon/5168914
Decorators are tricky to develop and use
As a reminder of drawbacks of decorators, here are some examples. If you are aware of those facts, feel free to jump to the next section.
Here is a simple decorator which prints "moo" then executes input function:
def moo(func):
def decorated(*args, **kwargs):
print 'moo'
return func(*args, **kwargs) # Run decorated function.
return decorator
You use it like this:
>>> @moo
... def i_am_a(kind):
... print "I am a {kind}".format(kind=kind)
>>> i_am_a("duck")
'moo'
'I am a duck'
Here is the same decorator, which allows you configure the value to print:
def speak(word='moo'):
def decorator(func):
def decorated(*args, **kwargs):
print word
return func(*args, **kwargs)
return decorated
return decorator
You use it like that:
>>> @speak('quack')
... def i_am_a(kind):
... print "I am a {kind}".format(kind=kind)
>>> i_am_a("duck")
'quack'
'I am a duck'
If you want to use default arguments for "speak", you have to use empty parentheses, i.e. you can't write @speak like you used @moo, you have to write @speak() instead.
I won't tell more about decorators here, there are plenty of articles about them on the web. I just wanted to highlight the fact that even simplest decorators are not as simple as they pretend.
But they could be! Let's introduce class-based decorators...
Hello world example
Here is a sample usage of the Decorator class:
class Greeter(Decorator):
"""Greet return value of decorated function."""
def setup(self, greeting='hello'):
self.greeting = greeting
def run(self, *args, **kwargs):
name = super(Greeter, self).run(*args, **kwargs)
return '{greeting} {name}!'.format(greeting=self.greeting, name=name)
The implementation is pretty simple, isn't it? So is the usage!
As a Decorator, you can use it without options.
>>> @Greeter
... def world():
... return 'world'
>>> world()
'hello world!'
The example above is the same as providing empty options.
>>> @Greeter()
... def world():
... return 'world'
>>> world()
'hello world!'
It accepts one greeting option:
>>> @Greeter(greeting='goodbye')
... def world():
... return 'world'
>>> world()
'goodbye world!'
greeting option defaults to 'hello':
>>> my_greeter = Greeter()
>>> my_greeter.greeting
'hello'
You can create a Greeter instance for later use:
>>> my_greeter = Greeter(greeting='hi')
>>> @my_greeter
... def world():
... return 'world'
>>> world()
'hi world!'
Which gives you an opportunity to setup the greeter yourself:
>>> my_greeter = Greeter()
>>> my_greeter.greeting = 'bonjour'
>>> @my_greeter
... def world():
... return 'world'
>>> world()
'bonjour world!'
In this example, all arguments are proxied to the decorated function:
>>> @Greeter
... def name(value):
... return value
>>> name('world')
'hello world!'
>>> @Greeter(greeting='goodbye')
... def names(*args):
... return ' and '.join(args)
>>> names('Laurel', 'Hardy')
'goodbye Laurel and Hardy!'
Wrapping functions with functools
functools [1] provides utilities to "wrap" a function, i.e. make the decorator return value look like the original function.
Here is another class-based decorator sample. It adds "functools.update_wrapper" features to Decorator:
import functools
class Chameleon(Decorator):
"""A Decorator that looks like decorated function.
It uses ``functools.update_wrapper``.
This is a base class which acts as a transparent proxy for the
decorated function. Consider overriding the ``run()`` method.
.. warning::
Take care of what you pass in ``assigned`` or ``updated``: you could
break the Chameleon itself. As an example, you should not pass "assigned",
"run" or "__call__" in ``assigned``, except you know what you are doing.
"""
def setup(self,
assigned=functools.WRAPPER_ASSIGNMENTS,
updated=functools.WRAPPER_UPDATES):
self.assigned = assigned
self.updated = updated
def decorate(self, func):
"""Make self wrap the decorated function."""
super(Chameleon, self).decorate(func)
functools.update_wrapper(self, func,
assigned=self.assigned,
updated=self.updated)
Again, the implementation is pretty simple.
Let's look at the result...
>>> @Chameleon
... def documented():
... '''Fake function with a docstring.'''
>>> documented.__doc__
'Fake function with a docstring.'
It accepts options assigned and updated, that are proxied to functools.update_wrapper.
Default values are functools.WRAPPER_ASSIGNMENTS for assigned and empty tuple for updated.
>>> def hello():
... '''Hello world!'''
>>> wrapped = Chameleon(hello)
>>> wrapped.assigned
('__module__', '__name__', '__doc__')
>>> wrapped.updated
('__dict__',)
>>> wrapped.__doc__ == hello.__doc__
True
>>> wrapped.__name__ == hello.__name__
True
>>> only_doc_wrapped = Chameleon(hello, assigned=['__doc__'])
>>> only_doc_wrapped.__doc__ == hello.__doc__
True
>>> only_doc_wrapped.__name__ == hello.__name__ # Doctest: +ELLIPSIS
Traceback (most recent call last):
...
AttributeError: 'Chameleon' object has no attribute '__name__'
>>> hello.__dict__ = {'some_attribute': 'some value'} # Best on an object.
>>> attr_wrapped = Chameleon(hello, updated=['__dict__'])
>>> attr_wrapped.updated
['__dict__']
>>> attr_wrapped.some_attribute
'some value'
Here we have a good replacement for decorators using functools.wraps.
As an example, the django-traditional-style decorator shown in Testing Django view decorators article...
from functools import wraps
from django.utils.decorators import available_attrs
def authenticated_user_passes_test(test_func,
unauthorized=UnauthorizedView.as_view(),
forbidden=ForbiddenView.as_view()):
"""Make sure user is authenticated and passes test."""
def decorator(view_func):
@wraps(view_func, assigned=available_attrs(view_func))
def _wrapped_view(request, *args, **kwargs):
if not request.user.is_authenticated():
return unauthorized(request)
if not test_func(request.user):
return forbidden(request)
return view_func(request, *args, **kwargs)
... would be written like this with class-based-style:
class authenticated_user_passes_test(Chameleon):
"""Make sure user is authenticated and passes test."""
def setup(self, **kwargs):
try:
self.test_func = kwargs.pop('test_func')
except KeyError:
raise TypeError('decorator requires "test_func" keyword argument')
self.unauthorized = kwargs.pop('unauthorized', UnauthorizedView.as_view())
self.forbidden = kwargs.pop('forbidden', ForbiddenView.as_view())
super(authenticated_user_passes_test, self).setup(**kwargs)
def run(self, request, *args, **kwargs):
if not request.user.is_authenticated():
return self.unauthorized(request)
if not self.test_func(request.user):
return self.forbidden(request)
return = super(authenticated_user_passes_test, self).run(request, *args, **kwargs)
The class-based way is a bit longer because we have to handle the required test_func manually. But it clearly separates setup from run. It makes the code readable, easy to test, and easy to extend.
The Decorator class
Here is the base class. Little magic inside.
class Decorator(object):
"""Base class to easily create convenient decorators.
Override :py:meth:`setup`, :py:meth:`run` or :py:meth:`decorate` to create
custom decorators.
Decorator instances are callables. The :py:meth:`__call__` method has a
special implementation in Decorator. Generally, consider overriding
:py:meth:`run` instead of :py:meth:`__call__`.
"""
def __init__(self, func=None, **kwargs):
"""Constructor.
Accepts one optional positional argument: the function to decorate.
Other arguments **must** be keyword arguments.
And beware passing ``func`` as keyword argument: it would be used as
the function to decorate.
"""
self.setup(**kwargs)
if func is not None:
self.decorate(func)
def decorate(self, func):
"""Remember the function to decorate."""
self.decorated = func
def setup(self, **kwargs):
"""Store decorator's options."""
self.options = kwargs
def __call__(self, *args, **kwargs):
"""Run decorated function if available, else decorate first arg."""
try:
self.decorated
except AttributeError:
func = args[0]
self.decorate(func)
return self
else:
return self.run(*args, **kwargs)
def run(self, *args, **kwargs):
"""Actually run the decorator.
This base implementation is a transparent proxy to the decorated
function: it passes positional and keyword arguments as is, and returns
result.
"""
return self.decorated(*args, **kwargs)
This base class transparently proxies to decorated function:
>>> @Decorator
... def return_args(*args, **kwargs):
... return (args, kwargs)
>>> return_args()
((), {})
>>> return_args(1, 2, three=3)
((1, 2), {'three': 3})
This base class stores decorator's options in options dictionary. But it doesn't use it... it's just a convenient mechanism for subclasses.
>>> @Decorator
... def nothing():
... pass
>>> nothing.options
{}
>>> @Decorator()
... def nothing():
... pass
>>> nothing.options
{}
>>> @Decorator(one=1)
... def nothing():
... pass
>>> nothing.options
{'one': 1}
Limitations
This Decorated implementation has some limitations:
- you can't use positional arguments to configure the decorator itself.
- required decorator arguments must be handled explicitely, because you can't use positional arguments.
- you have to remember the Decorator API, mainly setup() and run() methods.
Are there other limitations? Right now, I don't know...
Benefits
- As a decorator author, you focus on setup() and run(). It is easy to remember. It produces readable code.
- As a decorator user, you don't bother with parentheses. You just use the decorator depending on your needs, and it works.
- As a test writer, you can write tests for decorators internals: what is the state of the decorator after its own initialization, what is its state after a run...
Would you use it?
See also
- "Python et les décorateurs", by Gilles Fabio is a good article (in french). It ends with an list of useful links (most in english). It also provides a function-based implementation of decorators that work with or without arguments.
- Testing Django view decorators
| [1] | http://docs.python.org/2.7/library/functools.html |
