Python Decorators
A Brief Introduction
Decorators in Python are a way of applying code to other code. They turn this
@some_decorator
def some_function():
...
into this
def some_function():
...
some_function = some_decorator(some_function)
They
- can be used on functions or classes
- can be nested
- can be any expression resulting in a callable (as of Python 3.9, via PEP 614)
So this
@decorator_one
@decorator_two("extra information", 0)
class some_class():
...
is equivalent to
class some_class():
...
some_class = decorator_two("extra information", 0)(some_class)
some_class = decorator_one(some_class)
There is no more magic around decorators. There are some details to consider with other Python features. We’ll take a quick look at some of those features, then look at some possible uses of decorators.
Background
In Python, anything that can be given a name is an object. It can
- have methods called on it
- be passed to or returned from a function
- have its properties changed, added, or removed.
Here we demonstrate a variety of objects being passed into the id
function and having a property accessed:
for thing in [
42, # integer
"string", # string
tuple, # class
[1,2,3].pop, # method
(lambda x:x), # function
]:
print(id(thing), thing.__class__)
1681762578000 <class 'int'>
1681770874800 <class 'str'>
140715374168768 <class 'type'>
1681845528112 <class 'builtin_function_or_method'>
1681845306992 <class 'function'>
Some objects have a method named __call__
, which allows them to be called with syntax like this:
class C():
def __init__(self):
print("class called")
C() # Classes are callable
def f():
print("function called")
f() # Functions are callable
class called
function called
A decorator must accept (at least) one argument: the thing it’s decorating.
Whatever it returns is bound to the name of the thing that was passed in.
def decorator(thing):
print("decorating", thing.__name__)
return thing
@decorator
def my_meaningful_name():
print("meaningful stuff happened")
@decorator
class other_meaningful_name():
def __init__(self):
print("other meaningful stuff happened")
decorating my_meaningful_name
decorating other_meaningful_name
Note that the decorated function isn’t actually called yet, nor the decorated class instantiated.
This does not have to be related to the original thing in any way, or even be a callable itself.
def five(ignored):
return 5
@five
def important_function():
...
print(important_function)
important_function()
5
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~/ipykernel_17372/3447356210.py in <module>
7
8 print(important_function)
----> 9 important_function()
10
TypeError: 'int' object is not callable
Here, attempting to call important_function
fails because that name points to the integer 5
, which is not a callable.
Some useful decorators
Timer
Print the duration of each call to the decorated object.
from datetime import datetime as dt
def timed(thing):
def inner(*args, **kwargs):
time_pre_call = dt.now()
result = thing(*args, **kwargs)
time_post_call = dt.now()
print(f"{thing.__name__} lasted {time_post_call - time_pre_call}")
return result
return inner
@timed
def summation_to(high):
sum(x for x in range(high))
summation_to(10)
summation_to(50_000_000)
summation_to lasted 0:00:00
summation_to lasted 0:00:03.511591
Retry
If the first call to a function throws an exception, call it again with the same arguments.
def retry(thing):
def inner(*args, **kwargs):
# Instead of retrying once, consider using a loop and accepting an argument for a number
try:
print("Trying with args", args, kwargs)
thing(*args, **kwargs)
except Exception:
print("Retrying with args", args, kwargs)
thing(*args, **kwargs)
return inner
@retry
def div_zero(x):
return x/0
try:
div_zero(7)
except ZeroDivisionError:
print("Cannot divide by zero")
Trying with args (7,) {}
Retrying with args (7,) {}
Cannot divide by zero
Cache
Prevent extra work by storing the result of a function called with specific arguments, and return the old result if called with those same arguments again.
Here we reuse the above timed
decorator to show the duration of function calls. Repeat calls are much shorter.
def cached(thing):
_cache = {}
def inner(*args):
# This could be a time-expiring cache by storing a tuple of (timestamp, value) and checking the timestamp!
if not args in _cache:
_cache[args] = thing(*args)
return _cache[args]
return inner
import time
@timed
@cached
def slow_echo(arg):
time.sleep(2)
return arg
import random
for _ in range(10):
print(slow_echo(random.randint(1, 6)))
inner lasted 0:00:02.012066
2
inner lasted 0:00:00
2
inner lasted 0:00:02.006280
5
inner lasted 0:00:02.013154
4
inner lasted 0:00:00
4
inner lasted 0:00:00
2
inner lasted 0:00:02.007700
6
inner lasted 0:00:00
2
inner lasted 0:00:00
5
inner lasted 0:00:02.015261
3
Mock
Replace a function with one that simply returns the specified value without calling the mocked function at all. Here we bypass an infinite loop and just provide the answer.
def mock(mock_value):
def inner(thing):
# Consider having the mock value vary depending on the arguments
return lambda *args, **kwargs: mock_value
return inner
@mock(2)
def long_running_function():
current, total = 1, 0
# TODO: fix this infinite loop...
while True:
total += current
current /= 2
return total
print("long function returned", long_running_function())
long function returned 2
Trace
This decorator can be applied to specific functions to produce a visualized call stack featuring only those decorated functions.
Here we skip tracing function C
, but do show its child calls to function D
as if they were called by C
’s caller, either A
or B
.
def trace(thing):
def inner(*args, **kwargs):
print(trace.indent * " |" + "--" + thing.__name__)
trace.indent += 1
result = thing()
trace.indent -= 1 # BUG - throwing exceptions in thing() will skip this line
return result
return inner
trace.indent = 0
@trace
def A():
B()
C()
B()
@trace
def B():
C()
# Not decorated, not traced.
def C():
D()
from random import random
@trace
def D():
if random() < 0.25:
D()
if random() < 0.25:
D()
A()
--A
|--B
| |--D
| | |--D
| | | |--D
| | | | |--D
| | | |--D
|--D
| |--D
|--B
| |--D
| | |--D
| | |--D
Register
Suppose a web request dispatcher needs to know about functions which can handle requests and what kinds of requests they support. A traditional approach would be to add a new handler in one file and also a reference to it from the dispatcher file. This is fragile because it’s easy to add a new endpoint without noticing the need to update the dispatcher (and possibly other locations).
With this approach, implementing a new handler happens in only one place.
registry = {}
def register(name):
def inner(thing):
registry[name] = thing
return thing
return inner
@register("v1/orders/new")
def orders():
return [("tesla 6", 1),
("torus", 2)]
@register("v1/users/")
def users_legacy():
return ["User, Legacy"]
@register("v2/users/")
def users():
return ["User, Test", "Account, Guest"]
for endpoint in ["v1/orders/new", "v1/users/", "v2/users/"]:
print(registry[endpoint]())
[('tesla 6', 1), ('torus', 2)]
['User, Legacy']
['User, Test', 'Account, Guest']
Authentication
Rather than rewriting several if
statements at the top of every method, just write them all once and reuse them.
Never again worry that an old function didn’t get updated with new changes to the authentication logic.
class user():
def __init__(self, name, access_level):
self.name = name
self.access_level = access_level
def requires_auth_level(required_level):
def decorated(thing):
def inner(*args, **kwargs):
if args[1].access_level <= required_level:
return thing(*args, **kwargs)
else:
raise Exception(user.name + " not authorized!")
return inner
return decorated
class secured():
def __init__(self):
self.secrets = ["proprietary information"]
@requires_auth_level(2)
def read_data(self, user):
return self.secrets
@requires_auth_level(1)
def write_data(self, user, data):
self.secrets.append(data)
return self.secrets
user_list = [user("Alice", 1), user("Bob", 2), user("guest", 3)]
store = secured()
for user in user_list:
try:
print("read data:", store.read_data(user))
except Exception as e:
print("failed read attempt: " + str(e))
try:
print("appended data:", store.write_data(user, "written by " + user.name))
except Exception as e:
print("failed append attempt: " + str(e))
read data: ['proprietary information']
appended data: ['proprietary information', 'written by Alice']
read data: ['proprietary information', 'written by Alice']
failed append attempt: Bob not authorized!
failed read attempt: guest not authorized!
failed append attempt: guest not authorized!
Conclusion
Decorators are a powerful tool for using your language to express your problem domain.
They are cornerstones of popular frameworks such as Flask.
For example, the Register decorator above works the same way as Flask’s app.route
(but of course this implementation isn’t production ready).
You can and should use Decorators to model cross-cutting technical and incidental concerns, leaving the business logic for class and function bodies.