Python Decorators

Python Decorators

A Brief Introduction

Decorators in Python are a way of applying code to other code. They turn this

@some_decorator
def some_function():
    ...

into this

def some_function():
    ...
some_function = some_decorator(some_function)

They

  • can be used on functions or classes
  • can be nested
  • can be any expression resulting in a callable (as of Python 3.9, via PEP 614)

So this

@decorator_one
@decorator_two("extra information", 0)
class some_class():
    ...

is equivalent to

class some_class():
    ...
some_class = decorator_two("extra information", 0)(some_class)
some_class = decorator_one(some_class)

There is no more magic around decorators. There are some details to consider with other Python features. We’ll take a quick look at some of those features, then look at some possible uses of decorators.


Background

In Python, anything that can be given a name is an object. It can

  • have methods called on it
  • be passed to or returned from a function
  • have its properties changed, added, or removed.

Here we demonstrate a variety of objects being passed into the id function and having a property accessed:

for thing in [
    42,             # integer
    "string",       # string
    tuple,          # class
    [1,2,3].pop,    # method
    (lambda x:x),   # function
    ]:
    print(id(thing), thing.__class__)

1681762578000 <class 'int'>
1681770874800 <class 'str'>
140715374168768 <class 'type'>
1681845528112 <class 'builtin_function_or_method'>
1681845306992 <class 'function'>

Some objects have a method named __call__, which allows them to be called with syntax like this:

class C():
    def __init__(self):
        print("class called")

C()   # Classes are callable

def f():
    print("function called")

f()   # Functions are callable

class called
function called

A decorator must accept (at least) one argument: the thing it’s decorating.

Whatever it returns is bound to the name of the thing that was passed in.

def decorator(thing):
    print("decorating", thing.__name__)
    return thing

@decorator
def my_meaningful_name():
    print("meaningful stuff happened")

@decorator
class other_meaningful_name():
    def __init__(self):
        print("other meaningful stuff happened")
decorating my_meaningful_name
decorating other_meaningful_name

Note that the decorated function isn’t actually called yet, nor the decorated class instantiated.

This does not have to be related to the original thing in any way, or even be a callable itself.

def five(ignored):
    return 5

@five
def important_function():
    ...

print(important_function)
important_function()
5

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

~/ipykernel_17372/3447356210.py in <module>
      7 
      8 print(important_function)
----> 9 important_function()
     10 


TypeError: 'int' object is not callable

Here, attempting to call important_function fails because that name points to the integer 5, which is not a callable.

Some useful decorators

Timer

Print the duration of each call to the decorated object.

from datetime import datetime as dt

def timed(thing):
    def inner(*args, **kwargs):
        time_pre_call = dt.now()
        result = thing(*args, **kwargs)
        time_post_call = dt.now()

        print(f"{thing.__name__} lasted {time_post_call - time_pre_call}")

        return result
    return inner

@timed
def summation_to(high):
    sum(x for x in range(high))

summation_to(10)
summation_to(50_000_000)
summation_to lasted 0:00:00
summation_to lasted 0:00:03.511591

Retry

If the first call to a function throws an exception, call it again with the same arguments.

def retry(thing):
    def inner(*args, **kwargs):
        # Instead of retrying once, consider using a loop and accepting an argument for a number
        try:
            print("Trying with args", args, kwargs)
            thing(*args, **kwargs)
        except Exception:
            print("Retrying with args", args, kwargs)
            thing(*args, **kwargs)
    return inner

@retry
def div_zero(x):
    return x/0

try:
    div_zero(7)
except ZeroDivisionError:
    print("Cannot divide by zero")
Trying with args (7,) {}
Retrying with args (7,) {}
Cannot divide by zero

Cache

Prevent extra work by storing the result of a function called with specific arguments, and return the old result if called with those same arguments again. Here we reuse the above timed decorator to show the duration of function calls. Repeat calls are much shorter.

def cached(thing):
    _cache = {}
    def inner(*args):
        # This could be a time-expiring cache by storing a tuple of (timestamp, value) and checking the timestamp!
        if not args in _cache:
            _cache[args] = thing(*args)
        return _cache[args]
    return inner

import time

@timed
@cached
def slow_echo(arg):
    time.sleep(2)
    return arg

import random
for _ in range(10):
    print(slow_echo(random.randint(1, 6)))

inner lasted 0:00:02.012066
2
inner lasted 0:00:00
2
inner lasted 0:00:02.006280
5
inner lasted 0:00:02.013154
4
inner lasted 0:00:00
4
inner lasted 0:00:00
2
inner lasted 0:00:02.007700
6
inner lasted 0:00:00
2
inner lasted 0:00:00
5
inner lasted 0:00:02.015261
3

Mock

Replace a function with one that simply returns the specified value without calling the mocked function at all. Here we bypass an infinite loop and just provide the answer.

def mock(mock_value):
    def inner(thing):
        # Consider having the mock value vary depending on the arguments
        return lambda *args, **kwargs: mock_value
    return inner

@mock(2)
def long_running_function():
    current, total = 1, 0
    # TODO: fix this infinite loop...
    while True:
        total += current
        current /= 2
    return total

print("long function returned", long_running_function())
long function returned 2

Trace

This decorator can be applied to specific functions to produce a visualized call stack featuring only those decorated functions. Here we skip tracing function C, but do show its child calls to function D as if they were called by C’s caller, either A or B.

def trace(thing):
    def inner(*args, **kwargs):
        print(trace.indent * "  |" + "--" + thing.__name__)
        trace.indent += 1
        result = thing()
        trace.indent -= 1 # BUG - throwing exceptions in thing() will skip this line
        return result
    return inner
trace.indent = 0


@trace
def A():
    B()
    C()
    B()

@trace
def B():
    C()

# Not decorated, not traced.
def C():
    D()

from random import random

@trace
def D():
    if random() < 0.25:
        D()
    if random() < 0.25:
        D()

A()
--A
  |--B
  |  |--D
  |  |  |--D
  |  |  |  |--D
  |  |  |  |  |--D
  |  |  |  |--D
  |--D
  |  |--D
  |--B
  |  |--D
  |  |  |--D
  |  |  |--D

Register

Suppose a web request dispatcher needs to know about functions which can handle requests and what kinds of requests they support. A traditional approach would be to add a new handler in one file and also a reference to it from the dispatcher file. This is fragile because it’s easy to add a new endpoint without noticing the need to update the dispatcher (and possibly other locations).

With this approach, implementing a new handler happens in only one place.

registry = {}

def register(name):
    def inner(thing):
        registry[name] = thing
        return thing
    return inner


@register("v1/orders/new")
def orders():
    return [("tesla 6", 1),
            ("torus", 2)]

@register("v1/users/")
def users_legacy():
    return ["User, Legacy"]

@register("v2/users/")
def users():
    return ["User, Test", "Account, Guest"]


for endpoint in ["v1/orders/new", "v1/users/", "v2/users/"]:
    print(registry[endpoint]())
[('tesla 6', 1), ('torus', 2)]
['User, Legacy']
['User, Test', 'Account, Guest']

Authentication

Rather than rewriting several if statements at the top of every method, just write them all once and reuse them. Never again worry that an old function didn’t get updated with new changes to the authentication logic.

class user():
     def __init__(self, name, access_level):
        self.name = name
        self.access_level = access_level

def requires_auth_level(required_level):
    def decorated(thing):
        def inner(*args, **kwargs):
            if args[1].access_level <= required_level:
                return thing(*args, **kwargs)
            else:
                raise Exception(user.name + " not authorized!")
        return inner
    return decorated


class secured():
    def __init__(self):
        self.secrets = ["proprietary information"]

    @requires_auth_level(2)
    def read_data(self, user):
        return self.secrets

    @requires_auth_level(1)
    def write_data(self, user, data):
        self.secrets.append(data)
        return self.secrets


user_list = [user("Alice", 1), user("Bob", 2), user("guest", 3)]
store = secured()

for user in user_list:
    try:
        print("read data:", store.read_data(user))
    except Exception as e:
        print("failed read attempt: " + str(e))

    try:
        print("appended data:", store.write_data(user, "written by " + user.name))
    except Exception as e:
        print("failed append attempt: " + str(e))
read data: ['proprietary information']
appended data: ['proprietary information', 'written by Alice']
read data: ['proprietary information', 'written by Alice']
failed append attempt: Bob not authorized!
failed read attempt: guest not authorized!
failed append attempt: guest not authorized!

Conclusion

Decorators are a powerful tool for using your language to express your problem domain. They are cornerstones of popular frameworks such as Flask. For example, the Register decorator above works the same way as Flask’s app.route (but of course this implementation isn’t production ready).

You can and should use Decorators to model cross-cutting technical and incidental concerns, leaving the business logic for class and function bodies.