2/9/20235 min read

Context Managers

I was chatting with a colleague about Python context managers and how they work.

I thought it would be worthwhile writing down what I know to see if it helps other folks.

Motivating Example

One of the first context managers I came across was when I wanted to open a file with the open built-in. In many tutorials they show this brand-new keyword and syntax without much explanation: ^[1]

with open("my_file") as f:
    # do stuff with f

What does with and as f mean? This is a context manager in Python!

Resource Management

Before we jump into specifics, we should look at resource management. A common task in programming is keeping track of resources (or things) that your program is accessing. For example, files, sockets, mutexes, etc.

In each of these examples we have to perform some operation to get the resource and another operation to release the resource.

f = open("some_file") # Get a file
## do stuff with f
f.close() # Release the file

It can be a pain to keep track of the resources that are in use and when we should release them, especially when your language has exceptions or jumps.

Some programming languages have the concept of deferring some code until the end of the scope and others have finalisers or destructors. ^[2]

But we're here to talk about Python. After I open a file, how to I make sure that safely close the file when I'm done?

I know I can use a try/except block to handle exceptions:

f = open("some_file")
try:
    # do stuff with f
except:
    # Close the file on an error
    f.close()

But I also want to close the file even if there were no errors (or if I return, continue or break). Instead, I can use the finally keyword:

f = open("some_file")
try:
    # do stuff with f
finally:
    # Always close the file!
    f.close()

This snippet is almost everything that a context manager does! If you look at the original PEP you'll see this short snippet looks awfully similar.

It would be a pain to have to write this over and over, so we have the with statement as syntactic sugar. as f is a way to optionally name the resource we get.

with open("my_file") as f:
    # do stuff with f

Implementing a Context Manager

If we want to build our own context manager, we have two options:

Use a helper decorator.
Implement the context manager type.

Helper Decorator

The contextlib library contains a helper function that can be used as a decorator ^[3].

The @contextmanager decorator takes a generator function (a function that yields a value) and converts it to a context manager.

When using the decorator, you need to keep the try/except/finally statement inside the function to ensure the resource is released on any exception.

from contextlib import contextmanager

@contextmanager
def my_context_manager():
    resource = get()
    try:
      yield resource
    finally:
      release(resouce)

Implementing the Context Manager Type

The context manager type is implemented with two specific dunder^[4] methods: __enter__ and __exit__.

A key difference between implementing these methods and using the @contextmanager decorator is that you should notuse a try/except/finally statement.
Instead, any exception is passed to the __exit__ method.

`enter`

__enter__ is used for getting our resource and is called as we enter the with block.

Note: __enter__ doesn't have any parameters (except self), if you need any input arguments you can use the __init__method for your class.

def __enter__(self):
    # Get a resource and return it!
    return "hello"

`exit`

__exit__ is used for releasing our resource and is called as we exit the with block. __exit__ is called even if we raise an exception.

def __exit__(self, exc_type, exc_value, exc_tb):
    # Do any tidying up
    return True

__exit__ is a bit more complicated. It was 3 additional parameters: exc_type, exc_value, exc_tb. They're all related to exception handling:

exc_type is the type of the exception raised.
exc_value is the value of the exception raised. If you've ever caught an exception with except Exception as e:, this is the same as e.
exc_tb is the traceback object, you can inspect this with the traceback module.

If no exception was raised inside the with block, all three of these parameters will be None.

The final complication with __exit__ is its return value.

If an exception was called and __exit__ returns a truthy value, the exception is not propagated and execution continues as if it never happened. However, if it returns a falsey value, then the exception is propagated, and you must catch it before continuing.

Summary

Context managers can be used to manage resources, and we can use them in our code with the with statement. Implementing our own context manager is as simple as using the @contextmanager decorator or implementing __enter__ and __exit__.