James Clark

4 min read

2023: Python Development Workflow

I've had the chance to work on some personal projects recently and wanted to write down my workflow and tools. At work, the tools we use are made to suit everyone and we don't often change because our time is better spent on developing features.

When working on personal projects, it's the perfect time to try out new tools and libraries!

Project Setup

As with most modern Python projects, I use pyproject.toml to describe the project metadata.

I'm trying out hatch with hatch-vcs to pick up the version for the wheel.

Some choices I make when creating a new project:

  • src-layout.
  • hatch as the build-backend and test/coverage runner.
  • For personal projects, target Python 3.11.

I find using hatch new to be a great starting point, but a basic pyproject.toml looks like this:

[build-system]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"

[project]
name = "my-project"
dynamic = ["version"]

[tool.hatch.version]
source = "vcs"

Linting

My two main tools I use when developing in Python are Pyright and Ruff.

Pyright is a type checker for Python. I use Pyright for two reasons:

  1. It's pretty fast.
  2. A lot of the folks who use the code I develop use Visual Studio Code and Pylance is based on Pyright. If I can make sure my type hints work well with Pyright, then it should work for most people.

If you haven't used Ruff, stop reading, go to the GitHub repository, and download it.

Ruff performance chart
Ruff performance chart. Credit: charliermarsh/ruff.

Ruff it an all-in-one, extremely fast linter for Python code. I haven't had a chance to dive in and see how it works, but it is incredibly quick at returning diagnostics and has a bunch of rules that you can enable. My currently enabled rules are:

  • pycodestyle (E, Ruff default)
  • Pyflakes (F, Ruff default)
  • isort (I)
  • pyupgrade (UP)
  • flake8-builtins (A)
  • flake8-debugger (T10)
  • flake8-no-pep420 (INP)
  • NumPy-specfic Rules (NPY)

In a pyproject.toml, it looks like this:

[tool.ruff]
select = ["E", "F", "I", "UP", "A", "T10", "INP", "NPY"]

Editor

A code editor is a very personal choice, the best code editor is the one you like.

Usually, I use Neovim. However, on some recent projects I have been trying out Helix.

Diagnostics in the Helix text editor.
Diagnostics in the Helix text editor.

Helix Configuration

Having two language servers isn't currently supported in the released version of Helix, however there is a pull request with this feature.

Pyright is also broken with the PR, there was a fix in another PR.

With those two patches, I enable both Pyright and Ruff with this languages.toml:

[language-server.ruff]
command = "ruff-lsp"

[language-server.pyright]
command = "pyright-langserver"
args = ["--stdio"]
config = {}

[[language]]
name = "python"
scope = "source.python"
language-servers = ["pyright", "ruff"]

Debugging

Other than adding a breakpoint() call and dropping to pdb, I've used nvim-dap-python with Neovim. Helix introduced DAP support in 22.12, but I haven't tried it out yet.

Lately, I've taken to use PuDB as a debugger. It seems quite useful and with PYTHONBREAKPOINT="pudb.set_trace" it works nicely with breakpoint(). We'll see if this is my ongoing solution. I haven't found the perfect debugger in Python, so please message me on Mastodon if you have any tips! My wish-list is for rr for Python.

PuDB window debugging a simple function.
PuDB window debugging a simple function.

Evolution

I mainly get to look at new tools when developing personal projects, so next time I start something I may have a different set of tools to play with. I wouldn't recommend this approach when developing at work, in that case it's usually best to use what you're familiar with or what your team uses.

served from bos