Live reloading multi-module mitmproxy scripts using modd

Have you ever written a mitmproxy extension script which got too large for a single file, so you split it into multiple modules, only to realize you have broken mitmproxy’s live reloading functionality? Well, I have, and you might too in the future, so I’m writing this small tutorial on how to make it work.

For the purposes of this example, let’s assume your script is split into the following file structure:

main.py
baz.py
foo/bar.py

main.py is the main module, used as the entrypoint for the script and passed to mitmproxy using the -s flagmitmproxy -s main.py

. The other modules are imported and used by the main module.

In such a situation, mitmproxy will only reload the script upon changes to the main.py file. It will also only reload the main module, so any changes you make to other modules will not be reflected in the running instance.

To restore our previous live reloading workflow, we need to:

1. Explicitly reload the other modules from main.py using importlib.reload.
2. Watch for changes to other modules and execute touch main.py when they happen. Let’s solve them in order. It is important to note that any identifiers imported before the importlib.reload callWhich includes the module itself. So, if you import foo and then do importlib.reload(foo), the identifier foo still refers to the old module contents, not the reloaded one. Instead, you’d need to do foo = importlib.reload(foo).
   
   will not get updated. Hence, it’s best to first import all your modules at the start of your main module (so you have something to reload), reload them, and then import anything you need as usual.

In other words, if you had

import baz
from foo.bar import quux

You’d now do

import importlib

import baz
import foo.bar

modules = [baz, foo.bar]
for m in modules:
    importlib.reload(m)

import baz
from foo.bar import quux

A bit wordy, but it works! If you now change something in baz.py and then touch main.py, the changes you made will be reflected in the running mitmproxy instance.

This solves problem 1, but what about problem 2? For this, we will be using a small lightweight Go utility called modd designed for this exact purpose.

Install modd and then create a modd.conf file in the root directory of your script:

**/*.py {
    prep: touch main.py
}

Then simply run modd while inside that directory and it should work: every time you change and save any .py file located in that directory subtree, modd will run touch main.py for you.

In the future, mitmproxy might add built-in support for this.