Homebrew Python Is Not For You

Don’t use Homebrew Python. It’s not meant for you.

At some point Homebrew made changes that adversely affect Python development. While Homebrew’s Python formula has been the go-to choice for Python developers (including me) for a long time, that time is past — there are now much better options available.

What’s the Problem?

Perhaps you heard stories about why you shouldn’t use the system-bundled Python¹, so instead you use Homebrew to install Python and then use its interpreter to create a virtual environment. A month later, you activate that same environment, and when you try to use it, you see this inscrutable error:

dyld: Library not loaded: @executable_path/../.Python
Referenced from: /your/home/.virtualenvs/your-env-name/bin/python
Reason: image not found

What happened? The Python interpreter referenced by the virtual environment… no longer exists. But how can that be? You didn’t change anything!

You didn’t change anything… but Homebrew did. Of the adverse changes Homebrew made recently, two stand out as root causes of this problem.

First, Homebrew now upgrades dependent packages automatically. So if you upgrade any package that depends on the python formula² (and there are nearly 400 of them), then your Python interpreter will be upgraded also — whether you like it or not. You have no control over this.

Second, Homebrew now automatically deletes previous versions every 30 days, including of course any Python interpreters you used to create virtual environments.

The havoc wrought by the combination of these two changes is considerable. Those Python virtual environments you created over the last year? All of them are now broken. Why? Just because you ran brew upgrade ponysay? Yup.³

Homebrew Python Is Not For You

The web is littered with frustrated reports about this problem and replete with hacked-together Bash scripts that purport to repair damaged Python virtual environments. And all because of one core misunderstanding.

Homebrew’s Python is not for you. It exists to serve Homebrew, or more accurately, Homebrew’s other formulae. The primary purpose of Homebrew’s Python formula is to enable other Python-dependent Homebrew packages to work. If installing Homebrew’s Python allows you to run the occasional Python script or access the REPL⁴, that’s a nice side benefit, but anything beyond that — including developing software with Python — is squarely out-of-scope.

Once you come to this realization, it should be clear that this is the wrong tool for the job. So what is the right tool?

Best Tool For the Job

There are several solutions that put you back in control of when Python versions are installed, upgraded, and removed. Personally, I use asdf and its Python plugin to do just that. I prefer asdf to similar tools such as Pyenv and Pythonz because not only can you install multiple Python versions but also multiple Node.js versions, Ruby versions, and many more.

After switching to this solution, would you like to guess how many times I’ve had something break due to a Python upgrade? Right: Zero. Zero times. Because I am now in control of when older Python versions are swept away, I can upgrade virtual environments at my leisure, and then, once I’ve confirmed no other environments rely on older versions, I can then uninstall those old Python versions without worrying about something breaking.

Epilogue

There are hack workarounds, and I’ve tried them. None of them fix the core problem.

For example, the automatic monthly clean-up can be disabled by defining a HOMEBREW_NO_INSTALL_CLEANUP environment variable and setting it to “1”. But all it takes is one accidental brew invocation in a shell session where that variable is not set, and your environments are laid to waste. Plus, even if you avoid that footgun, there is no easy way to remove specific old versions that you may no longer need; you have no choice but to rm -rf directories in /usr/local/Cellar/ and hope you don’t break anything.

I also tried to avoid installing Homebrew Python entirely, with the idea that since I’m not using it anymore, I don’t need it to be installed. But assuming you use Homebrew to install other tools, that would be a fool’s errand — Homebrew Python will almost certainly be installed anyway when some other formula requires it. But that doesn’t mean we have to use it! Let the Homebrewed Python sit there and do its job serving other formulae, and we can forget it exists.

If you liked this article, follow me on Mastodon so you’ll know when I post new ones.

1. I’ve talked about some macOS-specific reasons to eschew its bundled Python, but Ubuntu/Debian is also affected. ↩︎
2. You can see which of your installed formulae depend on the python formula by running: brew uses python --installed … I have over 30 Python-dependent formulae installed on my workstation. ↩︎
3. Previously, you had to manually run brew cleanup to get yourself into this mess. Now the carnage happens automatically. Any time a new Python version is released, even patch releases such as from 3.8.6 to 3.8.7, there is the very real potential that any virtual environment that depends on the previous Python version will just break horribly and without any warning. And it’s not just your virtual environments — any software that depends on Python can have the carpet yanked out from under it, including Pipx, YouCompleteMe, and other components that you may not even realize depend on the presence of a stable Python foundation. Moreover, this problem isn’t even confined to Python. Need a specific version of Node.js to build static assets for your web application? Have an old Rails app that needs a previous Ruby version? Under the Homebrew regime, those can also be swept away at any time. ↩︎
4. Assuming you’re okay with bundled Python caveats, there’s no reason to install Homebrew just for Python, since macOS includes Python 3 in its command-line tools (CLT) package. ↩︎