SNAFU: Python Distribution Manager for Windows

Build status Documentation Status

SNAFU is a Python distribution manager for Windows. It helps you download, configure, install, and manage Python installations, [1] and provides utilities that can be integrated into your Python-related development workflows.

[1]Only CPython distributions from python.org at the moment.

Quick Start

Find an installer in Releases. Download and install to get a snafu command in a newly-opened command prompt.

Install Python 3.6:

$ snafu install 3.6

Run Python:

$ python3

Install Pipenv to Python 3.6:

$ pip3.6 install pipenv

And use it immediately:

$ pipenv --version
pipenv, version 9.0.1

Install Python 3.5 (32-bit):

$ snafu install 3.5-32

Switch to a specific version:

$ snafu use 3.5-32
$ python3 --version
Python 3.5.4

Switch back to 3.6:

$ snafu use 3.6
$ python3 --version
Python 3.6.4
$ python3.5 --version
Python 3.5.4

Uninstall Python:

$ snafu uninstall 3.5-32

Use --help to find more about SNAFU, or a specific command:

$ snafu --help
$ snafu install --help

Or read the Documentation.

Now you’re ready to use CPython on Windows like a first-class citizen, and ignore people telling you to get a Mac.

🤔😉😆

Topics

Install Python

SNAFU installs a Python version with a single command. To install <version> on you machine: [1]

snafu install <version>
[1]Use snafu list --all to find out what versions are available. See List Pythons for more information about the list command.

This automatically downloads the installer, and runs it with minimal user input possible. [2] If there’s need to install without Internet connection, you can download the installer yourself, and run:

snafu install --file=path\to\installer.exe <version>

to install directly from that installer. Either way, SNAFU sets up the Python installation on its own, and let you start using it right away.

[2]Generally only to confirm the UAC dialog, if needed.

SNAFU provides some extra executables for you. Say you have install Python 3.6 after you set up SNAFU. Now you can launch Python with:

python3

Install a package, say, Pipenv, with:

pip3 install pipenv

and have the pipenv command available immediately after.

Upgrade Python

SNAFU allows you to upgrade a Python installation to a newer micro version, e.g. 3.6.3 to 3.6.4. Run the following command to upgrade an installed version (if available):

snafu upgrade <version>

It takes some time for the developers to update the definition to a newer version. If you find a newer version released on python.org that is not available in SNAFU, send a pull request to update the definition files!

Uninstall Python

You probably guessed it:

snafu uninstall <version>

Similar to installing, this does nothing too fancy, but just runs the standard uninstaller. It does perform some additional cleanup if you are using the version. See :ref:manage about how you can manage/use versions.

Manage Pythons

The real benefits of SNAFU shows when you install more than one Python versions. Instead of manipulating the PATH environment variable, simply run the command with the appropriate version name.

# Python 3.6 (64-bit on available hosts, 32-bit otherwise).
snafu install 3.6
python3.6
pip3.6

# Python 3.5 (force 32-bit).
snafu install 3.5-32
python3.5-32
pip3.5-32

# Python 2.7.
snafu install 2.7
python2.7
pip2.7

No more python.exe shadowing because you have multiple versions in PATH.

Use Versions

When you have multiple Python versions installed, SNAFU only exposes the above two executables by default. If you want to expose other commands of a given Python version, You can tell SNAFU to use it: [1]

snafu use 3.6
[1]SNAFU does this automatically when you install your first ever Python version. This is why we had access to python3 without using 3.6 in Install Python.

So SNAFU publishes (almost) all commands available in Python 3.6, including

  • python3.exe
  • pip3.exe
  • easy_install-3.6.exe
  • All other scripts you installed via pip install.

As an exception, SNAFU blacklists python.exe, pip.exe, and easy_install.exe from being published, to encourage the best practice of always using versioned Python and Pip commands.

You can use multiple versions together to expose scripts installed across them:

snafu use 3.6 2.7

This exposes commands from both Python 3.6 and 2.7, with 3.6 taking precedence, i.e. if a given command exists in both versions, the 3.6 one will be called.

Manage Used Versions

To see what versions are currently in use:

snafu use

To reset using state (i.e. unuse all versions):

snafu use --reset

Misc. Commands

Upgrade SNAFU Itself

A special upgrade syntax to upgrade not a Python version, but SNAFU itself:

snafu upgrade self

This simply downloads the official installer and runs it.

List Pythons

This lists Python versions installed in your system:

snafu list

To list all Python versions available, including uninstalled ones, use:

snafu list --all

Either way, the output would be something like this:

o 2.7
o 3.4
  3.5
* 3.6
  • The o prefix means the version is installed.
  • * signifies an active version.
  • No prefix if the version is not installed.

Download Python

snafu download <version>

downloads the installer without exicuting it. The installer is saved to the current working directory by default, but you can also specify another directory with the --dest option.

Find Python Installation

snafu where <version>

prints where the installed python.exe really is, usually something like:

C:\Users\username\AppData\Local\Programs\Python\PythonXY\python.exe

This is useful when you need to pass the real executable somewhere else, e.g. set the path to an environment variable in a Powershell script.

Release History

Next (not released)

Nothing yet.

Unstable

Nothing yet.

Version 2.0

UI Changes
  • Remove confusing activate and deactivate commands in favour of the use command. The previous “reactivation” behaviour is now mapped to link --all.
  • A newly installed Python will be used if there are no other versions detected.
  • Add snafu upgrade self to perform in-place upgrade without manually downloading the installer.
  • Add snafu download <version> to download installer of given Python version without installing.
  • More complete help messages are provided to command arguments.
  • The uninstalling processes are now as interaction-free as installing. Previously some user intervention was needed (especially the EXE-based versions).
Behavioural Changes
  • snafu install now automatically uses the version if it is the only Python installation detected. This should simplify things even more for beginners.
  • Minor update to Python 3.4 definition.
  • Call snafu link --all automatically after installing with a shimmed pip command.
  • Upgrade Python 3.4 definition to 3.4.4.
  • Upgrade Python 3.6 definition to 3.6.4.
  • Improve behaviour when uninstalling versions not installed by SNAFU.
  • snafu link --all without active versions does not fail anymore. The warning message is still printed, but the exit code is now 0.
  • Py Launcher usage is reduced in favour of reading the registry ourselves.
  • Documentation is no longer installed with each Python version.
Installer Changes
  • Add an option to install and use a Python version after SNAFU is set up.
  • 64-bit variant does not carry x86 MSU files anymore, reducing installer size.
  • Correctly install Windows update KB2999226 on 32-bit Windows.

Version 1.0

UI Changes
  • New upgrade command to install a newer patch version on an installed version.
  • New link command to manually publish a script from the active Python versions.
Behavioural Changes
  • pip-X.Y are now published on install.
  • Automatically deactivate an uninstalling active version.
  • A Python exception will be raised early on download error, instead of failing later during installation.
  • Uninstallation now skips gracefully if launcher scripts do not exist.
Installer Changes
  • Bundle and trigger Windows update KB2999226 on installation to provide necessary runtime files on Windows Vista to 8.1 so the bundled Python 3.6 can run correctly.
  • Environment variables changes can now propagate correctly without OS restart.

Version 0.2

UI Changes
  • uninstall now attempts to use Windows’s uninstall feature to avoid re-download.
  • install and uninstall receives a --file option to allow using local installers without re-downloading.
     IMPORTANT: Correctness of the installer is not checked. The user is responsible for choosing the correct and valid installer file. Results of installing a faulty installer is undefined.
  • New command where to show where the actual interpreter is. This is useful if you need to pass it to another command (e.g. pipenv --python).
  • list shows activeness.
  • snafu --version shows program version.
Behavioural Changes
  • activate writes a pin file showing the current active versions.
  • Symbols in snafu list are changed.
Installer Changes
  • Environment variables are now set up automatically during installation.
  • Extract the py launcher MSI to make the distribution substentially smaller.
  • The installer now comes with both 64- and 32-bit flavours.
  • The uninstaller is now added to registry, so you can remove SNAFU in Control Panel.

Version 0.1

Initial release. Features I want the most are largely implemented. Only 64-bit Pythons are supported for now, and the installer is 64-bit-only.

An all-in-one installer that installs SNAFU into %LOCALAPPDATA%\Programs\SNAFU, and sets up the py launcher.

  • snafu install/uninstall <version>
  • snafu list [--all]
  • snafu activate/deactivate <version> [<version> ...]

Contribute to SNAFU

Development of SNAFU happens on GitHub.

Development Guide

Requirements
Optional Dependencies
  • NSIS 3.x if you want to build shims and the installer. makensis needs to be available in your shell.
Project Setup

Download and enter the project:

git clone https://github.com/uranusjr/snafu.git
cd snafu

Set up environment:

pipenv install --dev

Build the shims:

pipenv run invoke shims.build
Run Tests
pipenv run pytest tests

Unfortunately there are only very limited tests right now.

Run In-Development SNAFU
pipenv run python -m snafu [COMMAND] ...

This should have the same behaviour as an installed SNAFU command, but run inside the Pipenv-managed virtual environment.

Build Installer
pipenv run invoke installers.build

You can only build installers of your host’s architecture. Cross compilation is certainly possible, but I haven’t found the need to set it up.

After the command finishes you should get an EXE in the installers directory.

Build Documentation
pipenv run invoke docs.build

Documentation is managed with Sphinx, and hosted on Read the Docs with a custom domain.

Source Code Guideline

Try to follow the code style. For Python code, run the linter to check for issues before submitting:

pipenv run flake8 .

Format of text files are managed with EditorConfig. I recommend using one of the editor plugins to automatically format files. If you can’t/don’t want to do so, please at least make sure you’re using the correct format before sending in pull requests.

Explain SNAFU

The idea of SNAFU formed gradually from problems I faced when working on multiple development machines, constantly switching between Windows and macOS. I long a more consistent development experience, and a good way to manage my Python environment on Windows.

I tried out quite a few solutions. While a lot of them work for myself (to varying degree), none of them make me feel comfortable because I can’t teach anybody about them, which means they are all somehow wrong. Eventually I decided I need a complete managing system, which became SNAFU.

Below are some questions I either faced when discussing the Python setup problem with others, or through introspection. Each question represents an alternative solution I tried, but eventually couldn’t feel satisfied with. I hope they will answer why I think SNAFU is the best solution, and you should adapt it as well.

Why Not “Add Python to PATH”?

The standard CPython installer offers this option. It is not checked by default, but a lot of online tutorials tell you to do so. The CPython core team is correct; it shouldn’t be checked under most cercumstances.

CPython’s standard Windows build, unlike on Un*x, does not provide the “altinstall” build option. Every CPython distribution on Windows has only one Python executable, called python.exe, not versioned names such as python3.6.exe.

Adding Python to PATH stops being a good idea the moment you install a second Python. You can only access one Python at a time, and installed scripts from different versions start to mix, which is a bad thing. [1] It also become very tedious and delicate very quickly to manipulate the PATH environment variable.

[1]This is not a Windows-only problem, but also exactly why tutorials these days don’t recommand installing Python via python.org, but with platform-specific tools instead. Windows is the only mainstream operation system without a good Python verions management tool.

Why Not the Python Launcher (py.exe)?

Python introduced PEP 397 partly to solve the python.exe problem (also to interpret the shebang line on Windows). It installs a py.exe to your PATH, and instead of invoking python.exe directly, you should use, for example:

py -3.5 foo.py

to run foo.py with Python 3.5.

This is such a good idea SNAFU installs the Py Launcher during setup, and I encourage you to use it. But SNAFU also solves a few additional use cases that py.exe doesn’t:

  • Availability of versioned Python executables, e.g. python3.6.exe.
  • Managing commands other than python.exe.

SNAFU’s implementation also relies on a lot of the same values read by py.exe, so you can view SNAFU as an extension to it, not a replacement.

Why Not Install to All Users?

Authentication on Windows is difficult to manage, especially in the console. There really is no way to elevate priviledge inside a command prompt, and it is against common workflow to require opening a new console when you need to pip install something.

It is useful to have a system-wide Python setup. For developers, however, it is always recommended to install a seperate copy of Python for yourself, so you can manage it directly, without special priviledge.

Why Not Chocolatey?

Chocolatey is a package manager on Windows. A lot of SNAFU’s ideas are inspired by it: standard Windows installers, interaction-free installation, and shims for execution in command prompts. It is a very good tool, and I use it on my Windows machine—alongside SNAFU.

What SNAFU is to Chocolatey is similar to Pyenv to Homebrew and API etc. The aims are similar, but slightly different, so we can take an approach tailored to Python distribution.

Also I’m not very satisfied with Chocolatey’s user story. The setup is slightly complicated (due to Powershell’s execution policy), and requires administration priviledge to install packages. This is because it is fulfilling a different goal from SNAFU’s, but still makes me feel uncomforatble enough not to teach it to others.

Why Not Anaconda?

I considered Anaconda very hardly, but eventually decided against it. It is important to me that I need to feel comforatble teaching a solution to a relative beginner. Hey, let’s learn Python, but don’t download from `python.org`_, but this other website! This just feels wrong to me. I also have spent too much time answering beginners whether they should use Conda or Pip. I need to stick with CPython.

Anaconda is incredibly useful for those who need it, but for other people, maybe not so much. And since Anaconda also conforms to PEP 397, there’s nothing theoratically preventing SNAFU to co-live with Anaconda.

Implement SNAFU

Some implementation notes about SNAFU, in a series of questions. Nobody ever asked, but I want to answer them.

How are Pythons installed?

The official CPython installers are downloaded, and executed in a non-interactive manner. Check out the relevant documentation for more details:

Where are Pythons installed?

%LOCALAPPDATA%\Programs\Python\<version>. This is the standard “only-for-me” installation location for Python 3.5+, and we retrofit this rule to older versions as well for consistency.

How are executables linked?

They are not. Scripts are copied. .py files works as well because they have appropriate shebang lines, and can be handled by the py launcher, as specified in PEP 397.

A few wrapper executables (shims) are distributed with SNAFU, and are published into PATH to stub a few special executables, such as python.exe and pip.exe. When invoked, these shims rely on the registry to launch their real conterparts, and bridge all user interaction to them.

The shims minimise the need to expose internal DLLs, and, in the case of pip.exe etc., provide a chance to hook into extra machinery when you alter Python installations. This is inspired by pyenv and Chocolatey, and provides a more seamless experience.

Why the name?

Because Python is hard, Windows is harder, and setting up Windows for Python development is SNAFU. Or it’s Supernatrual Administration for You. Mosky says it sounds kind of like snake, so there’s that.