What’s wrong with my Python Environment in Linux, MacOS, Windows?

No problem, everything is fine…….As long as it is working…

https://xkcd.com/1987/

The four levels of Python Development Environment:

I. Interpreters → II. Tools → III. Dependencies → IV. Application

Photo by Ben Mathis Seibel on Unsplash

To ensure your application can run properly, there is four layers of environments (I. Interpreter → II. Tools → III. Dependencies → IV. Application). It’s like a maze. To solve a maze, it’s easier to walk from the exit. The layer above your application is the dependencies. So we start from there, III Dependencies.

III. Dependencies

$ python3 -m venv --prompt="project" .venv
$ . .venv/bin/activate
(project) $ pip install -r requirements.txt

You often hear people do bin activate with virtualenv / venv. What is bin activate actually doing? It adds path to your environment. If you have multiple environment, you might be adding wrong paths. And, it’s not easy to return to a clean environment again. (For more details, See link: https://gist.github.com/datagrok/2199506).

Figure extracted from PyCon TW 2018 Talk of TP Chung.

pipenv is a better way to manage venv. It combines pip and venv. It provides the flexibility of abstract environment dependency, and the certainty of concrete dependency.

pythonfinder:

requirements.lib ( convert pipfile → requirements.txt)

pipfile-cli

II. Tools

How to install pipenv? Should I use a virtual environment to install a tool for virtual environment? See video( first chicken or first egg?).

TL;DR. You don’t need to activate venv to run programs installed by venv.

$ mkdir -p ~/.local/bin ~/.local/venvs
$ python3 -m venv ~/.local/venvs/packagemanagers
$ ~/.local/venvs/packagemanagers/bin/pip install pipenv poetry
$ln -s ~/.local/venvs/packagemanagers/bin/pipenv ~/.local/bin
$ln -s ~/.local/venvs/packagemanagers/bin/poetry ~/.local/bin
$pipenv --version
  pipenv, version 11.10.3

Rule of Thumb:

• Do not use sudo . Ever.
• Always use a virtual environment.
• Use when pip install --user when desperate

I. Interpreters

How do I install python?

Linux machine:

• Ubuntu: use “deadsnakes” PPA
• Compile your own python
• Be careful with the package manager

MacOS

• Do not use System Python
• Do not use Homebrew Python
• You could do it manually. Compile your own python
• Or, download from (Python.org) tar xvz python.tar.gz...
• Instead, pyenv has a very nice plugin called python-build. Use that.

$ python-build 3.10.1 ~/.local/pythons/3.10
$ ln -s ~/.local/bin/pythons/3.10/python3.10 ~/.local/bin
$ ln -s ~/.local/bin/python3.10 ~/.local/bin/python3

Then, disable the command $ pythonwithout a specified version:

# Disable python if no version specified. To avoid using system python.
python(){
    local PYTHON="$(which python)"
    if [[ "$PYTHON" == /usr/* ]]; then 
        echo "nope! please specifiy python version." >&2 | echo > /dev/null
    else
        "$PYTHON" "$@"
    fi
}
# Do the same for pip
pip(){
    local PIP="$(which pip)"
    if [[ "$PIP" == /usr/* ]]; then 
        echo "nope! please specifiy pip version." >&2 | echo > /dev/null
    else
        "$PIP" "$@"
    fi
}

A nice tool wrapped all the above mention aspects into 3 lines listed below. Check it out: https://github.com/uranusjr/pythonup-macos

pythonup install 3.6
pythonup upgrade 3.6
pythonup use 3.6 # ln -s python3.6 to python3

Windows

• No global Python to worry about
• No support from pyenv
• Compiling/building is drastically more difficult
• However you can use the prebuilt version:

1. Go to python.org and download python of the version you like.
2. Install by clicking.

Or, download and use pythonoup: https://github.com/uranusjr/pythonup-windows

pythonup install 3.6
pythonup upgrade 3.6
pythonup use 3.6 # ln -s python3.6 to python3

Summaries

1. Always compile your own python (if possible)
2. Always use virtual environments. Link them to PATH if needed.
3. Pipfile and Pipenv (??)

Here, we didn’t discuss building of packages and delivering its proper environments. We will discuss that next time.

What about conda environment?

• It takes into account other programs outside of python. Does not use PyPI, but use it’s own distribution of all python packages. The version may be delaed than on PyPI.
• Anaconda, Inc. is a company. We aim to support community-build & community-driven projects as a open-source developer.

This medium post is based on a brilliant talk at PyCon TW 2018 from TP Chung. (See Reference 1.)

References:

1. PyCon TW 2018, Tzu-ping Chung, 這樣的開發環境沒有問題嗎？https://www.youtube.com/watch?v=6Nl0IYkU0hU
2. https://gist.github.com/datagrok/2199506