Different Python projects depend on different packages. To keep them separated and avoid polluting the global space with package installations we can use the concept of virtual environments. There are several ways to work with virtual environments in Python:

virtualenv
virtualenvwrapper
Conda (part of Anaconda)

virtualenv

Virtualenv — virtualenv 20.3.2.dev1+g7185e57 documentation

To use virtualenv we first need to install all necessary components:

$ sudo apt update
$ sudo apt install python3-dev python3-pip
$ sudo pip3 install -U virtualenv # system-wide install

Verification:

$ python3 --version
$ pip3 --version
$ virtualenv --version

or

$ virtualenv

usage: virtualenv [--version] [--with-traceback] [-v | -q] [--read-only-app-data] [--app-data APP_DATA] [--reset-app-data] [--upgrade-embed-wheels] [--discovery {builtin}] [-p py] [--try-first-with py_exe]

[--creator {builtin,cpython3-posix,venv}] [--seeder {app-data,pip}] [--no-seed] [--activators comma_sep_list] [--clear] [--no-vcs-ignore] [--system-site-packages] [--symlinks | --copies] [--no-download | --download]

[--extra-search-dir d [d ...]] [--pip version] [--setuptools version] [--wheel version] [--no-pip] [--no-setuptools] [--no-wheel] [--no-periodic-update] [--symlink-app-data] [--prompt prompt] [-h]

dest

virtualenv: error: the following arguments are required: dest

SystemExit: 2

$ virtualenv --help

[--extra-search-dir d [d ...]] [--pip version] [--setuptools version] [--wheel version] [--no-pip] [--no-setuptools] [--no-wheel] [--no-periodic-update] [--symlink-app-data] [--prompt prompt] [-h]

dest

optional arguments:

--version display the version of the virtualenv package and its location, then exit

--with-traceback on failure also display the stacktrace internals of virtualenv (default: False)

--read-only-app-data use app data folder in read-only mode (write operations will fail with error) (default: False)

--app-data APP_DATA a data folder used as cache by the virtualenv (default: /home/bojan/.local/share/virtualenv)

--reset-app-data start with empty app data folder (default: False)

--upgrade-embed-wheels trigger a manual update of the embedded wheels (default: False)

-h, --help show this help message and exit

verbosity:

verbosity = verbose - quiet, default INFO, mapping => CRITICAL=0, ERROR=1, WARNING=2, INFO=3, DEBUG=4, NOTSET=5

-v, --verbose increase verbosity (default: 2)

-q, --quiet decrease verbosity (default: 0)

discovery:

discover and provide a target interpreter

--discovery {builtin} interpreter discovery method (default: builtin)

-p py, --python py interpreter based on what to create environment (path/identifier) - by default use the interpreter where the tool is installed - first found wins (default: [])

--try-first-with py_exe try first these interpreters before starting the discovery (default: [])

creator:

options for creator builtin

--creator {builtin,cpython3-posix,venv}

create environment via (builtin = cpython3-posix) (default: builtin)

dest directory to create virtualenv at

--clear remove the destination directory if exist before starting (will overwrite files otherwise) (default: False)

--no-vcs-ignore don't create VCS ignore directive in the destination directory (default: False)

--system-site-packages give the virtual environment access to the system site-packages dir (default: False)

--symlinks try to use symlinks rather than copies, when symlinks are not the default for the platform (default: True)

--copies, --always-copy try to use copies rather than symlinks, even when symlinks are the default for the platform (default: False)

seeder:

options for seeder app-data

--seeder {app-data,pip} seed packages install method (default: app-data)

--no-seed, --without-pip do not install seed packages (default: False)

--no-download, --never-download

pass to disable download of the latest pip/setuptools/wheel from PyPI (default: True)

--download pass to enable download of the latest pip/setuptools/wheel from PyPI (default: False)

--extra-search-dir d [d ...] a path containing wheels to extend the internal wheel list (can be set 1+ times) (default: [])

--pip version version of pip to install as seed: embed, bundle or exact version (default: bundle)

--setuptools version version of setuptools to install as seed: embed, bundle or exact version (default: bundle)

--wheel version version of wheel to install as seed: embed, bundle or exact version (default: bundle)

--no-pip do not install pip (default: False)

--no-setuptools do not install setuptools (default: False)

--no-wheel do not install wheel (default: False)

--no-periodic-update disable the periodic (once every 14 days) update of the embedded wheels (default: False)

--symlink-app-data symlink the python packages from the app-data folder (requires seed pip>=19.3) (default: False)

activators:

options for activation scripts

--activators comma_sep_list activators to generate - default is all supported (default: bash,cshell,fish,powershell,python,xonsh)

--prompt prompt provides an alternative prompt prefix for this environment (default: None)

config file /home/bojan/.config/virtualenv/virtualenv.ini missing (change via env var VIRTUALENV_CONFIG_FILE)

Create virtual environment (we chose to use python3 as Python interpreter and venv as the directory where to store virtual environment):

$ virtualenv -p python3 ./venv_dir_name

$ virtualenv --system-site-packages -p python3 ./venv_dir_name

From virtualenv · PyPI:

If you build with virtualenv --system-site-packages ENV, your virtual environment will inherit packages from /usr/lib/python2.7/site-packages (or wherever your global site-packages directory is).

This can be used if you have control over the global site-packages directory, and you want to depend on the packages there. If you want isolation from the global system, do not use this flag. BK: This flag should be avoided as it defeats the entire point of virtualenvs (which is why --no-site-packages was made the default).

Example:

$ virtualenv -p python3 ./venv

created virtual environment CPython3.8.5.final.0-64 in 253ms

creator CPython3Posix(dest=/home/bojan/dev/github/python-demo/venv, clear=False, no_vcs_ignore=False, global=False)

seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/bojan/.local/share/virtualenv)

added seed packages: pip==20.3.3, setuptools==51.1.2, wheel==0.36.2

activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

I once experienced this error when tried to create a virtual environment:

$ virtualenv -p python3 ./venv

Traceback (most recent call last):

File "/usr/local/bin/virtualenv", line 5, in <module>

from virtualenv.__main__ import run_with_catch

File "/usr/local/lib/python3.8/dist-packages/virtualenv/__init__.py", line 3, in <module>

from .run import cli_run, session_via_cli

File "/usr/local/lib/python3.8/dist-packages/virtualenv/run/__init__.py", line 6, in <module>

from ..app_data import make_app_data

File "/usr/local/lib/python3.8/dist-packages/virtualenv/app_data/__init__.py", line 9, in <module>

from appdirs import user_data_dir

ModuleNotFoundError: No module named 'appdirs'

I then installed the missing module via pip but this showed another error:

$ sudo pip3 install appdirs

Collecting appdirs

Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)

ERROR: virtualenv 20.3.1 requires filelock<4,>=3.0.0, which is not installed.

Installing collected packages: appdirs

Successfully installed appdirs-1.4.4

I resolved this by reinstalling the virtualenv:

$ sudo pip3 install virtualenv

Requirement already satisfied: virtualenv in /usr/local/lib/python3.8/dist-packages (20.3.1)

Requirement already satisfied: appdirs<2,>=1.4.3 in /usr/local/lib/python3.8/dist-packages (from virtualenv) (1.4.4)

Requirement already satisfied: six<2,>=1.9.0 in /usr/lib/python3/dist-packages (from virtualenv) (1.14.0)

Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.8/dist-packages (from virtualenv) (0.3.1)

Collecting filelock<4,>=3.0.0

Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)

Installing collected packages: filelock

Successfully installed filelock-3.0.12

Once virtual environment is created, we'll have venv/bin/activate script created. Let's explore it:

$ cat venv/bin/activate
...

deactivate() {

...

}

...

VIRTUAL_ENV="/home/bojan/dev/github/tensorflow-demo/venv"
export VIRTUAL_ENV

_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

...

pydoc () {

python -m pydoc "$@"

}

...

We can see that we'll now have a new environment variable set - VIRTUAL_ENV and also a deactivate commmand.

To activate virtual environment:

$ source ./venv/bin/activate

We can also use a dot command:

$ . ./venv/bin/activate

When virtual environment is active, your shell prompt is prefixed with its name in form: (venv).

(venv) $ which pip

/home/bojan/dev/github/python-demo/venv/bin/pip

(venv) $ which pip3

/home/bojan/dev/github/python-demo/venv/bin/pip3

If we've saved requirements.txt, we can install all dependencies now with on of the following commands (see Python Package Management with pip | My Public Notepad):

$ pip install -r requirements.txt

$ pip3 install -r requirements.txt

$ pip3 install -r requirements.txt --verbose

Installing packages using pip and virtual environments — Python Packaging User Guide

It is possible to install a single new package in the virtual environment:

$ pip install package_name

python - How to install a package inside virtualenv? - Stack Overflow

Example:

(venv) $ pip3 install psycopg2

Collecting psycopg2

Using cached psycopg2-2.8.6-cp38-cp38-linux_x86_64.whl

Installing collected packages: psycopg2

Successfully installed psycopg2-2.8.6

To verify installation path:

(venv) $ pip3 show psycopg2

Name: psycopg2

Version: 2.8.6

Summary: psycopg2 - Python-PostgreSQL Database Adapter

Home-page: https://psycopg.org/

Author: Federico Di Gregorio

Author-email: fog@initd.org

License: LGPL with exceptions

Location: /home/user/dev/my-app/venv/lib/python3.8/site-packages

Requires:

Required-by:

To check the version of the package installed in virtual environment:

(venv) $ pip3 list | grep psycopg

psycopg2 2.8.6

To exit virtual environment:

(venv) $ deactivate

How to rename virtual environment?

directory - How to rename a virtualenv in Python? - Stack Overflow

From inside active virtual environment:

$ pip freeze > requirements.txt
$ deactivate

From python - Pip freeze vs. pip list - Stack Overflow:

pip list shows ALL packages.

pip freeze shows packages YOU installed via pip (or pipenv if using that tool) command in a requirements format.

Also, be aware of `$ pip freeze > requirements.txt` considered harmful.

Then delete old environment directory. E.g.

$ rm -r ~/python-envs/tensorrt-5.1.5.1-tensorflow-1.14.0/

...and create a new one with correct name:

$ python3 -m virtualenv -p python3 ~/python-envs/tensorrt-5.1.5.1-tensorflow-1.15.0
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/nvidia/python-envs/tensorrt-5.1.5.1-tensorflow-1.15.0/bin/python3
Also creating executable in /home/nvidia/python-envs/tensorrt-5.1.5.1-tensorflow-1.15.0/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.

Activate it and install all packages from the saved list:

$ source ~/python-envs/tensorrt-5.1.5.1-tensorflow-1.15.0/bin/activate

$ pip install -r requirements.txt

$ pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v43 tensorflow-gpu==1.15.0+nv19.12