Pages

Saturday, 3 October 2020

Managing Python Environments with Conda

At the end of my article How to install Anaconda on Ubuntu I verified that Anaconda was installed successfully by executing conda list. You've probably noticed (base) that appears in front of command prompt:

(base) $

This happens after we activate conda by executing ~/.bashrc:

$ cat ~/.bashrc
...
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/bojan/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/bojan/anaconda3/etc/profile.d/conda.sh" ]; then
        . "/home/bojan/anaconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/bojan/anaconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<
...




As conda is also an environment manager it is capable of setting a specified or a default environment. 

Each environment has associated:
  • name
  • collection of packages 
  • resources which are stored in a dedicated directory

When conda is activated, it sets a default environment unless some other is specified. Its name appears within parentheses before the prompt in terminal. In our case that was a default environment named base.

To list all available environments with their names and associated directories we can use:

$ conda env list
# conda environments:
#
base                  *  /home/bojan/anaconda3

Asterisk (*) appears next to the currently active environment.

Another way to get the same information is:

$ conda info --envs
# conda environments:
#
base                  *  /home/bojan/anaconda3

Here is the content of the base environment directory:

$ ls -1a /home/bojan/anaconda3
.
..
bin
compiler_compat
condabin
conda-meta
doc
envs
etc
include
lib
libexec
LICENSE.txt
man
mkspecs
phrasebooks
pkgs
plugins
qml
resources
sbin
share
shell
ssl
translations
var
x86_64-conda_cos6-linux-gnu

To deactivate the current environment:

(base) $ conda deactivate
$

To activate default environment again:

$ conda activate
(base) $ 


After installing Conda, base environment is set to be auto activated by default:

(base) $ conda config --show | grep auto_activate_base
auto_activate_base: True


(base) $ conda config --set auto_activate_base False

Let's verify it:

(base) $ conda config --show | grep auto_activate_base
auto_activate_base: False

Next time we open a new terminal, we should not see (base) in front of the prompt.

Note that when we have a base environment activated, Terminal shell uses Python from Anaconda installaton:

(base) $ which python
/home/bojan/anaconda3/bin/python

(base) $ which python3
/home/bojan/anaconda3/bin/python3

When no environment is activated, Terminal shell uses Python from a system installation:

$ which python
/usr/bin/python

$ which python3
/usr/bin/python3

Same applies to pip and pip3:

(base) $ which pip
/home/bojan/anaconda3/bin/pip
(base) $ which pip3
/home/bojan/anaconda3/bin/pip3

$ which pip
/home/bojan/.local/bin/pip
$ which pip3
/home/bojan/.local/bin/pip3


Each environment can be specified with its YAML (.yml) file. This file contains all necessary information for conda to recreate the same environment on some other computer:
  • name of the environment
  • channels
  • dependencies
Environment yaml file generic example:

name: my_env
channels:
  - channel1
dependencies:
  - python=3.6
  - some_package=1.2 # it is possible to add comments
  - some_package2
  - jupyter=1.0.0
  - jupyterlab=0.34.9
  - keras=2.2.2=0
  - matplotlib=2.2.3
  - ...


We can create this yaml file manually or can use conda command:

$ conda env export > my_environment.yml

To re-create the environment from a given yaml file:

$ conda env create -f my_environment.yml 
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 4.8.5

Please update conda by running

    $ conda update -n base -c defaults conda

Downloading and Extracting Packages
gmp-6.1.2            | 751 KB    | ##################################### | 100% 
prometheus_client-0. | 44 KB     | ##################################### | 100% 
libstdcxx-ng-9.3.0   | 4.0 MB    | ##################################### | 100% 
defusedxml-0.6.0     | 22 KB     | ##################################### | 100% 
ipython-7.16.1       | 1.1 MB    | ##################################### | 100% 
kiwisolver-1.2.0     | 87 KB     | ##################################### | 100% 
libgcc-ng-9.3.0      | 7.8 MB    | ##################################### | 100% 
...
Preparing transaction: done
Verifying transaction: done
Executing transaction: \ b'Enabling notebook extension jupyter-js-widgets/extension...\n      - Validating: \x1b[32mOK\x1b[0m\n'
done
#
# To activate this environment, use
#
#     $ conda activate my_environment
#
# To deactivate an active environment, use
#
#     $ conda deactivate


We can verify that this environment has now been added to conda:

(base) $ conda env list
# conda environments:
#
base                  *  /home/bojan/anaconda3
my_environment           /home/bojan/anaconda3/envs/my_environment
test-opencv              /home/bojan/anaconda3/envs/test-opencv


To activate this environment:

(base)$ conda activate my_environment
(my_environment)$


To check which packages are installed in the current environment:

(my_environment) $ conda list
# packages in environment at /home/bojan/anaconda3/envs/my_environment:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
absl-py                   0.10.0           py36h9f0ad1d_0    conda-forge
argon2-cffi               20.1.0           py36h8c4c3a4_1    conda-forge
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
async_generator           1.10                       py_0    conda-forge
atk                       2.25.90           hf2eb9ee_1001    conda-forge
attrs                     20.2.0             pyh9f0ad1d_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
...

To install additional package to the current environment:

(my_environment) $ conda install pillow
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/bojan/anaconda3/envs/my_environment

  added / updated specs:
    - pillow


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2020.10.14 |                0         121 KB
    certifi-2020.6.20          |           py36_0         156 KB
    olefile-0.46               |           py36_0          48 KB
    openssl-1.0.2u             |       h7b6447c_0         2.2 MB
    pillow-7.2.0               |   py36hb39fc2d_0         619 KB
    ------------------------------------------------------------
                                           Total:         3.1 MB

The following NEW packages will be INSTALLED:

  lcms2              pkgs/main/linux-64::lcms2-2.11-h396b838_0
  olefile            pkgs/main/linux-64::olefile-0.46-py36_0
  pillow             pkgs/main/linux-64::pillow-7.2.0-py36hb39fc2d_0

The following packages will be UPDATED:

  ca-certificates    conda-forge::ca-certificates-2020.6.2~ --> pkgs/main::ca-certificates-2020.10.14-0

The following packages will be SUPERSEDED by a higher-priority channel:

  certifi            conda-forge::certifi-2020.6.20-py36h9~ --> pkgs/main::certifi-2020.6.20-py36_0
  openssl            conda-forge::openssl-1.0.2u-h516909a_0 --> pkgs/main::openssl-1.0.2u-h7b6447c_0


Proceed ([y]/n)? y


Downloading and Extracting Packages
openssl-1.0.2u       | 2.2 MB    | ################################################################ | 100% 
certifi-2020.6.20    | 156 KB    | ################################################################ | 100% 
pillow-7.2.0         | 619 KB    | ################################################################ | 100% 
ca-certificates-2020 | 121 KB    | ################################################################ | 100% 
olefile-0.46         | 48 KB     | ################################################################ | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

We can verify that a new package is indeed installed in new environment:

(my_environment) $ conda list
# packages in environment at /home/bojan/anaconda3/envs/my_environment:
#
# Name                    Version                   Build  Channel
...
pillow                    7.2.0            py36hb39fc2d_0 
...

To install a new package from a specific channel in the current environment:

(my_environment) $ conda install -c channel_name package_name


To install a specific version of a new package from a specific channel in the current environment:

(my_environment) $ conda install -c channel_name package_name=version


To uninstall some package from the current environment:

(my_environment) $ conda remove pillow

To uninstall multiple packages from the current environment:

(my_environment) $ conda remove package1 package2 ...

To update an environment with (new) dependencies in (new) yml file:

(my_environment$ conda deactivate
$ conda env update --file my_environment.yml
$ conda activate my_environment


To update conda (I've deactivated the current environment):

$ conda update -n base -c defaults conda
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/bojan/anaconda3

  added / updated specs:
    - conda


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    conda-4.8.5                |           py38_0         2.9 MB
    ------------------------------------------------------------
                                           Total:         2.9 MB

The following packages will be UPDATED:

  conda                                        4.8.3-py38_0 --> 4.8.5-py38_0


Proceed ([y]/n)? y


Downloading and Extracting Packages
conda-4.8.5          | 2.9 MB    | ################################################################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done


To remove given environment:

$ conda env remove --name test-opencv

Remove all packages in environment /home/bojan/anaconda3/envs/test-opencv:


We can verify that its directory does not exist:

$ ls /home/bojan/anaconda3/envs/test-opencv
ls: cannot access '/home/bojan/anaconda3/envs/test-opencv': No such file or directory

References:


1 comment:

  1. BE SMART AND BECOME RICH IN LESS THAN 3DAYS....It all depends on how fast 
    you can be to get the new PROGRAMMED blank ATM card that is capable of
    hacking into any ATM machine,anywhere in the world. I got to know about 
    this BLANK ATM CARD when I was searching for job online about a month 
    ago..It has really changed my life for good and now I can say I'm rich and 
    I can never be poor again. The least money I get in a day with it is about 
    $50,000.(fifty thousand USD) Every now and then I keeping pumping money 
    into my account. Though is illegal,there is no risk of being caught 
    ,because it has been programmed in such a way that it is not traceable,it 
    also has a technique that makes it impossible for the CCTVs to detect 
    you..For details on how to get yours today, email the hackers on : (
    atmmachinehackers1@gmail.com ). Tell your 
    loved once too, and start to live large. That's the simple testimony of how 
    my life changed for good...Love you all ...the email address again is ;
    atmmachinehackers1@gmail.com

    ReplyDelete