Alimr.dev

What is a Virtual Environment?

"A virtual environment is a Python environment such that the Python interpreter, libraries and scripts installed into it are isolated from those installed in other virtual environments, and (by default) any libraries installed in a “system” Python, i.e., one which is installed as part of your operating system."(source)

Why Using a Virtual Environment Anyways?

Common scenario: you use this Python library called CommonLibrary very frequenty in your MS/PhD research. You also took this machine learning course this semester and most of the assignments require using this same library. You realize that you need to upgrade the library from version 2.1 to version 2.2 to be able to use this cool feature that was released lately. After the upgrade, you realize some of your research codes no longer work as expected, due to the upgrade. Downgrading the library means your course assignments can't benefit from the newly released feature so what do you do? You use a virtual environment to isolate the two work!

Installing Anaconda or Miniconda

You can find the build that fits your OS for installing Anaconda or Miniconda from Anaconda repositories. The script below shows an example for MacOS x86_64 installation of Miniconda using wget

wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.9.2-MacOSX-x86_64.sh
bash Miniconda3-py37_4.9.2-MacOSX-x86_64.sh

or you can use the same links to download the packages without using the command line.

The base environment

Successful installation of Anaconda or Miniconda activates a base environment on your machine with (base) showing up in your command line

(base)

Conda environment

Conda or other virtual environment management tools are useful in managing python packages and making sure that all the package dependencies are satisfied. Creating a new environment for a specific project isolates it from the other existing environments on your system. Installing packages on the base environment is not a good practice as it defeats the purpose of environment isolation.

Conda automatically handles the necessary package updates and makes sure that the updates will not affect the behavior of other packages that somehow make use of the updated package. Check this link to learn about the other benefits.

conda create -n new-env

Creating a new environemnt

You can specify and install the packages that you aim to use in the new environment by adding them to the end of the conda create command

conda create -n new-env numpy scipy

You can also specify the package(s) version while creating the environment, either the packages that are going to be created by default (e.g. python) or the ones that you are adding to the environment (e.g. numpy)

conda create -n new-env python=3.7 numpy=1.19.5

Finally, you can also specify the package build as

conda create -n new-env python=3.7 numpy=1.19.5=py37haa41c4c_1

If the package is not located in the default conda channels (anaconda free and anaconda main you should add the channel to your conda distribution. For example, you can add the conda-forge channel using

conda config --add channels conda-forge
conda config --set channel_priority strict

Note: The best practice is to install all packages at once to make sure that all of the dependencies are installed at the same time.

Useful flags

• n or --name to specify the name of the environment
• y or --yes proceeds the creation of environment or installation of the packages without asking for permission

conda create -n new-env scipy -y

Activation

Once the environment is created, you can activate the environment and start using it.

conda activate new-env

Similar to the activation of the base environment, activation of a new environment new-env leads to (new-env) appearing in the command line as

(new-env)

Dectivation

Whenever you are done with your coding, you can deactivate the environment.

conda deactivate

The same command can be used to deactivate the base environment.

Installing packages inside an environment

To install a package named python-package while you are in a conda environment you can simply

conda install python-package

Similar to before, you can specify the package build and version.

Installing Jupyter Notebook in a conda environment

You first need to install the jupyter package

conda install jupyter -y
jupyter nbextension enable --py widgetsnbextension

Next you install a new ipython kernel using python installer. You need to specify the conda environment this kernel points to and optionally, you can assign a name to this jupyter notebook kernel. Here, we use the names new-env and "new env kernel" to denote the conda environment name and the kernel name, respectively.

python -m ipykernel install --user --name new-env --display-name "new env kernel"

Note: You can check the name of all of the jupyter notebook kernels installed on your machine using

jupyter kernelspec list

Removing a package

To remove a package named python-package

conda remove python-package

Removing an environnment

To emove an wnvironment named new-env

conda remove -n new-env --all