Conda environments and Python virtual environments
Conda environments and Python virtual environments
This article describes two types of software environments that you can create in order to install packages for your own use on JASMIN. Typical examples why you may wish to do this is if you have asked us to add packages to Jaspy but wish to make use of them before the next release, or if they are not likely to be relevant to other users.
Separate pages explain the details of how to create and install Python virtual environments and Conda environments. This page gives an overview of what they are, and how to choose which one is most suitable for your needs.
To take an example of the size, a new Python virtual environment without
additional packages occupies about 10MB and contains under 1000 files (maybe
approximately twice this if using the --system-site-packages
option
explained in more detail elsewhere), whereas as a new conda base environment
occupies about 400MB and contains over 20,000 files.
To give an example of installing a Python package, the numexpr
library is a
numerical expression evaluator for NumPy. It is available as a pip package
called numexpr
, and also as a conda package called numexpr
. (For some
packages, the two may have slightly different names.) It can be installed
successfully into either a Python virtual environment using pip install numexpr
or a conda environment using conda install numexpr
. In either case,
the numpy
package on which it depends, amongst other things, will be
installed automatically if required.
To give an example of installing a non-python package, zsh
is a Unix shell
which combines various features of bash and csh. You cannot install this using
pip install
because it is not a python package, but it is available on
conda-forge, and can be installed into a conda environment using conda install zsh
.
We already provide a wide range of packages via Jaspy. This is in itself a conda environment, and it is important to note that although you can use a Python virtual environment to install additional packages when using a conda environment, you cannot have more than one conda environment activated at the same time.
Note that because you can install pip
packages into conda environment, it is
not generally useful to create a virtual environment in order to extend your
own private conda environment. The reason for creating one to extend Jaspy is
that users do not have write permission to add packages to Jaspy itself.
Because python virtual environments are much more lightweight that conda environments, and also give you access to packages that we provide via Jaspy, we would generally recommend that you start by trying a Python virtual environment. However, this might prove not to be possible, for example because:
If you do decide to install a conda environment, then remember to install additional packages that you might otherwise have used via Jaspy.