Jaspy Software Environments (Python 3, R and other tools)
This page provides details of the "Jaspy" software environments that provide access to Python 3, R and a range of other tools on JASMIN. The article includes:
- Working with Jaspy environments
- Discovering which environments are available
- Jaspy Python (and other tools) environments
- Jaspy R environments (Jasr)
- How Jaspy works
- Changes to software environments
Overview
Working with Jaspy environments
Quickstart for Python 3 environment
module load jaspy
Activating the environment in scripts
If you want a particular script to activate a Jaspy environment then add the "module" command to it, e.g.:
#!/bin/bash module load jaspy python do-something.py
Setting your profile to always use a Jaspy environment
If you want all your JASMIN sessions to use a particular Jaspy environment then you can add the "module load jaspy" command to your ".bashrc" file at: $HOME/.bashrc . In order to avoid issues with using "module load" on unsupported servers, please wrap the call in an "if" clause, such as:
if [[ $(hostname) =~ (sci[0-9]|host[0-9]|cylc) ]] ; then module load jaspy fi
Discover which environments are available
$ module avail jaspy ----------- /apps/modulefiles ------------- jaspy/2.7/r20190715 jaspy/3.7/r20190627 jaspy/3.8/r20211105 jaspy/3.7/r20181219 jaspy/3.7/r20200606 jaspy/3.10/r20220721 (D) jaspy/3.7/r20190612 jaspy/3.7/r20210320 jaspy/3.10/r20230718 (L) Where: L: Module is loaded D: Default Module
This lists all jaspy modules (i.e. environments) that can be loaded.
Jaspy Python 3.7+ (plus other tools) environments
Jaspy Python 3.7 Environment | Versioned list of software packages | Default? | Comments / Issues | |
jaspy/3.10/r20230718 | List of packages including versions |
Not yet (to be scheduled soon) | ||
jaspy/3.10/r20220721 | List of packages including versions | Yes (from: 18/10/2022) | NCO and NCL have now been moved to the "jasmin-sci" packages installation. | |
jaspy/3.8/r20211105 | List of packages including versions | No (was default: 16/11/2021 - 17/10/2022) | Known problem with NCL rendering Shapefiles (see issue). Some packages were removed in this release due to dependency problems: theano, pymc3, pystan, pyngl, pyferret (see issue). | |
jaspy/3.7/r20210320 | List of packages including versions |
No (was default: 20/05/2021 - 16/11/2021) |
Known problem with NCL rendering Shapefiles (see issue) | |
jaspy/3.7/r20200606 | List of packages including versions | No | ||
jaspy/3.7/r20181219 |
List of packages including versions | No |
Jaspy Python 2.7 (plus other tools) environments
This table lists all the Jaspy Python 2.7 environments provided on JASMIN and specifies the current (default) version.
Jaspy Python 2.7 Environment | Versioned list of software packages | Default? | |
jaspy/2.7/r20190715 | List of packages including versions |
Yes | |
Jaspy R environments (Jasr)
Environments for the "R" programming language are packaged into separate software environments, known as "Jasr". This table lists all the Jaspy R environments provided on JASMIN and specifies the current (default) version.
Jaspy R Environment ("Jasr") | Versioned list of software packages | Default? | |
jasr/4.2/r20230718/ | List of packages including versions |
Not yet (to be scheduled soon) | |
jasr/4.0/r20220729 | List of packages including versions |
Yes (from: 18/10/2022) |
|
jasr/4.0/r20211110 | List of packages including versions | No (was default: 16/11/2021 - 17/10/2022) |
|
The available R environments can be listed with:
module avail jasr
Understanding versioning with Jaspy/Jasr
module load jaspy/3.7/r20210320
module load jaspy/3.7
module load jaspy
How Jaspy works: managing Python and non-Python packages using conda
Jaspy is a framework for managing multiple Python (and other) environments simultaneously on a single platform. It was created in order to meet the requirements tabulated below.
Requirement | Details | Jaspy solution | Further info | ||
Reproducibility |
|
|
Conda: https://docs.conda.io jaspy-manager: https://github.com/cedadev/jaspy-manager/blob/master/README.md CEDA jaspy environments: https://github.com/cedadev/ceda-jaspy-envs |
||
Documentation | Provide an appropriate level of documentation detailing which software packages exist in each release. | We use Conda "environment files" to build the environments. These list the packages and versions and are stored in public GitHub repositories, so each environment is documented as a collection of packages/versions. | See: https://github.com/cedadev/jaspy-manager/blob/master/README.md Example package list: https://github.com/cedadev/ceda-jaspy-envs/blob/master/environments/py3.7/m3-4.5.11/jaspy3.7-m3-4.5.11-r20181219/packages.txt |
||
Multiple simultaneous environments | Allow multiple, but separate, software environments to co-exist on a single operating system. | Conda is designed to allow multiple environments to co-exist. Within jaspy it is possible to document each environment. Therefore, multiple environments can be deployed on one system. Key advantages are:
|
|||
Manageability | Provide tools to easily construct, test, deploy, document and reproduce software environments. | Jaspy builds upon a set of excellent Conda command-line tools that simplify the package management process. Jaspy wraps the Conda functionality so that command-line tools can be used to build, test, deploy and distribute Conda environments for use by our community. |
Updates and tracking of Jaspy/Jasr environments
History of environments on JASMIN
Which environment is "current"?
Please refer to the "Jaspy Python 3.7+ (and other tools) environments" section above for information about the current release on JASMIN.
Citing Jaspy environments
Can I cite a jaspy (conda) environment?
Requesting updates to a Jaspy environment
- E-mail the JASMIN Helpdesk with the subject: "Request for Jaspy update: <package name>"
- Get a GitHub account and add an issue to the `ceda-jaspy-envs` repository at:
Conda method of "activating" Jaspy environments
$ export PATH=/apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.6.14//bin/conda:$PATH $ source activate $ conda activate jaspy3.7-m3-4.6.14-r20210320
Using Jaspy beyond JASMIN
Jaspy is a versatile and generic tool for managing multiple conda environments. The code is open source, and more information is available at: