JASMIN software changes: migration to CentOS7 (2020)

This article explains the changes in the provision of common software on JASMIN ("sci" servers and LOTUS) when upgrading to main operating system from RedHat Enterprise Linux 6 (RHEL6) to CentOS7. It answers for the following questions:

• What is changing?
• Which systems are affected?
• Why is it changing?
• Comparison of packages in old JASMIN Analysis Platform and new Jaspy/"jasmin-sci" environments
• How do I get started (quickly)?

What is changing?

The upgrade from RHEL6 to introduces a new way of working with software environments on JASMIN. 

The previous system involved installations via RPM using a collection of packages known as the "JASMIN Analysis Platform" (JAP).

The current system uses two approaches to providing software environments and packages:

1. Jaspy environments – a collection of software environments that you can "activate" for use.
2. "jasmin-sci" environment – a single set of software packages installed separately because they were difficult to include in Jaspy.

Which systems are affected?

This change applies to all generally available JASMIN servers and the LOTUS cluster. These include:

• jasmin-sci*
• cems-sci*
• jasmin-cylc
• cron
• All LOTUS nodes (accessed via LSF)

Why is it changing?

The reason for moving away from the JAP was to provide a system that could support multiple versions of software packages, and environments, on a single platform. From the perspective of reproducible science, the Jaspy approach is more useful because it:

• keeps previous environments on the system when a new version of an environment is launched
• includes a listing of all packages (and their versions) that are provided in each date-stamped software environment

From a management point-of-view, Jaspy builds on the packaging tool "conda" and the community repositories known as "conda-forge". These tools are widely used in the scientific community and provide many components that are re-used in Jaspy.

Comparison of packages in old JASMIN Analysis Platform and new Jaspy/"jasmin-sci" environments

Most packages that were previously provided as part of the JASMIN Analysis Platform (JAP) are now included in Jaspy environments or the "jasmin-sci" environment. Some packages have however been dropped. The treatment of certain packages is documented on the "extra-sci-packages" GitHub repository. A full table of packages that were in JAP and their mappings to Jaspy/"jasmin-sci" is provided below.

How do I get started (quickly)?

If you want to access most of the packages you can find them in the most recent Jaspy environment. This can be activated using:

 module load jaspy

If you need packages that are provided in the "jasmin-sci" environment then you can activate it using:

 module load jasmin-sci

Full table of packages from JAP

Here is a full table of packages that were provided via JAP, and how they are provided for use on CentOS7 machines (as of initial release).

In the summary column:

• C - package is provided via Conda environment (i.e. Jaspy)
• R - package is provided via an RPM (as part of the base OS or via jasmin-sci)
• CR - package is provided both via Jaspy and via an RPM. The version in Jaspy is recommended for use with user code; the RPM version is also installed in order to satisfy an RPM dependency only, and may be subject to change.
• N - not provided (mostly these have been deprecated by the third-party sources)