Storage on the JASMIN Cloud

There are a few different types of storage on the JASMIN Cloud, each with their own advantages. These include:

• VM root partitions
• block storage, aka Cinder
• fileshare, aka Manilla
• object storage, aka Swift

Root Partitions  

When a VM is created, it comes with a root partition (usually mounted at /). The size of this partition is defined by the flavor which was used for the VM. See the STFC Cloud documentation  for up to date flavors.

This root partition usually uses the SSD on the hypervisor, meaning that access is fast, though the size is limited. It is also ephemeral, meaning that it is deleted when the VM is deleted. For these reasons, we suggest that root partitions are not used to store data.

Block Storage (Cinder)  

Block storage using Cinder is the traditional way of providing storage with OpenStack. Volumes can be created through the advanced tab in the Azimuth portal. Kubernetes clusters can also use Cinder using the Cinder CSI Driver, which is installed on Azimuth Kubernetes clusters by default.

Due to the limited amount of available block storage, we do not recommend that it is used for large volumes of data - fileshare should be used instead.

Fileshare (Manila)  

Fileshare (Manila) storage on the STFC cloud is the recommended way to store large data sets.

Currently Azimuth will provision a single fileshare for a tenancy, which we will resize to fill your fileshare quota. We are working on intergrating the management of fileshares with Azimuth more completely so that the size and quota can be viewed on the portal.

Fileshares are exposed as a Ceph Filesystem which can be mounted on multiple machines. In order to do this you need the Monitor IP address which can be found using the Openstack CLI, or given to you when your project was set up.

When tenancies are initially created, we will resize the created share to match the quota for the tenancy.

To avoid having to mount the fileshare yourself, you can use an Azimuth Workstation, or Slurm cluster where the share will be mounted under /project/ on each platform. Note that this share can be mounted in multiple places at once, allowing cross-platform data sharing.

We are currently working on adding fileshares as a CSI backend for Kubernetes persistent volumes (PVs) and allowing resizing, and surfacing information on the share through Azimuth.

Note that the only way to see the usage of a share is to check with df (or similar) on a machine where the share is mounted. This is an Openstack limitation and not something we have control over.

STFC Cloud has detailed documentation about fileshares  .

Mounting Instructions  

The kernel client requires the ceph-common package to be installed. To access the filesystem on a share, we can use the mount command of the form:

# Create a directory for your mount point
mkdir /mnt/share  # or any path you prefer

# Mount command (see below for breakdown)
mount -t ceph {mon1 ip addr}:6789,{mon2 ip addr}:6789,{mon3 ip addr}:6789/{manilla-volume} -o name={access-rule-name} secret={access-key} /mnt/path

• 130.246.208.244:6789,130.246.209.22:6789,130.246.209.142:6789/ is the Monitor IP address from the Openstack CLI.
• {manilla-volume} Manila volume UUID generated by Openstack. Obtainable through the Openstack CLI, and provided upon project creation.
• name={access-rule-name}
• secret={access-key}
• /mnt/path directory created above.

Once done, you can access the fileshare at the mount point.

Object store  

Openstack provides Swift in stfc cloud (see here  ), but we recommend that most users use the JASMIN object store.