Joint-storage Data Migration App (JDMA)
See the JDMA user documentation at: cedadev.github.io/jdma_client for more information about using JDMA.
The joint-storage data migration app (JDMA) is a multi-tiered storage system which provides a single API to users to allow the movement of data to a number of different storage systems, query the data they have stored on those storage systems and retrieve the data.
- The user experience for moving data, regardless of the underlying storage systems, should be identical.
- The user should not be responsible for maintaining the connection to the storage system in the case of asynchronous transfers.
- User and group ownership and permissions should be preserved and restored on downloading the data
- The user should receive notifications when the transfers are complete.
- Users should be able to transfer data from one storage system to another
- JDMA is only a request and query layer. Any cataloguing of data should be carried out by the backend system. So that, if JDMA fails, then the data is still available, independently of JDMA, from the storage backend.
See the JDMA user documentation at: cedadev.github.io/jdma_client/ for more information about using JDMA.
Quick guide to installing the JDMA client on JASMIN
If you are working on JASMIN and you wish to use the JDMA client, then you can install it as follows on a `sci` server:
module load jaspy python -m venv ~/venvs/jdma-venv source ~/venvs/jdma-venv/bin/activate pip install git+https://github.com/cedadev/jdma_client
You should then have the jdma command-line tool available in your terminal session.