JASMIN Help Site logo JASMIN Help Site logo
  • Docs 
  • Guides 
  • Training 
  • Discussions   

  •   Search this site  

Can't find what you're looking for?

Try our Google custom search, across all JASMIN sites

Docs
  • getting started
    • get started with jasmin
    • generate ssh key pair
    • get jasmin portal account
    • get login account
    • beginners training workshop
    • how to contact us about jasmin issues
    • jasmin status
    • jasmin training accounts
    • tips for new users
    • how to login
    • multiple account types
    • present ssh key
    • reconfirm email address
    • reset jasmin account password
    • ssh auth
    • storage
    • understanding new jasmin storage
    • update a jasmin account
  • interactive computing
    • interactive computing overview
    • check network details
    • login servers
    • login problems
    • graphical linux desktop access using nx
    • sci servers
    • tenancy sci analysis vms
    • transfer servers
    • jasmin notebooks service
    • jasmin notebooks service with gpus
    • creating a virtual environment in the notebooks service
    • project specific servers
    • dask gateway
    • access from vscode
  • batch computing
    • lotus overview
    • slurm scheduler overview
    • slurm queues
    • lotus cluster specification
    • how to monitor slurm jobs
    • how to submit a job
    • how to submit an mpi parallel job
    • example job 2 calc md5s
    • orchid gpu cluster
    • slurm status
    • slurm quick reference
  • software on jasmin
    • software overview
    • quickstart software envs
    • python virtual environments
    • additional software
    • community software esmvaltool
    • community software checksit
    • compiling and linking
    • conda environments and python virtual environments
    • conda removal
    • creating and using miniforge environments
    • idl
    • jasmin sci software environment
    • jasmin software faqs
    • jaspy envs
    • matplotlib
    • nag library
    • name dispersion model
    • geocat replaces ncl
    • postgres databases on request
    • running python on jasmin
    • running r on jasmin
    • rocky9 migration 2024
    • share software envs
  • data transfer
    • data transfer overview
    • data transfer tools
    • globus transfers with jasmin
    • bbcp
    • ftp and lftp
    • globus command line interface
    • globus connect personal
    • gridftp ssh auth
    • rclone
    • rsync scp sftp
    • scheduling automating transfers
    • transfers from archer2
  • short term project storage
    • apply for access to a gws
    • elastic tape command line interface hints
    • faqs storage
    • gws etiquette
    • gws scanner ui
    • gws scanner
    • gws alert system
    • install xfc client
    • xfc
    • introduction to group workspaces
    • jdma
    • managing a gws
    • secondary copy using elastic tape
    • share gws data on jasmin
    • share gws data via http
    • using the jasmin object store
    • configuring cors for object storage
  • long term archive storage
    • ceda archive
  • mass
    • external access to mass faq
    • how to apply for mass access
    • moose the mass client user guide
    • setting up your jasmin account for access to mass
  • for cloud tenants
    • introduction to the jasmin cloud
    • jasmin cloud portal
    • cluster as a service
    • cluster as a service kubernetes
    • cluster as a service identity manager
    • cluster as a service slurm
    • cluster as a service pangeo
    • cluster as a service shared storage
    • adding and removing ssh keys from an external cloud vm
    • provisioning tenancy sci vm managed cloud
    • sysadmin guidance external cloud
    • best practice
  • workflow management
    • rose cylc on jasmin
    • using cron
  • uncategorized
    • mobaxterm
    • requesting resources
    • processing requests for resources
    • acknowledging jasmin
    • approving requests for access
    • working with many linux groups
    • jasmin conditions of use
  • getting started
    • get started with jasmin
    • generate ssh key pair
    • get jasmin portal account
    • get login account
    • beginners training workshop
    • how to contact us about jasmin issues
    • jasmin status
    • jasmin training accounts
    • tips for new users
    • how to login
    • multiple account types
    • present ssh key
    • reconfirm email address
    • reset jasmin account password
    • ssh auth
    • storage
    • understanding new jasmin storage
    • update a jasmin account
  • interactive computing
    • interactive computing overview
    • check network details
    • login servers
    • login problems
    • graphical linux desktop access using nx
    • sci servers
    • tenancy sci analysis vms
    • transfer servers
    • jasmin notebooks service
    • jasmin notebooks service with gpus
    • creating a virtual environment in the notebooks service
    • project specific servers
    • dask gateway
    • access from vscode
  • batch computing
    • lotus overview
    • slurm scheduler overview
    • slurm queues
    • lotus cluster specification
    • how to monitor slurm jobs
    • how to submit a job
    • how to submit an mpi parallel job
    • example job 2 calc md5s
    • orchid gpu cluster
    • slurm status
    • slurm quick reference
  • software on jasmin
    • software overview
    • quickstart software envs
    • python virtual environments
    • additional software
    • community software esmvaltool
    • community software checksit
    • compiling and linking
    • conda environments and python virtual environments
    • conda removal
    • creating and using miniforge environments
    • idl
    • jasmin sci software environment
    • jasmin software faqs
    • jaspy envs
    • matplotlib
    • nag library
    • name dispersion model
    • geocat replaces ncl
    • postgres databases on request
    • running python on jasmin
    • running r on jasmin
    • rocky9 migration 2024
    • share software envs
  • data transfer
    • data transfer overview
    • data transfer tools
    • globus transfers with jasmin
    • bbcp
    • ftp and lftp
    • globus command line interface
    • globus connect personal
    • gridftp ssh auth
    • rclone
    • rsync scp sftp
    • scheduling automating transfers
    • transfers from archer2
  • short term project storage
    • apply for access to a gws
    • elastic tape command line interface hints
    • faqs storage
    • gws etiquette
    • gws scanner ui
    • gws scanner
    • gws alert system
    • install xfc client
    • xfc
    • introduction to group workspaces
    • jdma
    • managing a gws
    • secondary copy using elastic tape
    • share gws data on jasmin
    • share gws data via http
    • using the jasmin object store
    • configuring cors for object storage
  • long term archive storage
    • ceda archive
  • mass
    • external access to mass faq
    • how to apply for mass access
    • moose the mass client user guide
    • setting up your jasmin account for access to mass
  • for cloud tenants
    • introduction to the jasmin cloud
    • jasmin cloud portal
    • cluster as a service
    • cluster as a service kubernetes
    • cluster as a service identity manager
    • cluster as a service slurm
    • cluster as a service pangeo
    • cluster as a service shared storage
    • adding and removing ssh keys from an external cloud vm
    • provisioning tenancy sci vm managed cloud
    • sysadmin guidance external cloud
    • best practice
  • workflow management
    • rose cylc on jasmin
    • using cron
  • uncategorized
    • mobaxterm
    • requesting resources
    • processing requests for resources
    • acknowledging jasmin
    • approving requests for access
    • working with many linux groups
    • jasmin conditions of use
  1.   Data transfer
  1. Home
  2. Docs
  3. Data transfer
  4. rsync, scp, sftp

rsync, scp, sftp

 

Winscp   Scp   Filezilla  
Winscp   Scp   Filezilla  
Share via
JASMIN Help Site
Link copied to clipboard

Data Transfer Tools: rsync, scp, sftp

On this page
rsync over SSH   rsync example with JASMIN   scp   sftp   Note on performance  

This article tells you about some of the basic transfer tools available for use with JASMIN that work over an SSH connection:

  • rsync (over SSH)
  • scp
  • sftp

rsync over SSH  

rsync is a file synchronisation and file transfer program for Unix-like systems that can minimise network data transfer by using a form of delta encoding such that only the differences between and source and destination data are actually transmitted. rsync can compress the data transferred further using zlib compression and SSH or stunnel can be used to encrypt the transfer.

rsync is typically used to synchronise files and directories between two different systems, one local and one remote. For example, the command:

rsync mydata remoteuser@remotehost:/data/

will use SSH to connect as remoteuser to remotehost. Once connected, it will invoke another copy of rsync on the remote host, and then the two programmes will talk to each other over the connection, working together to determine which parts of mydata are already on the remote host and don’t need to be transferred over the connection.

The generic syntax is:

rsync [OPTION] ... SRC [SRC] ... [USER@]HOST:DEST
rsync [OPTION] ... [USER@]HOST:SRC [DEST]

…where SRC is the file or directory (or a list of multiple files and directories) to copy from, and DEST represents the file or directory to copy to. (Square brackets indicate optional parameters.)

For more information visit the official rsync website  .

rsync example with JASMIN  

Here is a simple example using rsync over SSH to your home directory copy a file to a Group Workspace on JASMIN:

exec ssh-agent $SHELL
ssh-add ~/.ssh/id_ecdsa_jasmin  # local path to your private key file
rsync myfile <username>@xfer-vm-01.jasmin.ac.uk:/gws/nopw/j04/myproject/data/

NOTE: The first two lines are the standard method for setting up the SSH agent in order to allow connections without prompting for a passphrase each time.

scp  

scp is another basic command-line tool for secure copying between two machines. It is installed as part of most SSH implementations and comes as standard on the JASMIN transfer servers. scp is the secure analogue of the rcp command. Typically, the syntax of scp is like the syntax of cp (copy):

To copy a file to a host:

scp myfile remoteuser@remotehost:directory/target

To copy a file (or directory) from a host to the local system:

scp remoteuser@remotehost:directory/source target
scp -r remoteuser@remotehost:directory/source_folder target_folder

Note that if the remote host uses a port other than the default of 22, it can be specified in the command. For example, copying a file from host over a non- standard SSH port (2222):

scp -P 2222 remoteuser@remotehost:directory/source target

For more information on scp please visit the following website  .

sftp  

sftp is a similar command-line tool to scp, but the underlying SFTP protocol allows for a range of operations on remote files which make it more like a remote file system protocol. sftp includes extra capabilities such as resuming interrupted transfers, directory listings, and remote file removal.

For basic transfer of a file on JASMIN to the local machine:

sftp remoteuser@xfer-vm-01.jasmin.ac.uk:/group_workspaces/jasmin/myproject/data/notes.txt ./

For more information see the Wikipedia page on SFTP  .

There are various 3rd-party tools and clients, for example, WinSCP, FileZilla, MobaXterm and others, which can do transfers using the SCP and/or SFTP protocols. Please refer to the documentation for that particular software, but the basic principles are the same, i.e that you can set up a connection to a remote host by specifying the same parameters that you would with the command- line tool, but often through a graphical user interface instead.

Note on performance  

While convenient and familiar to many users, the tools described above do not make efficient use of available bandwidth for transferring large quantities of data via high-speed networks over long distances. Please consult Data Transfer Tools to learn more about which might be the most appropriate tool to use in different contexts.

Last updated on 2025-02-07 as part of:  updated refs to xfer server & minor format tweaks (9db2e3e73)
On this page:
rsync over SSH   rsync example with JASMIN   scp   sftp   Note on performance  
Follow us

Social media & development

   

Useful links

  • CEDA Archive 
  • CEDA Catalogue 
  • JASMIN 
  • JASMIN Accounts Portal 
  • JASMIN Projects Portal 
  • JASMIN Cloud Portal 
  • JASMIN Notebooks Service 
  • JASMIN Community Discussions 

Contact us

  • Helpdesk
UKRI/STFC logo
UKRI/NERC logo
NCAS logo
NCEO logo
Accessibility | Terms and Conditions | Privacy and Cookies
Copyright © 2025 Science and Technology Facilities Council.
Hinode theme for Hugo licensed under Creative Commons (CC BY-NC-SA 4.0).
JASMIN Help Site
Code copied to clipboard