CEDA Archive

This article describes accessing the CEDA Archive, providing direct access to thousands of atmospheric, climate change and earth observation datasets. This is directly accessible as a file system from the shared science machines on JASMIN.via the JASMIN system. The local mounting of the CEDA Archives on JASMIN provides direct access to thousands of datasets available to users logged in to JASMIN on the local file system, though not all, as explained below.

Accessing the CEDA archive on JASMIN servers

The first step is to  get a CEDA user account, if required, and then link it to your JASMIN account via your JASMIN Account page. Once done, you will have access to large parts of the archive straightaway, but not all  - depnding on the UNIX access groups they are within (see below).

Access to many (but not all!) of the remaining datasets can be obtained by applying for specific access via the data centre (see  this article for more details). Use the CEDA Data Catalogue to find and apply for access to required datasets

If direct access is not possible the data can be obtained via standard FTP and web based access methods to the CEDA archive and transfered to a suitable group workspace on JASMIN. As the data centres use the same JASMIN infrastructure the transfer rates are high. See the CEDA data catalogue entries for dataset details.

The contents of the CEDA Archive are available on the file system under  /badc and /neodc. Note: do not access data via any symlinks that point to /datacentre/archvol* - these are not permanent links and may change when data are migrated to new storage. Please use the archive path names under /badc and /neodc.

Archive access groups

The UNIX access groups used within the CEDA archive are listed below with links to example datasets in the CEDA data catalogue for those wishing to use them:

  • open - Available to any logged in JASMIN user witha linked CEDA user account. See a full list of available datasets here.
  • cmip5_research- restricted CMIP3 and CMIP5 datasets
  • esacat1- Satellite data including MERIS, MIPAS and SCIAMACHY.
  • ecmwf- Access to the ECMWF Operational Datasets.
  • eurosat- Satellite data including IASI, AVHRR-3 and GOME-2.
  • ukmo_wx - Met Office observational dataset collections including LIDARNET, MIDAS, MetDB and NIMROD
  • ukmo_clim- Climatology datasets from the Met Office, including Central England Temperature dataset collection, HadISST.
  • byacl- These data have specific restrictions on them meaning that they can't be accessed directly from JASMIN, but can be obtained via FTP and web access.

Data Licencing

All use of data accessed directly from the CEDA Archive must be used in line with the relevant data licence in place for the relevant dataset for the purposes stated in the access application. Data licence information can be found on the relevant CEDA Data Catalogue page, a link to which can be found in the 00README_catalogue_and_licence.txt files found in the archive. For specific data licences granted for restricted datasets users should log into their MyCEDA page to view their granted licence and the associated usage purpose under which access was granted. Any required alternative use of the data beyond the original purpose stated in the original licence application can only be made with a freshly granted new licence application.

Accessing data in the archive

In the example below, the logged-in user is listing the contents of the CRU data sets within the BADC archive. These are "open" so all logged-in users can access them:

$ ls -l /badc/cru/data
total 320
-rw-r-----  1 badc open  396 Feb 18  2015 00README
drwxr-x---  8 badc open 4096 Mar 22 10:32 cru_cy
drwxr-x---  4 badc open 4096 Dec  6  2014 crutem
drwxr-x--- 12 badc open 4096 May  9 14:11 cru_ts
drwxr-x---  3 badc open 4096 Feb 18  2015 PDSI