External Access to MASS FAQ

Can I use my existing MASS account?

No. You need a separate MASS account for use on the Met Office internal network (CDN), Monsoon, ECMWF HPCs, and JASMIN. With these different account types, you can have permission to access different datasets specific to these computing environments.

When running "moo install", what does the "is this process running in the correct environment?" error message mean?

Occasionally, new users will report that, when running 'moo install' to set up their credentials-files locally, they get an error message like this:

Cannot read file: /home/user/<userid>/.moosedir/moose        
- is this process running in the correct environment?

This is almost certainly a result of the wrong combination of Unix user-id and UID having been used to encrypt the 'moose' credentials-file.

Credentials files for external users are encrypted using a key that involves the user's local (i.e. JASMIN or MONSooN) Unix user-id and UID before being sent via e-mail. The external (and MONSooN) version of the client decrypts the contents of the file every time the client is invoked, using the current Unix user-id and UID of the invoking user. If these don't match those used to encrypt the file in the first place, the above error is generated. This measure prevents the interception and misuse of a credentials-file by an unauthorised user.

If you encounter this error message, please send details to mailto:monsoon@metoffice.gov.uk so that a new credentials file can be generated for you.

When running "moo install", what does the "!: event not found" error message mean?

On running, moo install you may see the following error message:

/opt/moose/external-client-version-wrapper/bin/moo: line 14: !": event not found
/opt/moose/external-client-version-wrapper/bin/moo: line 14: syntax error near unexpected token `fi' 
/opt/moose/external-client-version-wrapper/bin/moo: line 14: `fi'

This is the result of a bug in the wrapper script that occurs when you have certain settings enabled in your $HOME/.bash_profile or $HOME/.bashrc file. To work around this temporarily remove the statement export SHELLOPTS from your $HOME/.bash_profile. This will be fixed in a future release (16/6/2015).

Why am I asked for a password when logging in to mass-cli?

There are two conditions that may result in you being prompted for a password when attempting to login to the MASS client machine mass-cli.jasmin.ac.uk

  • The first is if you don't have permission to access the machine. This may be when you first attempt to use the service. A quick method to check is to verify if you are a member of the "moose" user group. It should be listed when you use the "groups" command.
[login1]$ groups
users open ... moose
  • The second case can present itself if you forget the "-A" option for ssh when you first login to the login node. You can test for this condition by listing loaded identities on the login node, and finding you have none. e.g.
[login1]$ ssh-add -l
Could not open a connection to your authentication agent.

What does this error message mean?

Please see MOOSEErrorMessages for details.

How do I respond to the password expire message?

Occasionally on running a MOOSE command you will be told that your password is due to expire with a message of the form:

Your password is due to expire in 6 day(s).   
A new password can be generated using 'moo passwd -r'.
  • This refers specifically to your external MASS account, it does not affect your Linux login or your MASS account on MONSooN or CDN.
  • You need to run this command as advised in order to update your credentials.
  • You don't actually need to provide a new password, this is generated and hidden from you by the command.
  • You need to run the command on the client virtual machine, so you need to be logged in to mass-cli to do this.
  • If you have a retrieval in progress, it is safe to run this command as it won't affect processes already running.
  • Further details are in the MOOSE User Guide.

What action should I take when I get a MOOSE account expiring email?

By default, external access to MASS accounts have a 500 day expiry period, after which the account will be automatically disabled.

Shortly before the account is due to expire you will receive an email with the subject line " MOOSE account expiring". In order to continue having access to MASS data from JASMIN via MOOSE, you must forward the email to the Scientific Collaboration Service Manager (mailto:monsoon@metoffice.gov.uk) and confirm whether or not you still want to have access. If you do not contact the Service Manager, your access will be automatically disabled on the date specified in the email. Please act promptly on receipt of the email in order to ensure your account is not automatically disabled.

On receipt of the email, the Service Manager will confirm with your Met Office sponsor that access is still required, before extending the use of your account.

As a MASS project owner, how do I ensure that all the Sets associated with that Project are accessible to all members of the Project?

When a user is given access to a Project they do not automatically have access to all the Sets associated with the project, this can be due to the use of access control lists (ACLs) at the Set level. If you want to ensure that every Set in the Project is readable by people with access to the Project, as the Project owner you need to ensure that all the Sets are readable.

  • A user can check they are a member of a Project by
moo projlist
  • A member of a project can list all other members(users) of a Project using
moo projinfo –-members <projectname>
  • Symptoms that a member of a Project doesn’t have access to a Set within that Project:
moo ls moose:<datasetname> 
ls command-id=168986953 failed: (SSC_TASK_REJECTION) one or more tasks are rejected.   
moose:<datasetname>: (TSSC_UNAUTHORISED) permission denied. ls: failed (2)
  • Check the URI is a Set
moo test --is-set moose:<datasetname> 
  • List all the associated Sets and ACLs for a project
moo projinfo -–long <projectname>
  • Command for project owner to add a data Set to a Project (note that this is not particularly clear from the moose manual)
moo chown <projectname> moose:<datasetname>
  • Command for project owner to give read access to the data Set for all members of the Project
moo chacl <projectname>:r moose:<datasetname>

If you are feeling particularly confident you could try the following to give read access to all Sets associated with a Project:

moo projinfo --long <projectname> | perl -ne 'if(m/moose/ && !m/read/){ system "moo chacl <projectname>:r $_" }'

How do I check and modify the ACL of a Set?

You can see the Set's ACL by using

moo getacl URI

(note this is limited to internal users).

To give a user access to a Set you can either:

(a) add them to the membership of the owning project (though they would be prevented from writing to the Set if it's an external account);

(b) add the project to the ACL of the Set, with read-access: e.g.

moo chacl project-glomodel:r moose:crum/anrio

OR (c) add the user explicitly to the ACL: e.g.

moo chacl badc.user.name URI

For (a) you need to be an Admin, or a Superuser or Project-admin for the owning project.

For (b) & (c) you need to be an Admin or the owner of the Set.

Further information on changing ACLs can be found in the internal version of the MOOSE User Guide.

As a MASS data Set owner, how do I enable (or disable) external access to my data Set?

As a data Set owner, you can disable external access (also known as "deny-listing") by marking the set "InternalOnly". These commands can only be run by data Set owners internal to the Met Office (i.e. on CDN), guidance on the criteria for marking data Sets for internal access only is available here.

  • Check the URI is a Set
moo test --is-set moose:<datasetname> 
  • Check the current Set protection level
moo setinfo moose:<datasetname> 
Information for : moose:<datasetname> 
Owner: <mooseusername> 
Size: 13 
Cost: 0.00GBP 
Protection Level: Managed Protection Level
  • Change the Set protection level to "InternalOnly" to prevent external access to the Set.
moo chprot InternalOnly moose:<datasetname> 
### chprot, command-id=213814124
  • If you want the Set to be available for external access (also subject to Project membership and ACLs ("allow-listing")), set the protection level to "Managed"
moo chprot Managed moose:<datasetname> 
### chprot, command-id=213814052

How do I make sure my directory has all the available data retrieved from MASS?

The problem: You're running a model over a period of several days or weeks, and you need to analyse the output of the model as it runs. You have a moo get or moo select command that you run to fetch the data that's available. You want to be able to re-run it to fetch the files or fields that have been added to MASS since you last ran the command, but you don't want it to waste time re-fetching things you already have.

The solution: Use the -i or --fill-gaps option when you run moo get or moo select. This option tells MOOSE that you only want to fetch files that don't already exist in the specified local directory. Note that MASS works out where gaps are by doing checks to see if files of the expected name exist in your destination directory, so it won't behave correctly if you rename files after you've retrieved them, or if you use the -C option with moo select which condenses all the matching fields into a single file.

You might also find the -g / --get-if-available option to moo get useful. This tells MOOSE to get every file from your moo get list that is available, but ignore ones that aren't there rather than exit with an error. This could help if you're expecting files to be archived at some point, but aren't sure whether they'll be there when your job runs. If you use this option MOOSE will get as much as it can from your list without bailing out.

How can I directly login to the MASS client machine?

Although you can't directly login to the MASS client machine, you can create an ssh configuration that automatically enables you to jump through the intermediary login servers. Please note that this only works if you are using OpenSSH version 5.4 or greater, earlier versions do not support the "-W" flag. You can check your version using "ssh -v". Add the following to your home institute $HOME/.ssh/config file:

Host mass-cli   
  User <your_jasmin_userid>      
  HostName mass-cli.jasmin.ac.uk
  ProxyCommand ssh -YA -t <your_jasmin_userid>@login1.jasmin.ac.uk -W %h:%p 2>/dev/null

You should then be able to login directly using "ssh mass-cli".

How can I script my data retrieval from MASS?

There are restrictions on how to login to JASMIN and use of Linux utilities such as “cron” and “at”, but it is possible to remotely initiate a retrieval from MASS on to JASMIN, provided you have your ssh agent running on a machine local to you, e.g.

eval $(ssh-agent -s) 
ssh-add ~/.ssh/jasmin_id_rsa 
ssh -A -X sci1.jasmin.ac.uk 'ssh mass-cli my_script.sh'

or, if you have set up your $HOME/.ssh/config to allow direct access (as detailed above), then the following should work:

ssh mass-cli my_script.sh

This will run the script “my_script.sh” on the MASS client VM. You can put the moose retrieval commands into a script and it should work, e.g.:

moo get $SRC_URI jasmin_copy.pp 

if you have access to an appropriate JASMIN workspace, then you can scp data from the workspace directly through one of the dedicated data transfer VMs, again, you need the ssh-agent running locally, e.g.

eval $(ssh-agent -s)
ssh-add ~/.ssh/jasmin_id_rsa 
scp <userid>@xfer1.jasmin.ac.uk:/group_workspaces/cems/<project>/jasmin_file.pp my_local_copy.pp