How to compile and test jobs
This article explains how to compile and test code interactively on LOTUS. It covers:
- Interactive session on LOTUS
- Compile and test a code
- Debugging tools (GNU and Intel)
- Move from interactive mode to batch compute mode
Interactive session on LOTUS
Testing a job on LOTUS can be carried out in an interactive manner by launching a pseudo-shell terminal from JASMIN/CEMS sci3 servers:
$ bsub –n <no_CPUs_to_use> -q <queue-name> -m <hostgroup> –Is /bin/bash
where –n assigns the number CPU cores to your login session on an available host from a hostgroup. Note that no more than 16 CPU cores are launched in interactive mode.
Example 1 on how to request an interactive session for testing a serial job:
[fchami@jasmin-sci3 ~]$ bsub -Is /bin/bash Job <9113327> is submitted to default queue <short-serial>. <<Waiting for dispatch ...>> <<Starting on host304.jc.rl.ac.uk>> [fchami@host304 ~]$
The interactive session is launched on
host304 in the example above.
Example 2 on how to request an interactive session for testing an MPI parallel job
[fchami@jasmin-sci3 ~]$ bsub -n 4 -q par-multi -m ivybridge128G -Is /bin/bash Job <9113350> is submitted to queue <par-multi>. <<Waiting for dispatch ...>> <<Starting on host239.jc.rl.ac.uk>>
This will launch a pseudo-terminal on
host239and across four CPU cores
-n 4 . This can be verified from running
bjobs command on this host which outputs the
EXEC_HOST which are showing here in order to run the job interactively.
[fchami@host239 ~]$ bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 9113350 fchami RUN par-multi lotus.jc.rl 4*host239.j /bin/bash Jan 23 22:01
Note to specify similar host group to compiled executable of your MPI code when submitted to LSF batch compute.
Which compiler is available?
A code written in a programming language like Fortran, C or C++ needs to be compiled to an executable binary using an appropriate compiler. GNU is the default compiler on (gfortran, gcc). PGI and Intel compilers are available by loading the appropriate compiler environment module. An example is shown below for intel compiler:
$ module load intel/15.0
To use Intel C compiler
$ which icc /apps/intel/2015.0.090/composer_xe_2015.0.090/bin/intel64/icc
To use Intel Fortran compiler
$ which ifort /apps/intel/2015.0.090/composer_xe_2015.0.090/bin/intel64/ifort
An example of a Fortran code written in two versions: a serial single core code axpySerial.f90 and a parallel MPI code axpyMPI.f90 are provided. The Fortran code calculate a simple array arithmetic of a*x+y were a is scalar, x, y are vectors of n elements. Example of Fortran code axpyMPI.f90 axpySerial.f90 are in the following article.
Test a serial code
Compile a serial Fortran code using the Intel Fortran compiler command ifort. The output file is a binary with the ".ex" extension as a convention to denote that this file is an executable:
[fchami@host304 ~]$ ifort axpySerial.f90 -o axpySerial.ex
This binary file is executed directly at the command line as shown below:
[fchami@host304 ~]$ ./axpySerial.ex
Test a parallel code
Compile an MPI Fortran code using the Intel Fortran compiler mpif90. The output file is a binary with the ".ex" extension as a convention to denote that this file is an executable. The number of CPU cores and the executable are passed to the MPI LOTUS wrapper script as show below:
Compile your MPI Fortran code:
$ mpif90 axpyMPI.f90 -o axpyMPI.ex
Execute your MPI Fortran code:
$ mpirun.lotus -n 16 ./axpyMPI.ex<br>
The Fortran code examples are given here.
Testing MPI job on a higher CPU cores
This is a special case for testing an MPI job that is optimised for a greater number of CPU core > 16 . Requesting an interactive session across multiple hosts of the same CPU model might take time.
[fchami@lotus ~]$ bsub -n 22 -q par-multi -m ivybridge128G -Is /bin/bash Job <9113401> is submitted to queue <par-multi>. <<Waiting for dispatch ...>> <<Starting on host096.jc.rl.ac.uk>> [fchami@host096 ~]$ bjobs JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME 9113401 fchami RUN par-multi lotus.jc.rl 7*host096.j /bin/bash Jan 23 22:26 7*host265.jc.rl.ac.uk 7*host111.jc.rl.ac.uk 1*host064.jc.rl.ac.uk