Introduction

In this example we will compile an OpenMP executable in an interactive shell and then submit a job to the batch queue.

Exercise

Create an interactive shell with 10 cores allocated on the same node by executing the following command :-

You should see something similar to :-

[train1@login8(sciama) ~]$ sinteractive -p training.q --ntasks=1 --cpus-per-task=6
Waiting for JOBID 1691162 to start
......

[train1@node247(sciama) ~]$ 

Now select the modules you will need for this exercise:-

From the command line check which modules this has loaded ( module list ).

Now change into the “$HOME/training/src” directory and look at the file “openmp.c” . Try to understand the structure of the code.

We will now compile this file and create an executable to be stored in the “bin” directory:-

Check that you now have an openmp.exe in the bin directory.

We will now run try to the program using the 6 cores we have allocated:-

The output should be similar to :-

[train1@node106(sciama) bin]$ ./openmp.exe
Hello World from thread = 0
Hello World from thread = 2
Number of threads = 16
Hello World from thread = 5
Hello World from thread = 13
Hello World from thread = 1
Hello World from thread = 4
Hello World from thread = 6
Hello World from thread = 8
Hello World from thread = 9
Hello World from thread = 12
Hello World from thread = 10
Hello World from thread = 11
Hello World from thread = 14
Hello World from thread = 3

All threads finished. Back to single thread.

Notice that the program actually used more than 6 cores. By default, OpenMP is trying to use all cores available on the same node. Thus we need to be able to limit the number of cores used. Set the following environmental variable and rerun the program:-

The output should now be something like :-

[train1@node106(sciama) bin]$ export OMP_NUM_THREADS=4
[train1@node106(sciama) bin]$ ./openmp.exe
Hello World from thread = 0
Number of threads = 6
Hello World from thread = 5
Hello World from thread = 3
Hello World from thread = 2
Hello World from thread = 1
Hello World from thread = 4

All threads finished. Back to single thread.

Now exit from the interactive shell back onto the login node. Check that you are in your home directory ( cd ).

We will now submit the same program as a batch job. Using the knowledge from the previous exercises, try to write your own batch script to do so and store it in “training/scripts/” (hint: if you get stuck, you can find a commented solution in “training/scripts/openmp-job.sh.solution“). Now submit the job:-

Use squeue to monitor the job. When the job has completed locate and examine the output file(s).

The contents of the output files should be similar to the interactive output.