While Jupyter notebook support is installed as part of the supported cpython3 modules on SCIAMA, it should be avoided to run a notebook server directly on the login nodes to avoid unnecessary congestion there. Instead, it can be run on the compute nodes. This article describes how to do so and how to access the notebook server from either the login nodes or directly from the own computer.

So far, Jupyter notebook is supported for the cpython/3.7.1 for both intel_comp/2019.2 and gnu_comp/9.1.0. In order to start a new server, simply load this module i.e. e.g.

Now start a server on a compute node using the following command:

Once executed, you will have to wait until the resources have been allocated. On success, you will see the following output:

Waiting for remote jupyter server (SLURM JOBID 1627554) to start
....
Resources allocated. Waiting for server to finish init
...........
CLIENT LOCAL:44896 REMOTE:node108:53604 TOKEN:27630cd74b4b88de1fdc63e7ed8c7b084b08064aebf828a2

Jupyter nodebook server is RUNNING. You can access it via a browser on this node at the following URL:

http://localhost:44896/?token=27630cd74b4b88de1fdc63e7ed8c7b084b08064aebf828a2

Press CTRL-c to terminate server

The server keeps running as long as you keep this shell open or explicitly terminate it using CTRL-c. To access it, check out the sections below.

Managing resources

While the standard config (single core with 4 GB/core on the jupyter.q) should be suffienct for most users’ needs, you can also add the standard sbatch arguments to the initial sciama-notebook command to allocate more cores/memory or switch to an alternative queue e.g. the himem.q for extreme memory demands e.g.

In order to use jupyter notebook with the server on the compute node, simply open a browser on the login nodes (e.g. firefox) and enter the URL stated in the server output. X-forwarding may be sometimes a bit slow. In that case you may want to use a NoMachine login with a remote desktop instead.

You can also access the remote jupyter server with a browser on your own computer. This provides a more responsive experience, but also requires another step depending on the OS of your computer.

Linux

For Linux (and potentially Unix), we provide a convenient script. Simply copy this executable script from its location on SCIAMA

/opt/apps/pkgs/cpython/jupyter-sciama-remote.linux

onto your own computer and run it on YOUR computer with the login details as argument e.g.

./jupyter-sciama-remote.linux juser@login1.sciama.icg.port.ac.uk

where you have to use the SAME login node you used to start the server (as described above). If successful, you should see the following output:

CLIENT LOCAL:37652 REMOTE::40935 TOKEN:2736d2a30d799fe9d23e0df13b8abaecb55a0d5d3b92c178

Jupyter SSH tunneling: You can access it via a browser on this computer at the following URL:

http://localhost:37652/?token=2736d2a30d799fe9d23e0df13b8abaecb55a0d5d3b92c178

Now you can simply access jupyter notebook from a browser on your computer using the URL provided in the output.

macOS

For macOS, a similar script to the one provided for Linux can be found at/opt/apps/pkgs/cpython/jupyter-sciama-remote.linux

The only difference in the process is that you always have to provide a local port as a second argument (due to some issues with the automatic port allocation) e.g.

./jupyter-sciama-remote.macos juser@login1.sciama.icg.port.ac.uk 8888

Windows

On Windows, it is a bit more complicated as you will have to set up the SSH tunneling yourself. For detailed instructions, please see HERE (or google). As the remote port, use the one listed after ‘CLIENT LOCAL’ in the server output (e.g. 44896 in the example output shown above), for your local one, you can use any free port (e.g. try 8888).

Once the tunnel is established, copy the URL stated in the server output into the address bar of a browser on your own computer and replace the port number in it with the one picked by you (e.g. in our example replace http://localhost:44896/?token=... with http://localhost:8888/?token=....

Now open the URL and you should be greeted by the jupyter notebook interface.