Containerisation using Singularity

Please note that containerisation on BlueBEAR is currently in beta. This means that it may be unreliable and subject to operational changes. If you experience any issues then please submit a Service Desk ticket, describing the problem in as much detail as possible.

BlueBEAR supports containerisation using Singularity. Each computer node has Singularity installed, which means that the singularity command is available without needing to first load a module.

Singularity is preferred over Docker on high performance computing systems due to the administrative privileges that are required to run the latter. However, Singularity supports Docker images and can pull from Docker Hub in addition to its own online repository. This means that in many cases it can act as a drop-in replacement for users who are already familiar with containerisation using Docker. Please see below for an outline of Singularity operations.

Pull and Run

To pull an image from the Singularity Library:

singularity pull library://sylabsed/examples/lolcow

To pull an image from Docker Hub:

singularity pull docker://python
singularity pull docker://python:3.4.2 (to pull a specific tag)

The two common methods for using a Singularity container are exec and shell:
Exec is for where you want to run a command in a container and then exit the container as soon as the command completes.
Shell is for where you want to launch a container and then attach to its shell.

Examples:

singularity exec python_3.4.2.sif python --version
... will spawn a container from the specified image file, execute the command python --version, print the output and then exit the container.

 

Image architecture

An error will occur if you try to run an image that's built for a processor architecture that is different to that you are running on. In the case of BlueBEAR you may trigger this if trying to run an x86_64 image on an IBM POWER9 node. Please note that some of Docker's official images are available for ppc64le (the architectural name for POWER9). BlueBEAR defines the environment variable ${SINGULARITY_DOCKER_ARCHITECTURE} which ensures that any singularity pull docker://xxx commands will attempt to download images that are appropriate for the architecture on which you're running, if they're available.

Remote Builders

Singularity requires administrative privileges in order to build images locally and as a consequence this action is not available to BlueBEAR users. However, it is possible to build Singularity images using Sylabs' Remote Build service. The setup process for using this service is well documented at https://cloud.sylabs.io/builder but essentially you need to authenticate using an existing account (e.g. Google, GitHub etc.) and then generate a Sylabs cloud token. Here's an overview of the process:

  1. Execute: singularity remote login SylabsCloud
  2. Create a Sylabs cloud token (see above) and paste the key where prompted from the previous command
  3. Write a Singularity definition file (see here for information)
  4. Execute: singularity build --remote desired_image_filename.sif your_definiton_filename.def

Last modified: 18 June 2019