CaStLeS BlueBEAR QOS

CaStLeS (Compute and Storage for the Life Sciences) have purchased dedicated resources within BlueBEAR.

The CaStLeS BlueBEAR QOS (castles) includes:

  • Sixty icelake nodes, each with 72 cores and 500GB RAM
  • Twenty-two cascadelake nodes, each with 40 cores and 180GB RAM
  • Twenty-four broadwell nodes, each with 20 cores and 120GB RAM
  • One high-memory broadwell node with 20 cores and 1TB RAM

The CaStLeS GPU BlueBEAR QOS (castlesgpu) includes:

  • One icelake node with 56 cores, 500GB RAM, and two NVIDIA A100-40 GPUs
  • One icelake node with 56 cores, 500GB RAM, and two NVIDIA A30 GPUs
  • One broadwell node with 20 cores, 1TB RAM, and two NVIDIA P100 GPUs
Access to these resources is available for Life Sciences researchers. For further information, including how to apply for access see the CaStLeS overview. More information on the Intel CPU based CaStLeS GPU nodes is available on the BEAR GPUs page.

Using the CaStLeS QOS

Job Submission

To use one of the CaStLeS nodes, add

#SBATCH --qos castles
#SBATCH --account [projectname]

to your job submission script, where [projectname] is a project that has access to the CaStLeS QOS. If, when you submit a job, you receive an error message of

sbatch: error: Batch job submission failed: Invalid qos specification

then either you are not a member of a project that has access to the large memory service or the project you specified does not have access to the large memory service.

To specify the amount of memory for your job, add:

#SBATCH --mem [mem]

to your job submission script, where [mem] is the amount of memory your job requires - usually specified as the amount (in GB), e.g. 200G.

To request one of the GPUs for your job, add:

#SBATCH --gres [gputype]

to your job submission script, where [gputype] is the type of GPU you wish to use in your job - specified in the form gpu:p100:1 to request one NVIDIA P100 per node in your job and change the QOS line to specify the castlesgpu QOS.

Resource Utilisation

Some software can make use of multiple CPU cores (on BlueBEAR this means setting the '--ntasks' parameter in your job submission script up to a maximum of 40) or multiple nodes (setting '--ntasks' above 40). Programming for multiple nodes generally requires a different approach and so this is less common, especially in the type of software used by Castles researchers. Unfortunately, increasing the number of cores will not always make your job run faster. We provide some more information on this on the general BlueBEAR job submission page.

Currently, users submitting jobs to the castles and castlesgpu QoS are limited to:

  • A walltime limit of 10 days per job.
  • A maximum of 576 CPU cores per job and 4 Terabytes of RAM across all of their jobs. For e.g. this could be made up of:
    • A single job requesting 576 cores across 10 Cascade Lake nodes
    • A single job requesting 10 cores and 4 Terabytes of RAM
    • 8 jobs requesting 72 cores across 8 Ice Lake nodes

Colleges

Professional Services