Body
Requesting Specific Resources
SLURM allows you to request specific resources for your job to run on, beyond requesting resource amounts and queue availability. Be warned that this may significantly increase your queue time as you are artificially limiting the number of nodes that SLURM can assign your job to with each constraint.
This article will cover three specific constraints:
- A Node with a specific property
- A node with a specific resource
- A specific Node
A Node With a Specific Property
All nodes on the Mill are tagged with specific properties that can be used to filter using the constraint directive. For example, nodes with Intel processors or AMD processors are tagged with this information so that you can selectively run your job on the processor architecture best for your job. Example:
#SBATCH constraint=intel
SLURM supports complex constraints through the use of boolean operators. Constraints can be chained using & (and) or | (or). Complete constraints should be wrapped in double quotes. Example:
#SBATCH constraint="intel&EDR" will run on a node with an Intel processor and an Infiniband connection.
Valid Constraints
The Mill is fairly homogenous, so there are few meaningful constraints beyond processor brand. The following table contains all supported constraints:
| Feature |
Description |
| intel |
Node has Intel CPU |
| amd |
Node has AMD CPU |
| EDR |
Node has an EDR (100Gbit/s) Infiniband connection |
| gpu |
Node has GPU acceleration capabilities |
| [CPU Codename] |
Node is running the codename of cpu e.g. rome |
A Node With a Specific Resource
SLURM also tracks a field called GRES which stands for Generic Resource, and is used to track resources other than CPU and Memory. Most commonly, this is used to request GPUs. There are currently three types of GPUs on the Mill:
| Name |
Hardware |
GRES Name |
| V100 - (Only on Classes Partition) |
PCIe V100 |
gpu:V100-PCIE-32GB |
| V100 |
SXM V100 |
gpu:V100-SXM2-32GB |
| H100 |
SXM H100 |
gpu:H100 |
To get a list of all nodes and the GRES available on each node, you can use the following sinfo command:
sinfo -o "%N %G"
To request a node with an H100 GPU, you would use:
#SBATCH --gres=gpu:H100
To request multiple GPUs, you would specify the number after the GPU name, separated by a colon. For example, to request 2 H100 GPUs use:
#SBATCH --gres=gpu:H100:2
A Specific Node
Using the sbatch directive nodelist will allow you to specify which node(s) you want your job to run on. It accepts either a single node or a list of nodes separated by commas. Example:
#SBATCH --nodelist=compute-33-01
The opposite of the nodelist directive is the exclude directive, which can be used to run on any node(s) except the listed one(s). Example:
#SBATCH --exclude=compute-33-01