Gibbs Scheduler
Slurm
Gibbs is now using the Slurm scheduler.
Slurm is a free and open-source job scheduler for the Linux kernel used by many of the world's supercomputers and computer clusters. There is significant documentation available via the man pages. Try 'man sbatch' for submission related commands and 'man sstat' for status queries.
Partitions
(Can use sinfo to see partition. Jobs submitted will choose nodes in billing weight order, unless specified otherwise)
| Partition Name | #Nodes | Billing weights | ||
| cpu | mem | gpu | ||
| GTX670 | gibbs[01-08] | 0.125 | 0.031G | 0.25 |
| TITAN | gibbs[11-12] | 0.207 | 0.052G | 0.415 |
| GTX780 | gibbs[13-15] | 0.261 | 0.065G | 0.521 |
| K20 | gibbs[09-10] | 0.386 | 0.096G | 0.772 |
| GTX980 | gibbs[18-36] | 0.207 | 0.026G | 0.828 |
| K80 | gibbs[16-17] | 0.588 | 0.037G | 1.176 |
All the partitions have 4 GPU's except K80 with 8 GPU's.
Jobs can be submitted using the 'sbatch' command, generally followed by the name of the submission script. Common command line options are included in the sample scripts. A full listing can be found in the man pages ('man sbatch'). Status of queued or running jobs can be obtained via the 'sstat' command.
A generic sample script is available, as well as select submission scripts for specific applications, which can be found in their Application Pages.
Torque vs Slurm Commands
| Description | Torque | Slurm |
|---|---|---|
| Submit a job | qsub [file] | sbatch [file] |
| Job hold | qhold jobid | scontrol hold jobid |
| Cancel a job | qdel | scancel |
| Release a job | qrls [job_id] | scontrol release |
| Job status | qstat [job_id] | sstat OR scontrol show jobs |
| Queue list | qstat | squeue |
| Node list | pbsnodes -l | sinfo --Node OR scontrol show nodes |
| Cluster status | qstat | sinfo |
Job Specification
| Script directive | #PBS | #SBATCH |
|---|---|---|
| Queue | -q [queue] | none - see QoS |
| QoS | none | --qos= |
| Node Count | - l nodes=[count] | -N [min[-max]] |
| CPU Count | -l ppn=[count] OR -l mppwidth=[PE_count] |
-n [count] |
| Wall Clock Limit | -l walltime=[hh:mm:ss] | -t [min] OR -t [days-hh:mm:ss] |
| Standard Output File | -o [file_name] | -o [file_name] |
| Standard error File | -e [file_name] | -e [file_name] |
| Event Notification | -m abe | --mail-type=[events] |
| Email Address | -M [address] | --mail-user=[address] |
| Job Name | -N [name] | --job-name=[name] |
| Memory Size | -l mem=[MB] | --mem=[mem][M|G|T] OR --mem-per-cpu= [mem][M|G|T] |
| Account | none | --account=[account] |
Healey Library, Lower Level
UMass Boston
100 Morrissey Blvd.
Boston, MA 02125
Book a Consultation
617.287.5399
It-rc@umb.edu