Queues, Scheduling Policies, and Job Priorities
August 2016 Update
The following table summarizes the node types and limits associated with each queue:
|Max wall time||1 hour||4 hours||2 days||2 days||1 day||10 days||10 days||2 days|
|Max # nodes per user||4||8||296||576||256||120||46||4|
|Minimum # nodes per job||1||1||1||1||16||1||1||1|
|Maximum # nodes per job||2||8||288||576||262||120||46||4|
|# of 24 core 64 GB Haswell nodes||2||8||0||880||262||0||0||0||haswell|
|# of 24core 32 GB nodes||2||16||576||0||0||126||0||0||24core|
|# of 16core 32 GB nodes||2||8||196||0||0||152||0||0||16core, phi|
|# of 24core 64 GB nodes||2||8||198||0||0||0||80||0||64GB|
|# of 16core 256 GB nodes||2||2||0||0||0||0||52||0||256GB|
|# of 16 core 32 GB Phi nodes||0||0||0||0||0||0||0||10||phi|
|Total # nodes||10||34||970||880||262||278||132||10|
Nodes with Phi coprocessors, which are running the compute node image that allows use of these coprocessors, are available only in the phi queue.
Use the option -q queue-name on the qsub command or in your job script to specify what queue your job should go to with #PBS -q queue-name. Sample job scripts and the syntax for specifying the queue are available on the sample job scripts page.
Note that the debug, short, batch and long queues all have more than one node type available. If a particular node type is desired and the queue you are submitting your job to includes that node type, you should . If you ask for a node with a feature and there are no nodes of that type in that queue, you job may get stuck, so it is recommended that you don't ask for a specific feature unless you need it.
The large queue is intended for jobs that need a large number of nodes. Jobs that run in this queue must use at least 16 nodes. The run time limit in this queue is 1 day, to reduce the wait time for jobs that need a large number of nodes.
NEW! The long and bigmem queues are now open to ALL users.
The intent of the debug queue is to provide rapid access to nodes for code development, testing and debugging. Production runs are not permitted in the debug queue. User accounts are subject to suspension if they are determined to be using the debug queue for production computing.
A user can have a maximum of 1000 queued jobs (running/active, eligible/idle, blocked).
Priority is determined by queue wait time (the amount of time an eligible job has been waiting in the queue) and the size of the job (number of nodes requested). This enables larger jobs to run in the presence of back-fill scheduling. Larger jobs have higher priority not because we like them better, but because it helps the scheduler be more efficient planning resources for larger jobs.
When projects reach their allocation limit, jobs associated with those projects will run at very low priority, which will ensure that these jobs run only when there are no jobs waiting for resources from projects that have not yet reached their allocation limit.
If you have a short deadline or want to reduce the queue wait time for your job then please reference How to get High Priority for your Job.
The showq -i command may be used to look at your job's priority. The showstart -e all <JOBID> can be helpful in estimating when a job will run. The checkjob -v command can be useful in troubleshoot why a job is not starting.