Peregrine System compute nodes with Intel Xeon Phi Co-Processors
The Peregrine system has compute nodes that contain two Phi Co-Processors, and are available by submitting jobs to the “phi” queue. The Phi Co-Processors are Model 5120 with 60 physical cores available for computing on each card. However, hyperthreading on these cards makes 240 virtual cores visible (via, e.g., /proc/cpuinfo on the cards). The 4 hyperthreads per physical core are not equivalent, however. Instruction execution for a single thread is interleaved, meaning that useful computing is only done on every other clock cycle. To fully utilize the hardware's processing, then, at least 2 threads per core must be assigned. Additional subscription (i.e., having 3 or 4 threads on a core) is more analogous to standard hyperthreading on Intel Xeon processors. Further information on the cards is available on the Intel web site.
The Peregrine nodes are running the Intel MPSS 3.3 software stack, and the /home, /nopt, /scratch and /projects file systems are mounted on each Phi card. The card names for each compute node are <nodename>-mic0 and <nodename>-mic1. If you have work that may benefit from using these nodes and would like assistance, please open a ticket by sending email to firstname.lastname@example.org. The number of nodes in the queue can be adjusted to accommodate the workload up to 288 nodes.