X
Home High Performance Computing 2023 HPC Upgrade

2023 HPC Upgrade

During the Fall of 2023, we will upgrade the cluster with more and newer compute resources. This upgrade is part of a new 4-year cycle.

The Intel configuration consists of 88 Dell compute nodes with 3520 total CPU cores, 20736 GB total RAM, and 12 NVIDIA V100 GPUs from the old cluster. The AMD configuration consists of 32 Dell compute nodes with 5632 total CPU cores, 27648 GB total RAM, and 8 A100 GPUs. Overall, the cluster will have 120 compute nodes with 9152 cores, 48384 GB total RAM, and 20 GPUs.

  • Login: 2 PowerEdge R6625 dual socket AMD Epyc Genoa 9124 Login nodes with 384 GB DDR5 RAM and HDR100 Infiniband
  • Head: 1 PowerEdge R7625 dual socket AMD Epyc Genoa 9124 Head node with 384 GB DDR5 RAM and HDR100 Infiniband
  • Intel:
    • Thin: 78 PowerEdge C6420 dual socket Intel Skylake Gold 6148 Compute nodes with 192 GB DDR4 RAM and EDR Infiniband.
    • NVIDIA GPU: 6 PowerEdge R740 dual socket Intel Skylake Gold 6148 GPU nodes with 192 GB DDR4 RAM, 2 x NVIDIA V100 GPU and EDR Infiniband
    • Fat: 2 PowerEdge R740 dual socket Intel Skylake Gold 6148 Fat Memory Nodes with 768 GB DDR4 RAM and EDR Infiniband
    • Large Fat: 2 PowerEdge R740 dual socket Intel Skylake Gold 6148 Nodes with 1.5 TB DDR4 RAM and EDR Infiniband
  • AMD:
    • Compute: 24 PowerEdge R7625 dual socket AMD Epyc Genoa 9654 compute nodes with 768 GB DDR5 RAM, 1.6 TB NVME storage, and HDR100 Infiniband.
    • NVIDIA GPU: 4 PowerEdge R7625 dual socket AMD Epyc Genoa 9354 compute nodes with 768 GB DDR5 RAM, 1.6 TB NVME storage, 2 x NVIDIA A100 GPU and HDR100 Infiniband.
    • Fat: 4 PowerEdge R7625 dual socket AMD Epyc Genoa 9654 compute nodes with 1.5 TB DDR5 RAM, 1.6 TB NVME storage, and HDR100 Infiniband.
  • Parallel File System: Arcastream PixStor (GPFS) with 80 x 8 TB HDD (640 TB total raw storage) providing up to 7.5GB/sec read and 5.5GB/sec write performance, and 8 x 15.3 TB SSD (122.4 TB total raw storage) providing up to 80 GB/s read and write speeds. Total usable storage (without raid overhead) is 690 TB for home, project, and scratch directories.
  • Backup Storage: 1 PowerEdge R740XD2 with 24 x 20 TB HDD (480 TB total raw storage) for home and project directories.

All compute nodes are connected via HDR100/EDR Infiniband (2:1 Blocking) and 1/10/25GbE for host/OOB management. Head and Login nodes are connected via HDR100 Infiniband and 10/25GbE for host/OOB management.