From Beocat
Jump to: navigation, search
No edit summary
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
We currently have five classes of compute nodes. Starting with the oldest first we have
For a current list you can use kstat to request the hardware configuration
for any or all nodes.


== Mages ==
  kstat -h -l
[1,3,9,11] - Why are these numbered like this? There are actually 8 physical machines, however each pair (1 and 2, 3 and 4, etc.) is tied together with external [http://en.wikipedia.org/wiki/Intel_QuickPath_Interconnect QPI], making them appear as a single node.
  kstat -h -l wizard32


{| class="wikitable"
You can also use kstat to list all nodes with GPUs, or specify a type or minimum GPU memory.
|Processors
|8x 10-Core Xeon E7-8870
|-
|Ram
|1024GB
|-
|Hard Drive
|2x 300GB Hitachi 10,000rpm SAS
|-
|NIC 0
|Broadcom NetXtreme II BCM5709
|-
|NIC 1
|Broadcom NetXtreme II BCM5709
|-
|NIC 2
|Broadcom NetXtreme II BCM5709
|-
|NIC 3
|Broadcom NetXtreme II BCM5709
|-
| 10GbE and QDR Infiniband
|Mellanox Technologies MT27500 Family [ConnectX-3]
|}


== Elves ==
  kstat -h -l -g all
[1-56]
  kstat -h -l -g l40s
{| class="wikitable"
  kstat -h -l -g 40gb
|Processors
|2x 8-Core Xeon E5-2690
|-
|Ram
|64GB
|-
|Hard Drive
|1x 250GB 7,200 RPM SATA
|-
|NICs
|4x Intel I350
|-
| 10GbE and QDR Infiniband
|Mellanox Technologies MT27500 Family [ConnectX-3]
|}


[57-72,77]
We currently have 5 classes of compute nodes. Starting with the oldest first we have
{| class="wikitable"
|Processors
|2x 10-Core Xeon E5-2690 v2
|-
|Ram
|96GB
|-
|Hard Drive
|1x 250GB 7,200 RPM SATA
|-
|NICs
|4x Intel I350
|-
|10GbE and QDR Infiniband
|Mellanox Technologies MT27500 Family [ConnectX-3]
|}


[73-76,78,79]
== Heroes ==
[1-27,29-36,47-54]
{| class="wikitable"
{| class="wikitable"
|Processors
| Processors
|2x 10-Core Xeon E5-2690 v2
| 2 x 12-Core Xeon E5-2680 v3 --> 24 cores
|-
|-
|Ram
| Ram
|384GB
| 128 GB
|-
|-
|Hard Drive
| Hard Drive
|1x 250GB 7,200 RPM SATA
|1 x 1TB 7,200 RPM SATA
|-
|-
|NICs
|NICs
|4x Intel I350
|2 x Intel I350
|-
|-
|10GbE and QDR Infiniband
|40 GbE
|Mellanox Technologies MT27500 Family [ConnectX-3]
| Mellanox Technologies MT27500 Family [ConnectX-3]
|}
|}


[80-85]
[37-44]
{| class="wikitable"
|Processors
|2x 10-Core Xeon E5-2690v2
|-
|Ram
|64GB
|-
|Hard Drive
|1x 250GB 7,200 RPM SATA
|-
|NICs
|4x Intel I350
|-
| 10GbE and QDR Infiniband
|Mellanox Technologies MT27500 Family [ConnectX-3]
|}
 
== Heroes ==
[1-36,47-54]
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
Line 110: Line 38:
|-
|-
| Ram
| Ram
| 128 GB
| 512 GB
|-
| GPU
| 3 x nVidia RTX A4000
|-
|-
| Hard Drive
| Hard Drive
Line 120: Line 51:
|40 GbE
|40 GbE
| Mellanox Technologies MT27500 Family [ConnectX-3]
| Mellanox Technologies MT27500 Family [ConnectX-3]
|-
| Additional Notes
| 2 x Xeon Phi FPU
|}
|}


[37-46]
[45-46]
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
Line 144: Line 78:


== Dwarves ==
== Dwarves ==
[1-37]
[1-23,25-37]
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
Line 162: Line 96:
|}
|}


[38]
[36-38]
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
Line 171: Line 105:
|-
|-
| GPU
| GPU
| 2 x nVidia GeForce GTX 980 Ti
| 2 x nVidia RTX A4000
|-
|-
| Hard Drive
| Hard Drive
Line 243: Line 177:
|}
|}


[54-55,62-64]
[54-55,64]
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
Line 290: Line 224:
|-
|-
| GPU
| GPU
| 4 x nVidia GeForce GTX 1080 Ti
| 4 x nVidia GeForce GTX 1080 Ti (a few have only 3 GPUs left)
|-
|-
| Hard Drive
| Hard Drive
Line 309: Line 243:
| Ram
| Ram
| 1.5 TB
| 1.5 TB
|-
| GPU
| 2 x nVidia Quadro gp100
|-
|-
| Hard Drive
| Hard Drive
Line 356: Line 293:
|}
|}


[32-34]
[32]
{| class="wikitable"
| Processors
| 2 x 20-Core Xeon Gold 6130 -> 40 cores
|-
| Ram
| 96 GB
|-
| GPU
| 2 x nVidia RTX A6000
|-
| Hard Drive
| 946 GB
|-
| NICs
|
|-
| Network
| 40 Gbps RoCE
|}
 
[33-34]
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
Line 411: Line 369:
| Hard Drive
| Hard Drive
|
|
|-
| NICs
|
|-
| Network
| 40 Gbps RoCE
|}
[[Category:Information]]
[[Category:Hardware]]
[39]
{| class="wikitable"
| Processors
| 1 x 24-Core Xeon Gold 5317 -> 24 cores
|-
| Ram
| 256 GB
|-
| Hard Drive
|  450 GB
|-
|-
| NICs
| NICs
Line 451: Line 430:
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
| 2 x 64-Core AMD EPYC 7452 -> 128 cores
| 2 x 64-Core AMD EPYC 7742 -> 128 cores
|-
|-
| Ram
| Ram
Line 463: Line 442:
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
| 2 x 64-Core AMD EPYC 7452 -> 128 cores
| 2 x 64-Core AMD EPYC 7662 -> 128 cores
|-
|-
| Ram
| Ram
Line 475: Line 454:
{| class="wikitable"
{| class="wikitable"
| Processors
| Processors
| 2 x 64-Core AMD EPYC 7452 -> 128 cores
| 2 x 64-Core AMD EPYC 7662 -> 128 cores
|-
|-
| Ram
| Ram
Line 493: Line 472:
|-
|-
| GPU
| GPU
| 8 x nVidia GeForce GTX 3090 Ti
| 3-4 x nVidia GeForce GTX 3090
|-
| Network
| 40 Gbps RoCE
|}
 
[18-21]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 256 GB
|-
| Network
| 40 Gbps RoCE
|}
 
[22-24]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 512 GB
|-
| Network
| 100 Gbps Ethernet
|}
 
[25-26]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 9554 -> 128 cores
|-
| Ram
| 1.5 TB
|-
| GPU
| 4 x nVidia L40S
|-
| Network
| 40 Gbps RoCE
|}
 
[27-28]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 1 TB
|-
| Network
| 40 Gbps RoCE
|}
 
[29-30]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 2 TB
|-
| Network
| 40 Gbps RoCE
|}
 
[31]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 1 TB
|-
| GPU
| 1 x nVidia RTX 4500
|-
|-
| Network
| Network
| 40 Gbps RoCE
| 40 Gbps RoCE
|}
[32]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 1 TB
|-
| GPU
| 1 x nVidia L4
|-
| Network
| 40 Gbps Ethernet
|}
[33]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7713 -> 128 cores
|-
| Ram
| 1 TB
|-
| GPU
| 1 x nVidia L4
|-
| Network
| 100 Gbps Ethernet
|}
[34-37]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 9554 -> 128 cores
|-
| Ram
| 768 GB
|-
| Network
| 100 Gbps Ethernet
|}
[38]
{| class="wikitable"
| Processors
| 2 x 64-Core AMD EPYC 7452 -> 128 cores
|-
| Ram
| 512 GB
|-
| GPU
| 3 x nVidia L40S
|-
| Network
| 100 Gbps Ethernet
|}
|}


[[Category:Information]]
[[Category:Information]]
[[Category:Hardware]]
[[Category:Hardware]]

Latest revision as of 16:09, 22 August 2025

For a current list you can use kstat to request the hardware configuration for any or all nodes.

 kstat -h -l
 kstat -h -l wizard32

You can also use kstat to list all nodes with GPUs, or specify a type or minimum GPU memory.

 kstat -h -l -g all
 kstat -h -l -g l40s
 kstat -h -l -g 40gb

We currently have 5 classes of compute nodes. Starting with the oldest first we have

Heroes

[1-27,29-36,47-54]

Processors 2 x 12-Core Xeon E5-2680 v3 --> 24 cores
Ram 128 GB
Hard Drive 1 x 1TB 7,200 RPM SATA
NICs 2 x Intel I350
40 GbE Mellanox Technologies MT27500 Family [ConnectX-3]

[37-44]

Processors 2 x 12-Core Xeon E5-2680 v3 --> 24 cores
Ram 512 GB
GPU 3 x nVidia RTX A4000
Hard Drive 1 x 1TB 7,200 RPM SATA
NICs 2 x Intel I350
40 GbE Mellanox Technologies MT27500 Family [ConnectX-3]
Additional Notes 2 x Xeon Phi FPU

[45-46]

Processors 2 x 12-Core Xeon E5-2680 v3 --> 24 cores
Ram 512 GB
Hard Drive 1 x 1TB 7,200 RPM SATA
NICs 2 x Intel I350
40 GbE Mellanox Technologies MT27500 Family [ConnectX-3]
Additional Notes 2 x Xeon Phi FPU

Dwarves

[1-23,25-37]

Processors 2 x 16-Core Xeon E5-2683 v4 --> 32 cores
Ram 128 GB
Hard Drive 1 x 1TB 7,200 RPM SATA
NICs 4 x Broadcom BCM5719
40 GbE Mellanox Technologies MT27520 Family [ConnectX-3 Pro]

[36-38]

Processors 2 x 16-Core Xeon E5-2683 v4 --> 32 cores
Ram 512 GB
GPU 2 x nVidia RTX A4000
Hard Drive 1 x 1 TB 7,200 RPM SATA
NICs 4 x Broadcom BCM5719
40 GbE Mellanox Technologies MT27520 Family [ConnectX-3 Pro]

[22-25,29-30,35,39]

Processors 2 x 16-Core Xeon E5-2683 v4 --> 32 cores
Ram 128 GB
GPU 2 x nVidia GeForce GTX 1080 Ti
Hard Drive 1 x 1 TB 7,200 RPM SATA
NICs 4 x Broadcom BCM5719
40 GbE Mellanox Technologies MT27520 Family [ConnectX-3 Pro]

[31-34]

Processors 2 x 16-Core Xeon E5-2683 v4 --> 32 cores
Ram 128 GB
GPU 2 x nVidia GeForce GTX 2080 Ti
Hard Drive 1 x 1 TB 7,200 RPM SATA
NICs 4 x Broadcom BCM5719
40 GbE Mellanox Technologies MT27520 Family [ConnectX-3 Pro]

[40-53,56-61]

Processors 2 x 16-Core Xeon E5-2683 v4 --> 32 cores
Ram 128 GB
Hard Drive 1 x 1TB 7,200 RPM SATA
NICs 4 x Broadcom BCM5719
100 GbE Mellanox Technologies MT27700 Family [ConnectX-4]

[54-55,64]

Processors 2 x 16-Core Xeon E5-2683 v4 --> 32 cores
Ram 512 GB
Hard Drive 1 x 1TB 7,200 RPM SATA
NICs 4 x Broadcom BCM5719
100 GbE Mellanox Technologies MT27700 Family [ConnectX-4]

Moles

[1-120]

Processors 2 x 10-Core Xeon E5-2630 v4 --> 20 cores
Ram 32 GB
Hard Drive 1 x 1 TB 7,200 RPM SATA
NICs 2 x Intel I350
QDR Infiniband QLogic QLE7340

Wizards

[1-19,29]

Processors 2 x 16-Core Xeon Gold 6130 -> 32 cores
Ram 96 GB
GPU 4 x nVidia GeForce GTX 1080 Ti (a few have only 3 GPUs left)
Hard Drive
NICs
Network 100 Gbps OmniPath

[20-21]

Processors 2 x 16-Core Xeon Gold 6130 --> 32 cores
Ram 1.5 TB
GPU 2 x nVidia Quadro gp100
Hard Drive
NICs
Network 100 Gbps OmniPath

[22-28]

Processors 2 x 16-Core Xeon Gold 6130 -> 32 cores
Ram 96 GB
Hard Drive
NICs
Network 100 Gbps OmniPath

[30-31]

Processors 2 x 32-Core Xeon Gold 6130 -> 64 cores
Ram 768 GB
Hard Drive
NICs
Network 40 Gbps RoCE

[32]

Processors 2 x 20-Core Xeon Gold 6130 -> 40 cores
Ram 96 GB
GPU 2 x nVidia RTX A6000
Hard Drive 946 GB
NICs
Network 40 Gbps RoCE

[33-34]

Processors 2 x 20-Core Xeon Gold 6130 -> 40 cores
Ram 96 GB
GPU 4 x nVidia GeForce GTX 2080 Ti
Hard Drive
NICs
Network 40 Gbps RoCE

[35]

Processors 2 x 20-Core Xeon Gold 6130 -> 40 cores
Ram 192 GB
GPU 4 x nVidia GeForce GTX 2080 Ti
Hard Drive
NICs
Network 40 Gbps RoCE

[36-38]

Processors 2 x 24-Core Xeon Gold 6130 -> 48 cores
Ram 192 GB
GPU 4 x nVidia GeForce GTX 2080 Ti
Hard Drive
NICs
Network 40 Gbps RoCE

[39]

Processors 1 x 24-Core Xeon Gold 5317 -> 24 cores
Ram 256 GB
Hard Drive 450 GB
NICs
Network 40 Gbps RoCE

Warlocks

[1-4]

Processors 2 x 16-Core AMD EPYC 7281 -> 32 cores
Ram 128 GB
Network 40 Gbps RoCE

[5-8]

Processors 2 x 64-Core AMD EPYC 7742 -> 128 cores
Ram 386 GB
Network 40 Gbps RoCE

[9-10,15]

Processors 2 x 64-Core AMD EPYC 7742 -> 128 cores
Ram 1.5 TB
Network 40 Gbps RoCE

[11-13]

Processors 2 x 64-Core AMD EPYC 7662 -> 128 cores
Ram 256 GB
Network 40 Gbps RoCE

[14]

Processors 2 x 64-Core AMD EPYC 7662 -> 128 cores
Ram 1 TB
Network 40 Gbps RoCE

[16-17]

Processors 2 x 64-Core AMD EPYC 7452 -> 128 cores
Ram 512 GB
GPU 3-4 x nVidia GeForce GTX 3090
Network 40 Gbps RoCE

[18-21]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 256 GB
Network 40 Gbps RoCE

[22-24]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 512 GB
Network 100 Gbps Ethernet

[25-26]

Processors 2 x 64-Core AMD EPYC 9554 -> 128 cores
Ram 1.5 TB
GPU 4 x nVidia L40S
Network 40 Gbps RoCE

[27-28]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 1 TB
Network 40 Gbps RoCE

[29-30]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 2 TB
Network 40 Gbps RoCE

[31]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 1 TB
GPU 1 x nVidia RTX 4500
Network 40 Gbps RoCE

[32]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 1 TB
GPU 1 x nVidia L4
Network 40 Gbps Ethernet

[33]

Processors 2 x 64-Core AMD EPYC 7713 -> 128 cores
Ram 1 TB
GPU 1 x nVidia L4
Network 100 Gbps Ethernet

[34-37]

Processors 2 x 64-Core AMD EPYC 9554 -> 128 cores
Ram 768 GB
Network 100 Gbps Ethernet

[38]

Processors 2 x 64-Core AMD EPYC 7452 -> 128 cores
Ram 512 GB
GPU 3 x nVidia L40S
Network 100 Gbps Ethernet