Cloud server with extremely fast RAM: 6.9 TByte/s

With the Instinct MI300A, AMD supplies a so-called server APU, i.e. a combination of a computing accelerator, 24 x86 processor cores and 128 GB of fast HBM3 generation high bandwidth memory. Microsoft is now using a customer-specific variant of this for the Azure cloud, which does not have a computing accelerator. Instead, it has more CPU cores.

Microsoft announced in the Azure blog that the new HBv5 VM instance can have up to 352 AMD Zen 4 CPU cores with up to 4 GHz and up to 450 GB of RAM with a data transfer rate of 6.9 TByte/s.

Microsoft cites scientific and technical high-performance computing (HPC), such as molecular dynamics, computational fluid dynamics (CFD), weather models, simulation software and CAE as typical uses for Azure HBv5 VM.

At the SC'24 conference, Microsoft presented the hardware: a server mainboard with four SH5 sockets for four AMD processors. This is presumably the so-called Instinct MI300C, which AMD has sold exclusively to Microsoft to date.

Supermicro also exhibited the GPU A+ Server AS-4145GH-TNMR rack-insert with the H13QSH mainboard for four AMD Instinct MI300A at SC'24. There are no sockets for memory modules because the high bandwidth memory is located on the MI300A. Thanks to the built-in x86 cores, the MI300A can run an operating system itself. They also connect numerous PCI Express 5.0 lanes, for example for NVMe SSDs and network cards.

Intel also offers server processors with HBM RAM. However, the Xeon CPU Max from the fourth generation Xeon-SP Sapphire Rapids only has 64 GByte HBM2E and a maximum of 56 CPU cores. With four processors, a maximum of 224 cores and 256 GB RAM would therefore be possible. However, the Xeons can also connect DDR5 memory.

(ciw)