High Bandwidth Memory (HBM) is a type of computer memory built by stacking many DRAM chips vertically and connecting them with thousands of microscopic vertical wires called through-silicon vias. The stack sits right next to the processor, so data travels a very short distance over a very wide connection. The result is far more bandwidth - the rate at which data can be read and written - than the flat memory chips used in ordinary PCs and phones.
Bandwidth, not raw arithmetic, is often the real bottleneck in AI. A modern accelerator has thousands of cores that can multiply numbers far faster than memory can supply them, so the chip sits idle waiting for data. HBM exists to close that gap. SK hynix, one of the leading suppliers, announced in March 2024 that its HBM3E could process up to 1.18 terabytes of data per second - the company likened it to handling more than 230 full-HD movies every second - and framed HBM as essential infrastructure for AI systems that move huge amounts of data quickly.
Every high-end AI accelerator now ships with HBM stacked alongside the compute die, and the memory has become one of the scarcest, most strategically important components in the AI supply chain. Demand from NVIDIA and others made HBM a major profit driver for memory makers like SK hynix, Samsung and Micron.
Why business readers should care: HBM is a reminder that AI performance is limited as much by data movement as by computation. The companies that make this memory have become chokepoints in the AI economy, and HBM supply is now one of the constraints that gates how fast accelerators can actually be built and shipped.