HBM (High Bandwidth Memory) is a 3D-stacked DRAM architecture that provides dramatically higher bandwidth than conventional memory. HBM stacks multiple DRAM dies vertically using through-silicon vias (TSVs), connecting them to the processor via a silicon interposer. HBM3E — the current generation — delivers over 1 TB/s bandwidth per stack, roughly 5–10x faster than standard DDR5 memory.
AI training and inference workloads are fundamentally memory-bandwidth-limited. Large language models require massive parallel memory access to feed data to GPU compute cores fast enough. Without sufficient HBM bandwidth, GPU utilization drops and effective throughput collapses. Each NVIDIA H100 GPU uses 80GB of HBM3; the B200 uses up to 192GB of HBM3E. SK Hynix controls approximately 50% of the HBM market, Samsung roughly 40%, and Micron approximately 10%. The production process for HBM is significantly more complex than standard DRAM — TSV drilling, wafer thinning, and precision stacking reduce yields and increase costs by 3–5x per bit versus conventional DRAM.
HBM is the memory bottleneck in AI computing. Large language models and AI training workloads are memory-bandwidth constrained — the speed at which data moves between memory and GPU determines throughput. HBM solves this by stacking DRAM dies vertically and connecting them via through-silicon vias (TSVs), achieving bandwidth 5-10x higher than conventional memory.
The HBM market is dominated by SK Hynix (~50% share) and Samsung (~40%), with Micron as a distant third. HBM commands ASPs 5-10x higher than standard DRAM, making it the highest-margin memory product in history. Every NVIDIA Blackwell GPU requires multiple HBM stacks, creating a direct demand linkage between GPU shipments and HBM volume. The transition from HBM3E to HBM4 represents another cycle of capacity constraints and pricing power.
HBM is connected to CoWoS packaging (HBM stacks are bonded to GPUs via CoWoS), the Memory Wall thesis, and the Sentinel Ticker framework where Micron serves as a leading indicator.
