Glossary term
Memory Bandwidth
The rate at which data moves between memory and the processor; the specific bottleneck that high-bandwidth memory (HBM) exists to relieve.
Memory Bandwidth is the rate at which data moves between memory and the processor, the specific constraint that limits AI accelerator performance once compute is abundant. Modern accelerators are frequently memory-bandwidth-bound rather than compute-bound, which is why high-bandwidth memory (HBM) commands premium pricing and supply is tight. Closing the gap between compute speed and memory speed is the essence of the memory wall. See Memory Wall 101.