Glossary term
Bandwidth (Memory / Networking)
The rate at which data can be transferred; the constraint that binds AI performance after raw compute is satisfied.
Bandwidth is the rate at which data can be transferred between components, measured in GB/s for memory and Gb/s or Tb/s for networking. In AI systems, bandwidth — not raw compute — is increasingly the binding constraint, because feeding data to fast accelerators is harder than the computation itself. This is the technical core of both the memory wall and the AI networking thesis. See Memory Wall 101 and Networking & Optical 101.