HardwareTMEM
Tensor memory
In plain English
Blackwell storage used by tensor-core operations, separate from the ordinary register and shared-memory programming model.
Technical definition
Tensor memory is a Blackwell storage resource used by fifth-generation tensor-core operations, including storage of MMA accumulators.
Engineering details
Software explicitly manages allocation and access through the supported instruction interface. The article compares layouts with both inputs in shared memory against layouts with one input in TMEM. Changing placement can reduce pressure on a particular data path, but the benefit depends on instruction shape and synchronization.
Why it matters
Accumulator placement changes kernel design and resource accounting. TMEM capacity and access rules cannot be substituted with generic HBM capacity or assumed to behave like ordinary per-thread registers.
How to read it in InferenceX
The Blackwell port’s SS and TS comparisons isolate operand placement for specific MMA shapes. They do not establish that moving an arbitrary tensor to TMEM accelerates a complete model.