HardwareTMA
Tensor Memory Accelerator
In plain English
NVIDIA’s asynchronous bulk-transfer mechanism for moving tensor data while reducing per-thread address-generation work.
Technical definition
TMA is a hardware-supported transfer mechanism that uses tensor descriptions to move data asynchronously between supported memory spaces.
Engineering details
For the global-to-shared path studied in the Blackwell article, one thread can initiate a multidimensional transfer. The engine handles address generation and supported layout details, while synchronization tells consumers when the data is ready. Transfer shape and outstanding work affect attainable bandwidth.
Why it matters
TMA reduces some software overhead but is not automatically best for every access pattern or transfer size. Irregular page selection and small transfers can favor other copy mechanisms.
How to read it in InferenceX
The port compares two-dimensional TMA with LDGSTS under stated thread, stage, and box configurations. Its crossover behavior is workload-specific, and the port explicitly labels the source's byte-versus-KiB inconsistencies.