Parallelism
TMA multicast
In plain English
One tensor-memory transfer supplies multiple cooperating blocks, reducing repeated requests for shared input data.
Technical definition
TMA multicast distributes a transfer to selected blocks in a supported cluster, allowing them to receive shared input data in their local storage.
Engineering details
Explicit multicast describes the recipients to hardware. The Blackwell article separately tests independent requests to identical addresses, where request coalescing can provide some similar behavior. Equal effective delivery throughput does not imply equal L2 traffic, especially as the working set changes.
Why it matters
Reusing inputs across blocks can reduce memory traffic for GEMM-like computations. The benefit depends on access overlap, cluster placement, and the remaining compute or synchronization bottlenecks.
How to read it in InferenceX
Distinguish bytes delivered into all destination buffers from bytes fetched through L2 or HBM. A traffic-reduction factor is not automatically an application-speedup factor, and the source’s caption unit discrepancy is labeled in the port.