SparseCore
Also known as SparseCores, TPU SparseCore, SC
In plain English
SparseCores are small helper cores on a TPU chip that handle irregular jobs such as gathers and data movement so the matrix units can keep multiplying.
Technical definition
A SparseCore is a specialized TPU core for sparse and irregular operations such as embedding lookups, gathers, permutations, and data movement, running alongside the dense TensorCores on the same chip.
Engineering details
Ironwood has four third-generation SparseCores per chip next to two TensorCores. Dense matrix engines choke on ragged access patterns, so the TPU team moves that work to SparseCores. In the external serving stack, the MoE token permutation that gathers each expert’s tokens into contiguous groups, the top-k weight gather in the ragged gather-reduce path, and the ReduceScatter collective all run on SparseCore. The TensorCore keeps executing expert matmuls while the SparseCore rearranges data and moves partial sums, and double buffering overlaps the two.
Why it matters
Offload is not free. A collective that fits in VMEM can be slower on SparseCore than on the TensorCore because of launch overhead, so Qwen3.5 uses a VMEM-derived threshold to decide which all-reduce and all-gather operations move. Getting the split right is a recurring source of per-layer latency wins.
How to read it in InferenceX
The SparseCore MoE permutation rewrite reported 12% higher 8k1k serving throughput on Ironwood along with lower TTFT and TPOT. Moving the top-k weight gather cut TensorCore overhead from 29 to 14 microseconds in a DeepSeek-V3 microbenchmark, and the collective threshold gained 2.7% at concurrency 64 and 5.7% at concurrency 128.