Die-to-die link
Also known as die-to-die link, D2D link, intra-chip link, inter-die link
In plain English
A die-to-die link connects two compute dies inside the same chip package, faster than the network between chips but separate from a shared memory space.
Technical definition
A die-to-die link is the high-bandwidth interconnect that joins the two compute dies inside an Ironwood package, letting each die run as its own logical device while exchanging data faster than the chip-to-chip ICI network.
Engineering details
Ironwood packages two dies, each with a TensorCore, HBM, and SparseCores. The die-to-die link carries traffic between them without going through ICI. Because the link is faster than ICI, collectives are structured hierarchically: reduce within the chip first, then exchange partial sums between chips. The same idea appears on GPUs, where Blackwell joins two dies with NV-HBI, but Blackwell presents one device while Ironwood presents two.
Why it matters
Two-level collectives let kernels overlap the fast intra-chip stage of one microbatch with the slow inter-chip stage of another. In the Ironwood ReduceScatter kernel, the intra-chip DMA ScatterReduce for microbatch 1 runs concurrently with the ICI-dimension ScatterReduce for microbatch 0, shortening end-to-end collective time.
How to read it in InferenceX
The SparseCore ReduceScatter that uses the die-to-die link is part of the DP attention plus EP8 serving configuration that produced the Qwen3.5 397B results in the InferenceX Official Preview. Google reports the two-device-per-chip layout as a design change from MegaCore rather than as a separate benchmark line.
Source material
See the concept in real benchmarks
TPU Inference Externalization Full Steam Ahead
InferenceX, Up to 50% Better Performance per Dollar, Rapid Externalization of TPU stack, Growing Customer Base, Ironwood, TPUv8i, Reducing CUDA Moat
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton