Inter-Chip Interconnect
Also known as ICI, TPU interconnect, ICI fabric
In plain English
ICI is the custom network that lets TPU chips exchange data directly with each other across a whole pod without going through the host CPU.
Technical definition
ICI is Google’s chip-to-chip interconnect for TPUs, a point-to-point fabric that carries activations, gradients, and KV-cache traffic between chips without routing through PCIe or a general-purpose NIC.
Engineering details
Each TPU has ICI links to its torus neighbors: four on the 2D torus of v2 and v3, six on the 3D torus from v4 onward. Optical circuit switches join 64-chip cubes into pods of thousands of chips while preserving the wraparound links. The result is NVLink-class bandwidth across an entire pod rather than across eight or 72 devices. TPU 8i doubles ICI bandwidth to 19.2 Tb/s per chip and moves from the torus to the Boardfly topology.
Why it matters
Pod-wide bandwidth changes what parallelism is practical. A DeepSeek-V3-scale model can be sharded across a pod with tensor, expert, and data parallelism without pipeline stages, and Ironwood pods scaling past 1,000 chips can run disaggregated serving and ultra-wide expert parallelism that an NVL72 domain cannot. A torus has more hops than a single-hop NVSwitch, so latency depends on message size and topology.
How to read it in InferenceX
The InferenceX Official Preview notes that upcoming CollectiveX and NetworkingX results show the TPU torus often has lower latency than a single-hop NVSwitch for the small expert-parallel messages MoE decode produces. The ICI fabric is also where the SparseCore ReduceScatter overlaps die-to-die and chip-to-chip transfers.