Torus topology
Also known as 3D torus, twisted torus, torus network
In plain English
A torus is a grid network whose edges wrap around and connect to each other, so the farthest chip is never more than half the grid away.
Technical definition
A torus topology connects each chip to its nearest neighbors along every axis and adds wraparound links between the ends of each row, forming rings that halve the worst-case hop distance of a plain mesh.
Engineering details
TPU v2 and v3 used a 2D torus with four neighbors per chip. From v4 onward Google uses a 3D torus with six neighbors along the plus and minus X, Y, and Z axes, and Ironwood keeps that layout. The building block is a 4x4x4 cube of 64 chips sized to one rack. A twisted torus offsets the wraparound to shave the average hop count further. Optical circuit switches connect cubes while preserving the wraparound property, which lets Google rewire around a failed chip or link in seconds. Traffic that crosses many hops pays latency at each one, so collectives are scheduled to stay local where possible.
Why it matters
Before NVL72 racks, any model that did not fit in an eight-GPU node needed pipeline parallelism because inter-node InfiniBand was slow. The TPU torus gave NVLink-class bandwidth across an entire pod years earlier. The tradeoff is hop count: a 1,024-chip torus has a diameter around 16 hops, which is why TPU 8i moves to the flatter Boardfly network for inference.
How to read it in InferenceX
CollectiveX and NetworkingX measure collective latency directly. The InferenceX Official Preview reports that the TPU torus often beats a single-hop NVSwitch on latency for the small expert-parallel messages that dominate MoE decode, even though it has more hops.