uBenchX microbenchmarksBeta
Low-level accelerator microbenchmarks measuring fundamental hardware characteristics
Memory bandwidth utilization (MBU) vs message size MBU uses each device’s peak bandwidth
Shift+Scroll to zoom · Drag to pan · Double-click to reset
- Source
- SemiAnalysis InferenceX™
Huawei uses amortized rotating-buffer graph timing; NVIDIA/AMD use Triton copy timing. Small-message latency is not directly comparable.
Methodology
NVIDIA/AMD: times b.copy_(a) on float32 tensors using triton.testing.do_bench; bandwidth = 2 * bytes / time (read + write). Power-of-two sizes from 8 B to 16 GiB.
Huawei: FP16 rotating source/destination pools of at least 512 MiB each, timed as full rotating-buffer NPU graph replays. Three processes, 360 observations per size; full copied-bit checks before/after each block. Graph time is amortized per copy, not isolated-copy latency. Protocols differ from the GPU curves. No physical HBM-traffic counters were collected. The 8 B–2 KiB points were added on 30 October with CANN 9.2.1 and 4 KiB slot spacing; touched payload may fit in cache. Points from 4 KiB upward retain the original CANN 9.1.0 run.
Huawei MBU uses the 1,638.4 GB/s peak (8 HBM stacks × 256 bits × 6.4 Gbps), which Huawei rounds to 1.6 TB/s.
| Device | Hardware | Driver | PyTorch | Runtime | Container | Peak bandwidth |
|---|---|---|---|---|---|---|
| B300 SXM | NVIDIA B300 SXM6 AC | 580.159.03 | 2.14.0+cu130 | Triton 3.8.0 | pytorch/pytorch:2.14.0-cuda13.0-cudnn9-runtime | 8.00 TB/s |
| GB200 NVL72 | NVIDIA GB200 | 580.126.20 | 2.13.0+cu130 | Triton 3.7.1 | lmsysorg/sglang:nightly-dev-cu13-20260922-582389ce (arm64) | 8.00 TB/s |
| B200 SXM | NVIDIA B200 | 580.159.03 | 2.13.0+cu130 | Triton 3.7.1 | lmsysorg/sglang nightly 2026-09-22 (582389ce) | 8.00 TB/s |
| H200 SXM | NVIDIA H200 | 580.173.02 | 2.13.0+cu130 | Triton 3.7.1 | lmsysorg/sglang:nightly-dev-cu13-20260923-06008c17 | 4.80 TB/s |
| H100 SXM | NVIDIA H100 80GB HBM3 (SXM) | 580.159.03 | 2.5.1+cu124 | Triton 3.1.0 | pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime | 3.35 TB/s |
| MI355X | AMD Instinct MI355X | amdgpu 6.16.6 | 2.11.0+rocm7.2 | Triton 3.7.0 | sglang_local-rocm724-mi35x.sqsh | 8.00 TB/s |
| MI325X | AMD Instinct MI325X | amdgpu 6.16.13 | 2.10.0+rocm7.2.4 | Triton 3.6.0 | rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0 | 6.00 TB/s |
| MI300X | AMD Instinct MI300X | amdgpu 6.16.13 | 2.10.0+rocm7.2.4 | Triton 3.6.0 | rocm/pytorch:rocm7.2.4_ubuntu24.04_py3.12_pytorch_release_2.10.0 | 5.30 TB/s |
| Huawei 950PR | Ascend950PR_9579 | 25.7.rc1.10 | 2.10.0+cpu | CANN 9.1.0 · torch_npu 2.10.0.post4 | — | 1.64 TB/s |
Sources: PR #3879 · PR #3874 · PR #3880 · huawei-bandwidth.json