ROCm
Also known as AMD ROCm
In plain English
ROCm is AMD’s software toolbox for running AI and other high-performance programs on AMD GPUs.
Technical definition
ROCm is AMD’s open GPU computing software platform, including runtimes, compilers, communication libraries, and optimized math and AI kernels.
Engineering details
vLLM and SGLang use ROCm plus AMD-specific libraries and kernel projects to run on Instinct accelerators. Model support depends on compatible attention, MoE, quantization, collective, and graph-execution paths.
Why it matters
Software maturity can dominate cross-vendor inference results. Rapid kernel and engine work has produced multi-fold gains on unchanged MI355X hardware, while missing paths can leave strong theoretical silicon underused.
How to read it in InferenceX
InferenceX preserves engine versions and run dates so ROCm improvements can be measured over time. A point-in-time comparison should not be generalized to a later software release.
Source material
See the concept in real benchmarks
AMD MI355X Kimi K2.5 Inference: 7.7x Throughput, Up To 15x Interactivity in 25 Days on vLLM
vLLM PR #35850 Fixed AITER MLA Dispatch on MI355X CDNA4, Unlocking Kimi K2.5 Inference Performance at TP=8, Shipped in vLLM 0.18
MI355X DeepSeek-V4-Pro on SGLang: 110.5x Throughput per GPU in 26 Days
The amd/deepseek_v4 side branch shipped TileLang attention indexer, Triton sparse MLA, fused RoPE/Hadamard, FlyDSL MoE, and FP4 weights across 31 performance optimizations PRs — lifting first-light 20 tok/s/GPU at 2.4 tok/s/user into 2,256 tok/s/GPU at 9.4 tok/s/user on 8K/1K, with both throughput and interactivity climbing together
AMD MI355X Qwen3.5 397B-A17B Inference: Up to 19x Throughput per GPU in 3 Months on SGLang FP8
From v0.5.8 (Feb) → v0.5.10rc0 (Apr) → v0.5.12 (May), three AITER kernel landings on MI355X plus a TP=8 → TP=2/TP=4 retune push Qwen3.5 8k/1k peak from 1.3k to 6.4k tok/s/GPU and extend the curve out to 75 tok/s/user
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM