gpt-oss 120B · Chip comparison

gpt-oss 120B — H200 vs MI355X

Head-to-head AI inference benchmark comparison of H200 (NVIDIA Hopper) and MI355X (AMD CDNA 4) on gpt-oss 120B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.

Near the low end of the 38–255 tok/s/user interactivity band, at 92 tok/s/user on gpt-oss 120B: H200 runs 6,714 tok/s/chip at $0.050 per million tokens, MI355X runs 26,412 at $0.016. MI355X is 69% cheaper per token; MI355X delivers 3.9× the tok/s/chip.

Setting 147 tok/s/user as the target on gpt-oss 120B, H200 produces 4,482 tok/s/chip ($0.076 per million tokens) and MI355X produces 16,069 ($0.026). MI355X is 66% cheaper per token; MI355X delivers 3.6× the tok/s/chip.

At 201 tok/s/user interactivity on gpt-oss 120B, H200 delivers 2,812 tok/s/chip at $0.12 per million tokens; MI355X delivers 10,455 tok/s/chip at $0.040. MI355X is 67% cheaper per token; MI355X delivers 3.7× the tok/s/chip at this point. (Numbers reflect the default 8k/1k · fp4 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)

View performance per dollar →

Interpolated from real benchmark data. Edit target interactivity values below to compare at different operating points.

Metric
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Throughput (tok/s/chip)
H2006,714MI355X26,412
H2004,482MI355X16,069
H2002,812MI355X10,455
Cost ($/M tok)
H200$0.050MI355X$0.016
H200$0.076MI355X$0.026
H200$0.12MI355X$0.040
tok/s/MW
H2004,900,797MI355X12,637,175
H2003,271,417MI355X7,688,576
H2002,052,199MI355X5,002,206
Concurrency
H200~17MI355X~34
H200~16MI355X~13
H200~6MI355X~5