MiniMax M3

Overview

MiniMax-M3 is the flagship open-weights model of the M-series from Shanghai-based AI company MiniMax, described on its Hugging Face model card as "a native multimodal model with 1M context" with "~428B parameters and ~23B activated parameters" (MiniMaxAI/MiniMax-M3 model card). MiniMax's API release notes date the launch to "Jun. 1, 2026," describing M3 as "the latest M-series language model for agentic reasoning, tool use, coding, multimodal chat input, and long-context tasks" (MiniMax API release notes); the Hugging Face repository itself was created on 2026-06-02 (HF API metadata for MiniMaxAI/MiniMax-M3). The model is a mixture-of-experts vision-language model: it accepts text, image and video input and emits text, and the repository is published under pipeline_tag: image-text-to-text (M3 README source).

Positioning is explicitly agentic and coding-first. MiniMax's launch blog markets M3 as delivering "coding & agentic frontier" capability with autonomous task decomposition, tool invocation, multi-step reasoning, autonomous browsing and long-range coding, and calls it the first open-weight model combining frontier coding capability, million-token context and native multimodality (MiniMax M3 model page). The model card frames three headline capabilities: native multimodality ("mixed-modality training from the very first step"), context scaling through sparse attention, and "frontier-level performance across long-horizon agentic benchmarks, excelling in both coding and cowork" (M3 model card). NVIDIA's hosted model card adds concrete target workloads: long-form video understanding up to 30 minutes, long-horizon coding tasks of 8+ hours, and agentic/tool-use workflows, with two switchable reasoning modes (NVIDIA build.nvidia.com M3 model card); the card itself exposes three modes via the thinking parameter — enabled, adaptive, disabled (M3 model card).

The defining technical contribution is MiniMax Sparse Attention (MSA), "a high-performance sparse attention operator designed for million-token contexts" that, compared with GQA, "dramatically reduces the attention compute and memory footprint while preserving model quality," yielding "9× prefill and 15× decode speedups compared to M2 at 1M context, reducing per-token compute to 1/20" (M3 model card). The accompanying technical report describes MSA as blockwise sparse attention built on top of GQA, with a lightweight Index Branch that scores KV blocks and independently selects a Top-k subset per GQA group, a Main Branch performing exact block-sparse attention over selected blocks, exp-free Top-k selection, and a KV-outer sparse attention kernel; on a 109B-parameter native-multimodal testbed it reports 28.4× lower per-token attention compute at 1M context and 14.2× prefill / 7.6× decoding wall-clock speedups on H800 while performing "on par with GQA" (MiniMax Sparse Attention, arXiv:2606.13392). Licensing is not MIT: the repository ships the "MINIMAX COMMUNITY LICENSE," which grants rights for non-commercial purposes, requires prominent "Built with MiniMax M3" attribution for commercial use, requires a one-time notice to api@minimax.io below US$20M annual revenue, and requires separate prior written authorization above US$20M (M3 LICENSE); HF card metadata tags it license: other, license_name: minimax-community (M3 README source).

Architecture

• Total parameters: ~428B per the model card (M3 model card); Hugging Face safetensors metadata reports 427,040,140,160 parameters exactly (HF API metadata).
• Activated parameters: ~23B per token (M3 model card).
• Architecture class / model type: MiniMaxM3SparseForConditionalGeneration, model_type: minimax_m3_vl; the text tower is MiniMaxM3SparseForCausalLM (M3 config.json).
• Layers: 60 (num_hidden_layers: 60) (M3 config.json).
• Hidden size: 6,144; MoE expert intermediate size 3,072; dense-layer intermediate size 12,288; shared-expert intermediate size 3,072 (M3 config.json).
• Attention layout: 64 query heads, 4 key-value heads (GQA), head dim 128 (M3 config.json); MSA is explicitly "blockwise sparse attention built upon GQA" (arXiv:2606.13392).
• QK normalization: use_qk_norm: true with qk_norm_type: per_head (M3 config.json).
• Partial RoPE: rotary_dim: 64 with partial_rotary_factor: 0.5 and rope_theta: 5,000,000 — i.e. RoPE applied to half of each 128-dim head (M3 config.json).
• Sparse attention (MSA) configuration: use_sparse_attention: true, sparse_index_dim: 128, sparse_num_index_heads: 4, sparse_topk_blocks: 16, sparse_block_size: 128, sparse_score_type: max, sparse_local_block: 1; sparse attention is disabled in the first 3 layers and enabled in layers 4–60 via the sparse_attention_freq list (M3 config.json).
• MSA mechanism: lightweight Index Branch scores KV blocks and selects a Top-k subset independently per GQA group (group-specific sparse retrieval); Main Branch runs exact block-sparse attention over only those blocks; exp-free Top-k selection; KV-outer sparse attention kernel for tensor-core utilization (arXiv:2606.13392). MiniMax states MSA partitions KV into blocks more precisely than DSA/MoBA-style approaches and is "more than 4× faster than Flash-Sparse-Attention and flash-moba" (MiniMax M3 launch blog).
• Experts: 128 routed experts (num_local_experts: 128), top-4 routed per token (num_experts_per_tok: 4), plus 1 shared expert (n_shared_experts: 1); sigmoid routing scores with routing bias and routed_scaling_factor: 2.0; first 3 layers dense, layers 4–60 MoE per moe_layer_freq (M3 config.json).
• Activation / norms: hidden_act: swigluoai with swiglu_alpha: 1.702, swiglu_limit: 7.0; RMSNorm with use_gemma_norm: true, rms_norm_eps: 1e-06 (M3 config.json).
• Multi-token prediction: num_mtp_modules: 7, num_nextn_predict_layers: 1 (M3 config.json).
• Context window: max_position_embeddings: 1,048,576 (1M tokens) (M3 config.json); MiniMax markets "up to 1M tokens" with a guaranteed minimum of 512K (MiniMax M3 model page).
• Vocabulary: 200,064 tokens; image token index 200025, video token index 200026 (M3 config.json).
• Vision tower: CLIP-style vision encoder, 32 layers, hidden size 1,280, 16 heads, patch size 14, image size 2016, 3D RoPE position embeddings, projection to 6,144 dims, patch-merge token compression (spatial merge 2, temporal patch 2) (M3 config.json).
• Precision: torch_dtype: bfloat16 (M3 config.json); the HF card lists tensor types BF16 · F32 (M3 model card). MiniMax also publishes an MXFP8 checkpoint referenced in ATOM/MXFP4-MXFP8 usage guidance (M3 model card).
• Recommended inference settings: temperature=1.0, top_p=0.95; served via SGLang, vLLM, Transformers, KTransformers, unsloth, ATOM (M3 model card).
• Reasoning modes: thinking parameter with enabled / adaptive / disabled (M3 model card).

Official vendor eval scores

All rows below are MiniMax-reported scores taken from the official benchmark table published in the MiniMax-M3 Hugging Face repository (figures/benchmark.jpeg, referenced from the model card) or from MiniMax's own launch blog / model page.

Nemotron 3 Ultra figures are the vendor-reported numbers from the NVIDIA Nemotron 3 Ultra model card, shown as a common reference point across model pages. n.a. means NVIDIA does not report that benchmark, or reports a different variant of it that would not be a like-for-like comparison. % better is (score − Nemotron) / Nemotron, computed only where both sides are a plain number on the same metric; a rating scale, a range, or a prose cell shows — instead. Neither column is a head-to-head evaluation: each score is self-reported by its own vendor under its own harness. Additional vendor-reported figures from the same table (comparison context): the vendor chart also lists SWE Atlas-QnA 37.9, SWE Atlas-Test Writing 30.8, CL-bench 20.5, VIBE-V2 50.1, SVG-Bench 63.7, PaperBench 52.6, DRACO 73.2, GDPval rubrics 74.8, BankerToolBench 76.1, OfficeQA Pro 45.1, SpreadSheetBench-v1 89.4, YC-Bench 2.1M, LOCA-Bench (256k) 49.3, Claw-Eval 74.5, OmniDocBench 91.6 and VideoMME (w/ sub) 85.4 for M3 (MiniMax M3 vendor benchmark table).

Vendor evaluation methodology notes (from the same chart): SWE-Bench Verified was run on internal infrastructure with Claude Code as scaffolding, default system prompt overridden, 4 runs averaged; Terminal-bench 2.1 used an 8C16G sandbox, 2-hour timeout, 128K max output tokens and Terminus 2 scaffolding; SWE-fficiency used the open-source dataset/workflow in a 1C2G sandbox with a 2-hour timeout (MiniMax M3 vendor benchmark table).

Benchmark explanations

• SWE-Bench Verified — human-validated subset of real GitHub issues; measures whether the model's patch resolves the issue and passes the repo's tests (MiniMax reports it as an agentic coding benchmark run with Claude Code scaffolding).
• SWE-Bench Pro — harder, industry-grade repository-repair benchmark for coding agents (described as "industry-grade repository repair" in the MiniMax-M2 series report).
• Terminal Bench 2.1 — terminal/system-operation tasks executed in a sandboxed shell, measuring command-line agent competence (MiniMax describes Terminal-Bench as terminal and system-operation tasks).
• SWE-fficiency — measures a coding agent's ability to make repository changes that improve software efficiency, using the open-source SWE-fficiency dataset and workflow (evaluation methodology in MiniMax's M3 chart).
• NL2Repo — MiniMax's internal natural-language-to-repository synthesis benchmark: build a whole repo from a specification (MiniMax-M2 series report).
• LiveSQLBench — text-to-SQL agent benchmark (600 questions across 22 PostgreSQL databases in MiniMax's setup), scoring correct query results (M3 chart methodology).
• KernelBench Hard — hardest split of GPU-kernel generation tasks: write correct, fast CUDA-style kernels (reported by MiniMax as a coding/agentic frontier result).
• PostTrainBench — autonomous post-training benchmark: given pretrain-only base models, the agent must synthesize data, train, evaluate and iterate without human help (MiniMax M3 model page).
• BrowseComp — hard web-browsing benchmark measuring whether an agent can find difficult-to-locate facts online (used by MiniMax as a search/deep-research benchmark).
• MCP Atlas — tool-use benchmark over Model Context Protocol tool servers, measuring multi-tool orchestration (reported in MiniMax's M3 agentic results).
• Apex-Agents — Mercor's agentic evaluation suite of expert-designed real-work tasks (listed in MiniMax's M3 cowork block and as a leaderboard on the M3 card).
• OSWorld-Verified — computer-use benchmark: complete real desktop GUI tasks in a virtual OS environment (reported as a GUI benchmark with task completion rate in MiniMax's M3 materials).
• MMMU-Pro — multimodal college-level multiple-choice reasoning across images and diagrams, a harder variant of MMMU (listed as a multimodal benchmark on the M3 card).
• Video-MMMU / VideoMME — long-form video understanding benchmarks; MiniMax reports Video-MME at 512 frames, measuring comprehension of extended video input (M3 launch blog).
• USAMO 2026 / IMO 2025 — olympiad proof-based mathematics contests, scored out of 42 points, measuring frontier mathematical reasoning (MiniMax M3 vendor benchmark table).
• tau2-bench — customer-service style tool-use benchmark with simulated users and domain policies (retail/telecom splits); named among agent benchmarks in MiniMax's M-series report but not scored for M3 (MiniMax-M2 series report).
• LiveCodeBench — contamination-resistant competitive-programming benchmark using recently published contest problems; not reported by MiniMax for M3 (n.a.).
• AIME / GPQA Diamond / MMLU-Pro / HLE — competition math, graduate-level science QA, broad multi-domain knowledge, and frontier-difficulty open knowledge respectively; MiniMax defines them this way in the M-series report but publishes no M3 numbers for them (MiniMax-M2 series report).