Qwen3.5-397B-A17B

Overview

Qwen3.5-397B-A17B is the first open-weights model released in Alibaba's Qwen3.5 series, announced by the Qwen team as a native vision-language foundation model with 397B total parameters and 17B activated per forward pass (Alibaba Cloud "Qwen3.5: Towards Native Multimodal Agents"). The official announcement blog is dated February 2026 (Qwen blog), with third-party trackers placing the public release in mid-February 2026 — February 16, 2026 per OpenRouter and February 17, 2026 per Artificial Analysis. Weights are published on Hugging Face as a post-trained checkpoint compatible with Transformers, vLLM, SGLang and KTransformers, with a hosted counterpart (Qwen3.5-Plus) offering a 1M-token default context, built-in tools and adaptive tool use (Hugging Face model card). The repository ships the Apache License, Version 2.0 (LICENSE file on Hugging Face), also reported as Apache 2.0 by Artificial Analysis.

Positioning is explicitly agentic and multimodal: the vendor frames the model as targeting "reasoning, coding, agent capabilities, and multimodal understanding" (Alibaba Cloud blog), and the distribution description highlights language understanding, logical reasoning, code generation, agent-based tasks, image and video understanding, and GUI interaction, with "robust code-generation and agent capabilities" and strong generalization across agent tasks (OpenRouter). Qwen reports that early-fusion multimodal training achieves cross-generational parity with Qwen3 while outperforming the Qwen3-VL line on reasoning, coding, agents and visual understanding (Hugging Face model card). Independent evaluation placed it at 45 on the Artificial Analysis Intelligence Index — #3 among open-weights models at publication — with a GDPval-AA agentic ELO of 1,221 versus 860 for Qwen3 235B (Artificial Analysis).

On technique, the model unifies thinking and non-thinking behaviour in one checkpoint: it "operate[s] in thinking mode by default," emitting <think>...</think> content before the answer, and unlike Qwen3 it drops the /think and /nothink soft switches, so non-thinking responses are obtained via API parameters instead (Hugging Face model card); Artificial Analysis notes this reverses Qwen3's split instruct/thinking variants (Artificial Analysis). The checkpoint is trained with multi-token prediction (MTP) across multiple steps (Hugging Face model card), listed by Qwen as an efficiency feature of the underlying Qwen3-Next architecture (Alibaba Cloud blog), and serving stacks expose it directly as MTP speculative decoding, e.g. {"method": "qwen3_5_mtp", "num_speculative_tokens": 3} (vLLM Ascend docs). Training used a native FP8 pipeline over activations, MoE routing and GEMMs with BF16 preserved in sensitive layers (~50% activation-memory reduction, >10% speedup), plus a fully disaggregated asynchronous RL framework with FP8 end-to-end training, rollout router replay, speculative decoding and multi-turn rollout locking for a 3×–5× end-to-end speedup (Alibaba Cloud blog).

Architecture

• Total / active parameters: 397B total, 17B activated (Hugging Face model card; Alibaba Cloud blog)
• Type: Causal language model with vision encoder (native vision-language, early text-vision fusion) (Hugging Face model card; Alibaba Cloud blog)
• Base architecture: Qwen3-Next — hybrid linear attention (Gated Delta Networks) + high-sparsity sparse MoE (Alibaba Cloud blog)
• Layers: 60 (num_hidden_layers: 60) (Hugging Face model card; config.json)
• Hidden layout: 15 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE)); config shows three linear_attention layers per one full_attention layer, full_attention_interval: 4 (Hugging Face model card; config.json)
• Hidden dimension: 4096 (Hugging Face model card)
• Gated DeltaNet (linear attention): 64 value heads, 16 QK heads, head dim 128, conv kernel dim 4 (Hugging Face model card; config.json)
• Gated Attention (full attention): 32 Q heads, 2 KV heads, head dim 256, output gating enabled (attn_output_gate: true), no attention bias (Hugging Face model card; config.json)
• Rotary position embedding dimension: 64 (Hugging Face model card)
• MoE experts: 512 experts, 10 routed + 1 shared activated per token, expert intermediate dimension 1024 (Hugging Face model card; config.json)
• Vocabulary / embeddings: token embedding and LM output 248,320 (padded); Qwen describes the vocabulary as expanded to 250k from 150k, giving 10–60% encoding efficiency gains (Hugging Face model card; Alibaba Cloud blog)
• Context window: 262,144 tokens native (max_position_embeddings: 262144), extensible to 1,010,000 tokens; Qwen3.5-Plus defaults to 1M (Hugging Face model card; config.json)
• Precision: released weights dtype: bfloat16 (tensor types BF16/F32); an FP8-quantized repo uses fine-grained FP8 with block size 128; training used a native FP8 pipeline with BF16 retained in sensitive layers (config.json; Hugging Face model card; Qwen3.5-397B-A17B-FP8 card; Alibaba Cloud blog)
• Novel components: MTP trained with multi-steps, exposed for speculative decoding in serving (qwen3_5_mtp) (Hugging Face model card; vLLM Ascend docs)
• Multilingual coverage: 201 languages and dialects (up from 119) (Alibaba Cloud blog)
• Reported efficiency: decoding throughput 8.6× / 19.0× that of Qwen3-Max at 32k / 256k context; 3.5× / 7.2× versus Qwen3-235B-A22B (Alibaba Cloud blog)
• Thinking mode: thinking on by default with <think>...</think> traces; no /think /nothink soft switch; recommended thinking sampling temperature=0.6, top_p=0.95, top_k=20 (Hugging Face model card)

Official vendor eval scores

All values below are Qwen-reported (thinking mode, post-trained checkpoint) as published on the Hugging Face model card and the official Qwen3.5 announcement blog.

Vendor scaffold caveats stated by Qwen: τ²-bench follows the official setup except the airline domain, which uses the Claude Opus 4.5 system-card fixes; search-agent benchmarks use a 256k simple context-folding strategy, and BrowseComp's 78.6 uses the discard-all strategy also used by DeepSeek-V3.2 and Kimi K2.5 (Hugging Face model card).

Benchmark explanations

• SWE-bench Verified — human-validated set of real GitHub issues; measures whether the model's agentic patch actually resolves the issue and passes tests (SWE-bench Verified leaderboard reference on the model card).
• SWE-bench Multilingual — same issue-resolution task extended beyond Python to repositories in multiple programming languages (Alibaba Cloud blog).
• Terminal Bench 2 — end-to-end agentic terminal tasks: the model must drive a shell to complete real command-line workloads (Alibaba Cloud blog).
• TAU2-Bench (τ²-bench) — multi-turn tool-using customer-service style agent tasks across domains (including airline), scoring task completion under policy constraints (Hugging Face model card).
• BFCL-V4 — Berkeley Function Calling Leaderboard v4; accuracy of function/tool call selection and argument construction (Alibaba Cloud blog).
• BrowseComp — hard web-browsing research questions requiring an agent to search, navigate and synthesize evidence over many hops, here with context-management strategies (Hugging Face model card).
• LiveCodeBench v6 — contamination-controlled competitive-programming problems collected over time; measures code generation correctness (Alibaba Cloud blog).
• AIME 2026 — American Invitational Mathematics Examination 2026 problems; short-answer competition math reasoning (Alibaba Cloud blog).
• HMMT Feb 25 / Nov 25 — Harvard-MIT Mathematics Tournament problem sets from the February and November 2025 contests; olympiad-level math accuracy (Alibaba Cloud blog).
• GPQA (Diamond) — graduate-level, Google-proof multiple-choice questions in biology, chemistry and physics written by domain PhDs (Hugging Face model card).
• MMLU-Pro — harder, reasoning-heavy revision of MMLU with expanded answer options, testing broad academic and professional knowledge (Alibaba Cloud blog).
• HLE / HLE-Verified — Humanity's Last Exam, extremely difficult expert-written cross-domain questions; HLE-Verified is a revised version with a component-wise verification protocol and error taxonomy (Hugging Face model card).
• MMMU — massive multi-discipline multimodal understanding: college-level questions requiring joint image and text reasoning (Alibaba Cloud blog).

Caveats

• Neither the Qwen blog nor the model card states an exact calendar release day; only "February 2026" is official. Third-party dates disagree (Feb 15 / 16 / 17, 2026).
• The model card does not carry an explicit license statement in prose; Apache 2.0 is confirmed from the repository LICENSE file.
• Alibaba has not published a formal Qwen3.5 tech report; the architecture facts on this page come from the model card, its config.json, and the announcement blog.
• Vendor benchmark numbers use Qwen's own scaffolds and context-management strategies; the two BrowseComp figures illustrate how sensitive agentic scores are to harness choice.