DeepSeek R1 0528

Overview

DeepSeek-R1-0528 is a minor version upgrade of DeepSeek-AI's open-weights reasoning model DeepSeek-R1: per the official model card, "the DeepSeek R1 model has undergone a minor version upgrade, with the current version being DeepSeek-R1-0528," which "significantly improved its depth of reasoning and inference capabilities by leveraging increased computational resources and introducing algorithmic optimization mechanisms during post-training," with overall performance described as approaching leading models such as O3 and Gemini 2.5 Pro (DeepSeek-R1-0528 model card). DeepSeek's own release note, dated May 28, 2025, summarizes the update as improved benchmark performance, enhanced front-end capabilities, reduced hallucinations, and support for JSON output and function calling, with open-source weights on Hugging Face (DeepSeek-R1-0528 Release).

Positioning is reasoning-first, with strong secondary emphasis on code and agentic tool use. The card attributes the gains to deeper thinking: AIME 2025 accuracy rose from 70% to 87.5%, with average reasoning length increasing from about 12K tokens per question in the previous version to about 23K tokens in R1-0528; the release also cites a reduced hallucination rate, enhanced function-calling support, and a better "vibe coding" experience (DeepSeek-R1-0528 model card). Usage recommendations changed relative to earlier R1: a system prompt is now supported, and it is no longer necessary to prepend <think>\n to force thinking mode; the official web/app temperature is 0.6 (DeepSeek-R1-0528 model card).

The underlying technique lineage is DeepSeek-R1's reinforcement-learning recipe: the R1 paper argues that LLM reasoning ability can be incentivized through pure RL without human-labeled reasoning trajectories, producing emergent self-reflection, verification and dynamic strategy adaptation (DeepSeek-R1 paper, arXiv:2501.12948). R1 (671B total / 37B active MoE) is trained on top of DeepSeek-V3-Base (DeepSeek-R1 model card), whose architecture adopts Multi-head Latent Attention (MLA) and DeepSeekMoE with an auxiliary-loss-free load-balancing strategy and a multi-token-prediction objective usable for speculative decoding (DeepSeek-V3 model card). Both the code repository and model use are under the MIT License, and the R1 series supports commercial use and distillation (DeepSeek-R1-0528 model card). DeepSeek also distilled R1-0528's chain-of-thought into Qwen3-8B Base to produce DeepSeek-R1-0528-Qwen3-8B (DeepSeek-R1-0528 model card).

Architecture

• Total parameters: 671B; activated per token: 37B (DeepSeek-R1 series table) (DeepSeek-R1 model card); the Hugging Face repo metadata lists 685B params, which for DeepSeek-V3 is explained as 671B main weights + 14B MTP module weights (DeepSeek-R1 model card, DeepSeek-V3 model card)
• Architecture family: Mixture-of-Experts; architectures: ["DeepseekV3ForCausalLM"], model_type: deepseek_v3 (DeepSeek-R1 model card, config.json)
• Base model: DeepSeek-R1 and R1-Zero are trained based on DeepSeek-V3-Base (DeepSeek-R1 model card)
• Attention: Multi-head Latent Attention (MLA), adopted from the DeepSeek-V3/V2 architecture (DeepSeek-V3 model card); the R1-0528 config exposes MLA ranks kv_lora_rank: 512, q_lora_rank: 1536, qk_nope_head_dim: 128, qk_rope_head_dim: 64, v_head_dim: 128 (config.json). Note: the R1-0528 card itself does not name the attention mechanism (model card)
• Layers: num_hidden_layers: 61, with first_k_dense_replace: 3 (first three layers dense rather than MoE) and num_nextn_predict_layers: 1 (MTP block) (config.json)
• Attention heads: num_attention_heads: 128, num_key_value_heads: 128, hidden_size: 7168, dense intermediate_size: 18432 (config.json)
• MoE experts: n_routed_experts: 256, n_shared_experts: 1, num_experts_per_tok: 8, moe_intermediate_size: 2048, moe_layer_freq: 1, grouped routing with n_group: 8 / topk_group: 4, scoring_func: sigmoid, topk_method: noaux_tc, routed_scaling_factor: 2.5 (config.json)
• Load balancing: auxiliary-loss-free load-balancing strategy (DeepSeek-V3 lineage; noaux_tc in config) (DeepSeek-V3 model card, config.json)
• Context window: the DeepSeek-R1 series table lists 128K (DeepSeek-R1 model card); the R1-0528 config sets max_position_embeddings: 163840 with YaRN rope scaling (factor: 40 over original_max_position_embeddings: 4096, rope_theta: 10000) (config.json)
• Precision / quantization: torch_dtype: bfloat16 with an FP8 quantization config (quant_method: fp8, fmt: e4m3, dynamic activation scheme, weight_block_size: [128,128]); repo tensor types for R1 are listed as BF16, F8_E4M3, F32 (config.json, DeepSeek-R1 model card)
• Vocabulary: vocab_size: 129280, hidden_act: silu, rms_norm_eps: 1e-06, tie_word_embeddings: false (config.json)
• Multi-token prediction: MTP training objective from the V3 lineage, also usable for speculative decoding (DeepSeek-V3 model card); num_nextn_predict_layers: 1 in the R1-0528 config (config.json)
• Reasoning budget: maximum generation length used in DeepSeek's own evals is 64K tokens; average AIME reasoning length ~23K tokens per question (up from ~12K in R1) (model card)
• Distilled sibling: DeepSeek-R1-0528-Qwen3-8B has an architecture identical to Qwen3-8B but shares R1-0528's tokenizer configuration (model card)
• License: MIT (license: mit in the card front matter); commercial use and distillation supported (model card)

Official vendor eval scores

All values are DeepSeek-reported for DeepSeek-R1-0528 on its official Hugging Face model card. Eval settings: max generation length 64K, temperature 0.6, top-p 0.95, 16 samples per query for pass@1; SWE-Verified uses the Agentless framework; HLE is evaluated on text-only prompts; GPT-4.1 plays the user role in Tau-bench (model card).

Benchmark explanations

• MMLU-Pro — a harder, cleaned multi-subject multiple-choice exam suite measuring broad academic and professional knowledge and reasoning, scored as exact match (model card).
• MMLU-Redux — a relabeled/error-corrected version of MMLU, measuring the same broad knowledge with cleaner ground truth (model card).
• GPQA-Diamond — the hardest subset of graduate-level physics, chemistry and biology questions written to resist simple web lookup; measures expert scientific reasoning (model card).
• Humanity's Last Exam (HLE) — an extremely difficult expert-written exam across many disciplines used as a frontier-difficulty ceiling; DeepSeek evaluated text-only prompts (model card).
• SimpleQA — short factual questions testing parametric factual recall and calibration; measures the share answered correctly (model card).
• FRAMES — multi-document retrieval-and-reasoning questions requiring synthesis of several sources into one factual answer (model card).
• LiveCodeBench (2408–2505) — competitive-programming problems drawn from contests in a fixed recent window to limit contamination; measures pass@1 code correctness (model card).
• Codeforces-Div1 — an Elo-style rating estimated from Division-1 competitive-programming contest performance rather than a percentage (model card).
• SWE-bench Verified — human-validated real GitHub issues where the model must generate a patch passing the repo's tests; evaluated here with the Agentless framework (model card).
• Aider-Polyglot — multi-language code-editing tasks in the Aider harness, measuring the ability to apply correct edits across programming languages (model card).
• AIME 2024 / AIME 2025 — American Invitational Mathematics Examination problem sets: hard short-answer competition math, a core reasoning-depth test (model card).
• HMMT 2025 — the Harvard-MIT Math Tournament 2025 problem set, olympiad-style short-answer math (model card).
• CNMO 2024 — the 2024 Chinese National High School Mathematics Olympiad, measuring olympiad math in a Chinese-language setting (model card).
• BFCL_v3_MultiTurn — the Berkeley Function-Calling Leaderboard v3 multi-turn split, measuring correct tool/function invocation across a dialogue (model card).
• Tau-Bench (Airline / Retail) — simulated customer-service agent tasks requiring tool use and policy adherence over multi-turn dialogue; GPT-4.1 played the user (model card).
• Terminal-Bench — multi-step shell/terminal agent tasks; not reported by DeepSeek for R1-0528 (model card).
• tau2-bench — the successor to Tau-Bench with additional domains and a dual-control user simulator; not reported by DeepSeek for R1-0528 (model card).
• BrowseComp — hard-to-find information-seeking questions requiring persistent web browsing; not reported by DeepSeek for R1-0528 (model card).