Introduction

DeepSeek-V3.2 is easiest to understand as a coordinated attempt to make long reasoning and tool use both capable and affordable. The model changes how attention reads a long history, how reinforcement learning can be scaled without destabilizing a mixture-of-experts network, and how training environments teach an assistant to act rather than merely answer.

This book develops that story from first principles. Each chapter defines its prerequisites, explains one part of the system, works through the questions that part raises, and connects it to the next. You can read it in order or enter through a particular subject. The equations are accompanied by intuition, examples, and explicit assumptions.

The central source is DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models, arXiv:2512.02556v1, submitted on 2 December 2025. The complete TeX archive and its extracted files are preserved in this directory's parent. The book describes that report's experiments, supplemented by revision-pinned official configuration, inference, and encoding files. It is not a live leaderboard or a description of whatever model a changing API alias currently serves.

What kind of book this is

This is an explanatory technical report, not a training reproduction. Its aim is to let a reader understand the design well enough to reason about alternatives, spot misleading interpretations, and investigate implementation details.

Throughout the book:

• Reported means the authors or official release files state it.
• Derived means it follows from arithmetic or a mathematical argument whose assumptions are given.
• Interpretation means an explanatory hypothesis, practical implication, or proposed research direction.
• Illustrative means a deliberately small example, not a measurement of the released model.

Standard background explanations do not always carry one of these labels. When a statement could be confused with an experimental result, the distinction is made explicit. Missing recipe details are left missing.

Contents

Reading paths

For the architecture, read 2–7 and 22. For reinforcement learning, read 8–12 and 21. For agents, read 13–18, 20, and 23. For evaluation and model selection, read 1, 7, 19–21, and 24. Chapter 25 is a reference rather than a prerequisite.

Every path benefits from Chapter 1's distinction between DeepSeek-V3.1-Terminus, DeepSeek-V3.2-Exp, the final DeepSeek-V3.2, and Speciale. Their names refer to different stages or variants, not interchangeable measurements.

Source and numerical conventions

The source atlas records file locations, revisions, and SHA-256 checksums. Chapter source notes identify the report section and TeX file behind the discussion. Official supplemental files are preserved under ../_sources/; their content is evidence about that pinned release, not proof of the private production system.

Unless specified otherwise, K, M, and B denote powers of 1,000. For the training token-count arithmetic, the reported totals imply that 128K means 131,072 tokens. Context-window discussions retain the report's 128K label; implementation files have different configured maxima, discussed in Chapters 3 and 22. GB denotes $10^9$ bytes; GiB denotes $2^{30}$ bytes. Scores expressed as percentages differ by percentage points, while Codeforces ratings are rating points.

The source archive is preserved as ../2512.02556v1-source.tar.gz, with ../main.tex as its entry point. Original figures are retained under ../figures/; readable raster versions of selected PDF figures appear under assets/. Figures reproduced in the book are explicitly identified as the authors' figures. Small numerical examples and pseudocode are the book's teaching material.

1. The Problem and the Model Family

A language model can be good at answering a short question and still struggle to finish a complicated job. A software repair may require reading files, designing a patch, running tests, and revising a mistaken assumption. A research question may require searching several sources and resolving contradictions. A hard mathematical problem may require a long sequence of attempts before a useful idea appears.

These tasks expose three constraints at once: the model needs useful reasoning strategies, enough computational budget to practice them during training, and an affordable way to process the growing history during use. DeepSeek-V3.2 addresses all three. The paper's organizing idea is that architecture, learning, and agent environments reinforce one another.

Why long reasoning changes the economics

An autoregressive model generates one token after another. Every new token conditions on the preceding sequence. If the model writes a 20,000-token solution rather than a 500-token answer, it pays for many more forward passes. Those passes also operate on a progressively longer history.

An agent adds another source of growth: tool outputs. A search result, stack trace, or repository file becomes part of the context that the model reads. Even a concise assistant can accumulate a very long transcript if its tools return large responses.

Dense attention makes every query inspect every permitted preceding position. Processing a whole long prompt therefore involves a number of query–key interactions that grows quadratically with sequence length. During decoding, one additional query scans a history whose size grows with each step. This makes long trajectories costly both when serving users and when generating reinforcement-learning rollouts.

The paper introduces DeepSeek Sparse Attention, or DSA, to reduce the expensive part of that reading. A lightweight selector scores the history, and the main attention reads a limited set of token entries. The selector itself still scans the history. That distinction becomes central in Chapter 7.

Why architecture alone cannot produce an agent

Suppose attention suddenly becomes much cheaper. A model can now afford to read a larger transcript, but it may still call the wrong tool, forget a constraint, or stop before confirming success. Computational capacity and learned behavior are different things.

The authors argue that open models had invested too little in post-training compute and lacked sufficiently diverse agent training tasks. Their remedy is a more stable reinforcement-learning recipe and a task-synthesis pipeline that supplies unfamiliar, demanding environments with automatic verifiers.

Reported: the reinforcement-learning budget described by the final report exceeds 10% of pre-training cost. This is a relative compute-investment statement. The report does not disclose enough accounting information to turn it into an exact dollar amount or a complete GPU-hour budget.

Interpretation: cheaper long-context attention can make long rollouts less expensive, and stable optimization can make additional rollout compute more useful. Together, those changes make investing in harder training tasks more practical. The report does not isolate this entire causal chain with a single controlled experiment.

Four names that must stay separate

The architectural comparison is primarily Terminus versus Exp. The capability comparison in the final report is the final V3.2 versus other reasoning and agent models. Mixing these comparisons leads to a common mistake: attributing the final model's reasoning improvements entirely to DSA.

Reported: V3.2 uses the same architecture as V3.2-Exp. Relative to Terminus, the architectural change is DSA, introduced through continued training. The final release's stronger capabilities also reflect its post-training program.

Speciale needs another distinction. It is trained exclusively on reasoning data with a reduced length penalty and incorporates the dataset and reward method of DeepSeekMath-V2 for proofs. The official model card states that Speciale does not support tool calling. It should not be understood as the universal replacement for the main model.

Three forms of sparsity and selection

Readers often hear “sparse model” and assume there is one mechanism. Here there are several:

1. Expert sparsity: a token activates a subset of feed-forward experts. This reduces activated parameter computation.
2. Attention sparsity: a query reads a selected subset of historical token entries. This reduces core attention work.
3. Sampling truncation: top-p or top-k generation restricts the vocabulary actions considered during sampling. This affects rollout quality and the probability distributions used in RL.

These operate on different objects. Selecting eight experts does not select eight historical tokens. Selecting 2,048 attention entries does not restrict the output vocabulary to 2,048 words. Keeping a sampling mask is unrelated to keeping a causal attention mask, even though both are called masks.

What “open” provides here

The release provides model weights and implementation material. The official model card and license snapshot identify the weights and repository as MIT licensed. This supports inspection and local deployment.

That openness does not imply the complete training corpus, reinforcement-learning infrastructure, reward models, and environment-generation pipeline are reproducible from the release. A technical report can reveal important mechanisms without disclosing every ingredient needed to rebuild the checkpoint.

For this book, an open release is an opportunity to cross-check architectural descriptions against configuration and code. It does not remove the need to assess benchmark protocols or distinguish reported evidence from independent verification.

How to read the paper's headline claims

“Comparable to GPT-5” summarizes a collection of results, not equality on every task. V3.2 has strong math and agent scores, but the report also shows weaknesses in knowledge-heavy benchmarks and complex tool tasks. Speciale reaches higher scores on several reasoning tests at the cost of longer outputs and a narrower intended use.

“Gold-medal performance” means scores under an evaluation protocol reach competition thresholds. The report's protocol includes candidate generation, filtering, and, for proofs, iterative self-verification. Those details matter as much as the final score.

“Reduced attention complexity” is specifically about the main attention's expensive reading stage. The smaller indexer remains quadratic when evaluating a whole sequence. The authors acknowledge this directly.

The most useful reading habit is to ask what object a claim describes: the architecture, a checkpoint, a trained behavior, an inference framework, or an entire evaluation pipeline. Each chapter keeps that object explicit.

A question to carry forward

Imagine a model that spends half its serving time in attention and half elsewhere. Even an infinitely fast attention implementation could improve total speed by at most a factor of two under those assumptions. Now imagine that cheaper attention allows longer reasoning, which increases output length. The architecture may be more efficient per token while the full task becomes more expensive.

This tension is the book's recurring theme: the right unit of analysis is often a successfully completed task, not one token or one isolated benchmark score.

Sources: report §1 and §3, sections/01-0-intro.tex and sections/02-post.tex; official model card at the preserved revision. Continue with the foundations.

2. The Foundations the Report Assumes

DeepSeek-V3.2 is an autoregressive transformer trained to predict tokens and then adapted to produce useful answers and actions. This chapter supplies the vocabulary needed to follow the architecture and reinforcement-learning chapters. No familiarity with the DeepSeek family is required.

Tokens are the model's units

A tokenizer turns text into integer identifiers. A token might be a word, part of a word, punctuation, whitespace, or a special marker. Token counts are therefore not word counts, and different languages or tokenizers can represent the same content with different numbers of tokens.

The model maps each identifier to a learned vector, called an embedding. Successive layers transform these vectors into contextual representations. The final representation is projected to one logit per vocabulary item. A softmax turns logits into probabilities:

Here $\mathcal V$ is the vocabulary, $z_v$ is the logit for token $v$, and $x_{\le t}$ is the available prefix. Generation chooses a token according to a decoding rule, appends it, and repeats.

An output token may be ordinary prose or part of a serialized tool call. The probability model is still a next-token predictor; the external application determines whether a completed sequence is treated as text or executed as an action.

A transformer block has two main jobs

Attention lets a token read other positions. A feed-forward network transforms information within that token's vector. Residual connections add each module's output to an existing stream, helping preserve information and train deep networks.

An illustrative pre-normalized block is

Normalization stabilizes the scale of the inputs. The released inference implementation uses RMS normalization around its attention and feed-forward modules. Exact projection details matter later; this simplified form explains the division of labor.

Attention is a communication operation across positions. The feed-forward network is a local transformation, although its input already contains information obtained through attention. In a mixture-of-experts model, the feed-forward stage routes each token to selected expert networks.

Query, key, and value

For one attention head, a token at position $t$ produces a query $q_t$. A position $s$ produces a key $k_s$ and value $v_s$. The query–key dot product measures compatibility. Attention normalizes these compatibilities and forms a weighted sum of values:

The key helps decide where to read; the value supplies what is read. The query states what information the current position seeks. These are learned numerical representations, not explicit database fields.

Multiple heads use different projections and can attend differently. The model combines their outputs through an output projection. A head should not automatically be assigned a fixed human-readable function such as “syntax” or “arithmetic”; such interpretations require evidence.

Causality prevents looking ahead

When predicting a token, the model cannot use future tokens. A causal mask sets forbidden attention logits to negative infinity, so their softmax probabilities become zero.

For a prompt of length $L$, the number of allowed position pairs, including self-attention to the current input position, is

This is why dense causal attention still has quadratic pair count even though it reads only the lower triangle of an attention matrix.

The exact indexing convention can shift by one: a state at input token $t$ predicts token $t+1$. This does not change the computational argument. What matters is that no future input enters the prediction.

Prefill and decode are different workloads

Prefill processes the existing prompt. Many positions can be handled in parallel because the input tokens are already known, with a causal mask enforcing information flow.

Decode generates new tokens sequentially. After one token is chosen, the next state depends on it. This creates a serial component even when operations within one forward pass are highly parallel.

A KV cache stores the historical attention state so the model does not recompute all previous key and value projections for every new token. Ordinary multi-head attention stores separate historical vectors for its heads. MLA stores a smaller latent representation from which head-specific operations can be recovered algebraically.

The cache saves recomputation; it does not remove the need to read relevant history. DSA changes that reading budget. MLA and DSA therefore solve related but distinct problems.

Cross-entropy gives the base model its first objective

During language-model training, the known next token is the target. A standard loss is

This trains the model to assign higher probability to text in its training distribution. It can acquire syntax, facts, procedures, and patterns of reasoning through prediction. However, next-token prediction does not directly encode a user's definition of success for an interactive task.

Supervised fine-tuning, or SFT, uses demonstrations formatted as desired assistant behavior. Distillation uses outputs or distributions from teacher models to train a student. In this report, specialists produce domain-specific data for a final general checkpoint.

Reinforcement learning, or RL, evaluates sampled behavior with rewards and changes the policy to make more successful behavior likelier. The report uses GRPO, which compares multiple outputs for the same prompt.

A policy is a probability distribution over actions

For a language model, a policy $\pi_\theta$ assigns probabilities to next tokens. An entire response has probability

Here $q$ is the prompt and $o$ is the generated output. A response can have thousands of token-level decisions but receive one final outcome reward. GRPO assigns a response-level advantage to its tokens, as Chapter 9 explains.

For an agent, the trajectory alternates model actions and environmental observations. The model generates a tool request; a runtime executes it; the resulting observation becomes context; the model acts again. Tool output is conditioning information, not a token action sampled by the policy. An implementation must distinguish those roles when computing an RL loss.

Reward, advantage, and verifier

A reward evaluates an outcome. A verifier checks whether an answer or environment state meets stated conditions. An advantage says whether a sampled output performed better or worse than a baseline.

Illustratively, four responses score $[1,1,0,0]$. Their mean is $0.5$, so mean-centered advantages are $[0.5,0.5,-0.5,-0.5]$. The update encourages the better responses relative to the worse ones.

The verifier need not understand all the reasoning. A test suite can check a software patch. A constraint checker can validate an itinerary. This makes tasks with difficult search but cheap verification especially useful for scalable RL.

Verification quality still limits reward quality. Passing an incomplete test suite does not prove a patch is universally correct. A malformed constraint checker can reward an invalid solution. The reward is only as faithful to the intended task as its measurement procedure.

Three questions that organize the remaining chapters

The architecture asks: which computations and memory entries are necessary for a useful prediction? The learning procedure asks: which behaviors should become more likely, and how can updates remain stable? The agent system asks: what context, tools, and verifiers make success observable?

None of these questions can replace the others. A capable probability model can fail inside a poorly formatted runtime. A clean verifier can train the wrong behavior if its incentives are incomplete. Efficient attention can preserve information access while the agent still exceeds its finite context window.

Sources: background developed from the transformer and policy concepts used in report §§2–3; released model.py, classes MLA, Block, and Transformer. See the pinned official implementation. Continue with the backbone and experts.

3. Inside the Inherited Backbone

DeepSeek-V3.2 inherits a large mixture-of-experts transformer backbone. The headline architectural change is sparse attention, but understanding its costs requires knowing what remains around attention. Every token still passes through embeddings, projections, normalization, feed-forward computation, and a vocabulary output head.

This chapter uses the official release configuration to fill in details that the short architecture section assumes. Configuration facts describe the preserved checkpoint files; they are not new claims about an undisclosed training system.

A map of the released configuration

The configuration also records one next-token-prediction auxiliary layer. This field identifies inherited multi-token-prediction machinery; it does not mean the released demo generates two independent final answers at each step. Speculative decoding requires additional inference logic, and the report does not identify it as the new V3.2 contribution.

Why 128 heads do not imply a huge stored KV cache

In conventional multi-head attention, each head has its own keys and values. With 128 heads, directly storing those vectors can be expensive. V3.2 uses MLA, which stores a 512-dimensional latent and a separate positional key component instead. Head-specific transformations are recovered during computation.

The configuration's num_key_value_heads: 128 must therefore be read within the MLA architecture. It does not by itself describe the physical cache layout of a standard grouped-query-attention model. Generic memory calculators that multiply this field by conventional per-head KV sizes can overestimate or mischaracterize the cache.

Chapter 4 explains the algebra that makes this possible. For now, distinguish a logical head structure from a stored historical representation.

What an expert actually computes

An expert is a learned feed-forward network. The demo uses a gated form that can be written schematically as

The two input projections create intermediate vectors. SiLU applies a nonlinear activation to one; elementwise multiplication gates it with the other. The output projection maps the result back to the residual width.

The router chooses a small subset $\mathcal E_t$ of experts for token $t$. The MoE output combines them and adds a shared expert contribution:

Here $g_{t,e}$ is a learned routing weight. The shared expert is computed for every token; the routed experts vary with the input.

Routing in the demo

The released configuration uses sigmoid scores, eight expert groups, and selection from four groups. The demo applies learned routing biases for selection, ranks groups using the sum of their top two biased expert scores, restricts candidates to the chosen groups, and then selects eight routed experts.

The combination weights use the original unmodified scores for the selected experts, normalize them, and multiply by a routing scale of 2.5. Consequently, the values used to choose experts need not equal the values used to mix their outputs.

This distinction is useful when reading “Keep Routing” in Chapter 11. The active expert identities are a discrete part of the computation. Small numerical differences in scores can change those identities abruptly, even if the underlying score change looks small.

An expert is also not necessarily a named domain specialist. A routed FFN may develop patterns of specialization, but the paper does not provide an interpretability map assigning each expert to mathematics, Chinese, or Python.

Total parameters and active parameters

MoE separates stored model capacity from per-token activation. The inherited backbone is commonly described at approximately 671B total parameters and 37B activated parameters; these are backbone-scale figures, not a fresh exact accounting of every added V3.2 indexer parameter.

A transparent calculation explains their scale. Ignoring biases and norms, one expert has

weights. Across 58 MoE layers, 256 routed experts per layer contribute

weights. Per token, eight routed experts plus one shared expert contribute roughly

activated FFN weights. Attention, dense FFNs, embeddings, output projections, and other parameters supply the remainder of a broader count.

These calculations are derived approximations, not exact checkpoint totals. They explain why a model can store hundreds of billions of parameters while activating far fewer for one token.

They also explain why “37B active” does not make the full model fit into the memory footprint of a dense 37B checkpoint. Different tokens can need different experts. A deployment must keep the required weights accessible, usually distributed across devices.

Neural experts versus post-training specialists

The report uses another kind of specialization: complete specialist models trained for a domain, then used to generate distillation data. Those are full checkpoints. They are not the 256 FFN experts inside one transformer layer.

Keeping these separate prevents a misleading picture in which a router directly selects “the math teacher” for a question. Specialist distillation changes the student's parameters through training. The student's MoE router operates inside that trained network during inference.

The context-length discrepancy

The report's training and evaluations use 128K context. The preserved Hugging Face configuration sets max_position_embeddings to 163,840, while the demo's ModelArgs default maximum sequence length is only 16,384. Its positional settings include YaRN scaling from an original 4,096-token reference with factor 40.

These fields are not interchangeable guarantees. A positional configuration can describe a larger configured range than the paper's evaluated window. A runtime can allocate a smaller cache. The book uses 128K when discussing reported experiments and gives exact lengths when doing cache or training arithmetic.

What the backbone contributes to the overall story

MoE makes a high-capacity model less expensive to execute per token than activating every stored expert. MLA makes attention history cheaper to store. DSA makes expensive history reading selective. Post-training teaches the resulting model which reasoning and action sequences succeed.

Those mechanisms operate at different stages of the computation. Their benefits can combine, but none removes all remaining costs. This is why a complete model description must extend beyond the new attention mechanism.

Sources: report §2, sections/01-method.tex; released configuration; pinned inference-config.json and model.py, classes Gate, MoE, and Block; inherited backbone background in the local DeepSeek-V3 report. Continue with MLA.

4. Multi-Head Latent Attention

Multi-head latent attention, or MLA, compresses the attention state stored for each historical token. DeepSeek-V3.2 inherits MLA and places sparse token selection around it. Understanding MLA explains both what DSA selects and why sharing a selected entry across many query heads is useful.

The essential idea is to store a small latent from which head-specific keys and values can be obtained through linear maps. Because those maps are linear, part of their work can be moved to the query or output side.

From hidden vector to latent memory

Let $h_s\in\mathbb R^d$ be the hidden representation at historical position $s$. In a simplified notation,

The preserved release uses a latent width of 512. For head $i$, learned matrices expand that latent into a content key and value:

The keys and values can differ by head even though they share a source latent. Sharing a representation does not force all heads to read history in the same way.

The query has its own compressed representation, with width 1,536 in the release. It expands into head-specific query components. Query compression reduces projection structure; historical KV compression directly affects cache size.

Absorbing the key projection

Suppose the content query for a head is $q_{t,i}^C$. Its compatibility score with position $s$ is

Rearrange the multiplication:

The model can transform the current query once into a latent-space query $q_{t,i}^{A}$, then compare it directly with the stored latents. It does not need to reconstruct every historical head-specific content key for every decode step.

This is a change in how the computation is arranged, not a claim that all queries become identical. Each head has its own absorbed query.

Absorbing the value projection

If attention weights are $a_{t,s,i}$, the head output is

Linearity gives

The attention kernel first sums latents using that head's weights. It then transforms one aggregated latent into the head's output. The output projection can also be combined algebraically with the value expansion, subject to the execution strategy.

This avoids reconstructing a head-specific value vector for every historical entry during decode. The operation still produces distinct outputs for distinct query heads because the weights and transformations differ.

Position needs separate treatment

Attention needs to distinguish positions. Rotary positional embedding, RoPE, rotates query and key coordinates according to position. The dot product can then depend on relative position.

MLA separates a content component from a positional component. A simplified score is

followed by scaling, masking, and softmax. The release uses 64 positional coordinates and 512 latent content coordinates in the shared historical state.

Why not absorb everything into one fixed matrix? The positional rotations depend on token position. Moving them freely across learned projections is not the same algebraic operation as absorbing a position-independent linear key map. The separate RoPE path preserves the positional computation while allowing content compression.

The book's formulas omit some normalization and scale factors to expose the structure. Chapter 22 discusses implementation details that must be retained for numerical correctness.

MHA mode and MQA mode are execution views

The report's appendix contrasts two modes of MLA:

Reproduced from the report's Appendix A figure. The modes reorganize the same latent-attention computation.

MHA mode expands latent keys and values into head-specific representations. It can map well to conventional high-throughput attention operations during training and prefill.

MQA mode keeps the historical latent shared and absorbs transformations into the query/output computations. It is particularly useful during decode, when reading the cache efficiently matters.

“MQA mode” here does not mean MLA simply becomes ordinary multi-query attention with one conventional key and value head. It refers to the shared-latent execution of MLA, which still supports diverse query heads and head-specific transformations.

Reported: Terminus uses MHA mode for training and prefill, and MQA mode for decoding. DSA is formulated under MLA's MQA mode because sharing each selected KV entry across queries improves sparse-kernel efficiency. The report also uses masked MHA to simulate DSA for short-sequence prefill when that is faster.

A cache-size example

Assume 61 layers, a sequence of 131,072 tokens, and BF16 storage for a 512-dimensional latent plus a 64-dimensional positional key. The content-plus-position cache for one sequence would be

That is about 9.21 GB, or 8.58 GiB, before indexer memory, allocator overhead, sharding, and other runtime state. This is a derived example, not the production cache footprint.

The released demo comments say the actual deployment uses FP8 KV cache. Its demonstration simulates latent quantization by converting to FP8 and back, while retaining a higher-precision tensor for execution. A source-code tensor's storage therefore need not match production storage.

What compression does and does not mean

MLA compresses each token's feature representation. It does not merge several neighboring tokens into one entry. DSA in V3.2 selects individual token positions from this latent history.

That differs from an architecture that pools four tokens into a block summary and selects summaries. The distinction affects what “2,048 selected entries” means: here they correspond to token positions, not a fixed number of compressed multi-token blocks.

Latent compression also does not guarantee lossless reconstruction of arbitrary conventional KV vectors. The model learns to operate with this factorized representation. The algebraic equivalence is between different execution arrangements of that learned MLA structure.

Why MLA is a good foundation for DSA

Once a query has selected historical positions, a sparse kernel can load a compact shared entry at each position and reuse it across heads. Reuse improves the amount of useful work performed per memory load. Without it, a sparse pattern can require many irregular gathers of separate head-specific state.

MLA therefore supplies a favorable memory organization, while DSA supplies the position set. One compresses the representation; the other sparsifies the access pattern. Their combination is the architectural core of V3.2.

Sources: report §2.1 and Appendix A, sections/01-method.tex and sections/appendix.tex; pinned model.py, class MLA; official inference source. Continue with the indexer.

5. The Lightning Indexer and Sparse Selection

DeepSeek Sparse Attention, DSA, divides attention into a cheap ranking stage and a more expensive reading stage. The ranking stage is the lightning indexer. For each query token, it assigns scores to available historical positions. The reading stage runs MLA over the positions with the highest scores.

The separation lets the model inspect many candidates through a small representation while applying richer attention only to a selected subset. The ranking is learned, query-dependent, and performed inside each attention layer.

The architecture at a glance

Reproduced from the report's DSA architecture figure. Green paths identify the indexer and selector; the black paths are the MLA computation.

The main attention query and the indexer query are related but not identical. Both depend on the current hidden state and, in the demo, share the low-rank query latent. Their subsequent projections serve different purposes.

The historical indexer key is also separate from the main MLA latent. It is trained to help rank positions cheaply. The main latent carries the content that attention eventually reads.

The scoring equation

For current position $t$ and historical position $s$, the paper defines

The symbols have distinct jobs:

The release uses 64 indexer heads, each of width 128. The main attention has 128 heads and a different representation. The indexer's relative cheapness comes from its dimensions, shared keys, precision, and implementation, not from an absence of scanning.

ReLU sets negative dot products to zero. The authors choose it for throughput. ReLU does not make the complete aggregate score necessarily nonnegative: the displayed formula does not require the head weights $w_{t,j}^I$ to be positive. The implementation uses a linear weight projection.

A ranking example

Consider two illustrative indexer heads with weights $(0.75,0.25)$:

A top-two selector chooses A and one of B or C, subject to tie behavior. If the head weights change, the ranking changes. The same history can therefore produce different selected positions for different queries.

The main attention then calculates its own compatibility scores over those chosen positions. It might assign most weight to B even though A had the largest indexer score. The indexer proposes an available set; main attention decides the mixture within that set.

Selection is over token positions

The sparse training stage selects 2,048 KV tokens per query. If a query has fewer eligible positions, it can read all of them. For a long history, it reads at most the selected budget.

The set can be written

Using positional indices avoids an ambiguity in expressions that test whether a score value “belongs” to the top-k values: ties must still resolve to an actual set of indices.

The selected set is shared across the main query heads for a given token at that layer. Each main head can weight the set differently. Sharing the set limits head-specific retrieval freedom but makes the loaded entries reusable across heads.

Nothing in the reported prototype requires selected positions to be contiguous. A query can select an early definition, a recent result, and a distant code fragment. Nor does the report specify a separate always-on sliding window as part of this DSA prototype. Such features should not be added to a description merely because other sparse attention designs use them.

Causality still applies

The indexer must exclude future positions before selection. Otherwise it could choose a future token during training and leak the target into the current prediction.

The demo adds a causal mask to index scores during prefill. It also applies the causal restriction to main attention. When top-k is computed over a rectangular tensor containing rows with fewer eligible candidates than the tensor-wide selection count, masked future candidates can appear among filler indices; their attention must remain masked. Selection and causal validity are separate checks.

At decode time, the available cache ends at the current prefix, so there are no future entries in that slice. Correct cache indexing remains necessary.

The implementation's extra transformations

The demo adds useful details beyond the paper's compact equation. It derives indexer queries from the normalized query latent, derives keys from the hidden state with a separate projection and layer normalization, applies RoPE to part of each vector, and rotates query/key activations with a normalized Hadamard transform before FP8 quantization.

For an orthogonal transformation $R$,

Before quantization, applying the same orthogonal rotation preserves dot products. After quantization, the numerical result need not be identical. Such rotations can redistribute large coordinates and improve quantization behavior; this is an explanation of the technique, not a V3.2 ablation result.

The demo incorporates query quantization scales and head/attention scale factors into the aggregate weights. The book's scoring equation captures the conceptual operation; exact checkpoint execution requires the implementation's scales.

Why not just use the main attention scores to select?

Computing all full main-attention scores would already incur much of the expensive history-reading work that DSA wants to avoid. A separate small representation is worthwhile only if it can identify a useful subset more cheaply.

That introduces a retrieval bottleneck. A token excluded by the selector cannot receive main-attention weight at that query and layer. Main attention cannot repair a retrieval miss within the same selected set. Other layers and subsequent queries may access the information differently, but the local omission is real.

The training procedure therefore teaches the indexer to approximate the main model's attention distribution before enabling sparse reading. Chapter 6 explains how that teacher signal is constructed.

What “fine-grained” buys and costs

Selecting individual tokens avoids making a whole large block eligible merely because one token is useful. This can devote the budget more precisely to relevant positions. It also creates irregular memory accesses and top-k overhead.

Sparse arithmetic alone does not guarantee fast execution. Hardware favors regular operations and reuse. The shared MLA latent, common selected set, FP8 indexer, and optimized kernels are part of the practical design.

The architecture is best understood as a learned filter whose quality and execution efficiency must both be measured. A small set with high retrieval quality can preserve performance; a small set with poor retrieval quality can silently remove crucial evidence.

Sources: report §2.1, sections/01-method.tex; pinned model.py, class Indexer; official implementation. Continue with continued training.

6. Converting a Dense Model to Sparse Attention

DeepSeek-V3.2 does not learn its entire language model from scratch with DSA. It starts from a DeepSeek-V3.1-Terminus base checkpoint whose context has already been extended to 128K. It then introduces the indexer through two stages of continued pre-training.

This is a model-conversion problem: preserve the useful behavior of a mature network while changing which historical entries its attention can read. Enabling an untrained selector immediately would risk withholding important information before the model knows how to cope.

Stage one: preserve the reader and train the selector

During dense warm-up, all existing model parameters are frozen. Dense attention remains enabled. Only the lightning indexer learns.

For query $t$, suppose the main attention heads have distributions $a_{t,s,i}$. The authors sum across heads and normalize along the sequence dimension:

If every head distribution sums to one, this normalized sum is their average. It yields one distribution over positions, matching the indexer's shared selection decision.

The indexer's logits become a probability distribution through softmax, and the training loss is

Forward KL penalizes giving too little indexer probability to positions that the main attention uses. This makes the existing attention mechanism a teacher for the new selector.

An attention-teaching example

Suppose four positions receive main-attention target mass

An untrained indexer might assign

Its ranking is almost reversed. A top-two set would omit the position carrying 60% of the teacher's mass. Optimizing $D_{\mathrm{KL}}(p\|q)$ encourages a distribution whose ranking better matches the teacher.

The loss optimizes a distribution, while the eventual inference operation takes top-k indices. A lower KL generally indicates better distributional alignment, but it is not identical to a formal guarantee about top-k recall or retained attention mass. Those are distinct metrics.

The warm-up budget

Reported: warm-up uses learning rate $10^{-3}$, 1,000 steps, and 16 sequences of 128K tokens per step, for a stated total of 2.1B tokens.

Derived: taking 128K as 131,072,

This rounds to 2.1B. It is a short stage relative to the following adaptation stage, but not a tiny amount of text in absolute terms.

Freezing the main model keeps the teacher stable. A high learning rate for the new indexer is therefore not evidence that the entire mature backbone is being updated at that rate.

Stage two: make attention sparse and adapt the reader

After warm-up, the model enables top-k selection. Now all main-model parameters are trainable, and the network learns to predict text while reading only the selected history entries.

The selector continues to receive KL supervision, but only within its selected set:

The main attention distribution in sparse mode is defined on the selected support. For any abstract restriction of a larger distribution, it must be renormalized on that support before treating it as a KL probability distribution. The paper's notation is compact; a valid implementation needs compatible normalized distributions.

The teacher is no longer the unchanged dense model. The main network adapts under the sparse pattern, and its attention supplies a changing within-set teaching signal.

The gradient separation is deliberate

The paper says that the indexer's input is detached from the computational graph for separate optimization. The indexer receives training signal only from $\mathcal L^I$. The main model is optimized only by the language-modeling loss.

This has two implications:

1. The main representation is not changed directly to make the indexer's KL loss easier.
2. The discrete top-k operation is not described as a differentiable path from language loss into selector parameters.

One should therefore avoid calling this an ordinary end-to-end retrieval system trained through a smooth top-k relaxation. The reported recipe uses an auxiliary teacher objective for selection and language modeling for the reader.

Sharing a query latent in the forward computation does not contradict the separation. The selector can consume a detached copy of that latent; the main attention can consume the differentiable one.

The sparse adaptation budget

Reported: sparse adaptation uses learning rate $7.3\times10^{-6}$, 15,000 steps, 480 sequences of 128K tokens per step, and 2,048 selected KV tokens per query. The stated total is 943.7B tokens.

Derived:

That rounds to 943.7B. It is approximately 450 times the warm-up token count. Calling the conversion “a small added indexer” should not obscure the substantial continued-training investment used to adapt the network.

For both stages, the authors say the training distribution matches the long-context extension data used for Terminus. This supports a cleaner architectural comparison than introducing an unrelated corpus, though it is not a complete disclosure of the data recipe.

Why attention matching is a useful but imperfect teacher

Attention weights indicate how a particular layer combines values. They are not a direct ground-truth label of every token needed to solve the final task. A position with low weight in one head or layer may still matter elsewhere. Summing across heads also merges their differing preferences into one shared target.

Interpretation: the teacher gives the new indexer a strong initialization without requiring manually labeled relevant tokens. Continued language modeling then lets the main model adapt to the constrained access pattern. This is a practical compromise between maintaining a working checkpoint and learning retrieval from final-task rewards.

The sparse-stage loss supervises selected positions. It does not explicitly provide a fresh dense-attention comparison against every excluded candidate at every step. Studying retrieval errors outside the selected set would require additional measurements beyond the displayed objective.

What parity evaluation establishes

The report describes Exp as showing no substantial degradation against Terminus on standard benchmarks and unseen long-context evaluations. It also compares human preference through ChatbotArena with aligned post-training strategies.

Those results support the claim that the conversion preserves useful capabilities on the tested distributions. They do not prove equality of internal computation, absence of rare retrieval failures, or parity on every possible long document.

For a stronger analysis, one would want retained attention mass, retrieval recall on marked evidence, sensitivity to $k$, and task performance as context grows. The report's parity evidence is valuable precisely because the architecture introduces a real information-access constraint.

Sources: report §2.1.1 and §2.2, sections/01-method.tex. Continue with the cost model.

7. Computation, Memory, and Inference Cost

DSA makes the expensive attention reader sparse. It does not make every part of the transformer sparse, and it does not eliminate the indexer's full scan. The right cost model separates selection, reading, projections, experts, memory traffic, and the serial generation process.

This chapter explains why the reported long-context savings can be large while the overall asymptotic complexity remains quadratic for a whole sequence.

Main attention pair count

Let $L$ be the sequence length and $k$ the maximum selected positions. Dense causal attention has $L(L+1)/2$ allowed pairs. Sparse causal attention has at most

For $L\ge k$, this becomes

The early positions have fewer than $k$ predecessors. Once history is long enough, each query's main reading budget saturates.

With $L=131072$ and $k=2048$, the dense count is approximately 8.59 billion pairs. The sparse main-reader count is approximately 266.34 million, about 32.25 times fewer. This is pair-count arithmetic, not a claimed 32.25-fold end-to-end speedup.

For a single decode query at the end of that history, the reader visits 2,048 of 131,072 positions: a 64-fold reduction in candidate count. The full-prompt and final-query ratios differ because the full prompt includes many earlier, shorter histories.

The selector still scans

The indexer computes a score between each query and its eligible candidate positions. Its full-sequence work still grows as $O(L^2)$, but with a much smaller computational constant than dense main attention.

A schematic cost expression is

Here $a_I$ describes indexer score work, $a_A$ describes sparse main-attention work, $bL$ gathers token-local costs, and $C_{\mathrm{topk}}$ covers selection overhead. This is a teaching model, not a fitted performance equation.

Since $a_I$ is nonzero, the total remains quadratic as $L$ tends to infinity with fixed $k$. In the finite regime of the paper's experiments, a small $a_I$ can nevertheless produce substantial savings.

The distinction between an asymptotic class and a practical constant is not a technicality. Two quadratic systems can differ greatly at 128K, and a sparse system can still eventually become dominated by its quadratic selector.

Decode has a different cost expression

For one new token at history length $L$, a simplified form is

The main reader is bounded by $k$, but the indexer grows with history. Repeating this for many output tokens accumulates selector cost over increasingly long prefixes.

If a reasoning variant doubles output length, its total task cost can increase even if its main attention per step remains bounded. Output length, context length, and number of tool turns must all enter an application budget.

Sparse access does not imply sparse storage

Future queries may select any historical position. The model therefore retains the historical main latents and indexer keys. It cannot generally discard a token simply because the current query did not choose it.

A simplified indexer-key storage example uses 128 FP8 coordinates and one FP32 scale per token per layer:

That is about 1.06 GB, or 0.98 GiB, for one sequence under these assumptions. It is additional state beyond the main MLA cache. Actual layouts, scale formats, replication, sharding, and allocator overhead can change the number.

The useful distinction is that DSA reduces expensive reads without reducing historical entry count to 2,048. MLA reduces representation width; FP8 can reduce bytes per coordinate; neither turns the selected subset into the only retained memory.

Why a sparse kernel needs careful engineering

Dense attention can exploit large regular matrix operations. Token-level sparse attention needs irregular gather operations, selection, and sufficient reuse to compensate for less regularity.

DSA shares each selected latent across many query heads. This raises useful work per loaded entry. FP8 indexer computation reduces arithmetic and bandwidth costs. Optimized kernels avoid materializing large unnecessary intermediates.

The released demo is a correctness-oriented description, not proof that every operation is executed sparsely. Its prefill path can compute dense scores and then apply a sparse mask. Even its decode demonstration forms scores before masking. Production sparse kernels are necessary to realize the theoretical reduction in executed reader work.

That is why a benchmark of the demo cannot be substituted for the report's deployed-service measurements.

The paper's cost figures

Reproduced from the report's inference-cost figure. The vertical axes are cost per million tokens at the indicated token position.

Reported: the costs come from benchmarks of an actual H800 service, assuming a rental price of US$2 per GPU-hour. They are engineering cost estimates under that setup, not a current API price list.

The curves show much larger benefits at long histories than short ones. Short prefill uses a specially implemented masked MHA mode to simulate the sparse pattern efficiently. This illustrates that the fastest execution method can depend on sequence length even for one logical attention mechanism.

The plots should not be read as the cost of sending one complete million-token request. They show position-dependent token cost. An overall request estimate needs integration or aggregation over its token positions and an accounting of generated output.

Amdahl's law and the rest of the model

Suppose, illustratively, dense attention represents fraction $f$ of total time and becomes $S$ times faster while everything else stays fixed. Then

For $f=0.7$ and $S=10$, the total speedup is about 2.70, not ten. This equation ignores new selector overhead and changing bottlenecks, but it explains why attention pair reduction cannot be directly advertised as model speedup.

At short context, expert computation and projections may dominate. At long context, history reading can become more significant. Batch size, device bandwidth, interconnect, scheduling, and precision all influence which component dominates.

A useful measurement plan

To evaluate a DSA deployment, measure prefill throughput, decode throughput at several history lengths, memory per active sequence, latency percentiles, and completed-task cost. Keep the checkpoint, precision, framework, hardware, batch policy, and generation budget fixed when making comparisons.

For agent tasks, also count external tool latency and failed trajectories. A system that generates cheap tokens but performs many redundant searches may lose on time-to-success. The paper itself notes excessive self-verification and long trajectories as remaining problems.

The architectural result is compelling when stated precisely: DSA preserves useful tested behavior while reducing the costly main attention read at long context. It remains one component of an entire inference and learning system.

Sources: report §2.3, sections/01-method.tex; original figures/cost_prefilling.pdf and figures/cost_decoding.pdf; pinned model.py, Indexer and MLA. Continue with post-training.

8. Specialists, Distillation, and Mixed Learning

A pre-trained model predicts text. A useful assistant must also follow instructions, choose appropriate tools, solve problems, and produce responses that people judge helpful. DeepSeek-V3.2 obtains those behaviors through specialist training, distillation, and mixed reinforcement learning.

The report's post-training stage retains sparse attention. DSA is therefore present both when the model learns its assistant behaviors and when it uses them.

Why build specialists first?

A specialist is a full model optimized for a particular domain. The authors initialize all specialists from the same pre-trained V3.2 base checkpoint and invest substantial RL compute in each.

In addition to writing and general question answering, the report lists six specialized domains: mathematics, programming, general logical reasoning, general agents, agentic coding, and agentic search. It supports thinking and non-thinking behavior across the domains and uses different models to generate long-reasoning versus direct-response teaching data.

Interpretation: separate specialists make it easier to concentrate compute, reward design, and task difficulty on one skill. A mathematics specialist can use exact-answer or proof-oriented rewards; a writing specialist needs a different notion of quality. Training them independently avoids forcing one experimental reward mix to serve every domain from the beginning.

The cost is consolidation. A user usually wants one model that can decide among skills rather than a collection of independently served specialists. Distillation transfers their useful behavior into a final checkpoint.

What distillation means in this report

The specialists generate domain-specific data. The final model is trained on those data, integrating reasoning, agent behavior, and alignment examples.

A standard demonstration-learning objective would be

This equation explains how output-based distillation can work. The report does not specify every detail of its distillation loss, filtering, data mixture, or sampling procedure. It should not be rewritten as a claim that V3.2 uses a particular reverse-KL or on-policy distillation algorithm disclosed in a different report.

Reported: performance after distilled-data training is only marginally below domain specialists, and subsequent RL effectively eliminates the gap. The report states this result without supplying a complete per-specialist score table or all data proportions.

Distillation can teach a useful behavioral repertoire, but the student encounters its own states during deployment. Copying successful teacher trajectories does not guarantee recovery when the student makes an earlier mistake. Subsequent RL provides practice on sampled student behavior.

Thinking and non-thinking are behavioral modes

Thinking mode allows an extended reasoning process before the final response and, in V3.2, across tool interactions. Non-thinking mode produces more direct responses or tool actions.

These modes do not correspond to separate attention architectures. The same released backbone supports different formatted behaviors. A non-thinking model still performs complex neural computation; it simply does not emit the same extended reasoning trajectory.

A long reasoning trace can improve success on difficult tasks, but it consumes tokens and context. A direct response can be faster and sufficient for simpler tasks. Training both modes lets an application choose according to task and runtime compatibility.

The report's non-thinking agent results remain competitive, with small differences on SWE-bench Verified and larger differences on some long-tail tool benchmarks. Chapter 20 examines those measurements.

Mixed RL is the consolidation stage

The final model uses GRPO and merges reasoning, agent, and human-alignment training into one RL stage. It continues for thousands of steps.

Sequential fine-tuning can produce catastrophic forgetting: improving one task distribution damages another because the same parameters are reused. Mixing domains gives the optimizer ongoing exposure to multiple objectives.

That does not mathematically guarantee no forgetting. Domain balance depends on sampling, reward magnitudes, gradient interactions, and data difficulty. The report says the mixed approach balances performance and avoids the issues of multi-stage paradigms; it does not publish the full mixture schedule needed to independently reproduce that balance.

Different tasks need different rewards

For reasoning and agent tasks, the authors use rule-based outcome rewards, length penalties, and language-consistency rewards. For general tasks, they use a generative reward model with prompt-specific rubrics.

An exact answer or passing test can be checked mechanically. A useful explanatory response needs criteria such as completeness, relevance, clarity, and consistency. A generative reward model can inspect these criteria, but it also introduces evaluator errors and possible preference bias.

The training system therefore combines measurable outcomes with judged quality. It is neither entirely test-based nor entirely driven by a learned preference scalar.

Where the increased compute enters

RL compute includes generating candidate trajectories, executing tools, evaluating rewards, and updating the policy. Long trajectories can make generation expensive; executable environments add their own setup and runtime costs.

The report attributes the final reasoning gains to increased RL investment and describes improvements correlated with extending the budget beyond 10% of pre-training cost. This is evidence of a productive training direction, but it is not a controlled scaling law with fixed data, reward design, and algorithm across many compute points.

A “more compute” explanation should therefore be read alongside the stability interventions and improved task distributions. More samples are useful only if the optimizer can learn from them and the rewards measure the right behavior.

Why Speciale branches away

The main model integrates several domains and retains stricter length constraints to control serving cost. Speciale is trained exclusively on reasoning data with a reduced length penalty and additional proof-oriented data and reward methods.

This explores a different point on the quality–cost frontier. It does not establish that one can reproduce Speciale simply by changing an inference max_tokens field for the main model. Training affects the strategies and stopping behavior the model learns.

Chapter 21 treats Speciale as both a checkpoint variant and part of a higher-compute evaluation pipeline. That distinction matters when interpreting competition results.

What remains undisclosed

The report does not provide every specialist checkpoint, exact distillation dataset size, task mixture weight, reward coefficient, or RL hyperparameter. Its contribution is a description of the structure and several key stabilizing mechanisms.

A responsible reproduction would separately specify those missing choices and evaluate their consequences. It could implement the disclosed ideas without claiming to recreate the original run exactly.

The next chapters make the disclosed RL mathematics concrete. They explain what GRPO rewards, why KL regularization must respect the sampling distribution, and how expert routing and vocabulary masks enter the probability calculation.

Sources: report §3 opening paragraphs and §4.1, sections/02-post.tex. Continue with GRPO.

9. GRPO from First Principles

Group Relative Policy Optimization, GRPO, trains a model by sampling several responses to the same question, scoring them, and making better responses more likely relative to worse ones. V3.2 uses GRPO with additional stability measures described in the following chapters.

The central object is a policy $\pi_\theta$, a conditional probability distribution over next tokens. A rollout policy produces the training samples; the current policy is optimized using them.

Three policies have different roles

Immediately after copying parameters into a rollout worker, old and current policies may be close. After several optimizer updates on the same rollout batch, they diverge. Numerical differences between inference and training can also make them differ even at ostensibly identical parameters.

The reference has a different job: it discourages the current policy from drifting too far from a designated behavioral anchor. It is not merely another name for the rollout policy.

Groups provide a baseline

For a prompt $q$, sample $G$ outputs $o_1,\ldots,o_G$ from the old policy. Let their outcome rewards be $R_1,\ldots,R_G$.

The report gives the token advantage as

Every generated token in output $i$ receives the same outcome-based advantage in this displayed formulation. The report calls the process reward normalization, but its explicit equation is mean centering; it does not divide by a reward standard deviation. Describing the equation accurately matters because many other GRPO variants do use standard-deviation normalization.

An illustrative group with rewards $[1,0,0,1]$ has advantages $[0.5,-0.5,-0.5,0.5]$. If all rewards are one, the centered outcome advantages are all zero. If all are zero, they are also all zero.

This creates a curriculum pressure. Tasks that are too easy or too hard for every sample provide little relative outcome signal. Diversity and intermediate difficulty help produce informative groups. Other rewards and the KL term can still supply signal when the exact outcome reward is tied.

The likelihood ratio corrects for old samples

For generated token $o_{i,t}$ at its sampled prefix,

This compares how likely the same token is under the current and sampling policies. It is a token-level conditional ratio, not the product of all ratios over the response.

Without safeguards, highly changed probabilities can make old samples dominate the update. GRPO uses a clipped surrogate analogous to PPO.

The objective, term by term

A readable form of the displayed objective is

The policy maximizes this objective. A software implementation often minimizes its negative. $\varepsilon$ is the clipping range, $\beta$ is regularization strength, and $\widehat D$ is a sampled estimate of KL divergence to the reference.

The averages over $G$ responses and each response's length control how contributions are weighted. A response with ten times as many tokens does not automatically have ten times as much summed surrogate weight, because the paper includes $1/|o_i|$.

Length averaging does not remove the need for a length penalty. The reward still determines whether long successful behavior is preferable to shorter behavior, and generating the trajectory still costs compute.

What clipping does

Use $\varepsilon=0.2$ and advantage $A=0.5$ illustratively. If the ratio rises to $1.4$, the raw contribution is $0.7$, but the clipped contribution is $0.6$. Taking the minimum prevents additional reward for increasing that sampled token's probability far beyond the range.

Now use $A=-0.5$. If the ratio drops to $0.6$, the raw contribution is $-0.3$, while the clipped contribution is $-0.4$. Taking the minimum prevents unlimited benefit from pushing down the token probability based on that stale negative sample.

Clipping is sign-sensitive. It is not the same as simply clamping every gradient or every probability. Depending on the ratio and advantage sign, the unclipped branch can remain active.

It also does not impose a hard global bound on how much the policy changes. Other tokens, other samples, shared parameters, and multiple updates can still shift the distribution.

Why a KL term is useful

Successful answers might be rewarded even if they lose language consistency, become repetitive, or exploit evaluator weaknesses. KL regularization discourages excessive departure from a reference behavior.

For a fixed context, forward KL is

Computing this exactly over the whole vocabulary can be expensive when repeated at every rollout token. A sampled estimator is appealing, but it must account for which distribution generated the sample. Chapter 10 derives the report's correction.

The authors observe that domains need different regularization strengths. Mathematics can benefit from weak KL or none. This fits the intuition that tightly preserving an earlier answer style may hinder exploration of new successful reasoning behaviors.

Outcome reward gives imperfect credit assignment

If a long solution succeeds, its tokens all receive positive advantage in the simple outcome-based formulation. Some may be useful, some redundant, and some mistaken but later corrected. GRPO does not automatically identify the exact causal contribution of each token.

Across many samples, useful patterns can still be reinforced statistically. Verifiers, task diversity, and length incentives help shape the trajectory distribution. But a fluent trace should not be interpreted as a proof that every intermediate step was necessary or faithful to the network's internal computation.

Agents add action and observation boundaries

In a tool trajectory, model-generated reasoning, tool-call syntax, and final answers are actions. Tool results are supplied by the environment. An RL implementation normally masks environmental tokens out of the action loss while using them in later conditioning contexts.

The paper's compact $o_i$ notation does not specify every rollout packing or masking detail. A faithful engineering implementation must record which tokens were generated by the policy and the probabilities assigned at sampling time.

Why the stabilization chapters matter

The displayed objective assumes meaningful probability ratios. In practice, rollout reuse makes data stale; top-p changes action support; MoE routing changes the executed network; inference precision changes token probabilities. These are not peripheral implementation details. They affect the distributions the mathematics refers to.

V3.2's RL contribution is partly the effort to align those distributions and suppress harmful updates, making additional compute usable instead of merely generating more unstable training data.

Sources: report §3.1, sections/02-post.tex, displayed GRPO objective, ratio, and advantage equations. Continue with the corrected KL estimator.

10. The Corrected KL Estimator

KL divergence measures how much the current policy differs from a reference. In GRPO, however, the tokens available for training were sampled from an old policy. Estimating a current-policy expectation with old-policy samples requires a correction.

DeepSeek-V3.2 applies an importance-sampling ratio to the K3 KL estimator. This chapter derives what the correction does and the assumptions under which the mathematical claim holds.

Fix one context before doing the algebra

Let one sampled prefix be fixed, and abbreviate the next-token distributions as

These letters refer to distributions at that context, not entire sequence probabilities. Assume compatible support and nonzero denominators wherever terms are evaluated. For the clean derivation, all three distributions are normalized on the same action space, and $p$ covers the support needed by the estimator.

The desired divergence is

The K3 expression

Define

For $x>0$, $x-\log x-1\ge0$. Thus each K3 value is nonnegative. If the sample comes from $q$,

The cancellation depends on normalization and support. If $q$ has truncated support while $r$ places mass outside it, one cannot casually replace the restricted sum of $r$ by one. The definition of the action space must be made explicit.

Why old-policy sampling changes the answer

Training uses $v\sim p$. Then the uncorrected expectation is $\sum_vp(v)f(v)$, which generally differs from $\sum_vq(v)f(v)$.

Consider an illustrative two-token vocabulary:

The true divergence is approximately

K3 values are approximately $(0.31629,0.13000)$. Their expectation under the old policy is about 0.27903, rather than 0.22314. The old policy overweights the first token relative to the current distribution.

Importance weighting restores the intended expectation

The report uses

Taking the expectation under $p$ gives

In the example, the weighted sample values are approximately $(0.19768,0.32500)$. Their old-policy expectation is 0.22314, as required.

This is conditional next-token unbiasedness at a fixed prefix. It does not turn the old policy's distribution over prefixes into the current policy's distribution over prefixes. Full trajectory distribution correction would require additional reasoning and, potentially, products of ratios. The paper's correction targets the token-level estimator it displays.

The gradient claim needs the ratio to participate

An estimator can have an unbiased numerical expectation while a careless implementation produces a biased gradient. The dependence of $q$ in the ratio matters.

Rewrite the corrected estimator as

Holding $p$ and $r$ fixed, differentiating with respect to the trainable parameters gives

Its expectation under $p$ is

The exact KL gradient also contains a term $\sum_v\nabla_\theta q(v)$, but that is zero because $\sum_vq(v)=1$. Thus the expectations match under the stated conditions.

If one detaches the $q/p$ factor and only differentiates the inner expression, this cancellation changes. The proof above no longer applies. An implementation must state which factors receive gradients.

The report describes unbiased gradients as a consequence of its correction, but does not release the private RL loss implementation. This chapter explains the required mathematics rather than claiming to have inspected that implementation.

Why low current probability is troublesome

For an uncorrected K3 expression, the ratio $r(v)/q(v)$ can become very large when the current policy assigns tiny probability to a reference-favored token. If old-policy samples continue to present that token, its gradient can carry disproportionate weight.

The corrected form replaces the problematic sampling mismatch with an importance-weighted expression whose algebra has a more controlled expected gradient. That helps remove systematic estimation error.

It does not guarantee low variance in all circumstances. If $p(v)$ is very small where $q(v)$ is not, $q(v)/p(v)$ can be large. Stale data can still produce difficult importance weights. This is why the report also adds off-policy sequence masking.

Sampling masks complicate the action space

Top-p sampling renormalizes probabilities on a selected vocabulary subset. The saved old probability should describe the policy that actually sampled the token, including its truncation and temperature semantics.

V3.2 preserves the sampling mask and applies it to the current policy to keep old and current action spaces aligned. The report does not fully specify how reference-policy normalization interacts with every truncated KL calculation.

A reproduction must therefore define whether $r$ is normalized on the retained support or whether another exact correction is used. The full-support proof must not be silently applied to incompatible truncated distributions.

Regularization remains a design choice

An accurate KL estimate tells the optimizer what divergence it is penalizing. It does not tell the designer the best $\beta$. The authors find that different domains benefit from different strengths, including weak or omitted KL in mathematics.

Too much regularization can restrict improvement; too little can permit undesirable drift. The corrected estimator helps make that tradeoff intentional rather than contaminated by a mismatched sampling distribution.

Sources: report §3.1, “Unbiased KL Estimate,” sections/02-post.tex. Numerical examples and the fixed-context gradient derivation are derived teaching material. Continue with off-policy stability.

11. Keeping Off-Policy Learning Stable

Large RL systems generate trajectories in an inference framework and learn from them in a training framework. Reusing a rollout batch improves throughput, but each optimizer step makes the batch less current. Differences in precision, kernels, expert routing, and sampling can create further mismatch.

DeepSeek-V3.2 addresses three concrete sources of instability: harmful stale negative sequences, changed MoE paths, and changed vocabulary action spaces.

Two reasons samples become off-policy

The obvious reason is time. A rollout batch comes from an earlier checkpoint. After several updates, the trainable policy has changed.

The less obvious reason is execution. An inference engine can use different quantization, operation order, or routing behavior from the training engine. Thus a saved rollout probability may differ from a recomputed training probability even before a substantive model update.

The report defines $\pi_{\mathrm{old}}$ using the probability returned directly by the inference framework. This captures the distribution that actually generated the data instead of assuming a training-side recomputation will reproduce it exactly.

Sequence-level divergence proxy

For output $o_i$, define

This is an average sampled log ratio. At a fixed context, the expectation under the old policy corresponds to $D_{\mathrm{KL}}(\pi_{\mathrm{old}}\|\pi_\theta)$. A finite sampled sequence's value is an empirical statistic, and can be noisy or even negative. Calling it “the KL” should not imply it is an exact full-vocabulary divergence at every position.

Length averaging avoids mechanically making every long output appear more divergent simply because it contains more terms. It does not eliminate estimator variance or contextual differences across trajectories.

Which sequences are masked?

The report sets

The paper writes $M_{i,t}$, but the condition is sequence-level, so the same condition applies to the sequence's tokens. Only negative-advantage sequences with sufficiently large divergence are masked.

This is not a rule that discards every old sample or every failure. A fresh failure is useful: it teaches the model about a mistake it is still likely to make. The concern is a highly changed negative trajectory that may now push the policy in a misleading direction.

Illustratively, with $\delta=0.1$, a sequence with advantage $-0.4$ and $d_i=0.3$ is masked. A sequence with the same divergence and positive advantage remains eligible. A negative sequence with $d_i=0.02$ also remains eligible. The numerical threshold is illustrative; the report does not disclose its exact chosen value.

The mask does not remove every loss term

In the displayed masked objective, $M_i$ multiplies the clipped policy surrogate. The KL regularizer remains outside that multiplication.

Therefore, “masked sequence” should be read as “its specified policy-surrogate contribution is suppressed,” not necessarily “the sequence is completely absent from all optimization.” The placement of the mask is part of the algorithm.

Masking also changes which training evidence is used. It is a stability intervention with a potential bias/efficiency tradeoff, not a universal correction that makes all off-policy learning exactly on-policy.

Keep Routing: replay the executed expert paths

MoE routes each token through a subset of experts. Suppose a rollout token used experts A and B, while the training forward pass chooses A and C. The training system then evaluates and updates a different active parameter subset from the one that produced the sampled action.

Even small score changes can trigger such discrete routing changes. Numerical differences and policy updates can both contribute.

Reported: Keep Routing records the expert paths used by the inference framework and enforces those same paths during training. The authors call it crucial for MoE RL stability and say it has been used since DeepSeek-V3-0324.

Replaying paths makes the update target the experts actually involved in the rollout. It does not freeze their weights, guarantee identical activations across frameworks, or preserve every routing weight automatically. The report's statement is about path identities; details of routing-weight recomputation and router gradients are not fully disclosed.

An analogy is learning from a flight flown by one crew: training a different crew while treating the observed outcome as if it came from that crew confuses credit assignment. Routing replay keeps the responsible computational path present.

Keep Sampling Mask: replay the action support

Top-k sampling selects a fixed number of likely vocabulary items. Top-p sampling selects a subset whose cumulative probability meets a threshold. Both then sample from a renormalized distribution on that subset.

If the old policy sampled from set $S$, its probability is

Using an untruncated current probability in the numerator of an importance ratio compares different action spaces and normalization conventions. Recomputing a new top-p set from the current model can also change support.

The report preserves the old truncation mask and applies it to the current policy during training:

The two distributions then share the same retained token set for that context. This aligns the ratio with the sampled action space.

A normalization example

Suppose the untruncated old probabilities are $(0.6,0.3,0.1)$ and the saved set contains the first two tokens. The actual sampling probabilities are $(2/3,1/3,0)$.

Suppose current untruncated probabilities become $(0.4,0.4,0.2)$. On the same saved set, the current probabilities are $(0.5,0.5,0)$.

For the first token, the correct shared-support ratio is $0.5/(2/3)=0.75$. Comparing the raw current probability $0.4$ with the sampled old probability $2/3$ would give $0.6$, a different update weight caused solely by inconsistent normalization.

The example shows why recording only the chosen token is insufficient. The learner needs the sampling policy's probability semantics and, for this method, its mask.

The three interventions cooperate

The corrected KL estimator handles one expectation mismatch. Sequence masking reduces harmful stale negative updates. Routing replay keeps the neural path aligned. Sampling-mask replay keeps the vocabulary support aligned.

These solve different problems. Replaying a vocabulary set cannot fix expert-path mismatch. Replaying experts cannot stop a severely stale negative sample from producing a harmful update. Their shared purpose is to make the optimizer's probabilities describe the actual behavior it is learning from.

What an implementation must record

A reproduction needs generated-token masks, old log probabilities, reward components, output lengths, routing identities, and sampling masks or an equivalent replayable representation. It must also specify temperatures, normalization, reference semantics, and loss placement.

The report does not give a complete storage format or distributed rollout design. Its methodological lesson is nevertheless clear: in a large RL system, execution consistency is part of the learning algorithm.

Sources: report §3.1, “Off-Policy Sequence Masking,” “Keep Routing,” and “Keep Sampling Mask,” sections/02-post.tex. Continue with reward design.

12. Rewards, Length, and Reasoning Behavior

Reinforcement learning optimizes the reward that is measured. For an assistant, “good behavior” has several dimensions: correctness, useful explanation, valid tool use, faithful constraints, appropriate language, and reasonable cost. DeepSeek-V3.2 combines rewards to approximate these goals.

This chapter explains their roles without inventing coefficients or reward formulas that the report does not disclose.

Outcome rewards anchor success

For verifiable tasks, a rule can check the outcome: an exact mathematical answer, a passing software test, or a final environment state satisfying constraints.

An illustrative reward is $R_{\mathrm{success}}=1$ for passing and zero otherwise. The report does not state that every task uses exactly this binary scale. Some tasks can support partial credit or additional components.

Outcome checking has a major advantage: it does not require the evaluator to agree with every step of a long explanation. A valid final itinerary or correct program can be recognized through its properties.

It also has a limitation: if the verifier checks only a subset of intended requirements, the model can learn to optimize that subset. A patch that passes tests may still mishandle an untested input. A search answer can name the right entity but cite weak evidence.

Length penalties price the trajectory

Long reasoning can enable alternative attempts and verification. It can also become repetitive, slow, or self-defeating. The main V3.2 checkpoint uses length-related constraints; Speciale reduces the length penalty during reasoning-focused RL.

A schematic teaching objective is

This is an illustrative decomposition. The report identifies these reward families but does not publish a universal additive formula or $\lambda$ for every domain.

The important point is that the policy learns a stopping and exploration tradeoff. If extra reasoning increases success enough to compensate for its penalty, it can be favored. If it does not, the shorter trajectory can win.

Simply raising an inference output cap does not reproduce a model trained under a different length reward. The two models can learn different strategies long before either reaches a cap.

A small decision example

Consider two illustrative strategies. Strategy A succeeds 80% of the time with 10,000 tokens. Strategy B succeeds 85% of the time with 25,000 tokens. If success reward is one and the penalty is $0.002$ per thousand tokens, expected rewards are $0.78$ and $0.80$. B wins.

If the penalty becomes $0.005$ per thousand tokens, the expected rewards are $0.75$ and $0.725$. A wins. The model's preferred strategy depends on the cost assigned to extra exploration.

Real training uses sampled rewards, group-relative advantages, and nonlinear interactions. The example shows the incentive direction, not a fitted V3.2 reward model.

Language consistency rewards

A multilingual model can drift between languages during long RL trajectories. The authors use language-consistency rewards and report that top-p sampling with Keep Sampling Mask helps preserve consistency.

Consistency is not merely cosmetic. Unexpected language switching can reduce usability, obscure tool arguments, and make long explanations harder to follow. But the exact detection and scoring rule is not disclosed in the report.

A reward for consistency also needs to tolerate legitimate quoted text, code, names, and multilingual evidence. The book does not assume that the authors use any particular heuristic for those cases.

Generative reward models handle general quality

For general tasks, V3.2 uses a generative reward model with prompt-specific rubrics. A rubric can ask whether an answer addresses all requests, uses evidence correctly, follows format constraints, and avoids irrelevant detail.

“Generative” means the evaluator can generate an assessment conditioned on the prompt and response, rather than only compute a fixed scalar through a simple classification head. The report does not reveal its complete architecture or all rubrics.

Prompt-specific rubrics are useful because quality depends on the task. A concise answer may be ideal for a factual lookup; a long tutorial may be appropriate for a complex explanation. The same global preference for brevity would not serve both equally well.

The evaluator is still a model. It can miss factual errors, reward persuasive phrasing, or apply a criterion inconsistently. A rubric makes the criteria explicit; it does not make the evaluator infallible.

Search uses a hybrid signal

The search pipeline supplies difficult verifiable questions, then supplements them with helpfulness-oriented examples where search is beneficial. It uses multi-dimensional rubrics and a generative reward model for response quality.

This recognizes two notions of success. A difficult identification question may have one correct answer. A practical research request may need synthesis, clear evidence, and a response calibrated to the user's needs.

Optimizing only exact answers could teach a terse benchmark-solving style. Optimizing only judged helpfulness could reward plausible but unsupported prose. Combining them aims to retain factual anchors while training useful presentation.

Rewards can produce redundant self-verification

The report observes frequent redundant self-checking in tool tasks, leading to long trajectories and context overflow. It does not identify one reward component as the proven cause.

Interpretation: when correctness is heavily rewarded and the model is uncertain, repeated verification can appear attractive. If the training distribution lacks enough examples of efficient stopping, the model may keep checking after additional information has little value.

This hypothesis suggests experiments: reward success at fixed budgets, measure marginal value per tool call, or compare stopping policies under different length incentives. These are proposed investigations, not reported V3.2 ablations.

Outcome accuracy and reasoning quality are different measurements

A correct answer can be reached through an inefficient or partly mistaken path. A beautifully explained path can still end with a wrong answer. The reward and benchmark must specify which property they score.

For proofs, the distinction is sharper. Matching a short final answer does not establish a valid proof. Speciale incorporates DeepSeekMath-V2's proof-oriented dataset and reward method, and the competition evaluation uses a generate–verify–refine process. Chapter 21 separates those proof evaluations from answer-only math benchmarks.

What a reader should ask about any reward recipe

The useful questions are concrete: What is checked? What is merely judged? Can the model exploit the verifier? Does the reward penalize token count, tool calls, elapsed time, or some combination? Are components comparable across domains? How is a tied group handled?

The V3.2 report answers these at the level of reward categories and some algorithmic interventions. It leaves many exact implementation choices open. Those omissions limit replication, but the categories explain the model's central behavior tradeoffs.

Sources: report §3, §4.1, and §4.2, sections/02-post.tex and sections/06-high.tex. Continue with thinking and tools.

13. Reasoning with Tools and Conversation State

An agent does more than produce a tool-call string. It maintains a working plan across actions and observations. DeepSeek-V3.2 trains the model to reason between tool calls and changes how historical reasoning is retained during an ongoing task.

The crucial boundary is between a new user message and a tool result. They may both contain text, but they have different meanings for the model's conversation state.

The interaction loop

A typical tool trajectory is:

1. The user gives a task.
2. The assistant reasons and proposes a tool call.
3. A runtime executes the call.
4. The tool returns an observation.
5. The assistant updates its plan and acts again.
6. The assistant produces a final answer or completes the requested state change.

The tool runtime is outside the neural model. It parses a call, validates arguments, runs the function, and serializes the result. The model chooses subsequent actions based on the transcript it receives.

Thinking with tools makes the plan and observations part of one continuous problem-solving trajectory. The usefulness of that trajectory depends on preserving the state the model expects.

The report's retention rule

Reported: historical reasoning is discarded only when a new user message arrives. When only tool-related messages are appended, reasoning is retained throughout the interaction. When reasoning is removed, historical tool calls and results remain.

An illustrative sequence clarifies the distinction:

user: Repair the failing parser test.
assistant: [reasoning A] [tool call: read file]
tool: [file contents]
assistant: [reasoning B] [tool call: run test]
tool: [test result]
assistant: [final response]
user: Now explain the change.

Before the second user message, reasoning A and B can remain available during tool turns. Once the new user message is introduced, prior reasoning is dropped according to the formatting policy, while the substantive tool history remains.

The rule does not say all old messages disappear. It specifically distinguishes reasoning traces from action/observation history.

Why retaining reasoning can save tokens

If the assistant loses its working plan after every tool output, it may reconstruct the entire problem repeatedly. A multi-step task can then accumulate repeated planning and constraint analysis.

Retaining reasoning lets it continue from the established state: which files matter, which hypothesis failed, and what to check next. The report identifies repeated re-reasoning as a source of token inefficiency in a naïve extension of earlier reasoning-model context handling.

Retention also consumes context. A plan that becomes obsolete remains in the transcript unless the model or runtime manages it. The benefit is continuity, not infinite memory or guaranteed consistency.

This creates two separate context-management questions. Reasoning retention decides which state survives a tool turn. Overflow management, discussed in Chapter 18, decides what to do when the entire transcript approaches the window limit.

Why message roles can change measured capability

Some agent frameworks simulate tool outputs as user messages. Under V3.2's retention rule, that representation can trigger reasoning removal during what is logically still the same task.

The report names Roo Code and Terminus as examples that may not fully benefit from this reasoning persistence and recommends non-thinking mode for such architectures. This is a statement about the evaluated framework behavior in the report's time frame, not a permanent claim about all later framework versions.

A framework that preserves genuine tool-role observations can support the intended state continuity. A runtime that converts every observation into a user turn can make the model behave as if each result begins a new conversational request.

Agent performance therefore belongs to the combination of model and transcript semantics. Correctly encoded roles are part of the system's behavior, not decorative metadata.

Cold-start: combine existing abilities through prompting

Before robust agent RL, the authors have non-agent reasoning data and non-reasoning agent data. They use explicit prompts to encourage the model to combine them.

The appendix presents three illustrative templates: reasoning before a final program, tool use without extended reasoning, and tool calls interleaved within reasoning. The combined prompt asks the model to use a Python interpreter during problem solving and stop once it has a final solution.

These examples demonstrate format and behavior bootstrapping. They are not universal limits on the model. For example, the appendix's maximum of 20 code executions and its tool timeout belong to an example template, not a disclosed global V3.2 tool budget.

Reported: prompt-based integration is not initially robust, but it occasionally produces desired trajectories. Those successful patterns provide a basis for subsequent RL.

The approach depends on a model already capable of following instructions well enough to attempt the composite behavior. Prompting introduces a pattern; reinforcement learning improves the probability and reliability of useful trajectories.

Training examples must reflect the runtime

If a demonstration says a call returns a notebook result, the training trajectory needs that result at the proper boundary. A fabricated call with no observation teaches less about acting on evidence. A tool response incorrectly assigned the assistant role can confuse what the policy generated versus what the environment supplied.

The official release includes Python encoding and parsing code to map structured messages to model text. The model card says the release does not supply a Jinja-format chat template and cautions that its parser only handles well-formatted output without production-grade recovery.

An application should therefore distinguish successful formatting from robust runtime operation. Handling malformed tool calls, missing observations, and interrupted turns is a surrounding engineering responsibility.

Retained reasoning is operational state

For a model designed to retain reasoning during tool use, deleting it can alter behavior. That does not mean an application must expose the entire reasoning transcript to users or treat every emitted intermediate statement as correct.

The operational question is whether the state expected by the model is preserved when constructing subsequent inputs. User-facing presentation can separately focus on decisions, evidence, actions, and final results.

The report studies generated reasoning as a mechanism for improving task behavior. It does not establish that the emitted text fully reveals the neural model's internal computation or that every intermediate statement is a reliable explanation.

A practical compatibility check

A useful local check needs no large benchmark. Encode one user request, an assistant tool call with reasoning, a tool result, and a second assistant action. Confirm that the previous reasoning remains after the tool result. Then append a genuine new user message and confirm that the earlier reasoning is removed while tool history remains.

This tests the intended transcript rule. It does not verify model quality, but it catches a common source of avoidable mismatch before expensive agent evaluation.

Sources: report §3.2.1–3.2.2 and Appendix B, sections/02-post.tex and sections/appendix.tex; official encoding code; pinned model-card “Chat Template.” Continue with general environment synthesis.

14. Synthesizing General Agent Environments

Training a broadly useful agent requires more than many examples from one repository or one search tool. The model needs practice interpreting unfamiliar interfaces, retrieving relevant state, satisfying several constraints, and knowing when a task is complete.

DeepSeek-V3.2 creates synthetic task-oriented environments whose solutions are difficult to find but comparatively easy to verify. The basic training object is a tuple:

The environment supplies state. Tools mediate access. The task defines the requested outcome. The verifier checks whether a candidate meets the conditions.

Environment count and prompt count are different

Reported: the general-agent pipeline retains 1,827 environments and 4,417 tasks. The broader agent task table also includes 24,667 coding tasks, 50,275 search tasks, and 5,908 code-interpreter tasks.

Their sum is

This reconciles the introduction's “over 1,800 environments and 85,000 prompts.” It does not mean that all 85,267 prompts come from 1,827 fully synthetic general environments. Coding, search, and code interpretation use real tools, with different prompt origins.

Step one: construct useful state

Given a task category, a synthesis agent uses a sandbox with bash and search tools to create or retrieve relevant data. It stores that data in a sandbox database.

For a travel-planning category, the data might include cities, hotels, restaurants, attractions, prices, ratings, and transport connections. The eventual task-solving model is not granted direct database access. It receives a designed tool interface.

The setup agent can use external information, but a synthesized benchmark environment is still an artificial task setting. A database containing plausible travel options is not evidence that the resulting itinerary is a currently bookable real-world plan.

Step two: build a toolset

The synthesis agent implements task-specific functions: list hotels in a city, inspect restaurant properties, query intercity transport, and submit a final answer.

The tool interface creates the informational challenge. The model must decide what to ask for and how to combine results. If every relevant fact were directly included in the prompt, the task would mainly test static reasoning rather than interactive information acquisition.

Tool descriptions must expose enough semantics for a capable agent to solve the task. Unstated argument conventions or hidden constraints would make failure difficult to interpret.

Step three: generate a solution and verifier

The agent starts with a simple task and writes both a solution function and a verification function in Python. The solution may call the tool functions and perform logical computation, but cannot directly inspect the database or call arbitrary other functions.

This restriction makes the reference solution obey the intended interface. Otherwise, a task could appear solvable only because the setup agent uses privileged state unavailable to the policy being trained.

The solution's output must pass the verifier. If it does not, the synthesis agent revises the solution or verifier until they agree. It then increases task difficulty and updates both. If the toolset is insufficient, it adds tools.

This process generates increasingly demanding tasks with a known solution path. But agreement between an automatically generated solution and verifier is not independent proof that both reflect the intended natural-language specification. Correlated errors remain possible.

Why “hard to solve, easy to verify” matters

Suppose a three-day itinerary requires choosing one city, hotel, attraction, and two restaurants per day. If there are many options and cross-day uniqueness constraints, the search space can be large.

Checking one completed itinerary is simpler. A verifier can inspect each item, compare its city, check price and rating conditions, and track repeated identifiers. Search explores many combinations; verification checks one candidate's properties.

This asymmetry makes scalable outcome rewards possible. The evaluator need not reproduce the whole planning search for every trajectory.

A worked version of the report's planning idea

The report's example requires three days, no repeated cities or establishments, location consistency, and conditional day-two spending constraints. Consider an illustrative day-two candidate:

For the luxury-hotel branch, combined restaurant spending is 340 CNY, both ratings are at least 4.0, and the ticket is below 120 CNY. Those local constraints pass. The full verifier must still check location consistency and uniqueness across all three days.

If dinner rises to 195 CNY, total spending becomes 355 CNY and fails. A model might overlook the combined total while separately checking each restaurant. Such interdependent conditions make tasks harder than simply retrieving one hotel price.

Threshold boundaries need explicit definitions. A phrase such as “500–800” can overlap with “800 or more” unless the verifier defines half-open intervals. This is a general environment-design issue; the report's natural-language example does not fully specify every boundary convention.

A verifier sketch

def verify(plan, tools):
    if len(plan) != 3:
        return False
    if not unique_items_across_days(plan):
        return False
    for day in plan:
        if not all_items_belong_to_city(day, tools):
            return False
    day2 = plan[1]
    return conditional_budget_rules_hold(day2, tools)

This is teaching pseudocode, not the released synthesis verifier. Its purpose is to expose the difference between final-state checking and natural-language assessment.

Filter for reachable difficulty

After generation, the authors perform RL on the dataset with V3.2 and retain instances with nonzero pass@100. At least one successful attempt among the tested samples establishes empirical reachability under that policy and sampling setup.

It does not mean the task has a 100% success rate. Nor does it prove every retained task is moderately difficult. A task passed once in 100 attempts can still be extremely hard for a single rollout.

Filtering avoids spending all training on tasks with no observed success signal. It also biases the retained dataset toward what the generating/training model can sometimes solve. Truly novel tasks beyond its current ability may be excluded even if their verifiers are valid.

What can generalize across synthetic worlds?

The model can learn interface-reading, constraint tracking, decomposition, state inspection, and verification habits. These abstractions can transfer even when downstream tool names and databases differ.

The strongest version of this claim would require careful held-out environment families and independent checks of task validity. The report provides a narrower but useful transfer experiment in Chapter 17.

Synthetic diversity is an engineering tool for broadening practice. Its value depends on more than the number of environments: the tasks must present different information structures and decisions, with faithful verifiers and usable interfaces.

Sources: report §3.2.3, task table and “General Agent,” sections/02-post.tex. Continue with coding environments.

15. Coding Agents and Code Interpreters

Software work offers a powerful training signal: execute the code and observe tests. But turning a repository issue into a reproducible RL task is much harder than collecting a problem statement and answer. Dependencies must install, the old version must exhibit the failure, and the intended patch must fix it without breaking previously passing tests.

DeepSeek-V3.2 builds coding environments from GitHub issue–pull-request pairs and separately trains code-interpreter behavior in Jupyter notebooks.

Two kinds of code task

A coding agent needs navigation, patch construction, dependency awareness, and regression testing. An interpreter agent needs to translate a problem into computation and interpret the output correctly. They share tools and reasoning patterns, but their success conditions differ.

Mining issue–PR pairs

The authors begin with millions of GitHub issue–PR pairs. Heuristics and LLM judgments filter them for a reasonable issue description, a correlated gold patch, and a test patch suitable for validation.

The gold patch is the known fix associated with the task. The test patch supplies tests that characterize the intended behavior. A usable pair needs more than textual resemblance between an issue and a PR; the patch must actually address the described problem.

Filtering is necessary because repository history contains incomplete reports, unrelated changes, flaky tests, missing context, and dependencies that no longer resolve cleanly. Raw mining scale should not be confused with the final number of executable environments.

Reported: the task table contains 24,667 code-agent tasks, and the prose describes tens of thousands of reproducible issue-resolution environments.

An automated environment-setup agent

A setup agent powered by V3.2 installs packages, resolves dependencies, and executes tests. It converts results into JUnit format for consistent parsing across languages and frameworks.

This is an agent building environments for other agents. Its success is not measured merely by whether an install command exits successfully. It must produce an executable state whose tests distinguish broken and fixed behavior.

The report lists Python, Java, JavaScript, TypeScript, C, C++, Go, and PHP. That diversity broadens the training distribution beyond one language's tooling conventions.

It does not imply equal task counts or equal coverage in every language. The report does not publish a complete language-by-language breakdown.

F2P and P2F clarify the acceptance rule

A false-to-positive, or F2P, test changes from failing before the gold patch to passing afterward. A pass-to-fail, or P2F, test changes from passing before the patch to failing afterward.

An environment is accepted only when applying the gold patch produces a nonzero F2P count and zero P2F count.

Illustratively:

The third row would disqualify the environment under the stated rule. A task with no failing test that becomes passing would also fail the acceptance check, even if the gold patch looks plausible.

This criterion supports differential validity: the environment observes the intended transition. It does not prove complete software correctness beyond the tests.

What the policy learns in such an environment

A successful trajectory can involve identifying relevant files, reading the issue carefully, forming a hypothesis, making a focused patch, running targeted tests, and checking broader behavior.

The reward observes the final tested outcome. The model must learn useful intermediate actions through repeated experience. A test failure is not merely negative feedback; it can also reveal information that guides the next action within the same trajectory.

This makes repository repair a partially observed problem. The model does not initially know all code or environment state. Tool calls reveal it, and the transcript becomes the working memory for subsequent decisions.

Why executable environments are expensive

Dependency resolution can take time. Builds can consume substantial resources. Tests can fail for reasons unrelated to the patch: unavailable services, clock assumptions, nondeterministic ordering, or incompatible platform versions.

A robust training pipeline needs to distinguish task failure from environment failure. Otherwise the policy can receive negative reward for an infrastructure error or positive reward because a test was accidentally skipped.

The report gives its setup acceptance rule and standard output format, but not all retry policies, isolation details, or environment health checks. Those are important reproduction details left to an implementer.

Code interpretation as a reasoning aid

The code-interpreter tasks use Jupyter notebooks for mathematics, logic, and data science. The task table reports 5,908 tasks, with real environments and extracted prompts.

A notebook is stateful. Variables defined earlier can be reused, and later calculations depend on previous cells. This allows iterative computation but introduces hidden-state hazards: a stale variable or failed cell can make later results misleading.

The model must know when execution is useful and how to interpret the output. Running a brute-force search over a small example can reveal a pattern, but it is not automatically a proof for all inputs. A numerical approximation can support a hypothesis without establishing exact equality.

A small interpreter example

Suppose a task asks for the count of integer triples satisfying constraints. The model can implement enumeration over a finite specified range and inspect the count. It should verify that the loop bounds match the problem, that inclusive endpoints are handled correctly, and that duplicate representations are not counted twice.

The tool removes arithmetic burden, but the model still chooses the formalization. An incorrect program can execute successfully and return a precise wrong number. Tool use changes the source of potential error rather than eliminating it.

Rewards and gold-patch leakage

A training environment may use a gold patch to validate its setup without exposing that patch to the task-solving model. Keeping those roles separate matters: the gold solution is evidence that the environment can be solved, not part of the policy's allowed observation.

Likewise, evaluation should ensure that expected answers, hidden tests, or privileged environment metadata are not accidentally available through tools. The report describes the intended setup, but it does not provide a complete independent contamination audit.

What credible coding-agent evidence looks like

A strong evaluation pairs the final resolution rate with the framework, tool interface, test protocol, attempt budget, and environment conditions. A model can perform differently if one runtime preserves reasoning and another discards it after each tool result.

That is why the report's SWE-bench and Terminal Bench scores must be interpreted with their surrounding frameworks. Chapter 20 makes those qualifications explicit.

Sources: report §3.2.3, “Code Agent” and “Code Interpreter Agent,” task table, sections/02-post.tex; Appendix B cold-start examples. Continue with search-data construction.

16. Search Agents and Difficult Questions

A useful search agent must know what to look for, assess retrieved evidence, and decide whether the task is complete. It should not merely produce an answer that sounds likely from memory. DeepSeek-V3.2 trains search behavior using a pipeline that constructs difficult, verifiable questions and supplements them with helpfulness-oriented tasks.

The task table reports 50,275 search tasks, using real search tools with synthesized prompts. “Real environment” here refers to the tool setting, not to prompts collected directly from real users.

Start from informative long-tail entities

The authors sample entities across domains from large web corpora. A long-tail entity is less common than a highly familiar person, place, or concept. Questions about it are less likely to be answered reliably from memorized high-frequency patterns alone.

This sampling is meant to produce tasks where external information is useful. It is not a guarantee that every question is absent from pre-training or that the model has never encountered the entity.

The quality of the initial entity distribution influences the resulting task distribution. Breadth across languages and domains matters, but the report does not publish every sampling proportion.

A question-construction agent explores the entity

Using search tools, the construction agent gathers information with configurable exploration depth and breadth. It consolidates that information into question–answer pairs.

Depth can encourage chains of evidence: identify one entity, follow a related fact, then resolve a final attribute. Breadth can encourage comparison across sources or several candidate identities. The report names these configurable parameters without giving a complete generation schedule.

A good question needs a stable answer and enough available evidence. Difficulty should come from searching and combining facts, not from an ambiguous wording that allows several equally reasonable answers.

Heterogeneous answer agents test difficulty

Multiple answer-generation agents use different checkpoints, system prompts, or other configurations to produce candidate responses. A verifier with search capabilities checks the proposed ground truth and the candidates through multiple passes.

The pipeline retains samples only when the ground truth is judged correct and all candidate answers are verifiably incorrect.

This is a form of adversarial filtering. A question that all candidate configurations answer easily is not retained as a hard example. A question that exposes a shared weakness can survive and provide useful training pressure.

The criterion is about the candidate set tested by the pipeline. It does not prove that no model could solve the question or that the task is impossible without extensive search.

Why heterogeneous configurations help

If every answer agent uses the same prompt and checkpoint, they may fail in the same way for a superficial reason. Varying configurations broadens the attempted strategies.

For example, one agent might rely more on direct retrieval, another on decomposition, and another on a stronger initial knowledge prior. If all fail while an evidence-based verifier confirms the answer, the example is more plausibly challenging than a failure under one poorly chosen prompt.

This is a design rationale. The report does not provide an ablation showing exactly how much each source of heterogeneity contributes.

Verification must separate truth from plausibility

A search-enabled verifier can inspect supporting sources rather than judge only the language of an answer. Multiple passes can catch some inconsistencies.

However, web evidence itself can be wrong, duplicated, outdated, or mutually inconsistent. A verifier model can also misread a source. Repeated assessment improves confidence only to the extent that the passes offer independent useful checks.

Questions involving dates, renamed organizations, or ambiguous transliterations need careful answer definitions. A verifier should distinguish an incorrect answer from a correct alternative spelling or a reasonable answer at a different time reference.

The report states its correctness filtering rule but does not release a complete error-rate audit of that verifier. This limits how strongly one can claim every retained task has perfect labels.

An illustrative search task

Imagine a question that asks for the birthplace of the translator of an obscure edition of a book. A successful agent might first identify the edition, then verify the translator's identity, then find a biography, and finally cross-check the birthplace.

The failure modes are specific: confusing editions, assuming the translator is the original author, choosing the first similarly named person, or answering from an unsourced snippet. Such tasks reward entity resolution and evidence chaining rather than generic fluent explanation.

The example is illustrative and is not one of the released training questions.

Hard exact-answer tasks are not enough

The authors augment the verifiable dataset with filtered helpful-RL examples for which search provides measurable benefit. They use detailed rubrics across quality dimensions and a generative reward model.

This adds practical tasks whose success is not reducible to one short fact. A user may need a comparison, an explanation of conflicting evidence, or a plan grounded in current information.

The hybrid approach aims to teach both factual reliability and useful synthesis. It also means that the entire search training program cannot be described as an exact-match benchmark curriculum.

Training a search policy versus evaluating a search service

The model chooses queries and interprets observations. The external search API determines which results are returned, how snippets are formatted, and what pages are available.

The report evaluates search with a standard commercial search API. Results can change with provider behavior, query rewriting, indexing, and time. A reproduction needs to identify these environmental conditions.

Similarly, a tool that returns very long documents can create context overflow even when the model's search strategy is sensible. The runtime's result truncation and formatting influence the usable information budget.

What the pipeline is trying to teach

At its best, search RL encourages the model to use tools when memory is insufficient, formulate informative queries, pursue unresolved uncertainty, and stop when the evidence supports an answer.

At its worst, a poorly calibrated reward can favor excessive search or superficially convincing citations. The report's observations of redundant self-verification show that efficient stopping remains a challenge.

Chapter 18 examines how context management allows longer search trajectories. Before that, Chapter 17 asks whether the synthetic general-agent training transfers beyond the environments used to create it.

Sources: report §3.2.3, “Search Agent,” task table, sections/02-post.tex; report §4.1 search-evaluation notes. Continue with the synthesis evidence.

17. What the Synthesis Experiments Establish

The general-agent synthesis pipeline is useful only if its tasks are challenging enough to teach something and broad enough for learned behavior to transfer. The report investigates these questions separately: a small difficulty evaluation and an RL transfer experiment.

This chapter explains what each experiment controls and what remains uncertain.

The difficulty sample

The authors randomly sample 50 general synthesized task instances and test several models. The reported results are:

These are difficulty results on a sampled synthetic task set, not the final V3.2 model's downstream benchmark scores. The first column refers to V3.2-Exp.

The results show that the sampled tasks are not trivially solved by the model involved in synthesis and still challenge stronger systems. They also show that multiple attempts materially improve observed success.

What pass@k means

Pass@1 measures success for one attempted solution under a sampling setup. Pass@k asks whether at least one of $k$ candidates succeeds, with the benchmark's estimator and sampling procedure.

If a task had independent success probability $p$ on every attempt, the probability of at least one success would be

For $p=0.12$ and $k=4$, that illustrative probability is approximately 40%. The reported V3.2-Exp pass@4 is 26%. This does not create a contradiction: task difficulty varies, samples can be correlated, and aggregate pass@1 is not one identical success probability shared by every task.

For empirical pass@k estimated from $n$ sampled outputs with $c$ successes, a common estimator is

The report's table does not disclose all sampling details needed to identify the exact estimator behind every cell. The formula is background, not an assertion about a hidden implementation.

Why 50 instances deserve caution

A small random sample can establish that meaningful difficult tasks exist. It provides a less precise estimate of the full dataset's difficulty distribution.

Under a simplified independent Bernoulli model with $n=50$ and $p=0.5$, the standard error of a proportion is about seven percentage points. Actual benchmark aggregation can have different uncertainty, but the scale illustrates why small differences should not be overinterpreted.

The report does not provide confidence intervals or multiple independent task samples. The strongest supported conclusion is qualitative: the synthesis pipeline produces nontrivial tasks, and stronger models still fail many single attempts.

The transfer experiment starts from an SFT checkpoint

To test generalization, the authors apply RL to V3.2-SFT using only synthetic general-agent tasks in non-thinking mode.

This design intentionally removes two confounders: extended chain-of-thought generation and other RL task data. If downstream behavior improves, the explanation cannot simply be “the model learned to emit much longer reasoning traces” or “coding/search RL supplied all the improvement.”

The comparison includes V3.2-SFT and V3.2-Exp, with Exp described here as receiving RL only in search and coding environments.

What the training curves show

Reproduced from the report's synthetic-agent RL figure. Blue curves show the synthetic-data training run; dashed lines show SFT and Exp baselines.

The panels cover airline, retail, telecom, and overall $\tau^2$-bench; filesystem and PostgreSQL categories of MCP-Mark; and several MCP-Universe categories. The blue curves generally rise above the baseline levels, though they are noisy and not monotonically improving at every checkpoint.

The report's textual conclusion is that synthetic general-agent RL substantially improves transfer to these benchmarks, whereas restricting RL to code and search does not produce the same gains.

The figure provides evidence about a training intervention, not a complete table of exact final endpoint values. This book does not invent high-precision numbers by visually reading plot pixels.

What transfers if tools are unfamiliar?

An agent can reuse behaviors such as reading a schema carefully, retrieving missing information, checking conditional constraints, preserving state, and submitting a valid final artifact. Those are not tied to one exact tool name.

The report says the benchmark environments and toolsets were not encountered during RL. This supports transfer to unfamiliar interfaces. It is narrower than proving that every benchmark task, underlying document, or domain concept was absent from all pre-training and distillation data.

Novel tools and uncontaminated tasks are separate evaluation properties. A model can use an unfamiliar API to solve a concept it already knows.

How strong is the causal evidence?

The non-thinking synthetic-only intervention is more informative than a final-model comparison where architecture, compute, data, and reward all change together. It narrows the likely cause of improvement to training on those synthetic tasks under that setup.

Several questions remain: Are RL budgets exactly matched across comparison runs? How many seeds were used? How were checkpoints selected? Were all environment families held out at the appropriate level? How much benefit comes from verifier quality versus task variety?

The report does not answer every question. The evidence supports usefulness and transfer, but not a universal quantitative claim about the value of each synthesized environment.

What would strengthen the study?

Useful follow-up experiments would hold RL compute fixed while varying environment diversity, compare synthetic tasks with equal-cost real tasks, audit verifier errors independently, and evaluate entirely held-out tool families. Reporting task-level success and cost would show whether improvements come from better planning or simply more attempts.

These are proposed studies. They do not diminish the disclosed result; they specify the boundary between “this intervention helps” and “we understand exactly why and how far it will generalize.”

Sources: report §4.3, sections/05-thinking-with-tool.tex; tables/synthesis-eval.tex; figures/synthesis-rl-plot.png. Continue with context scaling.

18. Context Management and Test-Time Scaling

An agent can use a large amount of total computation while having only a finite context window. DeepSeek-V3.2's search evaluations expose that distinction: a substantial fraction of BrowseComp trajectories exceed the report's 128K window without context management.

The authors compare several ways to continue problem solving after the transcript becomes crowded. They also compare serial continuation with parallel independent attempts.

Three different budgets

Context budget is how much text the model can condition on at one time. Generation budget is how much output it can produce during a run or attempt. Task-compute budget includes all generation, repeated prefill, tools, summaries, and parallel candidates.

A context reset can keep each individual input within 128K while allowing a task to consume far more than 128K total tokens across successive segments. It does not create a native million-token attention window.

Similarly, several independent trajectories can multiply total compute without increasing any trajectory's context capacity. These forms of test-time scaling answer different resource constraints.

When the report triggers management

The context-management strategies activate when token usage exceeds 80% of the window. With a 131,072-token interpretation, that is approximately 104,858 tokens, leaving room for further actions and observations.

This arithmetic is illustrative. The report uses the 128K label rather than documenting every runtime threshold calculation. In practice, a system must count all formatted input, tool descriptions, outputs, and reserved generation space.

Triggering before full capacity avoids a hard failure when the next tool output is large. It also creates a decision: which parts of the trajectory should be preserved?

Summary

The Summary strategy summarizes the accumulated trajectory and restarts the rollout with that summary. It aims to preserve useful findings, plans, and constraints in a smaller representation.

Summaries can lose details, omit unresolved uncertainty, or convert a tentative hypothesis into a confident assertion. The summarizing model is performing another reasoning task, and its output becomes a compressed memory.

Repeated summarization can compound such errors. A high-quality summary needs exact identifiers, key evidence, completed actions, failed approaches, and remaining questions. The paper names the strategy but does not publish a universal summary schema.

Reported: Summary extends average steps to 364 and achieves a BrowseComp score of 60.2, but has relatively low overall efficiency in the studied comparison.

Discard-75%

This strategy discards the first 75% of tool-call history, freeing space while retaining more recent interactions.

Its attraction is simplicity: no extra model-generated summary is needed. Its risk is losing an early observation that remains essential. Recency is only a rough proxy for relevance.

The report describes which history fraction is discarded at a high level. It does not specify every handling detail for associated reasoning, system instructions, durable state, or partially completed actions. An implementation should not infer those choices from the name alone.

Discard-all

Discard-all resets context by discarding previous tool-call history. It allows a fresh search trajectory rather than continually carrying accumulated observations.

This can help when the existing run is stuck in an unproductive hypothesis or cluttered with redundant evidence. It can hurt when prior calls performed irreversible or state-changing actions whose results remain operationally important.

The reported experiment is search-focused. Extending full discard to a coding or transaction agent requires a different analysis of external state. Clearing the transcript does not undo a file edit, database update, or tool-side side effect.

Reported: Discard-all achieves 67.6 on BrowseComp with favorable step efficiency relative to the studied alternatives.

Parallel-fewest-step

The parallel baseline samples $N$ independent trajectories and selects the one with the fewest steps. This is a selection heuristic, not an oracle that knows which answer is correct.

Parallel sampling broadens exploration. It also spends compute on candidates that will not be returned. Wall-clock latency can improve if resources allow concurrent execution, while total GPU and tool costs rise.

Choosing the shortest trajectory can favor efficient successful runs, but it can also select a prematurely confident wrong answer. The measured benchmark result incorporates the behavior of that heuristic under the experiment's task distribution.

Reading the comparison figure

Reproduced from the report's search context-management figure. The horizontal axis is real steps and the vertical axis is BrowseComp score.

The figure shows that different strategies reach different accuracy–step tradeoffs. Discard-all rises strongly in the examined range. Parallel sampling can reach similar or higher accuracy at larger step totals. Summary improves over limited-context behavior but is less efficient in this experiment.

The lines are fitted visual trends, not precise guarantees that every extra step improves an individual task. The dots are aggregate experimental points.

“Real steps” is also not a universal monetary cost. One search call, one summary generation, and one long model continuation can have very different token counts and elapsed times. Fair cost analysis needs more than the number of interactions.

The important before/after distinction

The main evaluation table reports BrowseComp 51.4 without context management and 67.6 with it. The difference is 16.2 percentage points.

That improvement belongs to a changed inference procedure. It should not be described as the base model's single untouched trajectory accuracy. A model and a runtime strategy together produce the higher result.

The report notes that more than roughly 20% of search test cases exceed the original window. Context management gives those tasks room to continue and changes the exploration process for the run.

Why a reset can outperform remembering everything

More history is not always more useful history. A transcript can contain mistaken hypotheses, repeated calls, and distracting snippets. A reset changes the model's immediate conditioning and can encourage a different route.

Interpretation: part of Discard-all's gain may come from escaping path dependence, not only freeing token capacity. The report does not separately isolate those effects. A controlled test would distinguish resets that preserve key facts from resets that remove the reasoning trajectory entirely.

How serial and parallel scaling can combine

One can allocate several branches, allow each branch a limited number of context resets, and select among outputs using a verifier. This creates a richer search policy with several budget dimensions.

The report identifies finding an optimal serial–parallel combination as future work. It does not publish a universally best scheduling algorithm. The best allocation depends on task difficulty, verifier reliability, external tool cost, and available parallel hardware.

The practical lesson is to evaluate accuracy at a specified total compute and context policy. A leaderboard number without those conditions hides a large part of the system being measured.

Sources: report §4.4, sections/03-seach-agent.tex; §4.1 search notes and tables/eval_main.tex; original figures/search.pdf. Continue with reasoning evaluation.

19. Reading the Reasoning Benchmarks

The report evaluates several kinds of reasoning, but there is no single universal “reasoning score.” Knowledge-heavy questions, exact-answer mathematics, programming contests, and proof construction stress different abilities.

This chapter records the main non-tool results and explains how to interpret them. All values are the report's December 2025 experiment snapshot, not current model rankings.

The evaluation conditions

The authors set temperature to 1.0 and context to 128K. For math-related tasks, including AIME, HMMT, IMOAnswerBench, and HLE, they use a prompt asking for step-by-step reasoning and a boxed final answer.

For HLE, the authors additionally evaluate V3.2-Thinking with the official template and obtain 23.9, compared with 25.1 in the main table. That is direct evidence that formatting and instruction choices affect the reported result.

The report identifies LiveCodeBench's problem range as August 2024 through April 2025. Benchmark names without their date slices can refer to different sets and should not be treated as identical experiments.

The main reasoning table

Except for Codeforces rating, the values are percentage scores. The full source table also includes Claude-4.5-Sonnet and MiniMax-M2. Missing values mean unreported, not zero.

Knowledge-heavy versus search-supported performance

MMLU-Pro covers broad advanced knowledge and reasoning. GPQA Diamond emphasizes difficult science questions. HLE is a demanding broad expert-question evaluation; the report's non-tool HLE is text-only.

These tests require both problem solving and the right knowledge. More reasoning cannot reliably recover an obscure fact absent from the model's learned information. The authors acknowledge that V3.2's knowledge breadth lags leading proprietary models, associating this with fewer total training FLOPs.

That attribution is the authors' explanation, not a complete causal decomposition. Data coverage, training quality, and benchmark composition can also affect knowledge performance.

HLE with search tools appears separately in the agent table. V3.2's tool-supported HLE score is 40.8. It should not be mixed with the 25.1 non-tool score as if both were evaluations under the same information access.

Exact-answer mathematics is not proof evaluation

AIME and HMMT ask for answers that can be checked. IMOAnswerBench also measures answer-oriented mathematical problem solving; it is not the same as judging complete olympiad proofs.

A model can reach the correct answer through a valid derivation, numerical experimentation, or an erroneous path that happens to end correctly. Outcome accuracy measures the answer, not the universal validity of every step.

Speciale's IMO and CMO proof evaluations use a different procedure, discussed in Chapter 21. The distinction is essential when claims move from “high math benchmark accuracy” to “can construct rigorous proofs.”

Programming benchmarks stress different skills

LiveCodeBench measures solving coding problems from specifications. Codeforces rating is derived from contest-style performance and uses a different scale from percentage accuracy.

A 200-point rating difference is not a 200-percentage-point accuracy difference. Ratings summarize competitive performance relative to a scoring system, while pass@1 estimates one-attempt success under sampling.

Repository repair benchmarks require navigation and interaction with an existing codebase. Strong competitive programming does not automatically imply strong software maintenance. Chapter 20 treats those agent evaluations separately.

What “comparable to GPT-5” means in these rows

V3.2 is close to GPT-5 High on several reported reasoning tests and exceeds it on some HMMT and IMOAnswerBench rows. GPT-5 High leads on MMLU-Pro, GPQA, HLE, LiveCodeBench, Codeforces, and AIME in this table.

The paper's broad wording summarizes this mixed profile. It should not be turned into a claim that V3.2 wins every reasoning task or matches every aspect of a closed model's behavior.

Gemini-3.0-Pro leads V3.2 on all the displayed main-model non-tool rows. Speciale narrows or reverses some of those gaps, while spending more output tokens and retaining weaknesses elsewhere.

Token efficiency adds a second axis

The report gives output token counts in thousands for several benchmarks. On HMMT February, V3.2 uses about 19K tokens at 92.5, while Kimi uses 31K at 89.4. On LiveCodeBench, V3.2 uses about 16K at 83.3 versus Kimi's 29K at 82.6.

These comparisons support the authors' observation that V3.2 can obtain similar or higher scores with fewer output tokens than Kimi on the cited tests. They do not prove lower wall-clock cost across all deployments. Tokenizers differ, tokens have different runtime cost, and hardware/framework choices matter.

The published counts are rounded. The source table does not supply a full distribution of output lengths or tail latencies. An average can hide occasional trajectories near the context limit.

Small score differences need uncertainty

A benchmark score is an estimate produced by a finite task set and sampling procedure. Differences of one or two points can reflect genuine improvements, sampling noise, prompt sensitivity, or task selection.

The report does not publish comprehensive confidence intervals and repeated-seed results for every comparison. Therefore, small gaps should be interpreted as reported point estimates rather than definitive ordering of all future tasks.

One should also avoid averaging heterogeneous scores into one index without a clear rationale. A rating, a percent accuracy, and a resolution rate are not naturally commensurate measurements.

A useful way to compare models

Begin with the task class, then identify information access, output budget, sampling, and evaluation metric. Compare accuracy alongside token count and elapsed time. Finally, inspect failure types: missing knowledge, wrong reasoning, invalid code, or poor stopping.

This produces a much more useful model-selection picture than one headline rank. V3.2's report shows a strong open reasoning model whose behavior improves further with more expensive specialization, but whose knowledge and token efficiency remain uneven relative to the strongest compared system.

Sources: report §4.1, sections/02-post.tex; tables/eval_main.tex; §4.2 efficiency table in sections/06-high.tex. Continue with agent evaluation.

20. Reading the Agent Benchmarks

An agent score measures a model acting inside a runtime with tools, context rules, and an evaluator. The same model can score differently when those surrounding conditions change.

DeepSeek-V3.2's report is unusually explicit about several such differences. This chapter preserves those qualifications alongside the numbers instead of treating them as footnotes that can be discarded.

The main agent results

Scores are percentages under their benchmark-specific metrics. The full table also includes Claude-4.5-Sonnet and MiniMax-M2. For example, Claude scores 77.2 on SWE-bench Verified and 38.6 on Tool-Decathlon in that snapshot.

Terminal Bench's framework distinction

Reported: the V3.2 thinking score of 46.4 uses the Claude Code framework. The authors say their thinking-context strategy is incompatible with Terminus in the evaluated setup.

They also report 39.3 in non-thinking mode with Terminus. In the appendix's comparison using Claude Code, non-thinking scores 37.1.

These are three distinct conditions:

Comparing 46.4 with 39.3 does not isolate thinking alone because the framework also changes. Comparing 46.4 with 37.1 is the paper's more direct within-framework mode comparison.

The framework issue follows the message-role and reasoning-retention rule explained in Chapter 13. It is evidence that transcript semantics can affect benchmark success.

SWE-bench's main and robustness scores

The main SWE-bench Verified result of 73.1 uses an internal framework. The report says tests across Claude Code, RooCode, and non-thinking settings give results around 72–74.

This robustness range is encouraging: the main coding performance is not presented as depending on only one fragile setup. But it does not provide a complete framework-by-framework task-level analysis or uncertainty estimate.

A resolved rate measures whether a patch passes the benchmark's checks. It should not be read as the percentage of all real software issues the model can fix, nor as a guarantee of production correctness beyond those checks.

Search depends on context strategy

BrowseComp improves from 51.4 to 67.6 when context management is applied. The higher score includes a changed test-time procedure and additional opportunity to continue searching.

BrowseCompZh's 65.0 appears as a separate task set, not an automatic Chinese-language version of exactly the same experiment conditions. Tool-supported HLE is also separate from text-only HLE without tools.

The search API is part of the environment. Provider differences, unavailable pages, snippets, and returned text length can influence results. The report specifies a standard commercial API but does not offer a universal provider-independent score.

Tool-use evaluation has its own qualifications

The authors use standard function calls and thinking mode for tool benchmarks. Tool outputs are placed in the tool role rather than the user role for the MCP evaluations.

For MCP-Universe and MCP-Mark, all compared models are evaluated in the authors' internal environment because search and Playwright conditions can differ slightly from the official setting. These results support comparisons within that environment, but should not be silently equated with official-harness results produced elsewhere.

On $\tau^2$-bench, the authors use the model itself as the user agent. The category scores are 63.8 airline, 81.1 retail, and 96.2 telecom. Their unweighted mean is

The table reports 80.3; rounded component scores need not reproduce a rounded aggregate exactly. The important convention is averaging categories, not treating the reported number as a single homogeneous task rate.

Thinking versus non-thinking

The appendix supplies a useful paired comparison:

Thinking gives larger gains on several difficult tool benchmarks than on SWE-bench Verified. That suggests mode choice should depend on task type and cost, rather than assuming the gain is uniform.

The table does not report complete latency and token distributions for these pairs. A quality gain alone does not identify the best choice for a time-sensitive application.

Redundant verification is a measured limitation

The authors observe that V3.2 often checks itself redundantly, producing long trajectories. In tasks such as MCP-Mark GitHub and Playwright, this can exhaust the 128K context and reduce final success.

This is a useful reminder that tool competence includes stopping. A model may understand the interface and still spend its budget inefficiently. Long reasoning can become a liability when the marginal call adds little information.

Context management can alleviate the capacity problem, but it does not automatically teach efficient action selection. A separate stopping or budget-allocation improvement could be needed.

What “generalization” means here

The report says the benchmark environments and toolsets were not used during RL. The improved scores therefore support transfer beyond the exact trained interfaces.

This does not establish the absence of all benchmark-related material from pre-training, nor guarantee performance on every new business workflow. The synthetic-data ablation strengthens the transfer story, but remains bounded by its evaluated domains and setup.

Evaluate the whole system

A credible comparison records model revision, thinking mode, framework, tool roles, context strategy, attempt budget, external API, and evaluator. It reports success alongside tool calls, tokens, elapsed time, and failure categories.

V3.2 narrows the gap to the compared closed systems on many agent rows and leads some of them. It remains below the strongest comparison on several long-tail tool benchmarks. The right conclusion is a strong and uneven system profile, not universal agent dominance.

Sources: report §4.1 and Appendix C, sections/02-post.tex, tables/eval_main.tex, and tables/nonthink.tex. Continue with Speciale and competitions.

21. Speciale and Competition-Level Reasoning

DeepSeek-V3.2-Speciale explores what happens when training gives reasoning more room and places less emphasis on short outputs. It shares the V3.2 architecture but follows a narrower reasoning-focused post-training recipe.

The resulting scores are impressive, but understanding them requires separating the checkpoint's behavior from the additional compute and selection used during evaluation.

What changes in training

Reported: Speciale is trained exclusively on reasoning data with a reduced length penalty. It also incorporates the dataset and reward method of DeepSeekMath-V2 to improve mathematical proofs.

The official model card states that it does not support tool calling. It is designed for deep reasoning rather than the main model's broad agent role.

This is a training change, not merely a longer inference cap. A model trained to explore longer can learn different search and revision habits. Allowing the main checkpoint a large output budget does not establish equivalence to Speciale.

Accuracy and output length together

The report gives the following paired scores and output token counts, rounded to thousands:

The first number is a percentage except in the Codeforces row, where it is a rating. The second is the reported token count.

Speciale improves over the main model on every listed row. It exceeds Gemini on several math rows, while remaining below it on LiveCodeBench, GPQA, and HLE, and slightly below its Codeforces rating.

Thus “on par” summarizes a profile of strengths and weaknesses. It does not mean equal capability everywhere.

The cost of the improvement

Using rounded counts, IMOAnswerBench rises by 6.2 percentage points while output grows from 27K to 45K, about 1.67 times. Codeforces rises by 315 rating points while output grows from 42K to 77K, about 1.83 times.

These are derived comparisons, not cost-normalized causal estimates. Training changed as well as output behavior. Token counts across different models also use different tokenizers, so they are not exact hardware-cost measures.

The report explicitly says Speciale's token efficiency remains substantially worse than Gemini's and cites deployment cost and latency as reasons for stricter constraints in the main model.

The reported competition outcomes

These are reported offline evaluations. They should not be described as the model physically participating in the human contest or receiving an official medal.

The IMO per-problem scores are $[7,7,7,7,7,0]$. The CMO scores are $[18,18,9,21,18,18]$. IOI scores are $[100,82,72,100,55,83]$. These profiles show both successes and remaining unsolved or partially solved tasks.

IOI is a multi-candidate pipeline

The appendix sets maximum generation length to 128K and states that no tools or internet access are used during model generation, with contest time and attempt limits respected.

For IOI, the system first samples 500 candidate solutions per problem. It filters out submissions that fail provided sample tests or violate length constraints. V3.2-Exp is then used to remove candidates that explicitly express inability or refusal. From the remaining candidates, it chooses the 50 with the longest thinking traces for final submission.

The official rules allow up to 50 submissions per problem and use maximum achieved points across subtasks. The report designs its submission stage around those rules.

This is not a one-shot pass@1 result. It includes broad candidate generation, automatic filtering, and a heuristic favoring longer reasoning among valid candidates.

The appendix's “no tools” statement must be read alongside its filtering procedure. Candidate generation is tool-free; the surrounding evaluation still runs sample tests and submission checks. It would be misleading to describe the complete pipeline as involving no computation outside one model response.

What the IOI protocol leaves open

The appendix states adherence to contest time limits, but does not disclose enough scheduling, hardware, and candidate-generation timing to reconstruct the complete wall-clock budget independently.

Sampling 500 candidates and submitting at most 50 are different budgets. A fair comparison with another system should account for both, not only the final submission cap.

Choosing long traces is a heuristic, not an oracle for correctness. Its effectiveness depends on the candidate distribution. A stronger verifier or different selector could change results at the same generation budget.

ICPC uses a smaller pool

For ICPC, the authors generate 32 candidates per problem and apply the same filtering methodology. The table reports how many submissions were needed for each solved problem, with ten solved and two unreported as solved.

The protocol again combines model generation and surrounding selection. Competitive programming performance under multiple attempts measures more than the probability of a correct first response.

The report says the coding-competition results were obtained without targeted training. That is a statement about contest-specific training, not an assertion that the model lacked general programming or reasoning training.

IMO and CMO use generate–verify–refine

For proof tasks, the model iteratively generates and evaluates its solution, refining it until reaching a perfect self-evaluation or a maximum revision cap. The procedure follows the DeepSeekMath-V2 approach.

Self-evaluation is a search signal. It does not make the final proof correct by definition. A reliable result still needs an external grading standard, and the V3.2 report does not disclose every grader detail or revision cap in this appendix.

The official release provides final submissions for secondary checking. Those artifacts support further verification, but this book does not claim to have independently regraded every competition solution.

What the results establish

The competition results demonstrate that the checkpoint, combined with the disclosed sampling and revision procedures, can produce solutions reaching high contest thresholds. They show the potential of expensive open-model reasoning.

They do not establish a universal pass@1 gold-medal solver, an equal-cost comparison with every closed model, or a guarantee on future contests. The protocol is part of the achievement and belongs in any faithful description of it.

Sources: report §3 and §4.2, sections/02-post.tex and sections/06-high.tex; Appendix D, sections/appendix.tex; official model card and submission locations. Continue with the implementation.

22. Understanding the Released Implementation

The official release lets readers inspect the architecture and message encoding. It does not provide the complete private training system or the full production serving stack. The supplied inference code is best treated as an executable architectural description whose numerical and formatting conventions matter.

This chapter uses the revision-pinned files preserved under ../_sources/. No model weights were downloaded and no GPU inference was performed for this book.

Three sources with different authority

These sources can differ without being contradictory. For example, the paper evaluates at 128K; the checkpoint configuration permits 163,840 positions; the demo's default allocation is 16,384. One describes an experiment, another a configured positional range, and another a default runtime buffer.

The Hugging Face inference configuration is named config_671B_v3.2.json. Its dimension fields override many generic ModelArgs defaults. Reading defaults alone would describe a smaller example model rather than the released large checkpoint.

Follow the forward pass

Transformer embeds token IDs, constructs positional frequencies, applies the repeated Block modules, normalizes the final state, and projects to vocabulary logits.

Block applies normalized MLA attention with a residual addition, followed by a normalized dense FFN or MoE with another residual addition. The first three blocks use dense FFNs according to the large-model configuration; later blocks use MoE.

MLA computes the query latent and main queries, constructs the compressed KV content and positional keys, updates the cache, obtains indexer-selected positions, and applies the attention pattern. Indexer builds ranking queries and keys, updates its own cache, and computes top-k indices.

The important source distinction is that the demo can form dense score tensors and mask them afterward. That implements the logical sparse pattern without necessarily realizing the production sparse arithmetic savings. The report's service measurements depend on optimized kernels.

A map of useful code symbols

The reference chapter provides line locations for the preserved files. Named symbols are often more robust than line numbers when comparing another revision.

RoPE layout is a correctness detail

The demo uses non-interleaved RoPE layout in the indexer and interleaved layout in the MLA positional path.

Interleaved layout pairs adjacent coordinates. Non-interleaved layout groups the paired coordinate components differently. Applying the right mathematical rotation to the wrong coordinate arrangement changes the model's dot products.

The official Exp repository documents a November 2025 correction to this indexer layout issue. The preserved final-model implementation explicitly passes the non-interleaved flag for indexer query and key rotations.

This illustrates why implementing only the headline equation is insufficient for checkpoint fidelity. Projection dimensions, coordinate order, normalization, and scale factors can all affect selection and output.

Quantization is not one global dtype

The release configuration identifies FP8 quantization with dynamic activations, E4M3 format, UE8M0 scale format, and 128-by-128 weight blocks. Other configuration fields and tensors use BF16 or FP32.

The indexer keeps FP8 keys and separate scales. Its head-weight projection uses FP32 parameters in the demo for convenience, with a comment that the checkpoint stores them in BF16. Main KV latents are quantized and dequantized to simulate deployed precision, while the demo cache tensor itself can remain higher precision.

Thus “the model is FP8” should not imply every operation, parameter, cache field, or accumulation uses one-byte arithmetic. Low-precision systems mix storage and computation formats according to sensitivity and kernel support.

Quantization also changes numerical behavior. If two candidate index scores are close, rounding can swap their rank and change the discrete selected set. Stable implementations need to test both numerical agreement and task performance, especially around selection boundaries.

Weight memory still dominates a small-machine deployment

Using the inherited approximate 671B parameter count, a one-byte-per-weight estimate is about 671 GB before scales, higher-precision components, caches, workspaces, and runtime overhead. BF16 would approximately double the raw weight bytes.

This is a scale illustration, not a requirement for a particular deployment. Sharding, compression, and expert offloading alter placement and throughput. The model's smaller activated parameter count does not eliminate the need to access its much larger expert pool.

A deployment decision should therefore start from total weight storage and interconnect requirements, then add concurrency-dependent caches and workspaces. Simply comparing the active count to a dense model's memory footprint is misleading.

The encoding layer is part of the model interface

The official release provides Python code for encoding OpenAI-shaped messages and parsing generated text. It introduces a revised tool format and thinking-with-tools behavior.

The model card says there is no Jinja chat template in this release. It also says the example parser handles well-formatted strings only and is not production-ready without robust error handling.

The code serializes tool calls through dedicated markers and encodes function results separately. It uses the latest user boundary to determine which assistant reasoning survives. An application that manually concatenates messages can easily miss these conventions.

The model card's special developer role is dedicated to search scenarios and is not accepted by the official API in that release. It must not be assumed to have the same general semantics as a role with that name in another model provider.

A compatibility test before serving

Before a full agent benchmark, test a structured conversation containing a user request, an assistant call, a tool result, and continuation. Confirm marker order, retained reasoning, and the expected assistant prefix. Then introduce a new user message and confirm older reasoning is removed while action history survives.

Also test malformed outputs in the surrounding runtime: incomplete call syntax, invalid arguments, missing tool results, and interrupted generation. These checks evaluate integration behavior, not the model's reasoning quality.

For numerical fidelity, compare small fixed inputs against the reference path at the same precision and layout. Check selected indices as well as final logits. A final answer can appear plausible even when the indexer is wrong.

Sampling and generation settings

The model card recommends temperature 1.0 and top-p 0.95 for local use. The paper specifies temperature 1.0 for its evaluations. These are recorded source settings, not a claim that one sampling policy is optimal for every application.

Generation limits, repetition handling, tool budgets, and context policies need separate configuration. Speciale's tool-call limitation is a model-use distinction; a runtime cannot safely assume it supports the main model's agent interface merely because their architectures match.

What remains outside this book's checks

This book cross-checks source dimensions, code paths, encoding behavior, arithmetic, and document rendering. It does not benchmark H800 kernels, load all weights, reproduce RL training, or independently re-run the competition pipelines.

That boundary keeps architectural explanation separate from a deployment certification. The released code is valuable evidence, but measured production performance must come from a real deployment under specified conditions.

Sources: pinned official inference source, release configuration, model card, and encoding code; report §2 and Appendix A; Exp repository revision. Continue with a worked request.

23. One Request Through the Whole System

The preceding chapters explain individual mechanisms. This chapter joins them through an illustrative software-repair request. It is not a transcript generated by the released model and does not claim to reveal the model's private internal reasoning.

The purpose is to trace the data and decisions that a V3.2-style system must handle from input to successful completion.

The task

Suppose the user asks:

Repair a parser bug where an empty quoted field disappears. Preserve existing behavior for escaped quotes and delimiters, add a regression test, and run the relevant tests.

The runtime exposes tools to read files, edit files, and run commands. Its transcript includes a long repository guide and prior code observations. The task therefore requires instruction following, information retrieval, patch construction, and outcome verification.

The model does not receive the correct patch. It must discover relevant code through the tools. The runtime can enforce tool schemas and execution boundaries independently of the model.

Step one: encode the request and tools

The structured message list is serialized using the release's encoding conventions. Tool descriptions tell the model what actions are available and how arguments are formatted. The user request marks a genuine new task boundary.

If this is a continuation of an older conversation, previous assistant reasoning is removed according to the new-user rule, while relevant tool-call history can remain. The initial task instructions and observation history become model input tokens.

The tokenizer maps this text to integer IDs. Embeddings produce the initial vectors. Context accounting uses these actual tokens, including wrappers and tool descriptions, rather than only the visible user sentence.

Step two: prefill builds per-layer state

Each transformer layer processes the prompt causally. MLA builds a compressed historical content latent and positional key for each token. The indexer builds its own key cache.

The current hidden representation changes from layer to layer. Therefore, a token's layer-20 latent is not simply the same vector as its layer-1 latent. Every layer stores representations appropriate to its own computation.

For a long prompt, DSA selects up to 2,048 historical entries per query in the main attention. Early query positions can access fewer because causality limits the eligible history.

The efficient production kernel can perform sparse reading. A demonstration that calculates dense scores and then masks them has the same intended attention support but a different executed cost.

Step three: the first action is generated token by token

The assistant may choose to inspect parser files and tests. That choice emerges through autoregressive token generation. The model emits the serialized call name and arguments; the runtime recognizes a completed call and executes it.

At each decode step, the indexer scores the available history for the current query. The selected positions may include the user's empty-field requirement, a tool schema, or relevant repository instructions. Main attention assigns its own weights within that set.

This is an illustrative description of possible relevant information, not an interpretability claim that a particular head or layer definitely selects those exact tokens. Observing that would require instrumenting the model.

The FFN stage routes each token to eight routed experts plus the shared expert in MoE layers. Those experts are neural modules, not separately invoked domain-teacher models.

Step four: a tool result becomes an observation

The file-reading tool returns parser code and tests. The runtime appends the result with role tool. Previous reasoning within this user task remains available, allowing the model to continue its working plan.

If the runtime instead represents the result as a fresh user message, it can trigger reasoning removal. The same file contents can therefore lead to different subsequent behavior because the transcript boundary changes.

The observation is not a policy-generated action. In an RL rollout, its text conditions later decisions but should not be treated as tokens whose probability the policy chose. Generated-token masking is necessary when constructing the learning loss.

Step five: select evidence rather than all text

Assume the context now contains tens of thousands of tokens, including unrelated modules. The indexer ranks history using its lightweight query/key representations. The selected set is shared by the main query heads at that layer.

One head can focus on the user's constraint while another focuses on a function definition, but both operate within the selected support. A crucial excluded token cannot be recovered by that layer's main attention weighting.

The continued-training recipe was designed to reduce such retrieval misses by teaching the indexer from attention distributions and adapting the model under sparse access. It does not guarantee that a long codebase's every relevant detail is always selected.

In practice, a tool call that narrows the observed file region can also improve information availability. Architecture and agent information-gathering strategy work together.

Step six: propose a patch and a regression test

The model edits the parser and adds a test for the empty quoted field. It should preserve existing escaped-quote and delimiter cases.

The runtime applies the edit to actual files. Once applied, external state has changed. Deleting the conversation transcript would not undo the patch. This matters if a long trajectory later uses context resets.

The model then calls the test runner. The result may expose a mistake: perhaps the patch preserves empty fields but incorrectly creates an empty trailing field after every delimiter. The model must use that observation to revise its hypothesis.

The task is interactive reasoning, not merely producing a static program from a specification. Correctness depends on responding to evidence.

Step seven: stop at a verified outcome

Suppose the relevant tests pass after revision. A good assistant checks that the specific regression test ran and that the pre-existing tests did not fail. It then reports the change and its validation.

Repeatedly running identical tests may add little information. The report observes that V3.2 sometimes performs redundant self-verification, consuming tokens and context. Successful action selection includes deciding when additional checking is worth its cost.

The outcome verifier for a training task could require nonzero F2P tests and no P2F regressions relative to the prepared environment. That checks a specified test set; it does not prove the parser correct for all possible inputs.

Step eight: what if context approaches the limit?

A long debugging trajectory can fill the window through logs, files, and repeated attempts. A runtime may summarize prior work, discard old observations, or restart context.

For coding, any reset must account for persistent filesystem state. A summary should preserve applied changes, failing tests, and the current hypothesis. A full discard strategy that works well in search cannot be transferred blindly to state-changing software work.

Sparse attention makes reading more affordable, but the finite positional/runtime limit remains. DSA does not permit unlimited transcript accumulation.

Step nine: how the same trajectory becomes RL data

During training, the system can sample several trajectories for the same issue. A verifier gives each an outcome reward, possibly combined with length and consistency components. GRPO subtracts the group's mean reward to form advantages.

For a successful trajectory with positive advantage, the policy surrogate encourages the generated action tokens. For a failed trajectory with negative advantage, it discourages them within the clipped update regime.

The learner uses recorded old probabilities from the actual sampling framework. It replays expert-routing paths and vocabulary truncation masks. A highly off-policy negative sequence can have its policy-surrogate contribution masked, while the displayed KL term remains separately applied.

The report does not spell out how indexer parameters are handled during every RL update. Its separate attention-alignment objective is disclosed for continued pre-training, and sparse attention remains in use during post-training. A reader should not infer an additional direct agent-retrieval objective that the report does not specify.

Step ten: where specialists entered before the request

The coding specialist and other specialists supplied teaching data during post-training. Their behavior was distilled into the final checkpoint and refined with mixed RL.

At user inference time, the request does not normally trigger a hidden call to those complete specialist teachers. The released model executes its own learned parameters. Its neural experts remain internal FFN modules.

This distinction separates training provenance from runtime architecture. It is one of the easiest confusions to make when a report discusses both experts and specialists.

A compact accounting of the whole request

The final result depends on all of them. A wrong role assignment can damage a strong model. A weak verifier can approve an incomplete patch. A correct architecture can still be slow if executed through dense masking rather than sparse kernels.

The integrated view explains the paper's structure: efficiency makes long trajectories feasible, stable RL makes learning from them productive, and diverse environments teach the decisions that lead to useful outcomes.

Sources: this is an illustrative synthesis of report §§2–4 and the pinned release code, not a measured model transcript. Review limitations and research questions for the boundaries of these conclusions.

24. Limits, Practical Choices, and Research Questions

DeepSeek-V3.2 presents a coherent design: compact latent memory, selective attention reading, stable large-scale RL, and diverse agent practice. Its evidence supports strong reasoning and substantial agent improvements, with important limits in knowledge, efficiency, and complex-task reliability.

This chapter separates the authors' acknowledged weaknesses from further questions raised by the disclosed mechanisms.

The authors' three acknowledged limits

First, the report says the model's breadth of world knowledge lags leading proprietary systems and relates this to fewer total training FLOPs. It proposes scaling pre-training compute.

Second, token efficiency remains a problem. The model can need longer generation trajectories to match a system such as Gemini-3.0-Pro. Speciale makes the tradeoff especially visible: higher reasoning scores accompany substantially longer outputs.

Third, complex-task solving remains below frontier models in some settings. The authors propose further foundation-model and post-training improvements.

These are not incidental exceptions to an otherwise universal ranking. They identify where the final benchmark profile is uneven and where more efficient attention alone cannot supply missing capability.

Architecture limits

DSA can exclude relevant positions. Its selection quality matters as much as the reader's expressive power. A sparse attention model should be evaluated on tasks requiring several distant pieces of evidence, not only one obvious retrieval target.

The indexer remains quadratic across a full sequence. Extending the model far beyond the tested 128K regime would eventually make that scan increasingly important, even with efficient FP8 kernels.

Sparse reading also leaves linear cache growth. Each layer retains the historical latents and indexer keys needed for future selection. The architecture does not provide indefinite memory or automatic compression of the transcript into a fixed-sized state.

Learning limits

Group-relative outcome rewards can have weak signal when every candidate fails or every candidate succeeds. Task difficulty and sample diversity therefore matter.

Outcome rewards also give coarse credit assignment across long trajectories. A successful final result can reinforce redundant or mistaken intermediate steps. Length penalties and richer rewards can help, but the report does not fully isolate their contribution to efficient stopping.

Replay mechanisms improve consistency but do not remove all off-policy effects. Old contexts remain old contexts, importance weights can have high variance, and clipping/masking deliberately changes the learning signal.

Agent-data limits

Automatically constructed verifiers can share errors with automatically generated solutions. A task that passes the pipeline's internal check may still contain an ambiguous specification or unintended loophole.

Solvability filtering creates a curriculum that fits the current model's reach. It can exclude tasks that are valid but beyond the model's present strategies. This is useful for learning signal, but it bounds the dataset's difficulty diversity.

Real-tool environments add changing dependencies and service behavior. A search task's evidence can drift; a repository setup can fail for infrastructure reasons. Reproducibility requires preserving more than the natural-language prompt.

Evidence limits

The report does not release full confidence intervals, repeated-seed studies, and task-level logs for every benchmark. Several comparisons change more than one factor at once, such as checkpoint training and output length.

Some agent results use internal frameworks or environments. The report states those conditions, and a faithful reader should retain them when comparing with external scores.

Competition results include candidate sampling, filtering, and revision. They demonstrate a high-compute system's capability, not an untouched one-response model's gold-medal probability.

Choosing between main V3.2 and Speciale

The main model is the relevant release for broad assistant and tool-use tasks. Speciale is a reasoning-focused option when its additional output cost and lack of tool calling fit the task.

This is a conclusion about the preserved release descriptions, not a current service recommendation. A present-day choice should verify available model revisions, provider limits, and current implementation support.

For a simple coding edit or a framework that represents tool outputs as user turns, the report's non-thinking mode can be a sensible condition to test. For difficult long-tail tool tasks in a compatible runtime, thinking shows larger gains in several appendix rows.

The right choice is empirical: compare success, tokens, elapsed time, and context failures on the application's actual task distribution.

Research direction one: characterize the selector

Vary $k$ while holding other conditions fixed. Measure retained attention mass, marked-evidence recall, final task accuracy, and latency. Test contexts with many semantically similar distractors and requirements spread across several distant positions.

Such an experiment would reveal whether the 2,048-token budget is mostly generous, often binding, or disproportionately harmful for particular tasks. It would also distinguish ranking quality from reader quality.

A separate study could compare token-level selection with block-level selection at equal executed cost. Finer granularity may save wasted reading but increase gather overhead. The best tradeoff is hardware-dependent.

Research direction two: test the RL interventions independently

Use a fixed base checkpoint, task mix, rollout budget, and sampling policy. Compare corrected versus uncorrected KL, routing replay on/off, mask replay on/off, and negative sequence masking at several thresholds.

Measure divergence, gradient statistics, language consistency, reward, and held-out task success. An intervention can improve stability without improving final quality, or improve one domain while restricting another.

The report describes practical benefits, but a full factorial study would better separate interactions among the mechanisms. That is a proposed experiment, not something the report already contains.

Research direction three: make agent success cost-aware

Report accuracy as a function of total tokens, tool latency, tool count, and dollar cost. Distinguish successful first attempts from recovery after several resets. Analyze whether repeated verification is actually preventing errors or mostly wasting budget.

A learned stopping policy could compare expected value of another observation with its cost. A context controller could preserve task-critical state while removing low-value history. Both require evaluation on state-changing tasks as well as search.

Research direction four: audit synthetic environments

Independently inspect a sample of specifications, tools, reference solutions, and verifiers. Classify failures into ambiguity, unreachable solution, incorrect verification, leakage, and infrastructure error.

Then vary environment diversity at fixed task count and compute. This would test whether gains come from many interfaces, many prompts per interface, or particular task families.

The generalization experiment already supports synthetic data's usefulness. These studies would explain which qualities of the synthesis pipeline make that usefulness repeatable.

Research direction five: separate knowledge from reasoning

Compare closed-book and evidence-provided versions of knowledge-heavy tasks. If providing reliable context closes a gap, missing knowledge is a likely factor. If the gap persists, reasoning, instruction interpretation, or evidence integration may be more important.

Such experiments would refine the authors' broad training-FLOPs explanation and guide whether additional pre-training, retrieval, or post-training offers the highest return.

What remains necessary for a full reproduction

The missing pieces include detailed continued-training data composition, specialist datasets and budgets, distillation proportions, reward coefficients, RL hyperparameters, environment-generation prompts, verifier audits, and production kernel/scheduling details.

A reproduction can implement the disclosed algorithms and study them honestly. It should name its own choices and avoid claiming checkpoint equivalence solely because it uses DSA and GRPO.

The durable contribution

The report's contribution is not one isolated score. It shows how an open-model program can align memory efficiency, selective computation, stable learning, and environment diversity around long problem-solving trajectories.

The most valuable lesson for a reader is methodological: follow a claim through the architecture, objective, execution path, and evaluation protocol. When those levels are kept distinct, the model's strengths become clearer and its limitations become actionable research questions.

Sources: report §5, sections/02-post.tex; architecture, RL, agent, and evaluation sections discussed throughout this book. Proposed experiments are interpretation. Continue with the reference atlas.

25. Glossary, Equations, and Source Atlas

This chapter collects the book's terminology and factual anchors. It is designed for readers who want to check a number, locate an original equation, or resolve a confusion without rereading the entire narrative.

The central distinction remains the same throughout: a reported experiment, a released configuration, a mathematical derivation, and a practical interpretation are different kinds of evidence.

The source snapshot

The primary paper is DeepSeek-AI, DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models, arXiv:2512.02556v1, submitted 2 December 2025. The source archive was downloaded from the versioned arXiv source endpoint on 13 October 2026.

The unmodified archive is preserved as ../2512.02556v1-source.tar.gz. Its SHA-256 is:

ebac277dba6a19100bec4fc921e6716157bf96d48b2f417e27cc4289a3d61c7d

The extracted TeX entry point is ../main.tex; ../00README.json identifies it as the top-level source and specifies the original compiler environment. The archive also contains the class, macros, bibliography, sections, tables, and figures.

Official supplemental files were downloaded from the DeepSeek-V3.2 Hugging Face release at revision:

a7e62ac04ecb2c0a54d736dc46601c5606cf10a6

They include config.json, the model card saved as model-card.txt, inference-config.json, model.py, encoding_dsv32.py, and the release LICENSE. The inference GitHub repository was also identified at revision:

87e509a2e5a100d221c97df52c6e8be7835f0057

Metadata and per-file SHA-256 values are recorded in ../_sources/source-manifest.json. This makes later source changes distinguishable from the material used for the book. The supplements were retrieved later than the paper date; they illuminate the preserved release, not every detail of its original private training run.

Where the paper's parts live

All paths below are relative to papers/dsv32/ in the documentation source tree.

These line locations apply to the preserved archive. Named section headings and table labels should be used when inspecting another revision.

Where the implementation details live

The inherited approximate 671B-total and 37B-active figures are supported by the local DeepSeek-V3 source, ../dsv3/main.tex, and the DeepSeek-V3 report. They should not be interpreted as an exact fresh count including every V3.2 addition.

A factual ledger

Glossary

Equation map

The indexer ranks token positions:

The reader uses the selected set:

Indexer supervision uses attention-distribution alignment:

GRPO's displayed mean-centered advantage and token ratio are:

The corrected fixed-context KL sample is:

In that final equation only, $p,q,r$ denote old, current, and reference distributions. This $r$ is not the GRPO importance ratio. Chapter 10 defines compatible-support and gradient assumptions.

Frequently confused questions

Is V3.2 a completely new backbone?

It inherits the Terminus backbone and introduces DSA through continued training. The final capability gains also involve substantial post-training. Exp and the final V3.2 share architecture but differ in the learning program and measured behavior.

Is the entire attention system linear in context?

No. The main reader becomes $O(Lk)$ over a sequence, but the smaller indexer remains $O(L^2)$. Practical long-context savings and overall asymptotic class are different claims.

Does selecting 2,048 entries mean retaining only 2,048 tokens?

No. The history remains available because future queries can choose different positions. The selector limits the current expensive read, not total retained entry count.

Are selected entries compressed blocks of multiple tokens?

In this V3.2 design, they correspond to individual token positions represented by MLA latents. Feature compression and multi-token pooling are different operations.

Do the indexer scores become the main attention probabilities?

No. The indexer selects the candidate set. Main attention computes its own weights within that set. Its query heads can use the shared set differently.

Does the GRPO equation divide by reward standard deviation?

The displayed V3.2 equation subtracts the group mean. A reader should not import standard-deviation normalization from another GRPO implementation without stating the change.

Does an unbiased KL estimate make the whole RL update unbiased?

No. The derivation concerns a conditional token-level regularizer under specified sampling and differentiation assumptions. Old prefix distributions, clipped surrogates, truncation, and sequence masking introduce additional considerations.

Does Keep Routing freeze the MoE model?

No. It replays active expert paths for the sampled trajectory. Those parameters can still be updated. The report does not claim all numerical activations or routing weights remain identical.

Can tool results be placed in user messages without consequences?

The report's thinking-retention rule treats a new user message as a boundary that removes old reasoning. A runtime using user messages for tools can therefore alter behavior. Check the actual framework and encoding version.

Does non-thinking mode lack reasoning capability?

It still performs the same neural forward computation and can solve difficult tasks. The distinction concerns emitted extended reasoning and training/formatting behavior, not the absence of internal computation.

Does context management enlarge native attention capacity?

No. It keeps each segment within the window while allowing more total task work. Summaries and resets change which information remains available.

Is Speciale simply V3.2 with a larger output cap?

No. It follows reasoning-only training with a reduced length penalty and proof-oriented data/rewards. The official release says it does not support tool calling.

Are the competition scores pass@1?

No. IOI and ICPC use candidate generation and filtering; IMO and CMO use iterative generate–verify–refine. The complete protocol belongs with the score.

Can the full model fit wherever a dense 37B model fits?

The active parameter count concerns computation for one token. The much larger total expert pool still requires accessible storage. Memory, communication, and offloading need separate analysis.

Does the demo demonstrate production throughput?

It clarifies architecture and precision conventions. Some paths calculate dense tensors then mask them. Efficient sparse kernels and production scheduling are additional requirements.

Checks performed for this book

The source archive and supplementary files were retained and hashed. Configuration values were cross-checked against the chapters. The token-count, pair-count, cache-size, task-count, and KL examples were recomputed.

The official encoder was executed on a small synthetic conversation. It retained reasoning after a tool observation, removed old reasoning after a new user message, and preserved the tool-call/observation history. Python source syntax was parsed without loading the neural model.

Document checks passed for all 26 Markdown files, including frontmatter, internal chapter routes, seven image references, and 250 inline or displayed KaTeX expressions. The full Astro build also completed successfully, generated all 26 book routes, and reported all internal site links valid. Existing theme, missing-i18n/404, and sitemap-configuration warnings remain outside this book's scope.

No model weights, private training data, GPU performance run, full benchmark suite, or independent competition regrading was performed. The book's purpose is explanation grounded in preserved sources, with explicit limits on verification.

Return to the book's contents for a reading path or a particular chapter.