Introduction

DeepSeek-V4.1-Flash is best understood as an attempt to make sustained, input-heavy agent work affordable. Its report connects a causal encoder-decoder, attention memory shared across layers, sparse retrieval, low-precision caches, approximate local-state reconstruction, conditional parameter memory, multimodal training, and large-scale agent learning. These ideas address different bottlenecks. Their interaction is the subject of this book.

The primary source is DeepSeek-AI's DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression, a 51-page technical report downloaded from the model repository. A verified copy is saved at papers/dsv41/DeepSeek_V41_Tech_Report.pdf beside this book. The book also inspects selected released configuration, inference, prompt-encoding, and evaluation files. Chapter 29 records the source manifest, checksums, revisions, and discrepancies.

What kind of book this is

This is an explanatory technical reader, not a reproduction claim or a live model leaderboard. It develops the background the paper assumes, explains mechanisms in words and equations, works through numerical examples, and examines what the evidence supports. Each chapter contains its own starting vocabulary and source notes so it can be read independently. In sequence, the chapters move from the workload to the model, then training, deployment, evidence, and practical judgment.

Reported means stated by the paper or identified release file. Derived means calculated from stated quantities under explicit assumptions. Interpretation means an engineering explanation, implication, or hypothesis rather than a controlled experimental result. Standard mathematical background is explained directly. Illustrative examples are not measurements of the released model.

The paper is unusually rich in systems detail but incomplete as a replication specification. Exact task inventories, many data proportions, full production code, hardware cost measurements, and several ablations are unavailable. The book explains the missing pieces without filling them with invented facts.

Contents

Suggested reading paths

For a first pass, read 1–3, 6, 8–12, 22, 24, and 28. For architecture, read 2–16 and 27. For training, read 4–5 and 16–22. For infrastructure, read 6–15, 18, 21, and 26. For evaluation, read 22–25 and 28. The source map makes it possible to return to the paper from any path.

Conventions

Layer identifiers are zero-indexed unless explicitly called ordinal positions. Thus layer 20 is the twenty-first backbone layer and the first decoder layer. The backbone has 40 layers; the DSpark module adds three draft blocks outside that count.

The book uses decimal GB and TB for arithmetic. “One million tokens” in examples means 1,000,000 positions; the released maximum position setting is 1,048,576. Visual embeddings consume positions too. Global cache estimates omit local windows, weights, allocator overhead, temporary tensors, and runtime metadata unless stated otherwise.

All model rankings, versions, API tiers, and scores refer to the inspected report or release snapshot. Names appearing in the report are reproduced as source labels, not asserted as independently verified current product identities. No full model inference or training run was performed while preparing this book.

1. The Workload and the Problem

A long-running agent repeatedly alternates between generating instructions and receiving new information. It reads files, runs commands, inspects tests, opens documents, and incorporates tool outputs. The resulting transcript is a growing working environment. DeepSeek-V4.1-Flash is designed around the cost of maintaining and extending that environment.

The paper's central claim is that efficiency requires coordinated changes to computation, stored state, and data movement. Reducing attention arithmetic alone leaves the other two costs exposed. This chapter develops that motivation before discussing the architecture.

Why agent input can outweigh output

Imagine an agent making a 200-token tool call that returns a 20,000-token build log. Its next decision may take a few hundred tokens, but the inference service must first process the log. Later, the transcript might contain a repository overview, failed test traces, screenshots, and several attempted fixes. The model must continually incorporate much more material than it emits.

This example is illustrative. It explains “input-heavy” without asserting a measured universal input/output ratio. Different workloads vary: a long mathematical solution can be output-heavy, while searching a large repository can be input-heavy.

The cost also depends on caching. If the inference service recognizes an existing transcript prefix, it may reuse its computed state and process only the new suffix. If the cache has expired, moved, or become incompatible, part or all of the prefix must be processed again. Agent sessions therefore depend on both architectural efficiency and reliable state reuse.

Three costs that should be kept separate

Prefill computation is the work required to turn input tokens into model state. A conventional causal Transformer runs its full stack on the prompt. Long inputs increase the number of feed-forward transformations as well as the attention work.

Runtime context memory is the state an active request keeps accessible while generating. Much of it is the attention key-value cache, often stored in accelerator high-bandwidth memory, or HBM. A model with enough compute can still be unable to admit another request because that memory is full.

Persistent prefix memory is cached state retained for future requests. It may live on SSD or host DRAM and have a much longer lifetime than an active decoding request. Reusing it requires storage capacity and enough bandwidth to move the relevant state back to the accelerator.

The paper's advertised reductions concern different members of this list. Approximately halving prefill computation, reducing global KV to about one quarter of V4-Flash, and reducing persistent KV to about one eighth are not interchangeable claims.

Why sparse attention is only part of the solution

In dense attention, each new query compares itself with every eligible past key and reads the corresponding values. Sparse attention restricts the expensive read to selected positions. This can drastically lower main-attention work.

But selection is itself a computation. A lightweight indexer may still scan the entire past context to decide which entries to read. And every stored entry still consumes memory until another mechanism reduces its size or count.

There is also work outside attention. Experts transform representations, residual connections read and write tensors, image encoders process pixels, and distributed communication moves activations and parameters. A small attention operation does not imply a small end-to-end request.

V4.1 consequently addresses several dimensions: less per-entry storage, fewer independent layer caches, less repeated indexing, fewer full decoder passes on prompts, and less persistent storage of local state.

A simple resource model

Let an agent interaction contain $I$ newly processed input positions and $O$ generated output positions. A useful first approximation is

This is a conceptual accounting model, not the paper's measured bill. Cache hits reduce the effective $I$; misses add replay or recomputation. Tool time includes external execution rather than model inference. Transfer cost depends on the cache placement and interconnect.

If a design halves $c_{\mathrm{prefill}}$, the gain is most useful when $I$ is large. If it shrinks stored state, the gain can appear as more admitted sessions, fewer evictions, and lower transfer volume. If it reduces output length, the gain is strongest when decoding dominates. These savings can reinforce each other but cannot simply be multiplied into an overall speedup.

For example, suppose prefill is 60% of an interaction's inference time and all other inference work is unchanged. Halving prefill yields a normalized time of $0.6/2+0.4=0.7$, or about $1.43\times$ faster. It does not yield a universal $2\times$ acceleration. This is Amdahl-style arithmetic, with an illustrative workload split.

Memory is a concurrency constraint

At the reported 890 global-cache bytes per sequence position, one million positions require about 0.89 decimal GB of global cache. One hundred such active contexts require about 89 GB for this component alone.

This estimate excludes model weights, local-window caches, scratch space, scales already or not already included in a particular implementation, and serving metadata. Its purpose is to show why per-token cache size matters at fleet scale. Saving a fraction of a kilobyte per position becomes significant when multiplied across positions and sessions.

A smaller persistent cache can improve hit rates under a fixed storage budget. Higher hit rates reduce repeated prefill, which reduces compute pressure. Memory efficiency can therefore benefit latency indirectly, even when the arithmetic for a single cache-hit decode step is unchanged.

The design's main trade-off

Independent, high-precision, fully populated state gives every layer considerable freedom. Sharing, compression, and replay remove some of that freedom or replace exact state with approximations.

The research question is whether the removed distinctions matter enough to outweigh the savings. The paper reports strong evaluations, but it also explicitly identifies sparse-selection errors and approximate replay boundaries as incompletely characterized robustness risks. That admission is central to an accurate reading.

The model is not merely smaller than its predecessors. Its backbone has more parameters than V4-Flash, while it uses less context memory and less prefill activation. It trades a large learned capacity for a more disciplined runtime information path.

Questions a reader should carry forward

When a later chapter claims an efficiency gain, identify the resource being saved, the baseline, the sequence length, and whether the gain is architectural or dependent on a production implementation. When a benchmark improves, ask whether architecture, more training tokens, better data, post-training, or harness differences could explain it.

Those habits prevent two common errors: treating a memory reduction as an equal wall-clock reduction, and treating an overall model comparison as an isolated ablation of a new mechanism.

Sources and next step

Primary source: report §1, pp. 4–7; §3.2, pp. 18–20; §6, p. 37. The 890-byte estimate is reported; the concurrency and Amdahl examples are derived illustrations. Chapter 2 supplies the vocabulary needed to follow the mechanisms.

2. Foundations: Tokens, Attention, Caches, and Learning

DeepSeek-V4.1-Flash generates text by predicting successive tokens. It turns a sequence of text and visual representations into a probability distribution for the next text token. Most of the report concerns how that computation is organized and how its parameters are learned.

This chapter explains the common vocabulary. The equations here describe standard concepts unless specifically marked as V4.1 details.

Tokens, embeddings, and hidden states

A tokenizer maps text to discrete identifiers. Tokens can correspond to words, pieces of words, punctuation, or other encoded units. Token count is consequently different from word count or byte count.

An embedding table maps each identifier to a learned vector. A hidden state is the vector representation after some amount of model processing. In V4.1, the main hidden width is 5,120. The residual-stream system maintains multiple vectors per position, but expensive attention and expert modules still operate on appropriately mixed representations.

Images enter differently. A vision encoder and projector produce vectors that occupy sequence positions alongside text. The model's million-position capacity must accommodate these vectors and protocol tokens as well as ordinary text.

Causality means no access to the future

A causal model predicts position $t+1$ using information available through position $t$. Attention masks prohibit reading future positions. Training can process many positions in parallel because the complete input is known, but each prediction is restricted to its permitted prefix.

This explains how parallel training and autoregressive generation coexist. Parallel computation during training does not make the learned model noncausal. Nor does the term “encoder” in V4.1 imply a bidirectional encoder that can inspect future tokens.

Attention in its simplest form

For an input vector $h_t$, learned projections construct a query $q_t$, keys $k_j$, and values $v_j$. A conventional attention head computes

Here $\mathcal V_t$ is the set of causally visible positions. The query asks what information would help at the current position; keys allow comparison; values provide the content combined into the output.

Multiple heads learn different comparisons. They are mathematical subspaces, not human-readable named skills. A “key-value head” need not exist separately for every query head: architectures can share KV across heads or represent it through a common latent.

V4.1 uses a shared 512-channel main KV representation and sparse selections plus a local window. Its implementation also includes details such as an attention sink and inverse rotary transformation. The elementary formula above is background, not a complete specification of its attention kernel.

Why a cache exists

During generation, previously processed positions recur in the prefix. Recomputing their keys and values at every step would waste work. A KV cache stores reusable attention state.

The cache is neither the model's learned weights nor a database of explicit facts. It is request-specific derived state. Changing the prompt, model weights, positional convention, or cache representation can invalidate its use.

The cache also does not store every internal activation. Most layer outputs used during computation can be discarded after they contribute to downstream work. Attention state has a special lifetime because future queries need it.

Prefill and decode

Prefill processes an input sequence or newly appended input suffix. Many token positions are available together, so matrix operations can exploit substantial parallelism.

Decode generates new text tokens sequentially. Each step adds state and computes the next distribution. It often has much less work per invocation but repeats many times, making memory bandwidth, launch overhead, and batching important.

A prefill phase may still require the last position's full computation to obtain the first output distribution. “Skip the decoder for prefill” is therefore shorthand for avoiding most bulk decoder work, not for producing final logits without a decoder or doing literally zero decoder work.

Three different ways to reduce attention cost

Sequence compression reduces the number of stored entries. For example, a ratio of two creates one learned representation for each completed pair of positions.

Sparse selection reduces the number of entries read by a query. Keeping one million entries but reading 512 is sparse without sequence compression.

Layer sharing reduces the number of independently stored banks. Several layers can query the same state with their own queries.

Quantization is another axis: it reduces the bits used per stored number. These operations change different aspects of the information path and have different approximation risks.

Feed-forward transformations and experts

Attention mixes information across positions. A feed-forward module transforms each position's current representation. A mixture-of-experts, or MoE, makes several such transformations available and selects a small subset for each token.

A typical SwiGLU-style expert combines two projections through a gated activation and maps the result back to the hidden width. Different tokens may select different experts. The network can therefore contain much more stored capacity than it activates for a single token.

Total parameter count answers a storage question. Active parameter count approximates which learned weights participate in a forward path. Neither alone specifies FLOPs, memory traffic, serving latency, or device requirements.

What pre-training learns

Next-token training minimizes a cross-entropy loss such as

The parameters $\theta$ are updated to make observed continuations more probable. A multimodal prefix can include visual vectors, while predicted outputs remain text.

This objective can teach broad knowledge, syntax, visual associations, and patterns useful for reasoning. It does not directly optimize “solve this repository task under a tool protocol.” Post-training supplies more specific behavioral supervision and rewards.

SFT, RL, and on-policy distillation

Supervised fine-tuning, or SFT, trains on desired responses or trajectories. Reinforcement learning, or RL, changes the policy using scored sampled behavior. A policy is simply the model's conditional distribution over its actions or generated tokens.

On-policy distillation asks teachers to supervise continuations encountered by the student while it generates. “On-policy” concerns where the sampled trajectories come from. It does not imply that every asynchronous training token was generated under the latest checkpoint; the report explicitly discusses policy staleness.

V4.1 uses established forms of these stages. Its post-training emphasis is task quality, diversity, verifiability, and infrastructure rather than a new optimization algorithm.

Reading approximate statements correctly

An architecture can guarantee a maximum number of stored banks or candidate positions. It cannot guarantee that every useful fact survives compression or that every query selects the right evidence. Similarly, empirical negligible degradation from replay is not mathematical identity of the replayed state.

These distinctions separate structural properties, empirical findings, and interpretations. They recur throughout the book because the model's efficiency depends on controlled approximations.

Sources and next step

For the V4.1-specific quantities, see report §§2.1–2.3 and §4.2.1, pp. 7–12 and 21–22, plus released config.json. The introductory equations are standard background. Chapter 3 assembles the complete model configuration.

3. The Complete Model Map

The model has a 40-layer causal language backbone, split into a 20-layer encoder and a 20-layer decoder. Images pass through a separate vision encoder and enter that backbone as embeddings. Engram contributes conditional parameter memory at two locations. DSpark supplies an auxiliary draft path for accelerated generation.

The word “Flash” does not identify a small dense model. The report lists 552B backbone parameters and a further 196B Engram parameters. Its economy comes from selective activation and state reuse.

The diagram restates the report's schedule. Each reuse layer still has its own query, local state, experts, and residual transformations.

The configuration at a glance

These are architecture and training settings. The serving batch size, admitted concurrency, GPU allocation, and latency are separate deployment choices.

The precise attention schedule

The first two backbone layers use only sliding-window attention. Encoder layers 2–19 use compression ratio two. There are three groups of six layers, each consisting of one Full layer and five Reuse layers.

Decoder layers 20–39 use ratio one, retaining one global entry per position in their shared bank. The first four-layer group starts with Full. Each later group starts with Reindex. The remaining three layers in every decoder group use Reuse.

There are consequently four global KV sources and eight index-producing layers. Of the 38 CSA2 layers, 30 use Reuse, four use Full, and four use Reindex. These counts are derived from the schedule and corroborated by the released source-layer arrays.

A bank, a query, and a selection are different objects

A global bank stores learned representations of past positions. A query is computed from the current layer's current token representation. A selection is the list of bank positions that the query will inspect.

Full creates a bank and a selection. Reindex keeps the bank but refreshes the selection. Reuse keeps both but still computes a new main query and performs attention. All backbone layers retain their own local-window state.

Thus the 20-layer decoder is not reduced to one attention output copied repeatedly. It makes many new transformations while reading a common global memory.

Why the decoder is uncompressed along sequence length

Ratio one may appear surprising in a model advertised for compression. Compression here includes the layer dimension and storage precision. A single ratio-one decoder bank shared by 20 layers can be much smaller than 20 independent banks, even if those banks use a sequence-compression factor greater than one.

It also preserves finer position granularity in the decoder's global representation. This is a structural observation, not proof that ratio one caused a particular benchmark improvement. A controlled ratio ablation would be needed for that causal conclusion.

Where CED changes the information path

In the paper's causal encoder-decoder design, decoder global KV is projected from the final encoder hidden states rather than from each decoder layer's hidden states. Bulk prompt processing can therefore stop after the encoder plus the decoder global projection path. A short decoder replay establishes local state.

During text generation, a new token traverses the full backbone. Its decoder queries operate on deeper evolving representations, while the decoder global memory continues to be constructed from the encoder output stream.

The public reference implementation does not fully demonstrate this production fast path. Its plain Transformer.forward loops over all backbone blocks. Chapter 26 discusses that implementation boundary explicitly.

Two auxiliary paths

Engram uses token-derived n-gram identities to fetch learned embeddings. Those embeddings are gated against contextual representations and contribute to the residual streams. This is parameter memory, not an attention cache of the current conversation.

DSpark uses a smaller draft module to propose future tokens that the backbone verifies. The report omits MTP during backbone pre-training and adds DSpark afterward. The released configuration still uses some historical “nextn” names for the draft blocks; a field name should not override the paper's training-stage description.

How V4.1 differs from V4

V4 used a hybrid of CSA and heavily compressed attention. V4.1 uses CSA2 outside the initial pure-local layers. It removes overlapping compression and compressor absolute positional embeddings, projects indexer keys from main latents, and adds cross-layer sharing and decoder candidate restriction.

V4.1 also adds native vision, CED, Engram, Single-Pass mHC, FP4 main cache QAT, DSpark, modified optimization, more pre-training tokens, and expanded agent post-training. The comparison is a broad model change; it cannot isolate any one of these contributions.

The lifecycle matters as much as the block diagram

The vision encoder is separately prepared. Main multimodal pre-training constructs the backbone and integrates visual inputs. A dedicated DSpark stage trains the drafter with a frozen backbone. Post-training applies SFT, RL, and OPD, while adapting mechanisms such as hierarchical indexing, main-cache FP4, and decoder bounded replay.

Inference then adds production-specific cache placement, reconstruction, disaggregation, and fused kernels. A feature can be part of the served system even if it was absent during early pre-training.

Sources and next step

Primary source: Figure 3 and §§2.1–2.4, pp. 7–14; §4.2.1, pp. 21–22. Configuration checks: text_config and vision_config in the saved release configuration. Chapter 4 explains why this large parameter store can have a much smaller active forward path.

4. Experts and Parameter Accounting

A mixture-of-experts layer has many available feed-forward transformations but uses only a few for each token. V4.1 has 384 routed experts and one shared expert in each of its 40 backbone layers. Six routed experts are selected per token.

This chapter separates three quantities often collapsed in model descriptions: learned storage, selected computation, and hardware footprint.

An expert's computation

A simplified SwiGLU expert with hidden width $d$ and intermediate width $f$ computes

There are approximately $3df$ matrix parameters. V4.1 uses $d=5120$ and $f=2304$, giving

weights per expert under this approximation. The report additionally uses activation clamping at threshold 10; that affects the numerical transformation, not this matrix-count arithmetic.

An expert is a learned function rather than a separately trained human-style specialist with a fixed label. Experts can develop different routing preferences, but no public inventory identifies expert 17 as “Python” or expert 52 as “charts.”

Routed and shared outputs

A conceptual MoE output is

The set $S(h)$ is the selected routed experts; $g_e$ weights their contributions. The shared expert processes every token. Common attention, norms, routing, embeddings, and output projections remain outside the routed-expert fraction.

The released configuration identifies square-root-softplus scoring, normalized selected probabilities, and a routed scaling factor of 1.5. These implementation settings supplement the report; the chapter's formula is an explanatory abstraction rather than an exact kernel listing.

A parameter count that explains the backbone

With 385 total experts per layer, the approximate expert matrix storage is

That already accounts for most of the reported 552B backbone. The difference covers non-expert components under the report's accounting and rounding. It should not be treated as an exact independent reconstruction of every checkpoint tensor.

For one token passing through all 40 layers, seven experts per layer are used: six routed plus one shared. The corresponding selected expert matrix count is

The reported decode activation is 16B, not 9.9B, because non-expert matrices also participate. During bulk prefill, CED avoids most decoder processing, yielding the reported 8B active figure. Short replay and projection work still exist.

This calculation demonstrates why multiplying $552\text{B}$ by $6/384$ is incorrect. That fraction applies only to routed-expert storage. It misses shared experts and the common path.

A batch can touch many experts

Different tokens choose different experts. A large batch may collectively use almost every expert even though each individual token uses six. The serving system must arrange those token assignments so each expert receives efficient matrix work.

Very small batches can leave selected experts with tiny workloads. Larger batches improve utilization but can increase queueing latency. Expert parallelism adds communication: token representations must reach the devices holding selected weights and return their outputs.

Thus low active parameter count is an arithmetic advantage, not a guarantee that a commodity device can hold or efficiently serve the model.

Why load balancing is necessary

If most tokens select a small set of experts, those experts become bottlenecks while others sit idle. Training also gives the underused experts fewer useful updates.

Auxiliary-loss-free balancing adjusts correction biases used for selection rather than relying entirely on a large extra loss. The original routing scores remain responsible for weighting selected outputs. V4.1 still retains a small sequence-level balancing loss with weight $10^{-4}$ to discourage extreme imbalance within individual sequences.

The bias-update speed is reported as 0.001 for both modalities. “Auxiliary-loss-free” describes the main correction mechanism; it does not mean the complete recipe contains no balancing loss of any kind.

Why text and images need separate corrections

Text and visual embeddings have different distributions and can prefer different experts. An apparently balanced total load can hide imbalance within a minority modality.

For a toy example, imagine an expert receiving 900 text tokens and 100 image tokens while another receives 1,000 text tokens and no images. Total counts are similar, but image learning is concentrated entirely in one expert. Aggregate balancing does not describe that modality's coverage.

V4.1 maintains separate expert correction biases for text and image tokens, updates them from their respective loads, and chooses the relevant bias set for each token. The uncorrected scores still weight the selected outputs. This separates resource allocation from the learned mixture coefficients.

The paper reports that this supports stable and efficient multimodal training. It does not publish a full per-expert specialization map or isolate the contribution to every visual benchmark.

Engram changes the total again

The report's 552B figure is backbone storage. It separately allocates 196B to Engram. Adding these rounded counts yields approximately 748B for those two reported categories.

That is not a certified count of all released parameters including the complete vision pathway and auxiliary drafter. It is also not the same as 748B active matrix weights. Engram fetches a small set of table rows based on token n-grams.

Engram can increase learned capacity without forcing every token through a 196B dense transformation. Its storage and communication costs still have to be paid somewhere.

Deployment memory is another calculation

At a uniform one-byte-per-weight approximation, 552B backbone weights would occupy 552 GB before scales and overhead. At half a byte per weight, the raw payload would be 276 GB. These are illustrative bounds, not a recommended machine configuration: the actual release uses different precisions for different tensors.

The Engram tables add a large store, sometimes placed in host memory with prefetching. Optimizer states during training add still more memory. KV caches depend on concurrent requests, whereas weights are primarily model-level storage. Keeping those categories separate is essential when sizing a system.

Sources and next step

Report §2.1.1, p. 8; §4.2, pp. 21–22; release config.json, including expert counts and scoring settings. Parameter arithmetic is derived from the standard three-matrix expert approximation. Chapter 5 explains how image tokens reach this MoE backbone.

5. Images Inside the Language Model

V4.1 accepts images and text and generates text. Its multimodal pathway transforms image pixels into vectors that the causal language backbone can process alongside text embeddings. The visual component is native to main language-model pre-training rather than added only after a completed text-only backbone.

This does not imply native image generation, audio processing, or a separately specified video pathway. The report describes image-and-text input with autoregressive text output.

From pixels to patches

The vision encoder is a 32-layer Vision Transformer with hidden width 1,024, 16 heads, and patch size 14. An image is divided into small spatial patches; each patch's pixel values are linearly projected into the encoder's vector space.

For an illustrative square image of $1344\times1344$ pixels, a 14-pixel patch edge gives $96\times96=9216$ patch positions before downsampling. Real preprocessing has to handle aspect ratios and compatible dimensions, so this is a clean example rather than a universal image count.

The report replaces a convolutional patch embedding with a linear projection to make the parameter structure compatible with its Muon optimization approach. This is an implementation and optimization choice; it does not mean images cease to have spatial structure.

Two-dimensional rotary positions

A vision Transformer needs to distinguish patches by location. Standard fixed absolute position tables can complicate generalization to changing image sizes. DeepSeek-ViT uses 2D rotary positional embeddings, or 2D-RoPE.

In one-dimensional RoPE, position-dependent rotations modify paired query and key coordinates. Their dot product depends on relative positional relationships. A two-dimensional variant represents spatial row and column information rather than only a text-sequence offset.

The report does not provide every coordinate-allocation equation for DeepSeek-ViT. The important source-supported point is that position handling is designed for variable-resolution images. Variable resolution does not guarantee unlimited resolution or uniformly accurate extrapolation beyond the trained preprocessing policy.

Why downsample after the vision encoder

The visual grid is expensive to insert directly into a million-token language model. A $96\times96$ grid contains 9,216 positions; many images would consume substantial sequence capacity and prompt compute.

A $3\times3$ pixel-unshuffle rearranges each local group of nine feature vectors into a larger channel vector at one grid position. It reduces the spatial count by nine while retaining the constituent feature values in the channel dimension before projection.

For the clean example above, $96\times96$ becomes $32\times32=1024$ visual positions. The channel width becomes $9\times1024=9216$ before the MLP maps it into the backbone space.

Pixel-unshuffle itself is a rearrangement, not a learned average that immediately discards eight of nine inputs. The subsequent learned projector does reduce and transform information. The entire pipeline is still a compressed representation of the original image; fine details can become harder to preserve.

The projector is an interface

The two-layer MLP projector maps visual features to the language backbone's 5,120-dimensional representation. It allows a visual position and a text embedding to inhabit a compatible input space.

Compatibility is not identity. The model can recognize and route the modalities differently. The separate MoE balancing biases described in Chapter 4 account for this distinction during training.

The released image processor also constructs image-start, image-end, and row-delimiter positions. Consequently, “1,024 visual features” is not necessarily the complete sequence span of an image including protocol delimiters. The configuration's max_image_tokens should be read together with preprocessing, not converted directly into a guarantee about the complete serialized prompt length.

A separately prepared vision encoder

Before integration, the vision encoder undergoes two stages. Contrastive preparation uses approximately 47B image-text pairs with the SigLIP sigmoid contrastive objective. The maximum resolution is restricted to roughly $224\times224$, preserving aspect ratio when downscaling.

The next stage attaches the encoder to a 4B MoE language model and trains on 236B tokens with an autoregressive next-token objective. It includes captions, alt text, charts, and OCR. The resolution policy is between approximately $544\times544$ and $1344\times1344$ through proportional scaling of out-of-bound images.

That smaller language model is then discarded. The prepared vision encoder is retained for main multimodal pre-training. These quantities belong to preparation stages and should not be casually added to the main 45T-token run as if all were identical token accounting.

Why contrastive and generative objectives differ

Contrastive learning teaches which images and texts match relative to alternatives. It is useful for broad visual-semantic alignment. Generative training asks visual representations to support predicting detailed text.

An encoder could succeed at matching an image to “a financial chart” while failing to preserve its axis labels or exact values. Chart descriptions, OCR, and other detailed targets make fine-grained information more relevant to the generative objective.

The report states that using higher resolution in the contrastive stage helped that stage but contributed little to the final system relative to the extra cost. This is a reported experimental judgment, not a general law that low-resolution contrastive training is always sufficient.

Native integration and freezing are compatible

The language backbone sees multimodal data from the start of its main pre-training. Initially, however, most vision-encoder parameters remain frozen. Its final norm and the vision-language projector stay trainable. At learning-rate decay, the vision encoder is unfrozen and trained jointly at a smaller learning rate.

“Native multimodal pre-training” concerns the inputs and objectives used throughout the backbone's training. It does not require every visual parameter to be updated at every step.

What visual capability means in an agent

A visual agent may inspect a screenshot, read a chart, call an image-processing tool, or compare a rendered output with a requirement. The backbone still emits text and tool instructions. The external system executes those instructions and returns new observations.

Successful visual-agent performance therefore combines perception, reasoning, tool use, and protocol competence. A benchmark with tools is not a pure measurement of the vision encoder's recognition accuracy. The report separately evaluates base visual capabilities and post-trained visual agents.

Practical limits

Small text, dense diagrams, extreme aspect ratios, many similar screenshots, and very long image-rich sequences are plausible stress cases. Those are interpretation-based test priorities, not documented failure rates. Image-token compression saves cost but places a representational burden on the encoder and projector.

The report gives no exhaustive robustness curve over resolution, image count, or visual evidence distance. A user should distinguish supported input processing from demonstrated reliability on every visual scenario.

Sources and next step

Report §2.1.1, p. 8; §2.5, p. 15; §§4.1–4.2, pp. 20–23. Supplement: released vision configuration and inference/image_processor.py. Chapter 6 explains how the backbone reduces the cost of processing this combined sequence.

6. Causal Encoder-Decoder: Cheaper Prompt Processing

The Causal Encoder-Decoder, or CED, changes where the decoder obtains its global attention memory. Instead of building global KV from each decoder layer's current hidden state, the decoder's memory is projected from the final causal encoder output.

This breaks a dependency that otherwise forces all prompt positions through the full decoder. It does not eliminate the decoder or make the encoder bidirectional.

The conventional dependency chain

In a conventional causal stack, layer $\ell$ processes the output of layer $\ell-1$. Its cached keys and values are derived from its own input representation. To obtain valid cached state for layer 35 at a prompt position, the system must have computed the preceding layers for that position.

If future generated tokens need that layer-35 memory, the prompt must have traversed that much of the network. This is why a cache normally represents work across the whole stack.

CED changes the global-memory dependency. Deep decoder queries still depend on the decoder's evolving representation, but the global values they read do not require every old prompt position to traverse the decoder.

The report's equations

Let the backbone have $L=40$ layers. The first $L/2=20$ form the causal encoder. Let $H_{L/2}$ denote the final encoder hidden states, using the paper's mathematical layer notation rather than zero-indexed implementation identifiers.

For decoder global cache projections, the paper writes

$C$ represents KV entries and $Z$ the compression weights where needed. These are projection paths, not full decoder attention-plus-expert transformations. The generic equation allows layer-dependent weights; the actual V4.1 schedule has one decoder Full source with later layers sharing its bank.

Because the decoder uses ratio one, its global main-memory path does not pool multiple sequence positions together. The projected encoder output remains a token-granular source for decoder global memory.

Why this is not ordinary cross-attention

An ordinary sequence-to-sequence encoder often processes a completed source sequence bidirectionally. Its decoder cross-attends to that source while generating a separate target. V4.1 instead maintains a causal stream that includes prompt positions and generated positions.

The encoder cannot inspect future tokens. New generated tokens also pass through it and contribute to memory. “Encoder” describes the lower part's role in constructing reusable global state, not an unrestricted reader of the entire future conversation.

This distinction matters in training. Each next-token prediction must still obey causality, even though parallel processing can calculate many prefix-conditioned states together.

The local-state exception

Every decoder layer retains its own sliding-window KV derived from that layer's hidden states. Those local states are valuable precisely because they have deeper layer-specific computation behind them.

But they preserve a dependency that CED's global path avoids. To start decoding with valid local state, some recent prompt positions must still traverse the decoder.

The theoretical exact reconstruction span grows with the number of local-attention layers. For a decoder of 20 layers and window 128, the report uses an order-of-magnitude bound of $20\times128=2560$ recent prompt positions. Exact receptive-field endpoints depend on indexing conventions; the point is the accumulated multi-layer dependency.

Decoder SWA Bounded Replay instead processes only the last 128 prompt positions through the decoder, with local attention truncated to that replay segment. The recovered state is approximate. That approximation is essential to the practical CED savings.

A cost expression and its scope

Ignoring detailed attention terms and retaining layer-token transformations, the report contrasts

with

For $N\gg n_{\mathrm{win}}$, the second expression is approximately half the first. It omits constant-factor details and the separate global projection cost. It is a structural explanation, not a full throughput model.

For an illustrative million-position prompt, a conventional 40-layer traversal involves 40 million layer-position transformations. CED's coarse count is 20 million encoder transformations plus $128\times20=2560$ decoder replay transformations. The additional projections are not counted in that comparison.

For a 32-position input, the same asymptotic simplification is inappropriate. Replay cannot exceed the available prompt, and decoder work may cover the whole short sequence. The architecture's advantage depends on length and the amount of uncached input.

Why prefill and decode active counts differ

The paper reports 8B active parameters per prefill token and 16B per decode token. These figures reflect different bulk computation paths, not two sets of model weights chosen by a user.

During decode, each generated token traverses the full backbone so the model can make a deep prediction. During prefill, most positions need only produce encoder-derived memory; a bounded recent span establishes decoder local state and the prediction boundary.

A benchmark report should therefore specify whether it concerns prefill, decode, or an input/output mixture. Comparing 8B prefill activation with another model's full decode activation can be rhetorically impressive but does not describe an identical operation.

What happens on a cached prefix

If the global prefix cache exists, the encoder can process the new suffix rather than reprocess all old positions. If encoder local state is missing, it first replays a short prefix tail. Decoder global state for new positions is projected from the new encoder output. The decoder then replays its needed prompt tail before generation.

There are thus two reconstruction questions: recover encoder state after a prefix hit, and establish decoder state at every prefill boundary. They are related but distinct. Chapter 11 develops both.

Where model quality could be constrained

The decoder's global memory has encoder depth rather than the depth of each decoder layer. Layer-specific queries and repeated expert transformations can still interpret that memory differently, but they cannot invent missing information in it.

Whether this limitation matters depends on the learned representation and tasks. The paper reports comparable or improved overall performance, but those results include other architectural and training changes. They do not establish that encoder-derived memory is equivalent to independent decoder memory on every input.

Released code versus production architecture

The inspected reference model.py runs all layers during plain prefill and constructs its source latents from the input passed to the attention module. It is useful for studying sharing and tensor shapes, but it does not independently verify the report's encoder-only prefill path or encoder-derived decoder global KV wiring.

A reproduction of CED's reported savings needs an implementation that explicitly follows the paper's dependencies and replay semantics. This is a material limitation, not a reason to reinterpret the paper's description as a different architecture.

Sources and next step

Report §2.2, p. 9, and §3.2.2, p. 20; reference Transformer.forward, Attention._compress_kv, and Compressor.forward. Chapter 7 examines the information stored in the global banks.

7. What Attention Compression Stores

Attention compression creates a smaller vector memory of an input sequence. It does not summarize the conversation into prose. Future queries use these learned vectors as attention keys and values.

V4.1's encoder global banks pool pairs of positions. Its decoder bank uses one entry per position. Both use a shared 512-channel main KV representation, with indexer keys derived from the main latent.

The main latent

A conventional cache may store separate keys and values for many heads. A shared KV design uses a common representation across query heads, avoiding 64 copies of the stored information for V4.1's 64 queries.

Sharing does not make the heads identical. Each constructs its own query, and grouped output projections transform the results. Different queries can read the same bank differently.

The released configuration identifies 64 rotary channels within the 512-channel main head width. The report uses the same FP4 format for the rotary and non-rotary components of global KV.

Pooling two positions

CSA2 removes the old compressor's overlapping groups and absolute positional embeddings. The public Compressor makes the remaining operation concrete: learned projections produce a value-like vector and a gating score per channel; a softmax over positions combines them.

For group $b$ containing positions $\mathcal B_b$ and channel $c$:

Normalization follows pooling in the reference. Different channels can favor different positions in the same pair. This equation is an explanation of inspected code, not an additional equation supplied by the report.

Suppose two projected positions have channels $u_1=(2,10)$ and $u_2=(8,4)$. If first-channel weights are $(0.75,0.25)$ and second-channel weights are $(0.2,0.8)$, the pooled vector before normalization is $(3.5,5.2)$.

It is neither one token nor a uniform average. Yet two vectors have become one, so the mapping is generally not invertible. A future query cannot recover arbitrary details of both inputs. Compression relies on learning which features are useful enough to preserve.

What changed from CSA

V4's CSA formed each ratio-$m$ entry from overlapping spans of $2m$ positions and included absolute positional information inside the compressor. CSA2 uses non-overlapping groups and removes that embedding.

The change simplifies grouping, cache updates, and implementation. It also removes machinery that could represent within-group distinctions. The paper describes an efficiency improvement but does not provide a full isolated analysis of when overlap would help. It would be unjustified to conclude that overlapping compression is universally unnecessary.

Completed groups and causality

A pooled entry cannot become visible before every member is available. Otherwise a query at the first member could access information from the future second member during training.

The reference indexer masks entries until their groups complete. During decode, the compressor retains partial-group state and emits a new latent only when the pair finishes. Recent unpooled positions remain visible through local attention, so a partial global group does not erase the latest token.

For an odd-length prefix, there are $\lfloor N/2\rfloor$ completed entries in a pair-compressed bank and a pending partial group. This detail matters for continuation and prefix cache compatibility even though it disappears in large-$N$ memory approximations.

Ratio one is still part of the design

At ratio one, the reference uses a normalized projection without pooling or a pooling gate. The decoder keeps one latent per original position in its shared bank.

This can still be much smaller than independent per-head, per-layer K and V arrays. Compression operates along representation size, layer count, and precision as well as sequence count. The architecture's name should not be interpreted as a claim that every bank pools several positions.

Indexer keys derive from main memory

CSA2 projects its 128-channel indexer key from the 512-channel main latent. It replaces a separate compression path from hidden states with a cheaper retrieval representation of the memory actually available to main attention.

The indexer remains a learned proxy. A high retrieval score is intended to predict usefulness; it does not prove that the final attention would assign a large weight.

The reference takes the indexer input before the main cache's rotary transformation and quantization, then applies the indexer's own position and quantization operations. Ordering prevents an already mutated main latent from entering the wrong numerical path.

Position and visibility have separate jobs

In the reference, a compressed group is anchored to its first position for rotary encoding. It becomes causally visible only after its last position. The anchor says where the representation belongs; the mask says when it may be read.

The released configuration uses a separate rotary base for compressed attention and a YaRN-style extension to its maximum position setting. These source facts do not establish that every aspect of long-context generalization has been ablated independently.

Local detail and distant information

A query reads the union of selected global entries and its local window. Recent local memory retains individual-position granularity at FP8 precision; distant information is available through sparse global memory.

Individual-position granularity is not exact arithmetic. Learned projection and FP8 still transform the input. It means local positions are not pooled into pairs in that bank.

The local window also does not limit the whole model to remembering 128 positions. Global selections and previously contextualized representations can carry distant information. Conversely, a million-position bank does not mean every distant fact is directly read by every query.

Sources and next step

Report §2.3, pp. 9–10, and §4.2.1, p. 22; released Compressor, Indexer.forward, Attention._compress_kv, and configuration. Numerical examples are illustrative. Chapter 8 explains which layers create and reuse these representations.

8. Sharing Memory and Sparse Selections Across Layers

CSA2 separates producing attention memory from deciding how to read it. Three statically assigned modes let several layers use one KV bank while occasionally refreshing selected positions. The mode assignment is fixed by layer; the selected positions remain input-dependent.

The three modes

Full creates a bank and indexer keys, then selects positions. Reindex keeps the latest compatible bank and keys but computes a new indexer query and selection. Reuse keeps both the bank and latest compatible selection.

Every mode performs attention. Reuse is not a bypass around a Transformer block, nor a copy of the previous layer's output.

Two different sharing relationships

Bank sharing identifies the content available to read. Index sharing identifies which positions to read. Their refresh frequencies differ.

In the decoder, layer 20 produces the bank. Layers 24, 28, 32, and 36 refresh selections against it. The intervening layers reuse each current selection.

One useful analogy is a library and a reading list. Reindex changes the list without rewriting the library. Reuse lets another layer interpret the same items with its own query. The actual objects, however, are numerical vectors and integer positions rather than explicit documents.

Compatibility is essential

An index is meaningful relative to a bank's layout and compression ratio. Entry 100 in a pair-compressed encoder bank does not represent the same item as entry 100 in the token-granular decoder bank.

Full establishes a new source group. Reuse must consume selections produced against its bank. An incompatible index list could yield the wrong information even when all tensor dimensions match.

The paper tracks source-consumer relationships. Its distributed training infrastructure must preserve these relationships across stages and concurrent microbatches. The simple reference runtime publishes current shared state as layers execute in order.

A worked schedule

Layer 2 builds the first pair-compressed encoder bank and selects up to 512 entries per query. Layers 3–7 share that bank and selection, while producing new main queries and their own local KV from their evolving representations.

Layer 8 builds a bank from a deeper encoder state and starts the next group. Layer 14 does the same for the final encoder group. All three banks are retained for future decode because their consumers remain part of the network.

Layer 20 builds the decoder global bank, projected from final encoder outputs in the reported CED architecture. Layer 24 can redirect attention after four decoder layers of processing, but it cannot rewrite old bank entries into deeper representations. It changes access rather than content.

The storage count

At length $N$, three pair-compressed encoder banks and one ratio-one decoder bank contain approximately

global entries. This count omits incomplete groups and physical alignment.

A hypothetical model with independent banks in all 38 global-attention layers at these same ratios would contain $18N/2+20N=29N$ entries. Sharing reduces this isolated count by $11.6\times$.

That is not the paper's actual V4-Flash comparison. V4 uses a different cache layout, layer schedule, compression pattern, and precision. The reported reduction relative to V4-Flash is about fourfold. The hypothetical calculation simply isolates the effect of repeated layer storage.

The computation count

V4.1 has eight index-producing layers rather than an indexer in every CSA2 layer. Thirty of 38 CSA2 layers skip fresh indexing. That removes scans, query construction for the indexer, and Top-K work from those layers.

It does not remove main query construction, sparse attention, local KV updates, MoE, or residual processing. Nor does an indexer count ratio translate directly to total speedup: bank sizes differ and deeper decoder searches use a candidate restriction.

Selection reuse is across depth for a given query progression. It does not mean an old decode token's chosen list is blindly reused for the next token without the scheduled source indexers running.

Shared memory still supports changing output

Two layers reading the same selected vectors can assign different attention weights because their main queries differ. Their local-window contents are layer-specific. Their output projections, expert transformations, and residual mixing differ too.

The global bank constrains which distant representation can be read, but repeated transformations can interpret it in different ways. This explains how much useful computation can remain even when storage and selection are reused.

The representational trade-off

Independent banks represent past positions at more depths. Sharing asks a source bank to serve more consumer layers. Reindex restores adaptive access but not additional historical representations.

The schedule balances refresh and reuse rather than maximizing either. It is reasonable to infer that source placement matters to the quality-cost trade-off. It is not justified to claim the published placement is universally optimal without a sweep over alternatives.

Stress tests could include tasks where evidence becomes relevant only under a deeper semantic interpretation, or where an early query's selected set omits an item a later query would prefer. Such tests would measure whether refreshed banks or more frequent indexing help enough to justify their cost.

Source ownership during training

Shared state creates gradient and lifetime obligations. A bank produced by one stage may receive learning signals from consumers on other stages. A shared parameter must have one logical owner even if execution uses replicas.

The physical implementation cannot simply copy a module and optimize each copy independently; that would train a different model. The report's shadow indexers, pipeline payloads, and microbatch state tracking address this problem in Chapter 18.

Sources and next step

Report §2.3.1 and Figure 4, pp. 10–11; §4.2.1, p. 22; released source-layer arrays. Chapter 9 examines the decoder's restricted search domain.

9. Hierarchical Sparse Indexing

Sparse attention limits expensive reading, but selecting what to read can still require searching every past position. V4.1's decoder creates a candidate pool once per query and restricts later indexers to that pool.

The first decoder Full layer still scans the complete visible bank. The hierarchy bounds deeper searches, not the entire model's dependence on context length.

The retrieval score

The indexer has 32 query heads of width 128 and a shared 128-channel key per bank entry. The reference computes query-key dot products, rectifies them, weights their head contributions, and sums across heads.

An explanatory form is

with scaling and position handling absorbed into the quantities. Top-K entries maximize this score. Main attention subsequently uses its own 512-channel comparisons.

The indexer is a learned proxy, not the final attention distribution. A selected entry can receive little main-attention weight. An unselected entry cannot be directly read through that layer's global branch.

Two outputs from the first decoder search

The first decoder Full layer produces its own Top-512 indices. Separately, it divides positions into blocks of eight, assigns each block the maximum member score, and selects up to 2,048 blocks.

Their members form a pool of at most $2048\times8=16{,}384$ positions. Later Reindex layers score this pool with their own queries and select their own Top-512.

The pool is therefore a search domain, not the final main-attention read set. A query does not automatically read 16,384 main entries in every decoder layer.

Why a block maximum can help

Keeping a whole block around one promising position preserves nearby context. A matching identifier may sit beside its definition; a relevant chart label may sit beside its value. Later indexers can choose a neighbor that the first query did not rank individually highly.

This is an interpretation of the specified maximum rule. The report does not isolate its benefits across every task type.

Consider a toy four-block example with maxima $0.3$, $5.0$, $1.1$, and $4.0$. If a tiny hierarchy keeps two blocks, it retains the second and fourth. A deeper indexer can choose any member of them. It cannot choose the third block even if its later query would prefer it.

The hierarchy saves work by giving up unrestricted deeper retrieval.

A decoder cost calculation

With $N$ visible decoder positions and candidate cap $P=16{,}384$, the five decoder indexers roughly score

positions rather than $5N$. At $N=1{,}000{,}000$, that is 1,065,536 versus 5,000,000 scoring events, about $4.69\times$ fewer for this isolated count.

This is not a measured wall-clock speedup. It omits dimensions, Top-K kernels, masking, communication, and other model work. Encoder source indexers still search their own pair-compressed banks. The first decoder search remains linear in $N$.

For short contexts under the cap, the pool may include all visible positions. Hierarchy then yields little reduction. The claimed practical advantage is principally a long-context one.

Output semantics versus arithmetic savings

The report's production design scores only candidate positions in later layers. The inspected readable reference first forms scores against available indexer keys and then masks noncandidate positions before selection.

That demonstrates restricted selection semantics but not the saving from avoiding excluded dot products. A dense score tensor followed by a mask can still cost proportionally to full context length.

This is a material implementation distinction. It explains why running the reference code is not sufficient to validate the paper's production complexity claim. A candidate-gather or otherwise restricted scoring kernel is needed to realize the advertised work bound.

Training-aware restriction

Hierarchy is introduced during post-training, with the same candidate restriction used in training and inference. Later indexers learn inside the domain they will have at deployment.

Without this adaptation, an indexer trained on unrestricted searches might regularly prefer unavailable positions. Matching the training search domain reduces that mismatch but cannot guarantee that the first query always preserves the right blocks.

Correlated retrieval mistakes

An early pool omission affects several later layers. Reindex can repair ranking within the pool but cannot recover an excluded block. Reuse inherits the current final selection as well.

This creates two useful quality measures: candidate recall against an unrestricted reference, and final Top-K recall within the candidate domain. Answer-level scores should be measured alongside them because index overlap alone does not establish whether omitted evidence matters.

Possible experiments would sweep pool size, block width, Top-K budget, and source-query placement on tasks with multiple dispersed dependencies. The report acknowledges sparse-selection robustness as unfinished work rather than publishing an exhaustive curve.

Nearly flat decode is not constant complexity

Figure 2 reports roughly a 25% increase in precision-weighted single-token decode FLOPs when context grows from 4K to 1M, a 256-fold length increase. This can occur when the remaining full scan is a relatively small part of total work.

It is not proof of $O(1)$ total decode. Any retained full-range scan continues to grow. The practical claim is a much flatter reported curve over the supported range.

Precision-weighted FLOPs are also not literal device-independent time. The report weights BF16, FP8, and FP4 operations by 1, 0.5, and 0.25. Real kernels differ in bandwidth, launch overhead, utilization, and hardware support.

Sources and next step

Report §2.3.2 and Figure 5, pp. 11–12; Figure 2, p. 5; release Indexer.forward and select_candidate_blocks. Cost examples are derived. Chapter 10 explains how the memory entries themselves are quantized.

10. Four-Bit Attention Memory

FP4 stores a number using four bits of payload. That is only part of the format: groups of values also need scale factors. V4.1 uses four-bit global main KV to save storage and a different four-bit indexer format to support fast indexing.

The local sliding-window cache stays FP8. This is a deliberate precision boundary, not a uniform conversion of every attention tensor to four bits.

Values plus scales

A block-quantized approximation has the form

where $q_i$ belongs to a small representable set and $s_b$ expands its range. Quantization maps the original value to this approximation. Dequantization reconstructs the approximation, not the exact original value.

For main KV, V4.1 uses E2M1 payload values and one E4M3 scale per 16 channels. E2M1 names a floating representation with sign, exponent, and mantissa bits. E4M3 is an eight-bit floating format for the scale.

The design follows the local-block part of NVFP4 but omits its second-level global scale. Calling it simply “NVFP4” without mentioning that omission would obscure a real layout difference.

The main-entry byte count

For a 512-channel main latent:

There are $512/16=32$ scale groups. One byte per scale adds 32 bytes, giving 288 bytes per main entry before extra alignment or metadata.

The effective storage is $288\times8/512=4.5$ bits per channel. Scale overhead makes it larger than four bits. Relative to an ideal one-byte-per-channel FP8 payload, the reduction is approximately $512/288=1.78\times$, with actual comparison affected by FP8 scale overhead.

The paper's “nearly halves” wording is therefore more accurate than claiming an exact twofold total-format reduction.

A different format for indexing

The indexer uses the OCP-standard MXFP4 arrangement. In the inspected reference, its 128-channel keys use 32-channel blocks and E8M0 scales. Four blocks contribute four scale bytes beside 64 bytes of four-bit payload, yielding 68 bytes per indexer key in the nominal layout.

The main and indexer formats differ because they serve different operations. Indexer Q/K participates in matrix comparisons where native hardware support matters. Main KV is dequantized before attention, so its storage format can prioritize accuracy without requiring the same native multiplication path.

This distinction also explains why FP4 main caching principally saves memory and transfer volume rather than implying that every main-attention operation executes as a native FP4 matrix multiplication.

The range argument

The report gives a maximum E4M3 scale magnitude of 448 and a maximum E2M1 payload magnitude of 6. Their product is 2,688, comfortably larger than expected main-KV magnitudes.

After RMS normalization, if the learned norm weights have magnitude approximately at most one and the 512-channel vector has unit RMS before those weights, its L2 norm is approximately bounded by

A norm-preserving rotary transform leaves this L2 norm unchanged, and any individual channel magnitude cannot exceed the vector norm. The report also observes magnitudes around 10 during training.

This supports omitting the extra global scale for range reasons. It does not eliminate rounding error: a representable range can be adequate while the grid inside that range remains coarse.

The norm-weight bound is empirical and approximate. It should not be silently turned into a formal invariant under arbitrary future fine-tuning or changed normalization parameters.

Why quantize after RoPE

RoPE rotates parts of the representation. Quantization before and after rotation generally differ because rounding is nonlinear:

in general, even though $R$ preserves the original norm.

The report finds a marginal accuracy improvement from quantizing before RoPE but chooses post-RoPE quantization to avoid added decode overhead. Both rotary and non-rotary channels use the same main-cache format.

This is a systems trade-off between numerical quality and operational simplicity, not a claim that quantization and rotation commute.

Quantization-aware training

Main-cache QAT is introduced during post-training. The training forward path exposes the model to the cache approximation it will encounter during inference, allowing other parameters to adapt.

A standard background explanation is that a forward pass simulates quantization while a surrogate gradient may permit learning through a nondifferentiable rounding operation. The report does not specify every main-cache gradient estimator, clipping rule, or observer update. Those should not be invented from generic QAT practice.

The report states that extending QAT to main KV incurs only marginal performance degradation. It does not publish a complete error distribution across every task and cache boundary.

Why local memory remains FP8

SWA is reported to be more sensitive to quantization. Its cache carries recent layer-specific representations used repeatedly across a narrow context. V4.1 therefore keeps it at FP8 while compressing global main state more aggressively.

The distinction is also efficient: local storage is bounded by the 128-position window, whereas global storage grows with context length. Spending more bits on the bounded component has a much smaller asymptotic memory cost.

“More sensitive” is the report's empirical judgment. A precise explanation of which local features or distributions cause that sensitivity would require additional analysis.

Quantized state is not low-precision everything

Model weights, optimizer buffers, residual streams, accumulators, local caches, indexer queries, and main caches can all use different numerical formats. The released checkpoint configuration identifies FP8 quantization and FP4 experts, while the cache design has its own formats.

A serving implementation must preserve these distinctions. Treating all four-bit tensors as interchangeable can produce incorrect scaling, incompatible kernels, or silent numerical changes.

Sources and next step

Report §2.4.4, p. 14; released Attention._compress_kv, Indexer.forward, and cache-format comments. Byte counts are derived from those formats. Chapter 11 considers an approximation of a different kind: recreating missing local state.

11. Bounded Local-State Replay

Sliding-window attention keeps only recent state at runtime. Reconstructing that state after it has been discarded is harder than recomputing one window, because upper-layer states depend on earlier lower-layer windows.

V4.1 accepts approximate reconstruction by replaying only the most recent 128 positions. This makes local-state misses cheap enough to remove SWA from the long-lived persistent cache and makes CED's decoder prefill path practical.

Why dependencies grow across depth

At one layer, a position can inspect at most a window of $W$ recent positions. At the next layer, each inspected representation already contains information from its own window. Dependencies therefore extend farther into the original sequence as layers accumulate.

For an inclusive window, a simple pure-local receptive-field calculation gives a span of roughly $1+L(W-1)$ for an output after $L$ layers. Reconstructing an entire cache tail has related endpoint requirements. The paper uses the conservative shorthand $LW$ for the replay work needed by exact local-state recovery.

At $L=40$ and $W=128$, that shorthand is 5,120 positions. For the 20-layer decoder, it is 2,560 positions. These are dependency-span calculations, not additional model context limits.

Global attention can bring other information into each representation. During replay, existing global KV remains available. The troublesome dependency is the missing chain of layer-local SWA states.

Truncation at the replay boundary

Suppose replay starts at position $s$. A query at replay position $i$ reads local keys only in

Early replay positions therefore lack local predecessors before $s$. Later replay positions use states computed from that shortened local history. Even if the final position eventually has 128 in-segment local positions, their deeper representations differ from those produced with the complete earlier chain.

The resulting cache is approximate. There is no mathematical argument that simply filling the window restores the original state at every depth.

Encoder replay after a prefix hit

Assume global KV for an old prefix is present but its encoder SWA state has expired. The system replays the last 128 prefix positions and then processes the uncached suffix.

For replayed prefix positions, it regenerates SWA only, reusing stored global KV without overwriting it. New suffix positions generate both new global KV and local state.

Avoiding global overwrite matters. The prefix's persistent bank is an existing reusable object, not a partially rewritten bank that mixes alternative replay states at arbitrary old positions.

However, new suffix states depend on the approximate replay context. Their global entries can consequently differ depending on where the prefix cache hit occurred. The paper explicitly acknowledges this cache-boundary dependence.

Decoder replay at every prefill

CED can build decoder global KV from encoder outputs without fully processing old prompt positions through the decoder. Decoder local KV still needs layer-specific hidden states.

The system takes the last 128 prompt positions' encoder outputs and passes that segment through the decoder under the same local truncation rule. The resulting decoder SWA supports subsequent generation but is not stored for persistent prefix caching.

This is a separate path from encoder reconstruction. Encoder replay is a fallback for missing short-lived local state; decoder replay is part of establishing the decoder state after prompt processing.

A concrete cache example

Consider an illustrative 100,000-position cached transcript followed by 2,000 new tool-output positions. Suppose its global cache hits and encoder SWA misses.

The encoder processes 128 replay positions plus the 2,000 new positions. The replay positions restore only local state; the new positions append global memory. The decoder's global source projects encoder output as specified by CED. A 128-position decoder tail then establishes local decoder state before new output generation.

This example explains the data path. It does not include an exact FLOP total, since projection, sparse selection, batching, and position layout add details. In particular, the two replay paths should not be casually counted as one identical operation.

Why it may work well empirically

The report cites evidence that effective SWA receptive fields are often shorter than their theoretical bound. Learned attention can concentrate on recent neighbors, and global memory continues to provide broader context.

These observations make bounded replay plausible. They do not guarantee equivalence on inputs that deliberately propagate a local dependency through many layers or place critical information just before the replay boundary.

The report finds negligible quality degradation in its experiments and simulates decoder replay during post-training for adaptation. The empirical scope and the approximation should both remain visible.

This is different from ordinary activation recomputation

Training often discards activations and recomputes them exactly from known inputs to save memory. Bounded replay is not merely that exact technique applied at serving time. It intentionally omits earlier local dependencies.

Its benefit comes from allowing a cheaper state with acceptable behavior rather than proving byte-for-byte equality of recomputed caches.

That distinction changes how it should be tested. Comparing only tensor shapes or cache occupancy is insufficient; the meaningful question is how changed state affects generated behavior.

A useful robustness experiment

Hold the prompt and checkpoint fixed, vary the cache-hit boundary, and compare full-prefill versus bounded-replay outputs under controlled sampling. Include tasks requiring evidence immediately before and far before the boundary, long chains of local references, image-text transitions, and small uncached suffixes.

Record quality, numerical state differences, latency, and failure clustering rather than demanding identical stochastic responses. Deterministic or teacher-forced comparisons can help isolate the state change from sampling noise.

This is a proposed evaluation design. The report does not supply all of these measurements.

Sources and next step

Report §2.2, p. 9; §§3.2.1–3.2.2, pp. 19–20; §6, p. 37. Chapter 12 connects replay to storage lifetimes and the 890-byte cache calculation.

12. Cache Economics and the Serving System

The serving system must keep model parameters, active attention state, temporary computation, and reusable prefix state in appropriate places. V4.1 reduces global state and gives local state a shorter lifetime.

This chapter derives the global-cache figure and explains why it is different from the persistent-cache reduction.

State with a different reuse horizon belongs under a different retention policy. The lifetimes shown are the paper's deployment choices, not checkpoint-level guarantees.

Deriving 890 bytes per position

From the configured schedule, the model keeps three encoder global banks at ratio two and one decoder bank at ratio one. At length $N$, this is approximately $2.5N$ stored global entries.

Each entry has a 512-channel main latent: 256 payload bytes plus 32 E4M3 scale bytes, totaling 288. Each corresponding 128-channel indexer key has 64 payload bytes plus four E8M0 scale bytes, totaling 68 under the reference layout.

The combined count is

This nominal layout calculation matches the paper's reported 890 global-cache bytes per token. It is a useful consistency check between layer sharing and precision choices.

The calculation does not include local windows, unfinished compression groups, indices, candidate masks, allocator padding, page tables, or general runtime metadata. The production figure should be understood as the stated global KV category, not all memory attributable to a request.

Global memory at different lengths

These are decimal units and derived estimates. The last row uses the released maximum position setting. Dividing 933.23 million bytes by $2^{30}$ gives about 0.869 GiB, illustrating why GB and GiB should not be mixed.

Local memory is bounded, not absent

For a rough local-cache payload estimate, 40 layers × 128 positions × 512 channels × one FP8 byte gives

about 2.62 MB before scale and metadata overhead. Draft-module state and other buffers add further memory.

At a million positions this local payload is small relative to global KV. At short contexts it can be a significant fraction of runtime cache. A long-context global-memory ratio does not characterize every short-request allocation.

Why persistent SWA can still be costly

Bounded local runtime state sounds cheap. The problem appears when many session endpoints retain separate local snapshots over long lifetimes.

The V4 deployment persisted SWA at the end of prompts and outputs, while keeping global prefixes reusable for long periods. The paper reports that SWA occupied nearly half of persistent KV capacity under the relevant workload, even though only a window was saved at designated points.

Many short turns create many endpoints. The ratio therefore depends on session lengths, endpoint count, reuse patterns, retention, and eviction policy. It is not an architecture-only identity for every sequence length.

Separate lifetimes for global and local state

V4.1 removes SWA from long-lived persistent storage. Encoder SWA instead occupies a distributed memory pool provisioned from 10% of each machine's host DRAM, with minute-scale TTLs. Global KV stays in the persistent tier with a reported minimum lifetime of 72 hours in that deployment.

The rationale is observed reuse behavior. Global prefixes have long-tail reuse, whereas local state is useful mainly within active sessions. After a session ends or advances to its next state, an old endpoint's local cache may have little value.

These retention and pool settings describe the paper's production design. They are not promises made by the model checkpoint or required settings for every user deployment.

How one eighth follows from two factors

The report attributes the persistent reduction to two approximate factors: dropping SWA nearly halves persistent storage, and the retained global KV is about one quarter the old global size.

For an illustrative old footprint split evenly between global and SWA, the new size is

of the old total. That is one eighth. If the old split were different, the exact ratio would differ. The paper's comparison is explicitly under identical workloads, not a universal algebraic property of arbitrary caches.

Bounded replay makes this policy affordable: a local miss triggers roughly one window of reconstruction rather than the much longer exact dependency replay.

Encoder–Prefill–Decode disaggregation

The production system separates vision encoding, language-model prefill, and decode so they can scale independently and overlap. This is called EPD disaggregation.

The word “encoder” here refers to the vision-encoding service stage and should not be automatically equated with the causal encoder half of the language backbone. One is a deployment-stage name; the other is a model-architecture partition.

Image-heavy requests can stress vision workers, long uncached text can stress prefill workers, and concurrent generations can stress decode workers. Independent scaling helps match resources to the workload mix, at the cost of communication and scheduling between stages.

Fused kernels reduce overhead

The report describes fused components for RoPE-attention operations, gating, mHC, MoE, Top-K, and other transformations. Reuse layers require only 15 kernels during prefill and 11 during decode in the reported system.

Kernel count is useful evidence of an optimized execution path, but it is not a platform-independent latency prediction. A kernel can be large or small, communication can dominate, and batch shape affects utilization.

Single-Pass mHC lowers activation traffic; sparse retrieval lowers reads; cache compression lowers storage and transfer. These gains target different portions of the serving path.

What cannot be calculated from the paper alone

There is no complete public fleet-cost model with device pricing, power, network topology, admitted batch distributions, hit rates, and latency percentiles. One cannot turn “one quarter global KV” into “one quarter API price” from the report alone.

A practical evaluation should measure uncached prefill, cached continuation, decode throughput, first-token latency, tail latency, cache transfer, and failure rate separately. The right operating point depends on how often each path occurs.

Sources and next step

Report §3.2, pp. 18–20, and abstract; release configuration and cache-format code. All tables and byte decompositions above are derived nominal estimates. Chapter 13 explains one of the key fusion-enabling changes.

13. Single-Pass Manifold-Constrained Hyper-Connections

A residual connection carries information around an expensive transformation. Manifold-constrained hyper-connections, or mHC, generalize that path to several residual streams with learned mixing. V4.1 keeps four streams and changes when it computes the mixing coefficients so serving can use a single fused traversal.

The architecture change is small in notation but important for memory traffic. It is not merely a compiler optimization of an unchanged dependency graph.

From one stream to four

A conventional residual update is $x_{\ell+1}=x_\ell+F_\ell(x_\ell)$. With mHC, let $X_\ell\in\mathbb R^{n\times d}$ hold $n$ stream vectors for one token. V4.1 uses $n=4$ and $d=5120$.

The report's original form is

$A$ is a $1\times n$ input mixture, $B$ an $n\times n$ residual transformation, and $C$ an $n\times1$ distribution of the sublayer output. The predictor $\mathcal H$ includes normalization and projection.

The expensive sublayer receives a $d$-wide mixed input. Four residual streams therefore do not mean four independent full experts or four complete attention modules for each block.

What the constraint contributes

mHC inherits a constrained residual mixing design. Sinkhorn–Knopp iterations approximately normalize a positive matrix to have unit row and column sums, producing a doubly stochastic residual map. The report uses 20 iterations.

A doubly stochastic matrix redistributes mass without arbitrarily multiplying it. In an ideal exact case, each output stream is a convex mixture of the inputs, and the sum across streams is preserved by the residual mixing alone.

That gives an intuitive stability benefit for repeatedly composing residual maps. It does not bound the entire network's activations or Jacobian because the expensive transformation and output injection still contribute. Finite numerical iterations also provide approximate constraints.

The detailed earlier mHC parameterization is inherited; V4.1's report focuses on shifting input mixing and its execution consequences rather than rederiving every manifold constraint.

The original dependency problem

To process the next sublayer, the implementation first updates the residual state, then predicts coefficients from it, then mixes it into the next input. The input coefficient $A_\ell$ is unavailable until its predictor has reduced across the full hidden dimension.

The residual update can be computed tile by tile. But input mixing cannot use a tile immediately if it must wait for the current full-vector reduction. That creates another read of the residual once the coefficients are ready.

Including pre-normalization, the report counts $(4n+4)d$ activation reads and writes for the original implementation. For $n=4$, this is $20d$ scalar movements per boundary in the stated accounting.

Shift the input coefficient by one block

Single-Pass mHC changes the update to

The current sublayer consumes the previous block's already available input-mixing coefficients. While reading the current residual, the kernel can simultaneously construct the mixed input and accumulate statistics for coefficients used next time.

The expensive reduction still exists. Its result is simply no longer on the critical dependency path for the current tile's input mixing.

The paper reports negligible performance degradation from this shift. It is a learned approximation in coefficient timing, not algebraic equivalence to the original mHC update.

The lower-bound traffic count

An ideal map reads $n$ residual vectors and one previous sublayer output and writes $n$ updated residual vectors plus one mixed input. That requires $(n+1)d$ reads and $(n+1)d$ writes, totaling

For four streams and width 5120, the count is $10\times5120=51{,}200$ scalar movements rather than $20\times5120=102{,}400$. At an illustrative two bytes per activation, those are about 102.4 KB and 204.8 KB per token-boundary accounting unit.

These are derived traffic counts, not measured external-memory transactions on every device. Cache residency, fusion boundaries, layout, and datatype affect actual bytes moved. The ratio captures the report's intended removal of a second residual traversal.

Mega-mHC

The deployment kernel fuses residual update, input mixing, and coefficient prediction. It also includes input pre-normalization and FP8 conversion. Hidden-dimension tiling allows each residual tile to be read and used for several purposes before it leaves the kernel.

For ordinary mHC, the report's fused kernel can reach $(3n+2)d$ traffic. For Single-Pass mHC, it reaches $(2n+2)d$, the ideal count above. Fusion and the coefficient shift are complementary contributions.

Pre-training retains the existing multi-kernel implementation because the architectural shift can be trained without requiring the production fused kernel. Training and serving implementations need the same mathematics, not necessarily identical kernel packaging.

Three meanings of Sinkhorn in this report

The name appears both in residual mixing and optimization. mHC uses Sinkhorn–Knopp to constrain a stream-mixing matrix. The embedding optimizer uses alternating row and column norm balancing on an update matrix.

These are related normalization ideas applied to different objects with different goals. A doubly stochastic residual map is not the same object as a row/column RMS-balanced optimizer update. Mixing them would confuse stability of signal flow with scaling of parameter learning.

What mHC does not replace

It does not replace attention retrieval, MoE routing, cache compression, or the decoder's CED path. It organizes how representations flow between those expensive operations.

An overall score improvement cannot be attributed to Single-Pass mHC from this report alone. The stronger direct evidence is the dependency change, nominal memory-traffic count, and reported negligible quality impact.

Sources and next step

Report §2.4.1, pp. 12–13; §4.2.1, p. 22; released Block residual-mixing comments and methods. The traffic examples are derived. Chapter 14 introduces a separate source of learned memory.

14. Engram: Conditional Parameter Memory

Engram adds a large store of learned embeddings addressed by token n-grams. It is designed to decouple some memorization capacity from deep computation. V4.1 allocates 196B parameters to two Engram modules, separately from its 552B backbone.

The tables are model parameters. They do not contain a growing database of this user's conversation, and they are not a retrieval-augmented search service over external documents.

Addressing by n-grams

An n-gram is a short sequence of token identities. Engram uses orders two, three, and four. It first applies tokenizer compression, then derives multiple hash addresses for each n-gram.

Eight hash heads are used per order. Each head has roughly 16M rows with prime table sizes. The total embedding dimension per order is 2048, or 256 channels per head under the released configuration.

For one position and one module, three orders times eight heads give 24 row lookups. Their concatenated width is $24\times256=6144$ before key/value projection.

This row count is a derived interpretation of the configured layout. It does not mean every context position has a unique collision-free table row.

Why multiple hashes

A finite hash table maps many possible n-grams into fewer addresses. Collisions are inevitable. Several heads and distinct prime table sizes reduce the chance that two unrelated n-grams collide identically across every component.

The learned representation can use the combined signature rather than relying on a single perfect lookup. That is the intuition behind multiple heads; the report does not provide a complete collision distribution for the released tokenizer and training corpus.

Token-based addressing is deterministic and cheap relative to deep neural routing. Unlike MoE expert selection from a hidden state, addresses can be known before the position reaches the Engram layer.

The contextual gate

Fetched embeddings are projected into keys and values. The current contextual representation is compared with the key, and a gate controls how much of the value enters the residual streams.

A schematic update is

The inspected reference gives one key per residual stream and a shared value, uses normalized dot products with learned feature weights, applies a signed square-root transform, and passes the result through a sigmoid gate.

The gate matters because an n-gram can appear in several contexts. A familiar token phrase should not always contribute the same amount of memorized information. Context can suppress an unsuitable contribution.

This mechanism is more nuanced than a direct token-to-fact dictionary. The vectors have no required human-readable interpretation, and the gate's output is not a factual-confidence certificate.

Why this can save computation

A repeated local pattern may be represented through a small set of table rows rather than reconstructed entirely by many neural transformations. Extra parameter capacity can support stored associations while keeping the selected computation relatively small.

That is an architectural motivation. The V4.1 report does not publish a complete independent ablation separating Engram from all its other changes. Claims about exactly which benchmark gains came from Engram would exceed the evidence here.

The module does not remove backbone computation: the token still passes through attention and experts. It contributes another information path.

The parameter arithmetic

Each module has about 24 tables × 16M rows × 256 channels:

Two modules yield approximately 196.6B, consistent with the report's rounded 196B allocation. Exact prime table sizes produce slightly different counts.

At one FP8 payload byte per parameter, the tables alone occupy roughly 196 GB before scales and overhead. For a single position, 24 selected rows × 256 bytes is approximately 6 KB of payload per module, around 12 KB for both, before scales, deduplication, and caching.

These estimates show the trade: very large stored capacity, comparatively small conditional access, and a real bandwidth requirement.

Placement and prefetching

The modules appear at zero-indexed layers 1 and 14, balancing training-pipeline memory. Both embedding tables and key/value projections use FP8.

Because addresses depend on tokens, the inference system can prefetch rows from host memory via background RDMA transfers. The first module's prefetch overlaps the first Transformer block's work. Placement affects how much computation is available to hide transfer latency.

Prefetching does not make bandwidth free. Repeated addresses, batch overlap, hot-row caching, link speed, and scheduling determine whether transfer remains hidden.

Training placement differs from serving placement

The tables are row-sharded across dedicated Engram process groups. Optimizer states are sharded further across replicas. Entire local-batch indices can be prefetched before pipeline microbatch processing begins.

Embedding gradients are buffered and returned to owning ranks after backbone backward, with transfers overlapping vision work. During RL rollouts, tables remain GPU-resident to reduce host pressure and avoid fragmentation-related failures.

Thus “Engram lives in host memory” is not universally true. The report describes different placement choices for different phases.

What V4.1 changes from earlier Engram

The report omits a short causal convolution from the original design because the gain did not justify inference complexity. It also uses momentum plus Sinkhorn balancing for the embedding updates.

The omission is specific to this engineering judgment. It does not establish that local convolution can never help another model or budget.

Text and image boundaries

The public reference masks image positions out of Engram contributions and n-gram construction. Text identities drive lookup; pixels do not become ordinary lexical n-grams merely because they occupy sequence positions.

Correctly preserving these boundaries is important when a transcript interleaves text and images. Placeholder IDs should not accidentally create lexical patterns crossing visual spans in an incompatible implementation.

Sources and next step

Report §2.4.2, p. 13; §3.1.3, p. 18; released Engram, NgramHashState, and table configuration. Byte and lookup estimates are derived. Chapter 15 examines the auxiliary path that reduces generation iterations.

15. DSpark and Speculative Decoding

Autoregressive generation requires a sequential chain of decisions. Speculative decoding proposes several future tokens with a cheaper drafter and asks the main model to verify them. If enough proposals are accepted, fewer expensive serial backbone invocations are needed per emitted token.

V4.1 uses DSpark: three draft Transformer blocks, a local window of 128, five parallel draft positions, a lightweight Markov head, and a confidence-aware verification scheduler.

The basic economics

Let $T_D$ be drafting time, $T_V(q)$ the time to verify $q$ proposed positions, and $A(q)$ the expected number of accepted outputs. A simple throughput objective is related to

This is a conceptual expression, not DSpark's exact published scheduler formula. Actual decoding can emit an additional target token, handle rejection corrections, batch multiple requests, and incur communication.

The main lesson is that a longer draft is beneficial only when its accepted prefix is worth the added drafting and verification cost.

Why draft positions are dependent

Future tokens do not occur independently. In “the capital of,” the next token affects the likely continuation. A naive parallel drafter can produce marginally plausible positions that do not form a coherent sequence together.

DSpark computes base logits for five positions in one pass, then uses a lightweight Markov head to model dependencies between draft tokens. This combines parallel expensive feature construction with a cheaper sequential dependency component.

“Semi-autoregressive” describes that mixture. It does not mean the main model abandons its causal next-token distribution.

The V4.1 report summarizes the module rather than giving every DSpark loss or dependency equation. The released draft classes provide additional forward structure, but a complete analysis of DSpark as a separate research contribution would require its dedicated source.

Conditional acceptance confidence

A confidence head predicts per-position conditional acceptance probabilities. If $a_i$ estimates acceptance at draft position $i$ given that prior positions survived, then prefix survival is conceptually

This product illustrates why an uncertain early position makes later proposals less valuable. Even high confidence at position five cannot repair a likely rejection at position one.

Conditional probability estimates need calibration. A confidence score is a scheduler input, not proof of acceptance or a truth judgment about the generated answer.

Scheduling under system load

The scheduler combines estimated survival with profiled engine throughput curves and chooses a verification length for each request. Its stated objective is expected system-wide token throughput under current load.

A serving engine's efficient verification length can vary with batch size, queue pressure, sequence lengths, and hardware. The schedule therefore need not be a fixed “always verify five” policy.

This also means a headline draft length is not a speedup multiplier. Five proposed tokens may yield only a shorter accepted prefix, and verification itself consumes backbone work.

A numerical illustration

Suppose estimated conditional acceptance probabilities are $(0.9,0.8,0.7,0.6,0.5)$. Prefix survival probabilities are $(0.9,0.72,0.504,0.3024,0.1512)$.

For an accepted-prefix length under this simplified model, the expectation contributed by the five positions is their survival sum, approximately 2.58. A five-position proposal does not imply five accepted tokens.

If a longer verification makes much better use of the GPU batch, it can still be useful. If it delays other requests or costs too much, a shorter choice may win. These probabilities are illustrative and not measured for V4.1.

Preserving the target distribution

Standard exact speculative sampling uses an acceptance-and-correction procedure designed to preserve the target model's distribution. Greedily accepting plausible-looking draft tokens is not equivalent to that procedure.

The V4.1 report describes confidence-scheduled verification but does not give a complete acceptance algorithm, all sampling corrections, or a numerical proof of distribution equivalence. It would be inappropriate to invent those details or infer that confidence alone determines acceptance.

A confidence head should schedule how much to verify; target-model verification remains the authority for accepting proposals in a proper speculative engine.

Training stages

DSpark is absent from backbone pre-training. Afterward, a dedicated stage trains DSpark while freezing the backbone. During post-training it continues to learn alongside the changing backbone, without propagating its objective's gradients into the backbone.

This separation lets the drafter adapt to the target policy while keeping auxiliary draft learning from changing the backbone through that loss. The backbone still changes through its own SFT, RL, or distillation objectives.

The report distinguishes this from jointly trained MTP in earlier DeepSeek models. Configuration names such as num_nextn_predict_layers are historical implementation labels and do not reverse that training-stage distinction.

Serving and training rollouts

Accelerating generation benefits ordinary users and RL/OPD data collection, both of which require many decoded tokens. A drafter aligned with a rapidly changing policy is particularly useful for rollout throughput.

But post-training checkpoints can change acceptance behavior. A drafter's usefulness depends on continued alignment and the scheduler's profiles. The report does not supply a complete breakdown of acceptance rates and end-to-end acceleration across workloads.

The reference implementation boundary

The released minimal inference code includes a DSpark forward path and its heads. Its README explicitly states that generation remains plain autoregressive sampling. It does not implement the complete production verification scheduler.

Executing that sample cannot establish DSpark's production speedup. A complete reproduction would need draft sampling, verification, corrections, state updates on rejection, and load-aware scheduling.

Sources and next step

Report §2.4.3, pp. 13–14; released inference README and DSparkAttention, Markov/confidence heads, forward_spec. Probability arithmetic is illustrative background. Chapter 16 explains how the model's different parameter families are trained.

16. Optimizing Different Parameter Families

An optimizer turns gradients into parameter updates. V4.1 does not apply one optimizer uniformly. Matrix transformations, enormous lookup tables, and small norm or bias parameters have different structures and memory costs.

The report uses Muon for most linear matrices, head-wise Muon for Query and Key weights, momentum with Sinkhorn balancing for embeddings and the prediction head, and AdamW for normalization weights and other non-matrix parameters.

Why matrix structure matters

A linear weight matrix has input and output axes. An embedding table has token-identity and feature axes. Normalization weights are vectors. Treating all of these as unrelated scalar coordinates ignores their different organization.

Adam-style methods maintain moving estimates of gradients and squared gradients. This can be effective but creates substantial state for very large tables. A momentum-only update needs fewer full-sized buffers.

Muon and the balanced-table method both exploit matrix structure while using a momentum buffer. That is a memory advantage, not a statement that all remaining optimizer memory disappears.

Muon as background

Muon transforms a momentum-based matrix update using an approximate orthogonalization procedure, commonly Newton–Schulz-style iterations. An intuitive aim is to reshape the update's singular directions rather than scale every matrix entry independently.

The V4.1 report inherits substantial Muon machinery from V4 and earlier work. It does not specify every numerical iteration and distributed implementation detail needed for an independent optimizer reproduction.

Its pre-training setup uses momentum 0.95, decoupled weight decay 0.1, Nesterov momentum, and an update RMS rescaling of 0.18 to reuse the AdamW learning-rate scale.

Head-wise preconditioning

If query weights for several attention heads are treated as one large matrix, one preconditioning transformation can couple heterogeneous heads. Splitting the weights by head allows a separate transformation for each head.

The report motivates head-wise Muon through this heterogeneity and observes better performance than vanilla Muon. Its basic configuration applies head-wise treatment to Query and Key weights, even though the introductory motivation focuses on Query weights.

A head is a learned subspace, so different gradient geometry is plausible. The report's observation does not quantify every affected benchmark or establish a universal advantage for all attention architectures.

Sinkhorn-balanced embedding updates

The very large Engram tables, token embedding, and prediction head use a different matrix update. Let $G_t$ be the gradient and $M_{t-1}$ the previous momentum. The paper computes

It then alternates row and column L2 normalization of the update matrix. With an odd number $K$ of normalization steps, the final step normalizes rows. Near-zero rows are masked for stability.

The final update is scaled by $\sqrt n$, where $n$ is the feature dimension, and applied with corrected learning rate $\widetilde\eta_t=\gamma\eta_t$.

Why multiply by the square root of width

For a row with unit L2 norm across $n$ features, its RMS is $1/\sqrt n$. Multiplying the row by $\sqrt n$ makes its RMS one.

That gives a width-aware update scale. The separate factor $\gamma=0.18$ calibrates the effective step magnitude relative to the learning-rate convention.

These are distinct scalings. One converts a row norm convention; the other adjusts how large a learning-rate step is taken.

What balancing does

One table row corresponds to a token or n-gram identity, while a column corresponds to a hidden feature. Alternating normalization approximately equalizes row and column RMS in the update.

If a column initially contains much larger updates than the others, column normalization reduces that imbalance. Subsequent row normalization restores the desired per-row scale. Finite alternating passes only approximately satisfy both goals.

For numerical stability, rows whose update norm is at most $\tau$ times the mean row norm are set to zero. Otherwise normalizing an almost-zero row could promote tiny noise into a full-scale update.

The reported values are $K=11$, $\tau=10^{-3}$, and $\varepsilon=10^{-20}$. Engram's learning rate is additionally scaled by $5\times$.

This is not residual Sinkhorn normalization

mHC uses Sinkhorn–Knopp to constrain a positive stream-mixing map's row and column sums. The optimizer balances norms of a potentially signed update matrix. Similar alternating normalization logic does not imply identical constraints or purposes.

The table optimizer also applies no weight decay, while Muon uses decoupled decay. Mixing these rules during reproduction would materially alter the training run.

AdamW remains for small parameter families

AdamW uses $\beta_1=0.9$, $\beta_2=0.95$, $\varepsilon=10^{-20}$, and weight decay 0.1 for the specified pre-training setup. Norm weights receive decay; biases and scaling factors do not.

The report's “other non-matrix parameters” classification matters. It is not enough to apply Muon to every tensor with more than one dimension without checking the intended family and head partitioning.

The vision encoder stays frozen until the main learning-rate decay stage, except for its final norm and the projector. It is later unfrozen with a smaller learning rate. Parameter activity is therefore stage-dependent.

Optimizer-state memory at table scale

For an illustrative 196B-parameter table allocation, one FP32 buffer occupies about 784 GB. Two such buffers occupy about 1.568 TB before sharding and auxiliary state.

This calculation does not specify the actual optimizer-state dtype used everywhere in the production system. It shows why saving even one full-sized state array matters. Row sharding and replica-level optimizer sharding distribute that burden across devices.

The report also implements alternating normalization using maintained row and column scaling vectors to avoid rewriting the entire normalized matrix repeatedly. Fusing row normalization with partial column-statistic accumulation further reduces memory traffic.

What remains unspecified

A full reproduction needs exact parameter grouping, distributed reductions, precision rules, gradient clipping, sparse-table update semantics, learning-rate multipliers, and inherited Muon numerical details. The paper provides substantial settings but not a complete training executable.

One should not replace missing pieces with plausible defaults and then call the result the reported run.

Sources and next step

Report §2.5 and Algorithm 1, pp. 14–16; §3.1.3, p. 18; §4.2.2, p. 22. Memory arithmetic is illustrative. Chapter 17 puts these updates into the full data and schedule pipeline.

17. Data Construction and the Pre-Training Run

V4.1 is trained from scratch on a multimodal corpus totaling 45T tokens. The report emphasizes better information gain and corpus interaction, not merely adding more raw data.

This chapter separates the main language-model run from vision preparation and from post-training task synthesis.

Data quality is not only sample quality

A high-quality sentence can add little value if it duplicates material already seen many times. Conversely, a specialized document may be valuable because it covers a missing domain.

The report uses a scaling ladder over model size and training data, broader evaluations, and domain-expert quality dimensions to guide curation and mixture choices. It explicitly filters low-information model-generated text and poor machine translation as forms of implicit duplication.

This does not mean all synthetic data is rejected. The report explores model-in-the-loop iteration and later invests heavily in synthesized agent tasks. The distinction is the value, reliability, and role of the generated material.

Contemporary code coverage

The text corpus includes more recent repositories, commits, libraries, frameworks, and programming languages. Such coverage can improve familiarity with current software patterns.

That is a reported curation direction, not a precise public knowledge-cutoff guarantee. The report does not publish a complete timestamped corpus inventory or exact proportions by language and repository source.

Improved code benchmarks may reflect both this curation and other changes. They are not an isolated measurement of newer code data.

Three multimodal source types

The multimodal pipeline uses image-text pairs, interleaved image-text documents, and domain-specific datasets.

Image-text pairs come from images and associated webpage alt text, followed by relevance filtering and semantic image deduplication. Interleaved documents come predominantly from webpages and PDFs, preserving images and text in order. Domain data supports grounding, pointing, OCR, long-tail visual knowledge, image-code relationships, and computer-use trajectories.

The report says its initial crawler favored text-centric pages, so it re-bootstrapped from Common Crawl to improve multimodal coverage. That illustrates how collection infrastructure itself shapes the learned capability distribution.

Filter cheaply before expensive processing

Downloading and decoding images incurs CPU, storage, and I/O costs. The pipeline first applies heuristic, statistical, deduplication, and quality filtering to documents before image retrieval. Surviving documents receive image-aware filtering, assembly, and model-based quality scoring.

SmolVLM is used for strict image-text quality scoring. Some rejected interleaved documents are recycled into simpler pairs through additional screening and recombination.

The general engineering principle is staged selection: avoid paying the most expensive processing cost for every raw candidate. The report does not disclose all thresholds or classifier calibration.

Integrating two pipelines

Text-only and multimodal corpora are constructed separately, then united. When samples overlap, multimodal versions replace text-only versions, and the larger configured epoch count is used.

The resulting corpus uses a 7:1 token ratio of text-only to multimodal data. That is a token ratio, not an image count or necessarily the final loss weighting between modalities.

If a fixed ratio applied uniformly to exactly 45T consumed tokens, it would imply 39.375T text-only and 5.625T multimodal tokens. This is conditional arithmetic, not a separately reported allocation: schedules, repetitions, and the interpretation of multimodal token accounting can matter.

Long-document packing

Ultra-long documents are deterministically pre-split before mixing so shards and steps receive more uniform token distributions. Joint sample assignment reduces overlap between pre-training and context extension. Enhanced best-fit packing achieves a reported padding rate no greater than $10^{-4}$.

At this scale, small inefficiencies accumulate. A low padding rate means fewer allocated token slots are wasted, but it is not a guarantee that every consumed token contributes equally useful learning.

Sample-level attention masking prevents different packed samples from becoming one artificial causal document. Packing improves utilization while preserving intended information boundaries.

The main schedule

The batch size stays at 100.6 million tokens. Dividing 45T by that batch gives approximately 447,316 steps, ignoring final rounding. Warmup consumes about 201.2B token slots under this fixed batch approximation.

The report says the run had no instability. That is an observation about this run, not a universal stability proof for all scales or data mixtures.

Sparse from the beginning

Sparse attention is trained from scratch at 64K sequence length, without a dense warmup stage. At 34T tokens, context extends to 1M.

The remaining 11T tokens amount to about 24.4% of the main token budget. That shows the extension is followed by substantial training rather than being only a positional setting changed at release.

It does not establish that every remaining sample is one million positions long. Maximum sequence length, packed sequence lengths, and underlying document lengths are different quantities.

Vision preparation is separate

The separately trained vision encoder uses approximately 47B contrastive image-text pairs and a 236B-token generative preparation stage with a 4B MoE model. The main language model then uses that encoder on multimodal inputs from the start.

The encoder is largely frozen until main learning-rate decay begins at 28T, with its final norm and projector trainable earlier. At decay it is jointly optimized at a smaller learning rate.

The report does not state every smaller-rate value or multimodal mixture change. Reproduction would require those missing settings.

Mechanisms introduced later

DSpark is trained in a dedicated stage after backbone pre-training. Decoder hierarchical restriction and main-cache FP4 QAT are introduced in post-training, and decoder replay is simulated there for adaptation.

One should therefore distinguish the base model's original training path from the final serving path. A released checkpoint can embody adaptations absent from its initial 45T-token run.

What the scale numbers cannot establish

The report does not provide the complete main-run device inventory, wall-clock duration, energy use, or independent training-cost accounting. Tokens alone do not determine those quantities.

Nor can a 45T-versus-earlier-budget comparison isolate architecture. More data, changed corpus quality, native visual training, and optimizer changes occur together. Base-model evidence in Chapter 23 tests outcomes, while attribution remains limited.

Sources and next step

Report §§4.1–4.2, pp. 20–23; §§2.3.2 and 2.4.3–2.4.4 for later adaptations. Schedule arithmetic is derived. Chapter 18 explains how this run handles shared state and multimodal work across devices.

18. Distributed Training Infrastructure

An architecture is not fully useful until its dependencies can be trained efficiently across many devices. V4.1 has three unusual demands: multimodal work with variable image cost, attention state shared across layer boundaries, and very large conditional-memory tables.

The infrastructure preserves the intended model semantics while controlling communication and activation lifetimes.

Parallelism has several axes

Data parallelism processes different samples with replicated model logic. Tensor parallelism divides a transformation across devices. Pipeline parallelism places different layers on different stages. Context parallelism splits a long sequence. Expert parallelism distributes MoE experts. Engram parallelism row-shards its tables.

These are background categories. The report explains selected mechanisms but does not publish a complete production topology with all group sizes and stage placements.

The axes interact. A shared bank may cross a pipeline boundary while its positions are split by context parallelism. Its transfer must preserve both ownership and the correct positional partition.

Contrastive communication overlap

Vision preparation needs comparisons across a full batch of image-text pairs. Features must be gathered across data-parallel ranks. The text-feature gradient depends on gathered visual features, while the visual-feature gradient depends on gathered text features.

The report schedules visual feature gathering during the text forward pass and text feature gathering during the text backward pass. Useful computation covers communication that would otherwise stall the pipeline.

The reported schedule hides both gathers behind work. This depends on sufficient work and bandwidth; it should not be treated as a guarantee for every batch shape and network.

Disaggregated vision execution

The vision encoder is replicated outside the language model's parameter tree. A training step has three phases: vision forward, LLM forward/backward, and vision backward.

This prevents irregular visual work from disrupting the LLM's established text-style parallel schedule. Load balancing for images happens in the first and last phases, while the middle phase keeps the language pipeline focused on its own computation.

The separation is physical execution organization, not a claim that gradients no longer flow between modalities. Joint training still requires visual feature gradients to return to the encoder when it is unfrozen.

Why a long sequence can overload one host

An image-rich million-position sample can contain enough raw media to exhaust a single host's CPU, memory, or I/O during loading. Splitting only the language tensor across GPUs does not necessarily split image retrieval and decoding.

The system shards images across context-parallel ranks with balancing and loads each image once. That prevents redundant host work and spreads ingestion.

The report gives a criterion comparing raw bytes per token and compute per token. Its central algebraic point is that sequence length cancels when loading time and compute time both scale linearly with token count.

For dimensional clarity, let $\rho$ be bytes per token, $B_{IO}$ bytes per second, $C$ compute work per token, and $R_{GPU}$ matching compute work per second. Loading can be hidden if

This restates the report's criterion with units made explicit. The source calls its GPU rate $B_{GPU}$; its units must be consistent with the chosen $C$.

The conclusion is conditional on balanced loading and linear rates. Fixed overhead, data skew, and different device counts can still matter. The report observes production-scale models remaining compute-bound while smaller ablations can become storage-bound.

Incremental images in rollouts

RL engines receive new images incrementally rather than retransmitting all previous images every turn. CPU decoding and preprocessing outputs are cached on distributed storage for reuse across rollouts and training.

This saves repeated preprocessing and transfer. It also creates a cache-identity obligation: the reused media must correspond to the correct image, preprocessing configuration, and sequence order.

Sharing across pipeline stages

CSA2 consumers may lie on different stages from their source. A stage-local module assumption is therefore insufficient. The report uses shadow indexers: executable replicas on participating stages with one logical parameter owner.

The owner handles optimization and checkpointing. Synchronization and gradient aggregation keep replicas consistent. Independent optimization of each replica would change the model, so ownership is a semantic requirement rather than merely bookkeeping.

Pipeline payloads carry intermediate representations and sparse routing information along existing point-to-point paths. Context-parallel partitioning avoids replicating entire long sequences unnecessarily and preserves gradient flow.

Microbatch lifetime management

Several microbatches can be active in a pipeline at once. Shared state from microbatch A must not be overwritten by microbatch B before A's last consumer finishes.

The runtime tracks source states through forward computation, activation recomputation, and backward propagation. It releases them after final use. This both prevents incorrect reuse and avoids holding all shared intermediates for the entire iteration.

The abstraction handles physical stage placement so attention code can express logical sharing without hard-coding the pipeline layout.

Engram ownership and communication

Tables are partitioned by rows across dedicated process groups; optimizer state is further sharded across replicas. Token-based addresses allow lookup prefetch for the local batch before microbatch execution.

Backward embedding gradients are buffered and returned to owners after backbone backward. Transfers overlap vision phases. Fetched FP8 values and scales feed subsequent GEMM directly.

Sinkhorn update kernels maintain scaling vectors instead of repeatedly rewriting a full normalized table. Row normalization and partial column statistics are fused. The goal is to make the enormous table update less dominated by memory movement.

What remains beyond the public specification

The report omits complete stage maps, communication traces, device allocation, scaling-efficiency curves, and every checkpointing corner case. It describes a credible engineering design, but reproducing identical utilization requires additional system code and topology.

A reader can verify dependency logic and nominal memory reductions without claiming to have reproduced the training cluster.

Sources and next step

Report §3.1, pp. 16–18. The unit-consistent I/O equation is explanatory background based on the report. Chapter 19 moves from infrastructure to the interactive tasks used after pre-training.

19. Making Agent Tasks and Environments

An agent training example is more than a prompt and answer. It includes an environment the agent can change and a system that judges whether those changes solve the problem. V4.1's report makes the construction of these tasks its main post-training contribution.

The authors explicitly say they introduce no new post-training algorithm. Their emphasis is what the policy learns from: scalable, diverse, verifiable tasks and interactive environments.

The task triplet

The paper defines a task as (problem, environment, verification system). Each member can fail independently.

A clear problem can be impossible in a broken environment. A working environment can contain a verifier that accepts the wrong solution. A correct verifier can demand behavior the problem never specified.

For example, “add CSV export” is incomplete if tests secretly require a particular filename, character encoding, and column order absent from the description. The learning signal then rewards guessing hidden assumptions rather than satisfying a well-defined request.

Task quality is therefore assessed along difficulty and correctness. Difficulty should be nontrivial enough to teach; correctness requires agreement among the three components.

Learning to construct tasks

The report observes that the model can help build its own training tasks, though imperfectly. Difficulty and correctness become reward signals for improving construction and verification behavior.

This creates a feedback loop: models construct tasks, solver trajectories reveal defects, and auditing or repair improves the task pool. The loop is not automatically trustworthy just because several agents participate. Independent checks and lifecycle monitoring are central to its credibility.

Whenever a task enters a new RL run, fresh trajectories provide evidence for re-auditing. A task that was adequate for an earlier policy may become too easy, expose a shortcut, or reveal a verifier weakness under a stronger model.

General-agent workflows

Employees and external partners voluntarily return interaction data and feedback from routine work. The pipeline reconstructs relevant tool interfaces and behaviors, including input formats, output structures, API schemas, and constraints.

Mocked tools cover common SaaS and enterprise systems as well as specialized business backends. Negative feedback and failure cases help create environments that replay observed weaknesses.

The goal is not merely to train on a correct response text. It is to recreate the context in which the agent chose actions, interpreted tool returns, handled user interaction, and failed.

Faithful mocks can make training cheap and repeatable. Their limitation is fidelity: a model can learn behavior that works in the mock but misses latency, partial failures, hidden state, or permission semantics in the real system. The report does not publish a comprehensive mock-to-production transfer audit.

Coding-agent source selection

Coding tasks come from difficult or poorly solved interaction sessions and from public repositories meeting a star-count threshold. Session data is deduplicated by trajectory.

Repository popularity is a practical filter for viable projects, not a guarantee of representativeness. It can bias coverage toward certain languages, ecosystem conventions, and project types. That is an interpretation of the selection rule, not a measured bias study in the paper.

The construction pipeline

An initial agent determines whether a project can build and run in a container and be automatically verified. It selects a starting turn or commit, proposes complex implementation directions, and specifies fail-to-pass and pass-to-pass checks.

A separate agent prepares dependencies, working directories, tests, and descriptions in an isolated container. It self-tests, removes solution-revealing traces, and packages the environment into an image layer.

Several distinct agents attempt the task. An independent inspector checks the environment and solver trajectories for factual errors, broken dependencies, disagreement between instructions and tests, and hackability.

If inspection fails, a repair agent fixes defects and calibrates checks that are too easy or too difficult. The task returns to verification before entering training.

This is a task-production process described in the report. It does not authorize or require delegation while writing this book.

Why both test categories matter

Fail-to-pass tests check the new intended behavior. Pass-to-pass tests check that previously valid behavior remains valid.

A solver can satisfy a narrowly targeted new assertion while breaking unrelated functionality. Regression checks make that shortcut less rewarding. They also increase environment cost because more code must build and execute reliably.

Tests are still only proxies for requirements. A broad pass-to-pass suite can contain flaky assumptions or irrelevant environment constraints. The inspection phase must assess the verifier itself, not just whether it runs.

Remove solution leakage

Generated setup scripts, comments, cached files, git history, or reference artifacts can accidentally reveal the intended implementation. An agent rewarded for finding those traces may appear capable without solving the actual task.

Removing traces helps, but stronger policies can discover unexpected channels. The report's later discussion of package mirrors and environment exploits shows why task auditing must continue after initial construction.

A high score is meaningful only if the information and action boundaries match the intended problem.

Difficulty is relative to the policy

If almost every rollout succeeds immediately, the task may teach little new behavior. If none can succeed because the environment is flawed, learning can be actively misleading. If none succeeds because the task is genuinely difficult, the reward may be too sparse without a suitable curriculum or teacher.

The paper emphasizes controlled difficulty and ongoing calibration but does not disclose a full sampling schedule, difficulty distribution, or exact success thresholds for the production task pool.

“Millions of environments” is therefore a scale description, not proof that millions of independent, equally informative learning problems exist.

Distribution coverage and deduplication

Trajectory deduplication removes some repeated examples, but distinct trajectories can solve effectively identical tasks. Repository forks, recurring issue templates, and shared generated specifications can create semantic duplication.

The report says it applies rigorous filtering and deduplication. It does not expose all algorithms, so an external reader cannot independently certify the exact diversity or evaluation contamination status.

Useful research would compare raw task count with clusters of underlying requirements, environment families, and solution strategies.

What the claimed gains mean

The authors attribute essentially all observed post-training gains under a fixed conventional optimization recipe to systematic data and environment improvements. That is their interpretation and engineering experience.

The report does not give a complete factorial experiment separating every source of task quality, scale, scaffold diversity, and model merging. A careful reader can accept that the effort concentrates on data without turning the broad attribution into an exact causal decomposition.

Sources and next step

Report §5.1 and §5.1.1, pp. 25–26. Chapter 20 explains how trajectories from these environments update the policy and enter distillation.

20. SFT, Reinforcement Learning, and On-Policy Distillation

A base model predicts continuations. A useful agent must also follow instructions, choose tools, handle failures, preserve context, and stop at an appropriate outcome. V4.1 applies supervised fine-tuning, reinforcement learning, and on-policy distillation to shape that behavior.

The paper emphasizes conventional optimization with improved tasks and infrastructure. Details inherited from V4 should not be mistaken for newly specified algorithms in V4.1.

SFT supplies behavior examples

Supervised fine-tuning trains the model to make desired responses or trajectories likely. Examples can teach response formatting, tool protocols, solution structure, and interaction conventions.

A standard token loss is the negative log probability of the desired continuation, with masks identifying which positions receive supervision. The V4.1 report does not provide a complete SFT corpus inventory, stage duration, or every loss mask.

SFT is useful for initiating competent behavior, but imitation alone does not directly optimize every environmental outcome. A trajectory can look plausible while failing the hidden tests.

RL evaluates sampled behavior

In an interactive task, the policy emits text or tool calls, receives observations, and eventually gets a score. Rewards can reflect task success, correctness, resource use, and selected behavioral goals.

V4.1 uses large-scale asynchronous RL in synthesized tasks. Its reasoning-effort treatment explicitly describes group-relative advantages: responses sharing a prompt and effort level are compared within that subgroup.

A background mean-centered advantage is

The equation explains the comparison logic, not the complete policy loss. Clipping, normalization, KL terms, token masks, and off-policy corrections would require further specification.

Why a verifier changes what can be learned

An exact test suite gives a relatively objective success signal for coding behavior. A mocked enterprise workflow may judge whether the agent reaches a valid state while respecting constraints. Less verifiable writing or conversation tasks need other reward methods.

V4.1's report does not redefine every inherited reward model or specialist setup. It would be misleading to import a detailed earlier recipe and present every parameter as newly verified for this release.

The strongest source-grounded lesson is that task correctness and reward integrity determine what RL reinforces. A hacked verifier teaches the wrong behavior even when the RL optimizer itself is functioning perfectly.

Scaling across scaffolds

A scaffold is the surrounding agent software: prompts, tools, context management, turn logic, and termination conventions. The report trains within one scaffold, across versions of a scaffold, and across different scaffold families.

Figures 7 and 8 show improving task performance as cumulative RL steps grow. Some curve segments are disconnected because successive runs are reinitialized by merging checkpoints from different training paths.

The horizontal axis is cumulative RL steps, not a fully specified hardware-compute budget. Step cost can vary with trajectory length, group size, context, and environment work. Reading it as exact FLOPs would be unjustified.

Model merging between runs

The report combines improvements from checkpoints trained with different scaffolds or configurations to initialize later RL runs. This is a practical way to aggregate progress from different optimization paths.

It does not specify the complete merge coefficients or procedure. Generic weight averaging can cause regressions if parameter paths diverge. One should not infer that arbitrary checkpoints are safely mergeable.

The observed continued gains support the authors' pipeline, but do not identify model merging's independent contribution separate from additional RL and data changes.

On-policy distillation

The final stage uses full-vocabulary OPD across all domains with more than 40 teacher models. The best teacher for one domain may come from a different development stage than another. Teachers can differ architecturally from each other and from the student.

“Full vocabulary” means supervision can use distributions over all vocabulary tokens rather than only a sampled teacher answer or a short Top-K list. Architectural heterogeneity is compatible with matching output distributions, provided token spaces and interfaces are aligned appropriately.

The report says the infrastructure supports effectively unbounded teacher counts and low-cost switching. It does not publish every teacher identity, exact teacher-selection policy, or token-space alignment mechanism.

Why student-generated trajectories help

If a student makes an imperfect intermediate choice, the future states it encounters may differ from ideal teacher demonstrations. On-policy distillation obtains supervision on those student-visited states, reducing the gap between training examples and deployment behavior.

A standard distribution-matching background objective can be written as a divergence between student and teacher next-token distributions. For example, reverse KL takes

This illustrates one inherited OPD perspective, not a fully reproduced V4.1 loss specification. The report names standard OPD and full-vocabulary supervision but leaves important implementation details to prior work and infrastructure.

Continuous recipe changes

The training system tracks capabilities and can adjust dataset mixtures, per-dataset concurrency, and active teachers. In asynchronous generation, samples produced under different configurations can coexist.

Consistent transitions require recording which configuration generated a sample and how it should enter training. A recipe switch is not automatically clean just because the trainer loaded a new teacher.

The report describes infrastructure support for these transitions without exposing all policy thresholds.

Adapting the final inference path

Post-training introduces decoder candidate restriction and main-cache FP4 QAT, and simulates decoder bounded replay. These are not only systems options switched on after learning is complete. The policy learns under relevant approximation constraints.

DSpark is trained alongside the evolving backbone without backpropagating its auxiliary loss into that backbone. It can accelerate RL and distillation rollouts as well as final serving.

The resulting checkpoint therefore reflects a combination of behavioral learning and adaptation to the served numerical and retrieval path.

What post-training cannot establish

Success on synthesized tasks does not guarantee transfer to every real tool, security boundary, or scientific problem. A model can become fluent in a harness while still making reasoning errors or overusing tools.

The report's benchmark gains test several important transfers, but uncontrolled changes and unavailable task inventories limit causal attribution. Evaluation chapters examine those boundaries rather than treating “standard RL” as a universal explanation for success.

Sources and next step

Report §§5.1–5.2, pp. 25–32; §§2.3.2 and 2.4.3–2.4.4 for architecture adaptation. Generic loss equations are background, not a complete release recipe. Chapter 21 explains the asynchronous and sandbox machinery.

21. Asynchronous Learning and DSec

Agent trajectories have uneven durations. One task finishes after a few turns; another runs commands, waits for builds, and generates hundreds of thousands of tokens. Synchronous training that waits for every trajectory can leave resources idle behind a long tail.

V4.1 generates RL and OPD samples asynchronously. DSec, DeepSeek Elastic Compute, supplies the long-lived interactive environments. These are related systems, but environment execution and GPU policy training are distinct resource layers.

Maintaining rollout concurrency

Each task specifies an upper bound on in-flight samples. The system maintains concurrency while generation proceeds, then preempts rollout work when enough training data accumulates.

GPU rollout and training are colocated on the same physical devices and time-share execution. This avoids a fixed manual allocation between a permanent rollout fleet and permanent training fleet.

The sandbox and worker-container control layer run outside the preemptible GPU training pool. Long-lived environment state can therefore survive changes in fine-grained GPU scheduling.

Why dispatch granularity matters

The authors tried batch-level replenishment and observed severe training-metric oscillations. Prompt-level replacement still stalled behind long-tail members of a GRPO group.

The chosen sample-level scheme dispatches a new prompt when the number of newly completed samples reaches the next prompt's group size, regardless of which groups produced those completions.

Dispatching at sample granularity does not mean advantages are computed across unrelated prompts. Scheduling completion counts and defining reward-comparison groups are separate responsibilities.

Completion order biases the dataset

Short trajectories complete earlier. Early batches can therefore overrepresent short responses even if prompt dispatch was unbiased.

The report uses per-dataset concurrency caps to regulate incoming proportions and can discard early-returned short samples while entering steady state. These mechanisms mitigate length skew rather than assuming asynchronous arrival preserves the desired mixture.

A toy example makes the issue clear: if half the tasks take one minute and half take an hour, the first few minutes' completions mostly come from the short group. Training immediately on all arrivals changes the effective objective.

Policy staleness

A trajectory may begin under one checkpoint and finish after training updates. Some tokens are therefore off-policy relative to the current model. Long trajectories can span several checkpoints.

The system bounds excessive off-policy proportions through dispatch and training-wait logic and masks tokens with excessive staleness from the loss. The report does not give every threshold or correction coefficient.

These measures address a statistical issue distinct from completion-length bias. Balancing dataset arrival rates does not automatically make old-policy samples current.

Routing and cache resumption

The report uses concatenated routing replay: when a trajectory spans checkpoints, it retains the expert routing associated with each generated segment rather than replacing it with new routing everywhere.

Generation can stop at any token boundary. KV and routing state are persisted at token granularity and reused after a checkpoint switch, avoiding an expensive full re-prefill. State is garbage-collected when each sample finishes.

Reusing old state under changed weights is not the same as recomputing an exact cache for the new checkpoint. The report presents it as training infrastructure with associated staleness controls. It should not be generalized into a rule that arbitrary production caches remain valid across arbitrary model replacements.

A worker and sandbox split

The sandbox runs the scaffold and tools. A separate worker supplies scaffold-agnostic orchestration, converts heterogeneous interactions into a common trajectory schema, and communicates with the trainer.

When preempted, rollout execution can suspend and offload while preserving full state. This decouples the long life of an interactive task from temporary resource assignments.

The design supports many scaffolds without requiring each one to implement the learning algorithm itself.

Scaling DSec

The report describes millions of concurrent sandbox instances and partitions compute nodes into scale units. Sharding reduces centralized pressure and limits how far a bad experiment can affect unrelated work.

A custom placement engine uses independent replicas and recent resource measurements rather than strongly synchronized global state. Each node enforces local admission thresholds and rejects unsafe placements.

The design trades perfect global placement consistency for scalability while retaining hard local resource safety. Eventual placement consistency is acceptable only because nodes validate the final decision.

High density and NUMA locality

Worker VMs are bound to individual NUMA domains using hardware-supported partitioning. Containers inside each VM keep CPU and memory local to that domain.

Under comparable workloads, the authors report increasing supported density from roughly 1,000 to more than 2,500 live containers per physical node before measurable end-to-end degradation.

This is a workload-dependent result. “Live container” does not mean 2,500 simultaneously compute-saturated tasks, and the report does not provide a universal node specification or capacity formula.

Protecting time-sensitive evaluation

Dense background workloads can distort tasks with time limits. DSec adds a latency-sensitive class, assigns non-sensitive tasks SCHED_IDLE, and uses core scheduling so sibling hyperthreads do not mix priority classes.

A benchmark failure caused by interference is different from a model failure. Resource isolation and timing conditions are part of valid evaluation, not merely infrastructure convenience.

Misbehavior and reward integrity

The report observes agents trying reward shortcuts, breaking critical files, and exploiting environment weaknesses. It uses per-sandbox AppArmor profiles and eBPF-based network policies. Environment crashes become failed trajectories with a repercussion signal.

The book does not turn the cited vulnerability examples into exploitation instructions. Their relevance is methodological: increasingly capable agents can optimize around an inadequate verifier rather than solve the intended task.

Isolation helps protect the system, while verifier integrity protects the meaning of the reward. Both are necessary.

What the scale claims leave open

The report does not provide complete scheduling efficiency curves, staleness distributions, rejection rates, or precise end-to-end cost per useful trajectory. High concurrency is not identical to high learning efficiency.

An ideal audit would connect environment utilization, policy freshness, task diversity, verifier failures, and final quality. The paper offers the design and selected observations, not that complete audit.

Sources and next step

Report §§5.1.2–5.1.3, pp. 26–29, and §5.2, pp. 30–32. Chapter 22 explains the reward mechanism used to control generated length.

22. Controlling Reasoning Effort

V4.1 accepts an integer reasoning-effort signal from 1 to 100. Higher values request more thorough reasoning. The model learns to respond to that signal through effort-dependent length penalties during RL.

Effort is a learned conditioning variable, not a hard token limit, a new checkpoint, or a guarantee of correctness.

Conditioning the policy

The report prepends an effort instruction to the system prompt. For prompt $x$ and effort $b$, responses are sampled from

Training uses a finite set of effort levels. Deployment can use intermediate integers, and the authors observe interpolated behavior. “Continuous” control means fine-grained behavior along a scalar scale; the exposed numeric variable remains an integer.

The released encoder places this prefix only at the conversation beginning and only in thinking mode. Changing the field halfway through an already encoded conversation may not produce a newly encoded conditioning instruction in that reference format.

Compare within each effort subgroup

Responses sharing the same prompt and effort form a subgroup. Rewards are mean-centered within that subgroup to obtain relative advantages. Low-effort responses are not directly ranked against high-effort responses in one combined group.

This avoids automatically punishing a short response for being shorter than a differently conditioned response. Each subgroup learns its own trade-off because its length penalty depends on its requested effort.

The report does not disclose every sampled effort value, subgroup size, or complete reward formula across domains.

The capped penalty

For reasoning length $\ell$, the additional reward is

where $L_{\mathrm{norm}}$ is a reference length and $C_{\max}$ caps the deduction. The coefficient is

Raising effort by $\tau$ multiplies the coefficient by $e^{-1}$. Higher effort therefore makes extra reasoning less costly under the reward.

$k_0$ controls overall pressure toward brevity. Smaller $\tau$ makes effort levels separate more strongly. The paper gives the functional form but not all domain-specific numerical penalty settings.

A toy reward calculation

Choose illustrative values $k_0=1$, $b_{\min}=25$, $\tau=25$, and $L_{\mathrm{norm}}=10{,}000$. At effort 25, a 10,000-token trace incurs an uncapped deduction of one. At effort 50 it incurs about 0.368; at effort 100, about 0.050.

These are not the model's published training hyperparameters. They make the direction and exponential scale concrete.

The cap matters: once the deduction reaches $C_{\max}$, the marginal penalty from this term becomes zero. The reward does not impose an ever-growing charge on arbitrarily long trajectories. Other behavior incentives, termination rules, and output limits still matter.

Why an exponential coefficient is motivated

Appendix C considers a problem's solve probability $p_x(\ell)$ as a function of reasoning length. In the uncapped region, a preferred length maximizes

An interior optimum satisfies

Assume the marginal benefit declines approximately exponentially:

Substituting the exponential penalty and taking logs gives

Thus an exponential penalty can yield a locally affine preferred-length response to effort. It is a motivation for the schedule, not a theorem about actual generated lengths.

The assumptions are important

The derivation assumes an interior optimum, an inactive cap, a smooth solve-probability curve, and approximately exponential marginal utility. Actual generation is discrete and stochastic. Effort can change strategy, not merely extend the same reasoning trace. Agent trajectories also vary in tool calls and turns.

Appendix C explicitly cautions that measured lengths need not be linear or pointwise monotone. One cannot infer that effort 80 always emits exactly twice the tokens of effort 40.

Public tier mapping in the report

The model's numeric range also includes values below 50. “Low” is a preset, not the minimum supported numeric effort. The release encoder defaults to high, or 75.

These are report and release-snapshot facts. Endpoint behavior should be verified independently when using a live API; this book does not establish a permanent API contract.

The reported empirical trade-off

Increasing effort from 25 to 100 raises average Pass@1 across eight reasoning-intensive benchmarks from 67.1% to 76.3%. DeepSWE rises from 66.0% to 74.2%, and Terminal-Bench 2.1 from 82.4% to 90.6%, with roughly $2.5\times$ more output tokens in the summarized comparison.

Appendix B.3 gives task-specific examples: MathArena Apex moves from 25.3% to 65.6%, while output grows from 29.1K to 86.1K. AIME output grows from 4.6K to 11.4K and accuracy reaches 100% at the reported maximum setting.

Those are measurements on stated evaluation sets, not universal efficiency ratios. Output tokens measure generated work but omit prompt processing and external tool time.

What the appendix adds about agents

The main text emphasizes improving quality with more effort. Appendix B.2 is more qualified: trajectory length grows monotonically across its six scaffold/benchmark panels, but Pass@1 has plateaus and dips at intermediate settings.

This means effort is a stronger control of resource use than a guarantee of accuracy monotonicity. The same checkpoint can respond differently depending on prompts, tool schemas, and context management.

The reported 60–80 range recovers much of maximum accuracy at less than half the maximum token budget in the summarized study. It is a useful candidate range for calibration, not a rule that every task should use 75.

Choosing effort by expected value

An illustrative decision criterion compares the incremental success probability $\Delta p$ times the value of success with incremental compute, latency, and tool cost. A difficult high-value task can justify high effort; a routine task with saturated accuracy may not.

Retries complicate this decision. A cheap attempt followed by escalation can cost less than max for every request, but correlated failures can reduce its benefit. The report does not test every such policy.

Calibration should measure the actual workload, including tail latency and failure type, rather than use generated length as the only proxy for usefulness.

Sources and next step

Report §5.1.4, pp. 29–30; §§5.3.2–5.3.3, pp. 33–35; Appendix B.2–B.3 and C, pp. 48–51; release encoding README. Chapter 23 starts the systematic evidence review.

23. Reading the Base-Model Evidence

Base-model evaluation asks what the model can do before the final agent behavior is elicited through post-training. The report compares V4.1-Flash-Base with V4-Flash-Base and V4-Pro-Base under one internal evaluation framework.

The strongest reading is a capability profile, not a claim that V4.1 dominates every predecessor on every task.

The complete reported comparison

Table 1 reports the following scores. The authors treat differences no greater than 0.3 as the same level; this is their presentation convention, not a statistical confidence interval.

The backbone sizes are 284B, 1.6T, and 552B respectively. Active figures are 13B, 49B, and 8B/16B. The V4.1 backbone count excludes its separately reported Engram allocation.

What the metrics mean

Exact match, or EM, judges agreement with a target under the benchmark's matching procedure. It can be sensitive to answer extraction and formatting. F1 measures overlap and is not interchangeable with exact-match accuracy.

Pass@1 estimates single-attempt correctness under the stated sampling/evaluation procedure. Multiple sampled attempts may be used to estimate it; that does not make it Pass@8.

An LLM-judge metric uses another model to assess the response. It can evaluate flexible text but introduces judge dependence. Acc@0.5 in visual grounding is a benchmark-specific accuracy criterion tied to an overlap threshold, rather than 50% language-answer correctness.

Shot count specifies how many examples appear in the prompt. A 25-shot knowledge result and a zero-shot coding result are different evaluation conditions, even though each yields a percentage-like number.

Clear improvements over V4-Flash

V4.1 gains 5.8 points on MMLU-Pro, 12.2 on SimpleQA-Verified, 6.6 on SuperGPQA, 3.8 on BigCodeBench, and 9.9 on HumanEval. These differences support substantial progress in parts of knowledge and coding.

Some tasks improve much less. LongBench-V2 rises only 0.5 points. Several scores fall: MGSM drops 5.5, AGIEval 0.5, BBH 0.8, and DROP 0.7.

The report's broad progress claim should therefore be read alongside these exceptions. A useful evaluation chapter preserves regressions instead of selecting only favorable rows.

Comparison with the larger V4-Pro

V4.1 exceeds V4-Pro on MMLU-Pro, BigCodeBench, HumanEval, and GSM8K. It remains lower on SimpleQA-Verified by 12.9 points, MultiLoKo by 5.4, MATH by 3.4, and LongBench-V2 by 6.3.

These differences matter to users with knowledge-intensive or long-context workloads. “Comparable” describes an overall profile, not equality row by row.

The active-parameter comparison also has two valid phase choices: 16B decode versus 49B is about one third, while 8B prefill versus 49B is about one sixth. The introduction describes one-quarter activated parameters, which does not follow directly from either listed phase count divided by 49B. This book records that arithmetic mismatch rather than inventing a missing denominator. Exact ratios should name the phase.

Long context capacity is not long-context reliability

The model supports about one million positions, but its base LongBench-V2 score does not lead the comparison. A large context window is a capacity property. Reliable synthesis across that window is a behavioral property.

Sparse selection, representation compression, data mixture, and task difficulty all affect the latter. The report does not isolate their contributions in Table 1.

A million-token prompt that contains one easy retrieval target is different from one requiring several dispersed facts and a long chain of reasoning. A single long-context benchmark cannot characterize every such case.

Native visual evaluation

The visual rows establish that V4.1-Base can perform evaluated multimodal tasks. Earlier models have dashes because those entries are not supplied, not because their scores were measured as zero.

The table does not compare the visual base model with every contemporary multimodal system. It supports capability presence and selected scores rather than a universal visual ranking.

Bits per byte on held-out internal corpora

The authors also evaluate internal documentation, proprietary code, and academic materials. Figure 6 reports bits per byte, or BPB, with lower values better.

A standard BPB calculation is

Normalizing by bytes helps compare models with different tokenizations, though byte encoding and preprocessing still matter. It measures predictive compression of the held-out text rather than directly measuring agent success.

The report says V4.1 has the lowest BPB on all these task categories and describes 5%–10% improvements on held-out evaluations in its introduction. The figure does not supply a complete raw per-document dataset and uncertainty analysis. This book does not invent exact bar values from the graphic.

Why BPB and task scores can disagree

A model can assign better average probability to technical documents while still answering a particular knowledge question incorrectly. BPB aggregates token prediction; task evaluation tests a specific behavior under a prompt.

Internal corpora can also resemble the developers' workload more closely than a public benchmark. Their relevance is valuable but their private construction limits independent validation.

Attribution remains open

V4.1 changes capacity, pre-training budget, curation, vision integration, optimization, and architecture. Table 1 compares final base checkpoints, not a controlled architecture-only experiment.

The correct conclusion is that the new training-and-model package has a strong but uneven profile at lower active compute than V4-Pro. Assigning each gain to one component would require ablations not supplied here.

Sources and next step

Report §4.3, Table 1, and Figure 6, pp. 23–25; introduction p. 6. Scores are transcribed from the PDF, with derived differences explicitly discussed. Chapter 24 reviews post-trained reasoning and agent results.

24. Reasoning and Agent Benchmark Evidence

The instruct-model evaluation concentrates on reasoning and agent work. The report's maximum-effort results show large gains over V4-Flash, particularly in everyday coding and general workflows, with important gaps on harder scientific, security, and visual tasks.

All values below are claims from the inspected report. They are not independently rerun measurements or current leaderboard rankings.

Evaluation conditions

Reasoning evaluation uses temperature 1.0 and top-p 1.0 in §5.3.1. Code-agent evaluation uses temperature 1.0, top-p 0.95, and a 1M context window. Most code tasks use DeepSeek Harness Minimal; DeepSWE uses mini-SWE to align with the official setup. SEC-Bench Pro uses Claude Code for its session compaction.

Visual agents use Claude Code with a 512K context window and the same agent sampling settings. Agents' Last Exam and AutomationBench use their official scaffolds.

The downloaded model card summarizes instruct sampling somewhat differently, applying 0.95 in its general presentation. The PDF's per-category setup governs the tables in this chapter. Such differences belong in a reproduction log rather than being silently smoothed over.

The main comparison

Table 3 names the comparator settings as maximum effort. Model names below are source labels. † denotes the text-only HLE subset.

NL2Repo's PDF score is 65.4; the separately downloaded model-card snapshot reports 64.0. This book preserves the discrepancy and uses the PDF for this table.

The strongest practical gains

Relative to V4-Flash, DeepSWE improves by 19.8 percentage points, Terminal-Bench 2.1 by 7.9, AutomationBench by 17.1, and Agents' Last Exam by 6.6.

These gains support the paper's emphasis on coding and general workflows. They are much larger than some reasoning changes, such as GPQA's one-point increase.

A model's distinctive profile can matter more than a single average. A workflow agent may care more about resolved repository tasks than competition mathematics, while a research assistant may prioritize the opposite.

Harder tasks preserve a gap

V4.1 trails the strongest listed values on Terminal-Bench 3.0 and 4.0, ProgramBench, ExploitGym, and several visual-agent tasks. Its HLE result is also substantially below the strongest full-set comparator.

The table's 90.6 on Terminal-Bench 2.1 does not imply the same success level on later, harder suites. Benchmark versions test different distributions and should not be treated as one interchangeable score.

The report explicitly says everyday performance can be close while difficult frontier tasks remain separated. Its conclusion mentions additional product labels not present in Table 3; those narrative comparisons are not treated here as a second numerical evaluation.

Subsets cannot be silently compared

V4.1 has both full HLE and text-only results. V4-Pro, V4-Flash, and GLM rows identify text-only values. Comparing 36.8 full-set directly with 42.7 text-only as if they were identical task sets is invalid.

Likewise, HLE with tools is a different protocol from closed-book HLE. The large increase to 63.9 reflects a changed system with tool access, not a pure improvement in memorized knowledge.

Missing entries are unavailable comparisons, not zeros.

Metrics are not interchangeable

Codeforces rating is not a percentage. NL2Repo uses a score, ProgramBench an Almost@1 criterion, and ZeroBench Pass@5. Averaging these raw values would have no coherent meaning.

Pass@5 permits multiple attempts and differs from single-attempt reliability. With independent per-attempt success $p$, a toy at-least-one-success probability is $1-(1-p)^5$, but real benchmark estimators and correlated attempts can differ. This formula is background, not a conversion of the reported ZeroBench score to Pass@1.

Even percentage-like metrics can measure different outcomes: passing tests, completing workflows, or matching answers.

Small margins need uncertainty

DeepSWE 74.2 versus 74.0 is a 0.2-point reported lead, not proof of a meaningful underlying superiority. Sampling, finite tasks, correlated attempts, and setup differences can affect the difference.

For independent Bernoulli observations, a basic standard-error approximation is $\sqrt{p(1-p)/n}$. Real repeated attempts within tasks require more careful clustered analysis. The report does not publish full confidence intervals for every comparator.

The large predecessor gains are more persuasive than tiny ranking margins, but they still reflect a changed model-and-training package rather than one isolated intervention.

Evaluation defenses

The authors restrict network access, remove Git histories, and purge build/package caches to reduce answer leakage and reward shortcuts. They still observe exploit-seeking behavior during testing.

This shows why hidden tests and containers alone do not certify valid evaluation. An agent can use environmental artifacts to obtain answers or alter the verifier. Restricting channels improves validity but does not guarantee zero contamination or gaming.

What remains necessary for reproduction

A valid reproduction needs the checkpoint, reasoning effort, sampling parameters, benchmark version and subset, scaffold version, tool permissions, context limit, generation-round limit, timeout policy, dependency state, and scoring procedure.

The repository supplies a focused DeepSWE reproduction guide, not a complete rerun package for every number. Public release is valuable evidence access, but it is distinct from independent replication.

Sources and next step

Report §§5.3.1–5.3.2 and Table 3, pp. 32–34; saved model card for discrepancies. Chapter 25 examines how the same checkpoint behaves under different surrounding agent systems.

25. Agent Scaffolds and Multi-Agent Teams

An agent is a model embedded in software that supplies prompts, tools, context management, and stopping rules. V4.1's report evaluates both variation in this scaffold and collaboration among several copies of the policy.

These experiments show that system design matters alongside model weights. They do not establish one universally best harness or prove that adding agents always helps.

The same checkpoint in eight configurations

Table 4 compares six scaffold families, with three DeepSeek Harness modes. All use effort 100, temperature 1.0, top-p 0.95, Linux containers, a 1M context limit, and at most 500 model-generation rounds.

DeepSWE uses eight samples per task, and Terminal-Bench 2.1 uses three. These repetitions estimate single-attempt metrics; the reported rows are not “succeeds at least once in eight attempts.” Terminal-Bench runs without network access.

The range is 8.7 points on DeepSWE and 6.5 on Terminal-Bench. These are derived differences between highest and lowest reported values.

What changes with a scaffold

The model may see a different system prompt, shell tool, file tool, task delegation interface, context-compaction method, or result format. One shell-only interface can expose flexible command composition, while another offers many structured functions.

The report uses native runtime prompts and adds no experimental system prompt. It identifies Codex v0.147.0 with adapted schemas, OpenCode v1.18.15, Pi v0.84.2 with a search extension, and a mini-SWE port pinned to commit 04d809ceab9d.

DeepSeek Harness Minimal has a single bash tool; Standard starts with 26 function tools; PTC exposes TypeScript programs over 24 underlying tools. Tool count alone therefore does not predict performance: Minimal leads the Terminal-Bench comparison.

These are study-specific versions and integrations, not verified statements about current installed products.

Version variation inside Claude Code

Appendix Table 5 reports four versions:

Table 4 uses 2.1.251, not this average. The average was calculated from unrounded scores. Recomputing from displayed rounded values may not reproduce it exactly.

Version changes can matter even with the same checkpoint and benchmark. A reproduction should pin the scaffold rather than identify only its family name.

Transfer is robust but not invariant

The report interprets consistently strong scores across scaffolds as generalization beyond one harness. That is reasonable: the model remains useful under several interfaces.

The observed range also shows material dependence on the surrounding system. “Robust transfer” should not mean that harness choice is negligible. Native delegation, context management, and prompt conventions can change the effective task.

Training across diverse scaffolds plausibly supports transfer, but this experiment does not isolate its causal contribution from task synthesis and other post-training changes.

Effort depends on the scaffold

Appendix Figure 11 shows that increasing effort raises mean trajectory length in every tested scaffold/benchmark pair. Accuracy responds less uniformly, with plateaus and intermediate dips.

Claude Code has a flatter DeepSWE response and spends relatively fewer additional tokens; Minimal starts lower and gains more at larger token cost; mini-SWE falls between them. On nearly saturated Terminal-Bench 2.1, scaffold choice can matter at least as much as the effort tier.

A single numeric effort setting is consequently not a fixed computational budget across agent systems.

The Agent Team experiment

DeepSeek Harness Agent Team lets a lead create named persistent teammates. A teammate starts with a fresh task or a snapshot of the lead's completed history. Agents share one repository checkout and communicate through durable mailboxes.

The lead monitors status and the shared task board, manages dependencies and advisory write scopes, can interrupt teammates, and reviews and tests the combined result. Shared state enables collaboration but also creates conflict risks when several agents edit the same files.

These are the report's harness semantics, not evidence that every multi-agent implementation works identically.

Training for useful parallelism

The reward combines task performance, a collaboration bonus encouraging delegation and communication, and a derived-latency penalty.

Execution events and dependencies form a directed acyclic graph. Token counts are converted to costs at fixed prefill/decode rates, tool time contributes measured costs, and the critical path estimates derived latency.

For a simple example, two independent 10-minute subtasks plus a five-minute integration step have a 15-minute critical path if run together, versus 25 minutes sequentially. If the second actually depends on the first, the path remains 25 minutes. More agents do not remove real dependencies.

The surrogate reduces sensitivity to incidental queueing and batching, but its fixed rates are still an approximation to production latency.

Preliminary results and their subsets

The ProgramBench study retains 172 “golden” tasks whose reference solutions pass at least 95% of hidden tests. Up to three rollouts per task gives 516 planned rollouts per configuration. Almost@1 measures individual rollouts achieving score at least 0.95.

At one hour, multi-agent and single-agent scores are 13.59% and 12.79%. At the reported eight-hour peak, they are 30.04% and 20.39%. The final difference is 9.65 points.

FrontierSWE v2 uses the currently public no-GPU subset and Mean@5. At one hour, scores are 13.50% versus 10.50%; at 20 hours, 32.90% versus 28.20%, a 4.70-point difference.

These subsets and metrics differ from Table 3's broader comparisons. The ProgramBench team score cannot be directly substituted into the main table as if it used identical tasks and compute.

Wall-clock equality is not compute equality

Several agents can spend more total tokens and device time within the same deadline. A team advantage at equal wall-clock time does not by itself establish superiority at equal compute or equal cost.

The authors compare the strongest observed team configurations with the strongest available single-agent baselines and label the results preliminary. Configuration search and filtering affect interpretation.

Useful further studies would match token budgets, agent counts, hardware throughput, and configuration-search effort, while measuring coordination overhead and edit conflicts.

Sources and next step

Report §§5.3.4–5.3.5, Table 4, and Figure 10, pp. 34–36; Appendix B.1–B.2 and Table 5, pp. 47–49. Chapter 26 explains what the public repository exposes for inspection and use.

26. The Released Model, Implementation, and Protocol

The paper explains a model and production system. The repository supplies weights, configuration, reference inference, prompt encoding, and a focused benchmark reproduction guide. These artifacts overlap but are not identical specifications.

This chapter describes what was actually inspected and what remains unverified. It does not claim that the full checkpoint was downloaded or executed.

The inspected snapshot

The repository metadata returned revision 5254448103b6254fc646dacbca1cbbf20e9216c1. Reference inference, encoding, evaluation files, the license, and the Jinja file were downloaded at that revision.

The PDF, main model card, and top-level configuration were initially fetched from main. Their hashes are recorded in the source manifest, with retrieval provenance distinguished from pinned files. The PDF's SHA-256 matches the value published on its Hugging Face file page.

The source directory contains selected artifacts only. It is not a complete clone or model-weight download. Reading code provides stronger specificity than relying on platform-generated usage widgets, but does not certify runtime compatibility.

Configuration facts

The top-level architecture is DeepseekV41ForCausalLM, with model type deepseek_v41. Nested text and vision configurations describe the two pathways.

The text vocabulary contains 129,280 entries; the maximum position setting is 1,048,576. The configuration has one main KV head, width 512, 64 query heads, and the source-layer arrays described in Chapter 3.

The precision configuration specifies FP8 quantization, dynamic activation handling, 32×32 weight blocks, UE8M0 scales, and FP4 expert dtype. This is checkpoint/runtime metadata, distinct from the main-cache E2M1/E4M3 layout.

The listed Transformers version is 5.6.0. A version string records the release environment; it is not proof that a generic package installation on any platform can execute the model.

What the reference does implement

The inference README calls it a readable reference, not a production engine. Code covers vision and projection, local plus sparse attention, two-level selection, Engram lookup, MoE, hyper-connections, and the DSpark forward path.

It is valuable for checking dimensions, cache ownership, causal group completion, image masks, output projection groups, and the flow of shared selections.

Its small self-test uses uninitialized weights and checks shapes and kernel plumbing. That is not a numerical correctness test of the trained checkpoint or a quality benchmark.

What it does not establish

Plain Transformer.forward traverses all backbone layers. Source compression uses the attention module's supplied hidden input. It does not demonstrate the paper's optimized encoder-only bulk prefill, encoder-output decoder global projection wiring, or bounded-replay production scheduler.

Later indexers compute full available scores before candidate masking, rather than demonstrating candidate-only dot-product work. Generation is plain autoregressive sampling despite a DSpark forward method.

These differences mean the reference is insufficient to reproduce several headline production savings. They are documented as artifact boundaries, not silently treated as proof that the deployed system has no such optimizations.

Weight conversion and hardware assumptions

The released conversion guide produces one checkpoint shard per tensor-parallel rank. Its example uses eight ranks and FP4 expert conversion. Distributed generation uses torchrun; multi-node execution adds the usual rendezvous and node arguments.

Eight ranks are an example partition, not a complete minimum-memory recommendation. Device capacity, kernel support, Engram placement, active cache length, batch size, and conversion overhead determine feasibility.

A responsible deployment estimate must inspect actual tensor bytes and scales, then add runtime state and headroom. Active parameters alone cannot size weight storage.

The book does not supply an untested one-line “works on any GPU” command. The source's examples require their dependencies and compatible low-precision kernels.

Prompt format is part of model behavior

The encoding reference handles system, user, assistant, tool, and reminder roles. It inserts model-specific delimiters, thought boundaries, tool schemas, and ordered image placeholders.

Changing this format can alter behavior even if the weights are unchanged. A generic chat-template assumption can misplace reasoning instructions or erase state needed across tool calls.

The encoder accepts thinking_mode="thinking" or chat mode and numeric effort 1–100, with low/high/max aliases 50/75/100. High is the default. The effort prefix is rendered at the beginning in thinking mode.

Tool serialization

V4.1 DSML tags include a space before tag names, for example <｜DSML｜ calls> and <｜DSML｜ invoke>. This differs from older V4 tag conventions. String parameters are serialized as strings; other types use JSON-form values.

Tool results are represented through the encoder's merging rules and ordered according to corresponding calls. Namespace support qualifies function names and checks agreement between explicit namespaces and qualified names.

These details are protocol contracts for serialization and parsing. They do not make the model's generated tool arguments automatically valid. A caller still needs parsing and schema validation before execution.

Reasoning retention

By default, the encoder can drop earlier reasoning before the final user turn in ordinary conversations. With tools, that dropping is automatically disabled so the policy can retain multi-step context.

This helps explain why agent conversations can become input-heavy: internal reasoning and tool history remain in the encoded context. A caller that independently strips them changes the deployment protocol and may change behavior.

The parser is designed for well-formed outputs and does not promise to repair malformed generation. Its README recommends additional production error handling.

Images have two representations

The prompt string contains an image placeholder, while a separate media list preserves the actual image records in prompt order. The inference processor expands images into the visual sequence spans and supplies pixels to the vision path.

Counting placeholders as one language token per image would underestimate sequence length. Conversely, changing media order while leaving placeholder order fixed can associate the wrong image with the user's text.

The reference requires image spans in the initial prefill chunk. That is an implementation limitation of this sample, not a general proof that the production system cannot accept new images in later agent turns; the report describes incremental image transfer during rollout.

Source discrepancies to preserve

The saved model card says no Jinja-format template is included, while the pinned repository contains chat_template.jinja. Presence alone does not certify its equivalence to the Python encoder. The contradiction remains unresolved in this book.

The PDF and downloaded card differ on NL2Repo, and sampling summaries differ from the PDF's per-category description. Main-fetched artifacts can also move independently of pinned files. Hashes and provenance prevent a later reader from assuming one perfectly synchronized release specification.

What can be reproduced with the supplied guide

The evaluation README gives pinned Pier and DeepSWE commits, container setup, a reference integration patch, mini-SWE and Minimal runs, and result-file locations. It discusses CPU caps, loopback behavior, network restrictions, and preserved logs.

That is enough to prepare a focused reproduction attempt with access to the model endpoint and suitable resources. It does not prove the report's score before the attempt is run, nor does it reproduce all training or serving infrastructure.

Sources and next step

Selected release files: config.json, inference README and model code, encoding README and code, evaluation README, model card, and template inventory. Pinned remote links and hashes are in Chapter 29. Chapter 27 integrates the described mechanisms in one request.

27. One Request from Beginning to End

Consider a user asking an agent to fix a chart-rendering bug in a repository. The conversation already contains a long project overview. The user adds a screenshot, the agent reads files and runs tests, and later resumes after its short-lived local state has expired.

This is an illustrative trace of the paper's proposed system. It is not a recorded V4.1 execution, and it does not claim that the public reference implements every production step.

1. Encode the conversation

The caller supplies system instructions, tool definitions, user messages, prior assistant actions, tool results, and the screenshot in a structured format. The model-specific encoder creates role delimiters, thinking boundaries, and tool-schema text.

Assume thinking mode and effort 75. The effort instruction appears at the beginning of the conversation. Tool use preserves earlier reasoning under the reference's retention rule.

The screenshot's placeholder is paired with a media record. Correct ordering ensures it is associated with the request rather than an earlier image. Prompt strings and image pixels are separate transport objects until preprocessing constructs the model input.

2. Construct visual positions

The image is resized under the preprocessing policy and divided into 14-pixel patches. The vision Transformer contextualizes patch features with two-dimensional position information.

If its compatible grid were $96\times96$, pixel-unshuffle would produce $32\times32=1024$ visual feature positions. The projector maps them to 5,120-wide vectors. Delimiters and row markers can add positions beyond the pure feature count.

The model sees an interleaved sequence, not a text caption substituted for the screenshot. It can learn relationships between the image and neighboring text directly through the multimodal backbone.

3. Locate reusable prefix state

Suppose the previous transcript contains 200,000 positions and the new material adds 10,000 text positions plus the image span. A compatible global prefix cache can avoid recomputing the old 200,000 positions.

Compatibility includes the checkpoint, prompt identities, positional conventions, and cache representation. This trace assumes a valid hit; it does not infer that every production prefix cache uses a particular hashing algorithm.

If encoder local state is still in the short-lived DRAM pool, processing can continue from it. If it has expired, the system replays the last 128 prefix positions to reconstruct approximate local state while retaining old global KV.

4. Process the suffix through the causal encoder

New embeddings expand into four residual streams. The first two backbone layers use local attention. Engram at layer 1 contributes gated token-n-gram memory for text positions.

At layer 2, Full mode constructs a pair-compressed global bank, projects indexer keys, and selects up to 512 entries for each query. Layers 3–7 reuse that bank and selection while computing new main queries, local state, expert outputs, and residual updates.

Layers 8 and 14 create the next encoder banks. Engram contributes again at layer 14. Image tokens use modality-appropriate routing corrections, while lexical Engram lookup is masked for visual spans in the reference.

Only completed pairs become visible in global banks. The local window provides recent individual positions while an encoder group is incomplete.

5. Build decoder global memory

In the reported CED architecture, final encoder outputs project into decoder global KV. The decoder source has ratio one, so each position contributes a token-granular latent rather than a pooled pair.

That one bank serves all 20 decoder layers. Bulk suffix positions do not need full decoder transformations merely to populate it.

The main latents are quantized after rotary position encoding with the E2M1/E4M3 layout. Indexer keys use their own MXFP4 format. These are different numerical objects even though both have four-bit payloads.

6. Establish decoder local state

The system sends the latest 128 prompt positions' encoder outputs through the decoder. Local attention is truncated at the segment boundary, so the resulting decoder SWA is approximate.

This replay provides the layer-specific recent states that global CED projections cannot supply. It also provides the deep boundary computation needed to start generation.

No persistent decoder SWA snapshot is required for the long-lived prefix tier. The state is useful for this decoding continuation and can later be reconstructed.

7. Generate the first action

A generated token traverses all 40 backbone layers. Each MoE layer uses one shared and six routed experts. The full decode path uses the reported 16B active parameter category.

At the first decoder Full layer, the indexer searches visible decoder global memory, selects its own Top-512, and builds up to 16,384 candidate positions from high-scoring blocks.

Layers 24, 28, 32, and 36 can select different Top-512 lists within that pool. Reuse layers share the current list but make new main-attention comparisons. Single-Pass mHC organizes residual flow between sublayers.

The resulting distribution may emit a tool call to inspect the chart component. The model itself does not read the file until the external harness executes the call and returns its contents.

8. Optionally speculate

In the production DSpark path, the drafter proposes several future positions. Its confidence estimates and engine profiles choose how much to verify. The backbone verifies proposals before they become accepted output.

Rejected proposals must not be treated as final conversation state. Correct cache and position updates around rejection are part of a complete speculative engine. The public sample's plain generation does not demonstrate this whole step.

9. Execute tools and return observations

The harness validates and executes a file read or test command. It returns source code, build output, and exit status under the protocol. The next model call processes those observations as new input.

A 200-token command can return a 15,000-token test log, making the interaction input-heavy. CED reduces the new-input computation; prefix reuse prevents old turns from being reprocessed; global compression keeps the growing context affordable.

Tools still consume external time and can fail. A model-side cache reduction cannot accelerate a ten-minute build itself.

10. Resume after a pause

The user returns later. Global state remains in the long-lived cache, but encoder local state has expired from the short-TTL pool. Encoder bounded replay recovers a 128-position tail, then the new suffix is processed.

New suffix states can differ from full-prefill states because the replay boundary truncates local dependencies. The paper reports small quality impact but does not promise identical logits.

The system again establishes decoder local state through bounded replay. This is how cache policies and architecture cooperate rather than simply storing every layer's local state indefinitely.

11. Verify and finish

The agent edits code, runs relevant tests, inspects the rendered chart, and produces a final explanation. Higher effort can encourage more exploration or verification, but cannot guarantee it chooses the right tests.

Success is determined by behavior and the environment's checks. An answer that sounds confident does not establish a correct patch. Likewise, passing a weak test suite may miss a visual regression.

A resource ledger for the trace

The global cache at 211,000 illustrative positions would be approximately $211{,}000\times890=187.79$ MB, ignoring exact image-span count and overhead. Local windows remain bounded. Weights and Engram storage are independent of this request length.

The ledger has separate entries for newly processed input, output tokens, replay, transfers, vision work, and tools. These categories explain why no one architectural ratio fully predicts the session's latency or price.

Sources and next step

This illustrative integration draws on report §§2–3 and §5.1.4, plus released encoding and image-processing documentation. Numerical sequence lengths are invented for explanation. Chapter 28 turns the trace into practical limits and research questions.

28. Limits, Research Questions, and Practical Decisions

V4.1's report supports a coherent engineering result: a large multimodal MoE can retain strong evaluated capability while making long context substantially cheaper to compute, store, and reuse. It does not prove universal reliability or isolate every mechanism's contribution.

The important conclusion is a package-level advance with specific remaining boundaries. This chapter turns those boundaries into questions a researcher or practitioner can actually test.

What is structurally well supported

The published schedule has four global KV sources rather than independent global banks in every layer. Its specified formats and entry counts reproduce the nominal 890-byte-per-position figure.

CED changes the decoder-memory dependency so bulk prefill can avoid most decoder transformations. Single-Pass mHC removes an input-mixing dependency and supports a lower activation-traffic count. Hierarchical indexing caps deeper decoder candidate searches. Bounded replay limits reconstruction work by accepting approximate local state.

These properties follow from the described design. They should be distinguished from measured end-to-end performance, which also depends on implementation and workloads.

What is empirically supported

The reported base and instruct evaluations show improved capability in many important domains, including substantial gains over V4-Flash in repository work and general agent tasks. Some reasoning, knowledge, and long-context rows remain lower than larger comparators.

The authors report small degradation from cache quantization, shifted mHC mixing, and bounded replay, plus improving outcomes with increased RL and effort. The scope is their tested settings, not an exhaustive distribution of all possible inputs.

The public artifacts make selected details auditable, but this book has not rerun the full checkpoint or production cluster. Its validation concerns the source-grounded explanation and documentation artifact.

The global representation bottleneck

Sharing historical memory across many layers reduces independent representational views of old positions. Pair pooling further reduces encoder granularity. Quantization adds numerical error.

These approximations can interact. A useful feature weakened by pooling may receive a worse indexer score; an omitted candidate cannot be recovered by later Reindex layers; approximate replay can alter the new suffix representation that constructs future memory.

The report does not give a complete joint error analysis. Testing each approximation alone may underestimate combined failures at difficult boundaries.

Sparse retrieval stress cases

Useful tasks include multiple distant facts that must be combined, contradictory documents that require attribution, repeated near-matching identifiers, evidence surrounded by distractors, and queries whose relevant concept emerges only after deeper reasoning.

Measure candidate recall, selected-entry recall, answer correctness, and attribution accuracy. Also vary Top-K, candidate cap, block size, and cache-source frequency.

These are proposed stress tests, not published failure rates. The authors explicitly identify sparse-selection robustness as future work.

Replay boundary stress cases

Vary cache-hit positions while keeping the logical prompt fixed. Place critical local dependencies just before the replay segment, interleave images and text near the boundary, and use short new suffixes after very long prefixes.

Compare full-prefill and replay paths with controlled sampling or teacher-forced log-probabilities. A state-difference metric can identify numerical changes, while answer-level metrics determine whether they matter.

The key question is not whether approximate state differs—it does—but how frequently and how severely that difference changes useful behavior.

Multimodal limits

Variable resolution and native visual training do not imply reliable reading of every tiny label or complex diagram. Visual-token reduction creates an information bottleneck. Long transcripts with many images also place demands on ingestion and global retrieval.

Task-specific checks should vary resolution, text size, image count, aspect ratio, and the distance between visual evidence and the question. Pure perception and tool-assisted reasoning should be measured separately.

The report provides selected base and visual-agent results, not a complete visual robustness envelope.

Reward and environment limits

Agent learning optimizes a verifier. If that verifier accepts shortcuts, agents may learn shortcuts. If environment failures are frequent, the training signal may reflect infrastructure rather than intended difficulty.

The report's task audits, isolation controls, and evaluation defenses address these concerns, but the private task inventory prevents complete independent inspection.

For a real deployment, success should include valid actions and preserved constraints, not only a final score. That is a practical evaluation principle derived from the task-triplet discussion.

Attribution limits

V4.1 changes model size, attention, prefill organization, Engram, visual integration, optimizer, data, token budget, tasks, and serving. A predecessor comparison cannot tell how much each contributes.

A useful controlled program would compare otherwise matched runs with and without CED, sharing, Engram, hierarchy, FP4 main cache, replay, and scaffold diversity. It would also hold training budget and data fixed where feasible.

Some interactions may make a full factorial study too expensive. Even targeted ablations with explicit controls would improve causal understanding.

Cost claims that need direct measurement

Nominal cache bytes are comparatively easy to audit. Dollar cost, energy, latency percentiles, and fleet throughput require hardware and workload evidence.

Measure uncached prefill throughput, prefix-hit continuation, local-state-miss replay, decode under load, cache transfer, image encoding, and tool execution. Include temporary allocations and effective concurrency rather than only steady-state global KV.

A memory improvement can increase concurrency without shortening one request. A decode improvement can be hidden behind tool time. A smaller persistent footprint can improve hit rates. These are different practical benefits.

Choosing a workload fit

For a coding assistant dominated by repository reading and tool output, the architecture targets relevant costs and the agent results are promising. For a difficult scientific reasoning workload, the reported frontier gaps deserve attention. For factual closed-book questions, the base knowledge profile is uneven compared with V4-Pro.

For visual workflows, native inputs are a meaningful capability, but pure image recognition and tool-assisted chart reasoning should not be conflated.

These are evidence-based interpretations of the report, not current product recommendations. A real purchasing or deployment decision needs contemporary availability, pricing, and workload tests beyond this snapshot.

A concrete local evaluation plan

First, collect representative tasks with independently reviewable success criteria. Include ordinary cases and expensive failure cases. Preserve the actual prompts, tool interfaces, and environment setup.

Next, test several effort values and at least the intended production scaffold. Record success, latency, input and output tokens, tool time, retries, and cache status. Treat invalid tool calls, stale state, and environment errors as distinct failure categories.

Then exercise cold starts, long-prefix hits, local misses, image-rich inputs, and maximum-context boundaries. Compare cost per completed useful task, not merely token price or benchmark average.

Finally, inspect failures to decide whether they need a better model, improved tool definitions, stronger verification, different effort, or a cache-policy change. Increasing all budgets is not always the most informative fix.

Reproduction boundaries

The released reference helps with tensor structure and protocol. It does not establish the full CED fast path, candidate-only scoring, bounded replay scheduler, or DSpark verification system. A reproduction of the serving claims must implement or obtain those paths.

Similarly, the paper supplies enough training detail to understand the strategy but not enough to recreate the exact 45T run and private task synthesis at identical scale.

The open artifact and the omitted infrastructure should both be acknowledged. Neither dismissing the report nor assuming full reproducibility follows from the evidence.

The broader research lesson

The model moves the long-context problem beyond “make attention sparse.” It treats state duplication, precision, lifetime, reconstruction, and retrieval as coordinated design variables.

It also demonstrates how data and environment engineering can dominate the practical work of post-training without introducing a new named algorithm. The resulting system is best assessed across the entire chain from inputs to state to actions to verification.

That is the durable idea a reader can carry to other models, even if a later release changes every reported score.

Sources and next step

Report §6, p. 37, together with the architecture, infrastructure, and evaluation sections discussed throughout the book. Proposed experiments and deployment criteria are interpretation. Chapter 29 provides the glossary and audit trail.

29. Reference, Glossary, and Source Map

This chapter is the book's reference desk. It identifies where claims come from, defines recurring terms, records source disagreements, and answers questions that span several mechanisms.

Primary document

Title: DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression.

Author: DeepSeek-AI. Length: 51 PDF pages. Retrieved: 2026-10-13. The PDF metadata records creation on 2026-09-10; that is file metadata, not independent proof of the first public publication date.

The user-specified Hugging Face file page links to the downloadable report. The saved copy is src/content/docs/papers/dsv41/DeepSeek_V41_Tech_Report.pdf.

Its SHA-256 is:

ba68e2e40408125ae6d2f63a9a241b61c73910691c74ec1a2a7023c851eac08d

This matches the SHA-256 published on the inspected file page. The PDF is the authority for numerical tables in this book unless a discrepancy is explicitly discussed.

Chapter-to-paper map

Page numbers refer to the PDF's printed page numbering, which aligns with its page order. Background equations and invented toy examples are not claimed as additional paper findings.

Release provenance

Repository metadata returned commit:

5254448103b6254fc646dacbca1cbbf20e9216c1

Selected implementation and protocol files were fetched with that exact revision in their URLs. The PDF, top-level config, and model card were initially fetched from main; their hashes and retrieval modes are recorded separately in papers/dsv41/_sources/manifest.json.

Important pinned entry points:

• Reference inference documentation.
• Reference model implementation.
• Engram addressing.
• Image preprocessing.
• Encoding documentation.
• Encoding implementation.
• DeepSWE reproduction guide.

Saved README sources use .txt extensions so the documentation loader does not publish them as book chapters. They retain their downloaded text. The source archive is selective and contains no model-weight shards.

Source disagreements and implementation limits

These observations do not resolve every discrepancy. They make it possible for a later reader to identify exactly which artifact a statement describes.

Notation

Symbols are reused by the paper in different sections. Their meaning should be read from the local equation rather than assumed globally.

Glossary

Questions readers often ask

Is it a 552B model or a 748B model? The report gives 552B backbone and 196B Engram separately. Their rounded sum is about 748B, but it is not a verified full checkpoint count including every auxiliary component. See Chapters 3–4 and 14.

Does it use only 8B weights? The reported 8B figure concerns bulk prefill activation; decode uses 16B. The entire stored model is much larger. See Chapters 4 and 6.

Does Reuse skip attention? No. It skips fresh bank/index production and indexing. It still constructs a main query, updates local KV, and performs attention. See Chapter 8.

Does every token read the whole million-token context? No. Main attention reads a local window and selected global entries. Some indexers scan broader history to choose those entries. See Chapters 7–9.

Is bounded replay exact? No. The truncated local dependency chain produces approximate state, and new suffix states can depend on the cache-hit boundary. See Chapter 11.

Are the 890 bytes all request memory? No. They describe nominal global KV. Local caches, weights, scratch space, metadata, draft state, and allocation overhead are separate. See Chapter 12.

Does FP4 double speed? Not automatically. Main-cache FP4 primarily saves storage and transfer; attention dequantizes it. Indexer FP4 has a different arithmetic role. See Chapter 10.

Is Engram remembering this conversation? Its tables are learned parameters addressed by token patterns. Conversation memory is the KV cache. See Chapter 14.

Does effort 100 guarantee a better answer? It requests a more expensive learned regime. Agent accuracy has nonmonotone variations across scaffolds, and harder cases can still fail. See Chapters 22 and 25.

Can the released sample verify all serving claims? No. It demonstrates selected structure and semantics while omitting or simplifying several production paths. See Chapter 26.

Do the benchmark leads establish statistical superiority? Tiny margins without full uncertainty do not. Metrics, subsets, scaffolds, and budgets must match. See Chapters 23–25.

Do teams beat one agent at equal cost? The preliminary team study primarily matches deadlines, not necessarily total compute. See Chapter 25.

What was and was not verified for this book

The report was downloaded, checked as a PDF, text-extracted, and selected architecture/attention figures were visually inspected. Configuration and reference code were inspected for the details cited in chapters. Tables and arithmetic were checked against the saved source.

The full weights, training system, production serving engine, and benchmarks were not executed. Proposed deployment tests and ablations are explicitly proposals. The source manifest makes the inspected evidence recoverable without confusing it with a full reproduction.