Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking

Chunking schemes for RAG are almost always compared on one static corpus, yet the documents in a knowledge base keep changing. Each chunk is embedded on its own, so if an edit moves a chunk boundary, the chunks on either side become new embedding inputs even when not a single word in them changed. This 6-page short paper measures that update cost directly: the authors take 120 pairs of adjacent revisions of Chinese and English Wikipedia articles, allow an old vector to be reused only when a chunk's complete embedding input is byte-identical to one in the old version, and compare three chunking methods with a frozen BGE-M3 encoder. On the 116 articles where all three methods succeed, non-overlapping fixed-size chunking recomputes 57–68% of the new version's embedding input, paragraph-first recursive chunking 38–56%, and content-defined chunking (CDC) 43–54%; under a 2048-token retrieval budget, the three differ in evidence recall by less than 2 percentage points within each language. Fixed-size chunking shifts everything after the insertion point, so none of the downstream windows can match the old version, which is why it recomputes the most. Which of recursive chunking and CDC is cheaper flips as the chunk-length parameters change.

1. Why measure update cost

An English article had only 0.13% of its text edited, yet fixed-size chunking recomputed 49% of its embedding input while paragraph-first chunking recomputed only 8%. This example from the paper's dataset shows that the amount edited and the amount recomputed can be far apart, and the gap is determined by the chunking rule.

For a knowledge base that is continually updated, the amount recomputed maps directly to embedding compute and to writes into the vector index. Earlier chunking evaluations only compared retrieval quality, but two schemes with similar retrieval quality can differ a lot in update cost: on the English articles in this paper, fixed-size chunking's mean recompute fraction is 1.5 times that of paragraph-first chunking (57% vs 38%). The authors want to answer three questions: how much each of the three chunking methods must recompute; whether that amount is driven by edit size, edit position or cut-point rules; and whether the scheme that recomputes less also retrieves worse.

2. Measurement

The reuse rule is strict: a chunk's complete embedding input (its text plus the encoder's special tokens) must be byte-identical to the input of some chunk in the old version for the old vector to be reused; a single differing character makes it a new input. The cache holds only the previous version of the same document and does not match across documents. The recompute fraction is defined as:
$$R_c=\frac{{\sum }_{x\in M_c}T(x)}{{\sum }_{x\in U_c(D_{\mathrm{new}})}T(x)}$$
Here $U_c(D_{\mathrm{new}})$ is the set of distinct embedding inputs that chunking method $c$ produces on the new version, $M_c$ is the part of it not found in the old version's cache, and $T(x)$ is the number of input tokens of chunk $x$. $R_c=0$ means everything is reused and $R_c=1$ means everything is recomputed. The authors also run a no-edit control: chunking the same version twice must give $R_c=0$, otherwise it counts as a pipeline failure.

All three chunking methods see only the current version and fixed parameters, never how the old version was chunked:

• Fixed-size chunking: greedily cuts non-overlapping windows of at most 304 tokens.
• Recursive chunking: splits any span longer than the 400-token target, preferring blank lines, then line breaks, then sentence-ending punctuation, and falls back to fixed windows only when none works; it then greedily packs the small units up to the target length.
• CDC: uses the Steadlith implementation, which computes Rabin rolling fingerprints over the word sequence and places a cut wherever the fingerprint meets a condition, with chunk lengths limited to 176–352 tokens. The method was originally built for file-system deduplication; cut points depend only on nearby content, so inserting bytes near the front of a file does not move the cut points after it.

The chunk-length parameters of the three methods were tuned in advance so that their median chunk lengths are close (the actual maximum ratio is 1.17).

Retrieval quality is measured as follows: each article has two questions, 240 in total, and annotators mark on each version the character spans that answer the question. At retrieval time a batch of articles is put into one candidate pool, chunks are ranked by dot product, and whole chunks are packed into a 2048-token budget in rank order; recall is 1 if the packed chunks cover one complete evidence set.

3. Experiments

3.1 Main result: fixed-size chunking recomputes the most, recall differs little

This experiment answers how much each method must recompute and whether retrieval quality differs. The authors average the recompute fraction and evidence recall with fixed parameters over the 116 articles where all three methods succeed (paper Table I):

Paired by article, recursive chunking recomputes 12.7 percentage points less than fixed-size chunking (Chinese, 95% interval −19.7 to −6.2) and 19.6 points less (English); CDC recomputes 14.4 and 14.7 points less than fixed-size chunking. Paired differences in recall range from −1.9 to +0.9 points, with all intervals including or touching 0. The authors state explicitly that these intervals cannot rule out a drop in recall and do not prove equivalence, so they do not rank the methods by recall.

Recursive chunking's advantage is smaller in Chinese than in English because of its cut-point rules: Chinese sentence-ending punctuation is usually not followed by a space, so overly long Chinese paragraphs cannot be cut at sentence boundaries and fall back to fixed windows.

The smaller the edit, the larger the gap. On the 28 English articles with small edits (at most 5% edited), fixed-size chunking recomputes 46% on average, recursive chunking 25% and CDC 27%; on the 23 Chinese small-edit articles the figures are 62%, 45% and 36%. When more than 50% is edited, all three must recompute 60–90%.

3.2 Edit position: why fixed-size chunking recomputes so much

This experiment tests whether "boundary shift" is the main source of recomputation. It changes only one variable: the same 1% insertion is placed at the start, middle or end of the article, with everything else unchanged. Result: fixed-size chunking falls from 100% for an insertion at the start to about 12% for one at the end, because every window after the insertion point moves back by the length of the insertion and no downstream chunk can be reused. Recursive chunking and CDC stay below 30% at all three positions, as paragraph breaks and content-fingerprint cut points realign quickly after the edit.

The authors also run a verbatim check: how much of the recomputed input is text that already exists unchanged in the old version. On English articles, 50% of the tokens fixed-size chunking recomputes are text present verbatim in the old version, 33% for CDC and 12% for recursive chunking (47%, 45% and 37% in Chinese). This text was not changed at all; it was only cut into different chunks, so recomputing it is caused entirely by the cut-point rule.

3.3 Whether recursive chunking or CDC is cheaper depends on parameters

This experiment checks whether the ranking of the two content-aware methods is stable. Without retuning, the authors try 6 target chunk lengths between 256 and 496 tokens for each method and compare paired settings whose median-length ratio is at most 1.15. Result: the CDC-minus-recursive recompute difference ranges from −2.4 to +6.2 points in English and from −0.7 to +5.5 in Chinese, flipping sign in both languages. For example, in English raising recursive chunking's target length from 400 to 448 raises its recompute fraction from 37.9% to 46.2%, while CDC at the same 448 is 43.8%. The reason is that recursive chunking packs several small paragraphs into one chunk; once an edit changes one paragraph's length, the packing of several following paragraphs can change too, and the larger the target length, the farther this chain reaction propagates. Fixed-size chunking recomputes the most in every setting.

The authors checked the cost once more on the arXivEdits paper-revision data: the mean recompute fractions of fixed-size, recursive and CDC chunking are 94.5%, 81.4% and 80.2%, with fixed-size still highest. The text-extraction noise of this dataset is unquantified, so it serves only as side evidence on cost.

3.4 Overlapping windows are not the fix

A common remedy is to let adjacent fixed windows overlap by 32 or 64 tokens. The result is a recompute fraction within 1.3 points of the non-overlapping version, 10–26% more total embedding input, and English recall down by 3.0 and 3.9 points, because the duplicated text uses up the 2048-token retrieval budget.

4. Limitations

The authors list these limitations: Wikipedia is the only source; the annotations of the 240 questions were not independently adjudicated by multiple people, and when the authors spot-checked 60 labels themselves, 57 agreed; the median chunk-length ratio of the three methods is 1.17, above the preset 1.15; only one encoder is used; and the related method KARA was not run. CDC failed on 4 Chinese articles because CDC splits words on spaces, so a long run of Chinese text without spaces is treated as a single unit of 403–550 tokens, exceeding the 352 maximum.

Two further points deserve attention. First, all three methods recovered all the evidence in 105 of the 116 articles, and recall is generally above 95%, close to the ceiling, so this retrieval setup cannot distinguish small quality differences between methods. Second, the recompute fraction measures embedding input and is not converted into latency or cost; if embedding is a small part of the whole update pipeline, for example when index rebuilding or review costs more, a gap of a few tens of percentage points is diluted in the total cost.

5. Practical implications

First, if a knowledge base is edited often and in small amounts, do not use non-overlapping fixed-size chunking. Switching to paragraph-first recursive chunking takes the mean recompute fraction on English articles from 57% to 38%, and from 46% to 25% on small-edit articles, with retrieval recall essentially unchanged. The precondition is to use the byte-identical input as the cache key, so that vectors are reused whenever the text has not changed.

Second, when evaluating chunking schemes, report the recompute fraction together with retrieval quality, and measure both under your own chunk-length parameters. The ranking of recursive chunking and CDC flips with parameters, so no ranking in the paper can be carried over as is. For Chinese corpora, check whether the cut-point rules rely on spaces; otherwise recursive chunking falls back to fixed windows and CDC fails outright.