Learning Marginal Viewpoint Utility for Budgeted Evidence Selection in RAG

Ask a question about election participation, and a complete answer should cover three kinds of people: those who voted, those who did not, and those who were not eligible; if the context holds only voters' answers, it represents only one viewpoint, however accurate and relevant each one is. Under a 1024-token context budget, classic MMR repeatedly selects evidence supporting the same three viewpoints on a corruption-perception query from the World Values Survey (WVS), and misses the viewpoint "there is no corruption". This paper from Nanyang Technological University and the University of Toronto proposes LMVU (learned marginal viewpoint utility), a linear ranker with only 18 coefficients that picks evidence by "how many not-yet-covered viewpoints this piece adds to the current context". The authors evaluate with recall viewpoint coverage (RVC, the fraction of annotated viewpoints the context covers). On four viewpoint-annotated benchmarks, LMVU reaches the same full coverage as the exact optimum on the protected WVS test fold, and averages RVC 0.9588 across the four benchmarks, beating all 16 non-oracle methods, including two neural rerankers that cost orders of magnitude more to train.

1. The problem: relevant is not the same as covering

RAG usually ranks passages by relevance to the query and puts the relevant ones into the context first. But some queries need coverage of different answers or stances. The authors call "an answer category or stance supported by a set of evidence" a viewpoint. Highly relevant evidence may all support the same viewpoint, leaving other viewpoints unrepresented. Methods like MMR trade off relevance against "how similar this is to already selected evidence", and recent diversity-seeking RAG selection methods follow the same idea. But semantic diversity is not viewpoint coverage: evidence worded differently can support the same viewpoint, and evidence worded similarly can take opposite stances.

The authors frame the task as a coverage problem under a budget. Each query has a full candidate pool and an annotated list of viewpoints; each piece of evidence has its own token cost and supports some of those viewpoints. The goal is to have the selected evidence cover as many viewpoints as possible while total cost stays within budget, counting each viewpoint once, and this fraction is RVC. The key constraint is that annotations are invisible at inference: viewpoint labels are used only for training and evaluation, and the selector can only estimate how many new viewpoints the next piece of evidence will bring from visible evidence features and what has already been selected. Because labels are available at evaluation, the authors can compute the exact full-pool optimum with dynamic programming, and so measure how far any selection method is from the optimum, which they call headroom.

2. Method: learning each piece of evidence's marginal viewpoint utility

What the selector really needs at each step is a candidate's marginal gain given the current selection, that is, how many not-yet-covered viewpoints it fills in. The same candidate's gain changes as the selection changes, and once all the viewpoints it supports are covered, its gain drops to zero. This gain requires labels to compute, so it cannot be used in deployment, but it can be computed on the development set, so the authors use it to train a surrogate scorer that imitates it using only features visible at inference.

The training data comes from the following procedure. On a fixed development candidate pool, the authors make selections with five strategies: greedy by dense relevance, live MMR, two fixed-seed random orders, and greedy by true gain. Each selection run is cut into at most its first 9 steps, giving many "half-selected" states; after removing duplicates, they take up to 20 candidates per state and compute each one's true gain from the labels.

The scorer itself is a linear combination of 18 features. The features fall into four groups. The first describes the candidate (4): retriever score, cosine to the query, length, and number of atomic pieces; atomic pieces are smaller units the dataset already annotates, such as single claims split out of a piece of evidence. The second compares the candidate with the selected evidence (5): max, mean and min cosine to the selected items, novelty as 1 minus the max cosine, and the dot product with the mean selected vector. The third does the same at the atomic level (5). First compute $m_a$ for each of the candidate's pieces: its max cosine to any selected piece. The five features are the mean of $m_a$, the max of $m_a$, 1 minus the max $m_a$, the mean of $(1-m_a)$ and the max of $(1-m_a)$. The last three measure how many claims the candidate adds that the selection lacks. The fourth describes the selection (4): how many items are selected, tokens used and remaining, and the mean retriever score of the selection. All similarities reuse the frozen embeddings already computed for dense retrieval, so no new model is needed. The last group of features is identical for all candidates in the same state, so it cancels out in comparisons and its fitted coefficients come out exactly zero; the budget therefore acts only through "does it fit", not through the score.

The authors do not regress the gain directly, because the range of gains differs across states, anywhere from zero to the number of viewpoints still uncovered. They compare only in pairs within the same state: if candidate A's true gain is larger than B's, the scorer must give A a higher score; tied pairs are dropped, and the model is fit with an L2-regularized pairwise logistic loss. Comparing this way cancels each state's own overall offset, and what gets optimized is exactly the decision the selector faces at each step. This differs from MMR in that MMR uses a fixed λ to trade off relevance and novelty; LMVU's weights are also fixed at inference, but they act on geometric features that update with the selection, so the novelty penalty changes with what has already been covered. The same candidate may rank high early on, and once its viewpoints are covered by other evidence, its rank drops.

Inference starts from an empty context. At each step it picks the highest-scoring candidate among those that fit entirely in the remaining budget, adds it and updates the state; when no candidate fits, it stops, and it never truncates evidence. Each step needs only one 18-dimensional dot product per candidate, maintained with incremental statistics, without building a similarity matrix over the whole pool. The authors also state plainly that LMVU only guarantees the selected set stays within budget, not how close its coverage is to the optimum.

3. Setup

The authors use four benchmarks. Two come from social surveys: the World Values Survey (WVS) and the European Social Survey Round 10 (ESS10). Perspectrum has a "claim plus different perspectives" structure, and ArgKP has an "argument plus key points" structure; the authors use it as a control where coverage is already saturated. These benchmarks vary widely in size, with average candidate pools ranging from 9 to 507 items, and evaluation queries are few: WVS and ArgKP have only 6 each. Budgets are 512, 1024 and 2048 tokens, with 1024 for the main results. WVS evaluation uses a protected respondent fold that is used only for pre-declared evaluation and read-only audits, never for fitting. The encoder is a frozen local all-MiniLM-L6-v2.

Besides breadth RVC, the authors report a "repeated support" version, RVC₂: a viewpoint supported by only one source scores half, and only two or more distinct sources score full, to measure depth. There are more than a dozen baselines, including BM25, dense retrieval, cross-encoder reranking, MMR, several clustering and submodular diversity methods, the diversity selection methods MMKP and AdaGReS reproduced under the same protocol, and the neural reranking models JPR and RichRAG. The authors audited that no label field was read at run time. They also state that because they had seen the held-out set before, this full-pool expansion is exploratory analysis, not a new confirmatory test.

4. Main results

RVC (breadth) and RVC₂ (repeated support) at a 1024 budget, excerpt:

(M0 is an ablation that uses only three static features: relevance, cosine similarity and length.)

First, how far is MMR from the optimum? On ArgKP the gap is zero, meaning any selection already achieves full coverage; the most headroom is on ESS10, where MMR still falls short by about 0.16. LMVU closes the gap entirely on WVS, recovers about a third of it on ESS10, and matches the ceiling on ArgKP. The table shows three more things. BM25, which relies only on word matching, trails embedding methods on WVS and Perspectrum. Structured diversity heuristics depend heavily on the benchmark: MMKP beats MMR on Perspectrum but collapses to 0.25 on ESS10 because of its per-cluster selection cap. Only the learned, state-dependent utility beats MMR on all three benchmarks that still have room to improve.

Statistical significance needs caution, though. Compared with MMR, LMVU's difference is non-negative on every benchmark, but only the repeated-support gain on Perspectrum is statistically significant; the confidence intervals for the other differences all cross zero, largely because there are too few evaluation queries. Against weaker baselines, significant advantages show up on benchmarks with more queries such as ESS10. There are counterexamples too: on ESS10, RichRAG significantly beats LMVU. Each neural method wins on only one benchmark, JPR on ArgKP and RichRAG on ESS10, but both trail MMR on WVS and Perspectrum.

The cost gap is huge. On one L20 GPU, JPR takes 150 seconds per fit and RichRAG (SFT plus DPO) 963 seconds, and neural inference during evaluation takes about another 11 GPU hours. LMVU refits its 18 coefficients on a CPU in seconds, 1.3 seconds measured on the largest benchmark, and inference is just one dot product per candidate.

5. The role of state information, transfer and limits

LMVU's advantage comes mainly from the state information of "what has already been selected". Compared with M0, which uses only static features, LMVU is 0.20 higher on WVS and clearly better on ArgKP and ESS10 too; Perspectrum is the exception, where M0 is slightly higher. To rule out the explanation "it just has more parameters", the authors run a control of the same capacity: keep all 18 features but randomly shuffle the 14 features tied to the selection among the candidates of the same state, so they no longer match their candidates, leaving the 4 candidate features unchanged. Ranking quality on the development set collapses back to the static level, so the gain really does come from state information. Removing the group of atomic-granularity features also sharply lowers ranking quality on the WVS development set, which shows fine-grained geometry matters.

The patterns the coefficients learn transfer across benchmarks. Training on the other three benchmarks and testing on the fourth, LMVU still beats MMR on WVS, Perspectrum and ESS10, and WVS is still at full coverage.

With repeated-support RVC₂, the ordering of methods stays largely the same; only on WVS does FacLoc, a submodular method that ignores relevance, slightly beat LMVU. There is more room to improve in depth: LMVU's breadth on WVS is already optimal, but its depth still falls short by about 0.10.

The tighter the budget, the larger LMVU's advantage. On WVS, its lead over MMR is largest at 512 tokens and shrinks by about two thirds at 2048 tokens, because with a loose budget MMR can also buy in the missing viewpoints. The authors also find that the realized gain roughly grows with the available headroom and vanishes where the ceiling is already reached, though with only four benchmarks this correlation is descriptive only. The conclusion is also sensitive to the encoder: with bge-base, the gains on Perspectrum and ESS10 are significant; with bge-m3, gains are smaller and not significant, and on WVS LMVU is even slightly below MMR.

6. Limitations and practical value

The authors acknowledge several limitations: results depend on the embedding pipeline and on the viewpoint labels needed for training and evaluation; the full-pool expansion is exploratory; the learned scorer has no approximation guarantee on coverage; and they have not evaluated the effect on downstream generation or studied how to adapt to domains that need a new viewpoint list. Note also that the evaluation is very small: WVS and ArgKP have only 6 queries each, and WVS's full coverage was obtained on those 6 queries; most differences from MMR are not significant; and changing the encoder changes the conclusion.

For engineering teams building RAG, the paper offers a cheap, auditable selection layer: no neural reranker to train, just 18 features from existing embeddings, trained in seconds, with inference that is only a dot product, so it can directly replace MMR between retrieval and context assembly. It suits scenarios like survey analysis, public-opinion monitoring and policy Q&A where "every side's view must appear", provided you have a batch of development data with viewpoint labels. What is more worth borrowing is its evaluation method: first compute how much headroom remains using the exact optimum, confirm whether and by how much coverage can improve, and only then decide whether to switch selectors. In a setting like ArgKP where there is no headroom left, no method will bring any gain.