AI SEARCH · 2026-10-25
AI Search — Paper Daily, 2026-10-25
Today's AI search theme is a fresh look at evaluation. Adobe shows that in agent assistants the model reads retrieval results first, so click-based attribution breaks; AutoSim tests whether simulated users can replace real ones in evaluation; and Cohere's RCP replaces sparse human labels with calibrated LLM judgments. In addition, LMVU picks evidence by which viewpoints are still missing, and ULME has one model do both retrieval and generation well.
4.60
LEAD STORY ·
AdobeAdobe In Adobe Express's AI assistant, retrieval metrics that assume the user saw a ranked list and acted on it stop working. The chat entry point has a reformulation rate of 44.4%, higher than the template gallery search served by the same retrieval system, at 33.1%. The reason is that reformulation means different things on the two sides. In the gallery, a user reformulates because the results were irrelevant and they try other words. In chat, most reformulations come after the user sees a generated result and wants to tweak it. More surprising still, the assistant actually called retrieval on only 0.31% of prompts. The paper's evaluation framework measures in three layers: it scores the returned results against relevance labels when retrieval fires, then checks whether the returned assets were used, and finally whether the user exported. Four log fields record the retrieval ID, the full returned list, which asset each action used, and which retrieval that asset came from, linking every use and export back to that retrieval.
4.35
Ask a strong LLM to play a search user, and a discriminator that never saw the training data recognizes its utterances with AUC 0.88. The conversational search user simulator AutoSim is in exactly this state: the order in which it ranks retrieval systems by retrieval quality matches the official NIST evaluation, yet the way it talks is far from human. The authors have the simulator improve itself toward real users. For prompts, another model writes edits aimed at the measured gaps to human behavior, and an edit is kept only if its score on a calibration set does not drop. The scores come from evaluators that each check one gap type, such as utterances that are too long and polite or that fail to lean on earlier turns. For weights, the model is first fine-tuned on real user utterances, then repeatedly fine-tuned on the generated utterances the discriminator finds most human. Changing only the prompt cannot push discriminator AUC below 0.85; changing the model weights brings AUC down to 0.45. Sounding human changes evaluation conclusions: where real users retrieve poorly, a prompt-driven simulator overestimates nDCG by 0.136.
4.35
Ask a question about election participation, and a complete answer should cover three kinds of people: those who voted, those who did not, and those who were not eligible. If the context holds only voters' answers, it represents only one viewpoint, however accurate and relevant each one is. LMVU is a linear ranker with only 18 coefficients. At each step it computes novelty and similar features from how a candidate relates to the evidence already chosen, to estimate how many not-yet-covered viewpoints it adds, with coefficients learned on a viewpoint-annotated development set. The authors evaluate with the fraction of annotated viewpoints the context covers. On four viewpoint-annotated benchmarks, LMVU averages a coverage of 0.9588, beating all 16 comparison methods, including two neural rerankers that cost orders of magnitude more to train.
4.35
In RAG, in-context example selection and agent memory, the same query usually goes through two models: an embedding model first encodes it into a vector for retrieval, and then a generation model reads it again to answer. The two models are trained separately, and their understanding of the same query never connects. To write a coherent answer, a generation model's hidden states must already contain an answer-relevant understanding of the query, which is exactly what a good embedding should have. ULME inserts a chain of learnable slots after the query in a causal decoder and adds a loss that aligns the last slot's hidden state with the mean of the answer's hidden states from the same forward pass. Gradients flow back only to the slot side, so information from the answer flows one way into the embedding. A single ULME checkpoint beats the strongest open-source baseline on the same backbone by 3.3–4.1 points on MTEB, while matching or beating generation-only fine-tuned models on benchmarks such as MMLU.
4.35
CCohere
Across the rerankers the authors evaluated, nDCG cannot tell them apart on 45.6% of NanoBEIR queries. The root cause is that human relevance labels are sparse, noisy and limited to a few discrete grades. RCP first has an LLM score 10 documents at a time and puts each document into several batches. The more one document outscores another in a batch, the likelier it is more relevant, and a Bradley-Terry model uses this to build one score per document. It then has the LLM answer five yes-or-no questions of rising difficulty about each document, from whether it is on topic to whether it answers the query in depth. Item response theory shifts and scales the document scores into ability values; each question has its own difficulty, and the further ability exceeds it, the higher the predicted pass rate. Question difficulties are shared by all queries; only each query's shift and scale are fitted to match the actual answers, putting all queries on one ruler. Checked against blind ratings from external annotators, RCP ranks a useful document above a useless one with probability 0.910, against 0.651 for the original human labels.
4.20
Search agents often start answering before they have gathered all the evidence, because they don't know what they are still missing. During training, the authors have the model look back over its own search, point out which requirements already have evidence and which are still missing, and then keep searching. It is rewarded if this check turns a wrong answer into a right one, and penalized more the more checks it makes. At test time the check step is not needed: the model gathers the missing information on its own. In a control where the check is replaced with random feedback, average accuracy is 41.2; training with the real check gives 55.5.
4.00
In graph retrieval, extracting only entity-relation-entity triples throws away context that reasoning needs. The authors instead split the source text into passages and link passages that are adjacent or semantically similar. Starting from the passages most relevant to the question, they spread scores along the edges with personalized PageRank, prune low-scoring passages, and string the rest together in the text's own order as a chain of evidence. On HotpotQA, with everything else unchanged, simply swapping the triple graph for this text-preserving graph raises the share of fully correct answers from 0.288 to 0.414.
4.00
CLIP's text encoder reads only short text and cuts off the rest of a long caption; training on just twenty thousand image-text pairs for one epoch clearly improves its long-caption retrieval, with no change to the model at inference. During training, a long caption is split into sentences and each is encoded separately. They are merged by averaging or by a small weighting network into one caption vector for contrastive learning, with an extra loss keeping the image and short-caption vectors close to those from frozen original CLIP, so that zero-shot classification is not lost. At inference there is no sentence splitting: the model is plain CLIP and long captions are still cut at 77 tokens. The full captions are used only to tune the encoders during training. Long-caption retrieval R@1 rises by 6.5 to 11.2 points. Long-text retrieval does not always need a new model; feeding long captions in training already helps.
3.90
Visual document retrieval does not need the largest multi-vector representation for every query; picking a suitable smaller one per query saves about a fifth of the compute with almost no loss in quality. One retrieval model is trained to yield a family of smaller representations that skip some encoder layers, shrink the vector dimension or use fewer vectors. The query is first encoded up to the shallowest layer that every candidate representation passes through. The system averages the token hidden states at that layer and gives them to a small predictor, which estimates the nDCG ranking quality each representation would reach for this query. A tunable compute preference then trades score against cost, one representation is chosen, and encoding stops at its layer. On the industrial subset of ViDoRe V3, compute fell 20.4% while nDCG@5 dropped only 0.86 points. One retrieval model can adjust its compute as needed without swapping models.
3.90
Document-image retrieval has to store many patch embeddings for each page, and the biggest risk when compressing them is that a few representative vectors end up absorbing the whole page. The authors compress each page into a small number of representative vectors: they repeatedly assign similar patches to the representatives and then update each representative from its assigned patches. Each patch is weighted by the total attention it receives in the vision-language model's self-attention, and the patch weights each representative receives must add up to roughly the same amount. When each page's vectors are compressed 50 times, nDCG@5 is 19.4 points higher than Light-ColQwen, which compresses by clustering.
EMNLP 2026
ECCV 2026