CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation
Large language models are attractive rerankers, but running an expensive model on every request is costly and often hurts users with short histories.
Looking for a title? Use site search.
Large language models are attractive rerankers, but running an expensive model on every request is costly and often hurts users with short histories.
Robot manipulation does not have to rely on specialist policies trained on large sets of demonstrations.
Cross-modal retrieval, such as searching images with text or text with images, is often trained on data where some samples lack one modality, and the class labels contain errors.
When a large model plans a recommendation workflow, it can be given memory of how a group of users has behaved.
Generative retrieval outputs a Semantic ID (SID), but several products can share one SID, and which of them are finally retrieved directly affects online gains.
Ranking general-purpose robot policies is hard because real-robot experiments are difficult to reproduce: change the site or the object layout and the result changes.
Different retrievers can return passages with different value for answering the same question.
Ranking already scores many candidates that never reach the screen.
Generative retrieval predicts the next item's Semantic ID (SID) one code at a time from user history, but ordinary next-code training checks only the probability assigned to the correct code.
RAGLog reads only the API logs each run of a RAG pipeline leaves behind, with no reference answers and no code changes, and points out which stage of the run went wrong.
The paper builds a multi-hop graph retriever, QueryFlow, that revises the question as it searches.
When labeled data is limited, how good a retriever can get follows a law you can compute ahead of time.
Quietly adding one sentence that mentions a brand to a web article can be enough to get that brand into a generative search engine's answer.
In composed image retrieval, having a multimodal LLM first represent the edited image in a few parts and then gather them into the retrieval vector works better.
Deep Research systems split a question into subquestions, search the web repeatedly and then write an answer, so a small change in wording can push an early slip all the way through to a wrong answer.
Worth reading for anyone who evaluates recommenders, because the problem it uncovers is not in the models but in how the data files map onto each other.
Many teams decide which content categories to promote from the average watch ratio per category in their recommendation logs, but that dashboard is skewed by the serving policy.
When picking recommendation reasons for a user, earlier methods often fail to beat a simple baseline that only counts which statements appear most in the user's history.
The same training job can often run on several accelerator types, and which one is cheapest depends on its throughput on each.
In vector search, switching to another embedding form or quantizer often means switching libraries.
Long-context inference is bound by the KV cache, and squeezing keys to very few bits would save a lot of memory if the error stays small.
When doing speculative decoding on models that mix Gated DeltaNet (GDN, a linear-attention layer with a recurrent state) and MoE layers, how many draft tokens to verify per round cannot be copied from dense-model settings.
Teams putting generative retrieval into production should read this: it shows how to reuse large-model compute per query while leaving personalization to a small online model, and it is backed by an online A/B test.
Engineering teams serving generative recommenders should read this: it shows that most constrained-decoding compute goes to candidates that will be discarded, and gives a filter-first system implementation.
Teams working on reranking and mix-ranking should read this: it turns two stages that have long run on rules into a generation problem that improves with more compute, and the system is fully deployed.
LLM-based recommenders often give the user a written profile, but items get no matching description; the user profile only goes into the reranker prompt, where it competes for length.
The hard part of conversational recommendation is that users rarely state all their requirements up front. τ-Rec-Books keeps each book request fixed and changes only when the user reveals requirements: all at the start, one at the start and the rest only when asked, or some never.
The homepage of a local-services app must generate a few queries out of millions of possibilities, with no two alike, matching where the user is right now.
Robot imitation learning usually starts with teleoperating a robot to collect data; AIRoA-0 collects no robot data at all.
WeSearch at WeChat Search replaces the old set of parallel retrieval paths with one model: it first finds the document clusters relevant to the query, then scores individual documents inside those clusters, separating coarse pruning from fine relevance judgment.