GENERATIVE · 2026-10-23
Generative — Paper Daily, 2026-10-23
Community search over short user-generated posts, questions, answers and comments. Key metrics are search relevance (nDCG) and CTR. Focus: semantic matching, query understanding, short-text retrieval, ranking models, and LLM applications in search.
4.50
LEAD STORY ·
XXJTLU /
University of Liverpool /
PolyUXXJTLUUniversity of LiverpoolPolyU Generative recommenders emit item-ID tokens left to right, so temporary diversity at inference can only rewrite one position mid-decode and often produces IDs that do not exist in the catalog. ReGen uses a bidirectional Transformer to predict all 4 ID tokens at once, subtracts a score only on the coarsest positions for token values common in the user history, remasks low-confidence positions and repredicts, and uses a prefix tree of all legal IDs to drop token values that cannot complete a real item—without changing model parameters. On Amazon Instruments, top-10 category coverage rises from 0.41 to 0.59 while Recall@10 falls only from 0.0427 to 0.0394 (TIGER-MMR falls from 0.0433 to 0.0319 in the same range); without the prefix tree only 27.63% of IDs map to real items.
4.30
SpotifySpotify wants the home feed to surface podcasts the user has not yet formed a habit around (under 10 minutes on a show in the past 28 days). GLIDE, based on Llama 3.2 1B, first learns to translate between episode semantic IDs and text, then reads recent episode semantic IDs, region and other text fields, a soft-prompt token projected from a collaborative-filtering user vector, and an instruction to push familiar versus unseen content, and directly generates 30 episode candidates. Offline Recall@30 is 29.9% above a semantic-ID-only variant; online A/B tests show up to +5.4% non-habitual listening and +14.3% new-show discovery while meeting latency SLOs.
3.90
When asked what to study, which career to pursue, or which product to build, large models often recommend AI-related options. On 20 open-ended advice questions (10 phrasings each), the share of answers with AI/ML terms is already 27% for base models and rises to 59% for the released instruction-tuned pairs. The authors' own post-training on 11 base models finds that generic instruction tuning does not change this share; dropping the 7% of AI-related instruction samples cuts it by 14 points; preference data rewarding "helpfulness" raises it by 9 points, while ranking by diverse simulated user preferences lowers it by 9. Phrasing matters too: asking for "the best" adds 8 points, and "what should I do now" adds 18.
3.50
For RAG over financial transaction data, should you use a specialized transaction foundation model or a general text embedder? On IBM synthetic AML transaction data the authors template 1,026 QA pairs with gold documents and compare NVIDIA's 29M-parameter TFM with five general and finance text embedders at matched scale. TFM Recall@5 is 0.81 versus 0.25–0.39 for same-scale text embedders, and end-to-end accuracy 66.7% versus 27.5%. But manually summarizing transactions into text and encoding with a general sentence embedder matches or beats TFM overall, and is clearly better on low-cardinality attributes such as bank, currency, and payment method.
3.40
Diffusion language models denoise a fully masked span rather than generating left to right. On ToTTo table-to-text, the authors compare MDLM and LLaDA with GPT-2 and GPT-Neo under the same retrieval few-shot prompts, plus a QLoRA-tuned LLaDA, scoring reference similarity (BLEU) and NLI entailment of the source table. The two metrics move in opposite directions: as LLaDA examples increase, BLEU rises from 11.3 to 22.0 while NLI entailment falls from 0.78 to 0.52; the QLoRA variant reaches the paper's highest BLEU at 24.15 without better faithfulness. Human labels show autoregressive models often invent off-table content, while LLaDA more often swaps entities inside the topic.
3.40
Long-context continued pretraining (LCCP) is usually a short stage after short-context pretraining. Tsinghua and OpenBMB fit an additive power law over 5 model sizes and 100+ short/long token budgets, writing long-context performance as a function of model size and the two token budgets, and extrapolating to sizes left out of the fit. To track one training trajectory they train with a constant learning rate and evaluate each point as the average of the latest 4 consecutive checkpoints (Constant-Last4); that procedure accurately predicts a 1.22B model's trajectory at about 100B LCCP tokens. Downstream evals such as RULER do not saturate in the measured range, and gains remain after extending to 128K and after SFT.
3.40
Knowledge distillation asks the student to match the teacher's next-token distribution, but different tokenizers misalign vocabularies and split points. CSD first aligns both sides on the same text spans; when tokens are one-to-one it compares only shared tokens, otherwise it uses the chain rule for multi-token continuations, adds an "other" mass so both distributions stay normalized, then matches with one objective. Across four setups with Llama-3.2-1B or Gemma3-1B-PT as students and Qwen3-4B or Phi-4-mini as teachers, CSD leads all cross-tokenizer methods by up to 1.8 points; with a Llama-3.1-8B student it leads the strongest baseline by up to 4.2 points.
3.30
Deep-research agents must plan, retrieve, and write long reports; autoregressive models emit token by token, so users wait a long time. DiffResearch freezes the pipeline, retrieval code, prompts, and evals, and only swaps the backbone between the diffusion model diffusiongemma-26B-A4B and the same-size autoregressive gemma-4-26B-A4B. On 100 Deep Research Bench tasks, end-to-end mean latency falls from 71.45 s to 11.89 s, about 6×, mostly in planning and writing. Letting the sub-query step read intermediate denoising states without waiting for generation to finish alone speeds the median by 1.99× with no drop in report quality.
3.30
Some closed models return only an answer or a short summary, not a full reasoning trace, so they cannot be used directly to distill chains of thought. TraceForge trains a generator that, given a problem and answer, fills in a reasoning trace in the style of the student family (currently Qwen), first with supervised training and then with adversarial training against a judge of whether a chain is genuine. Fine-tuning Qwen3.5-9B-Base on the same 3,500 problems and teacher answers, the filled-in chains beat training on Fable 5's own thinking content by 3.7 on HumanEval+ and 7.1 on MBPP+, and lead on MBPP+ at every checkpoint; on general tasks the data sources differ little.
3.20
OpenAIOpenAI's Decisions API is generally available for the many small, finite-answer judgments inside apps and agents—classifying an input, routing a request, choosing the next action—backed by GPT-6 Luna. Developers write questions and allowed answers, pass text, images, or JSON, and get one of three results: a predicate probability, a choice among options, or a scored tier. One request can ask up to 400 questions and is up to 10× faster than the same model via the Responses API; billing is input-only at $0.10 per million tokens ($0.01 on cache hits). Sierra uses it to pick the next customer-service action with p95 under 280 ms.
EMNLP 2026