Weekly Top 10 · W46

2026-11-09 ~ 11-15 · Infra

Generative · Personalization · AI Search · Infra · Embodied AI

  1. 1. RecoGEM: A Serving System for Production-Scale Generative Recommendation · 📖 Deep dive

    Generative recommendation has the model generate an item's SID token by token like an LLM, so LLM serving stacks look reusable, yet they run poorly because they assume every step has the same shape and attention cost…

    Appeared in: 11-11

  2. 2. Firepool: Brokering Warm GPU Capacity for Agentic and RL Workloads at LinkedIn · 📖 Deep dive

    Agents and RL training issue thousands of GPU jobs that run for seconds, yet starting a Pod takes twenty to thirty seconds even with the image cached, so waiting outweighs the work…

    Appeared in: 11-11

  3. 3. Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving · 📖 Deep dive

    Serving agents depends on the prefix cache: on a hit each turn only computes the newly added content, and on a miss the whole context of tens of thousands of tokens is recomputed…

    Appeared in: 11-12

  4. 4. ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding · 📖 Deep dive

    A parallel draft model writes a whole block of draft tokens in one forward pass. Current systems write the full block first, then pick the most promising few for the target model to verify…

    Appeared in: 11-12

  5. 5. One-Sided Expert-Parallel All-to-All for Mixture-of-Experts Inference over NVLink · 📖 Deep dive

    In MoE inference every layer runs two all-to-all exchanges: first it sends tokens to the GPUs that hold their experts, then it collects the experts' outputs back. During inference each GPU handles only a few tokens at…

    Appeared in: 11-12

  6. 6. Inside the Trap: A Renewal Law for Prefix-Cache Collapse in LLM Agent Serving · 📖 Deep dive

    Multi-turn LLM agents rely on prefix caching to skip repeated work, but this paper finds a trap. After a sudden burst of sessions, an engine can fall into a state where every turn misses the cache and the backlog never…

    Appeared in: 11-10

  7. 7. Turbocharging PiPNN for Proximity and k-NN Graph Building

    Graph indexes for vector search, such as HNSW and Vamana, answer queries fast but take hours or even days to build on large datasets. PiPNN speeds this up by splitting the vectors into small blocks…

    Appeared in: 11-09

  8. 8. Dynamic Graph Indexing for Label-Filtered Approximate Nearest Neighbor Search · 📖 Deep dive

    Vector retrieval often comes with a filter, such as finding nearest neighbors only within a few tags or categories. The hard part is that once a graph index like HNSW is filtered by label, many edges get cut…

    Appeared in: 11-13

  9. 9. e-means: outperforming k-means (in one-tenth the epochs)

    Large-scale embedding clustering still often uses Lloyd's k-means, which takes many epochs and is awkward to tune. This paper turns the updates into mini-batches with an exponential moving average…

    Appeared in: 11-10

  10. 10. How We Built TLX: A Hardware-Native GPU Compiler for Production ML at Scale

    The performance of recent GPUs depends more and more on how groups of threads divide the work, for example one group moving data while another multiplies matrices. CUDA can express this division but needs large amounts…

    Appeared in: 11-11

‹ Previous week · Next week ›