- 1. ATTUNE: Agentic Throughput Tuning for LLM Training · 📖 Deep dive
Move a large model to a different GPU or framework version, and the training recipe that used to be fastest often no longer is. The recipe settles how the model is split across GPUs, which low-level routines to use…
Appeared in: 10-25
- 2. RemoteANN: A Native Vector Search Engine for Cloud Object Storage
RemoteANN co-designs vector indexing and query execution for cloud object storage. Quantized inverted lists, reordered data, merged range reads and local SSD caching reduce remote I/O…
Appeared in: 10-21
- 3. uGDS: A User-Space Development Kit for High-Performance GPU Direct Storage · 📖 Deep dive
When NVIDIA's GDS reads data from SSD straight into GPU memory, the authors measured that 99.6% of the time for asynchronous reads was stuck at the step of submitting requests…
Appeared in: 10-25
- 4. TokenGate: Adapting LLMs for Fast and Memory-Efficient Prefill with Learned Token Pruning
TokenGate trains an existing model to skip less useful tokens in later layers. Token-gated attention supplies a learning signal for token selection. Across the tested tasks…
Appeared in: 10-21
- 5. E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks
Grouped weight quantization avoids an explicit codebook search by projecting onto a structured lattice. At each format’s largest feasible batch, decoding reaches 1.97 times BF16 throughput…
Appeared in: 10-22
- 6. Large Flow Language Models
Lafa adapts pretrained autoregressive models into blockwise continuous-flow language models. A single model proposes flow-based drafts and verifies them autoregressively, reaching 2.54 times autoregressive throughput…
Appeared in: 10-22
- 7. Is 2:4 Sparsity Still Worth It? Semi-Structured Sparsity versus Quantization in LLM Inference · 📖 Deep dive
An independent researcher compares "add 2:4 sparsity" versus "drop one precision tier" on Llama-3.1-8B-Instruct and Qwen3-8B under matched storage bits (including sparse indices) and matched nominal compute tiers…
Appeared in: 10-23
- 8. Cache Engineering for Agents · 📖 Deep dive
When an agent goes back and forth with tools such as search, web pages and code execution for hundreds of turns, it pays a recurring cost. On every turn the model re-reads the KV cache of everything that came before…
Appeared in: 10-25
- 9. RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies · 📖 Deep dive
The KV cache a large model keeps in GPU memory grows with the text, and one way to compress it is to make the vector stored for each word shorter, but most existing methods only save storage…
Appeared in: 10-25
- 10. Stratified Sparse Attention: Reducing the Sparse-to-Dense Gap at Extreme Sparsity · 📖 Deep dive
In long-context decoding, every generated token reads the whole KV cache from HBM, so the longer the context, the slower attention gets. Sparse attention reads only a small subset of tokens…
Appeared in: 10-25