INFRA · 2026-10-23
Infra — Paper Daily, 2026-10-23
Community search over short user-generated posts, questions, answers and comments. Key metrics are search relevance (nDCG) and CTR. Focus: semantic matching, query understanding, short-text retrieval, ranking models, and LLM applications in search.
4.40
An independent researcher compares "add 2:4 sparsity" versus "drop one precision tier" on Llama-3.1-8B-Instruct and Qwen3-8B under matched storage bits (including sparse indices) and matched nominal compute tiers, with measurements on L40S, H100, and RTX PRO 6000. As long as a lower precision is still available, adding 2:4 is both less accurate and slower: 2:4 + INT8 costs the same storage as INT5. At the FP8 floor, a retrained 2:4 model served by vLLM on H100 raises prefill throughput 27% over dense FP8 with matched GSM8K; but during retraining perplexity recovers first while reasoning accuracy recovers much more slowly.
4.30
SSGLang
AMD
NVIDIA SGLang, NVIDIA, and AMD use a plugin framework to plug specialized fused kernels into SGLang, cutting inter-kernel waits and cross-GPU communication at low concurrency. On AgentX replays of DeepSeek-V4.1-Flash + DSpark, per-user P90 generation rises from 671.2 to 928.4 token/s on 4×GB300 with 1 session, and from 331.6 to 516.0 token/s on 4×MI355X with 4 sessions. On NVIDIA, median time for one decode layer falls from 96.8 µs to 68.4 µs with almost the same kernel count; the speedup mainly comes from less waiting between kernels and overlapping work.
4.25
Hisense's RCSD continues training a speculative-decoding draft model that was distilled with HASS, using DPO or GRPO: sequences from the target alone are "good," sequences alternating target and draft are "bad," and the GRPO variant scores by acceptance rate. On Qwen3-1.7B and Llama-3.1-8B, acceptance length exceeds HASS on four datasets and draft steps 3–6—for example Qwen3-1.7B on GSM8K at step 6 rises from 2.176 to 3.246, and Llama-3.1-8B ShareGPT throughput from 142.3 to 176.6 token/s. The paper reports no generation-quality evaluation.
4.15
Scitix.AI's UlaCache maps every GPU KV pool and CPU memory into one logical address space so GPU and CPU each store different history, and removes MLA's cross-GPU duplication under TP; the GPU pool is shared across requests with a bucketed parallel LRU for eviction. Under SGLang + GLM-5.3 with 64K–128K inputs, concurrent decodeable requests on 8×B300 TP rise from HiSparse's 37 to 452; with speculative decoding, peak throughput is 1.16–2.51× HiSparse. At low concurrency latency is higher than pure-GPU SGLang.
4.15
Intellifusion's SPARE decides how many MoE experts to prune per layer: on calibration data it computes Top-K gate-weight entropy per token per layer, and layers whose weights are diffuse lose little when one expert is dropped, so they are pruned more. It needs no training and composes with methods such as REAP that choose which experts to drop. Under the same selection method and budget, at 50% expert pruning Qwen3-30B-A3B's eight-task average rises from 57.2% to 63.8%, and DeepSeek-R1's math/code/science average from 80.1 to 82.8; on Mixtral it also beats uniform allocation.
4.15
CASIA finds that on reasoning tasks the weight-update norm of LoRA-style PEFT is far smaller than full fine-tuning, and treats that as the main cause of the gap. NormTune adaptively amplifies under-updated layers by per-layer update norm. Training on S1K-1.1, rank-8 LoRA, Bone, and MoRA raise the average of AIME 2024, MATH 500, GPQA, and AIME 2025 by 24.24%, 11.33%, and 16.21%; at rank-256 all three match or slightly beat S1 full fine-tuning.
3.95
Snowflake specializes a training system for long-horizon software-engineering agent RL on Qwen3.8-Flash-Next, based on prime-rl, generating trajectories with vLLM and training the full 48-layer model in BF16 on one 8×B200 node (freezing the first two layers and other parts). After host offload, gradient accumulation, and MoE/sparse-attention kernel work, per-GPU training throughput rises from about 395 to 4,438 token/s and context from 32K to 131K. With three inference nodes, a 1,024-trajectory batch averages 22.4 minutes of real training per step.
3.85
CUHK-Shenzhen and collaborators propose TrainBridge to optimize training programs on Ascend NPUs: it first extracts optimization intent from execution produced by a GPU compiler stack (operator fusion, etc.), lets an agent rewrite the NPU program, then iterates from NPU profiles. Across four workloads covering LLM pretraining, image classification, and vision–language training, mean speedup is 1.7×; on 27 Modded-NanoGPT speedrun records mean speedup is nearly 2.0×.
3.80
MBZUAI and Sony study "looping" in uniform diffusion LMs (any position may be replaced by any token and can be revised during generation): reuse the first few Transformer layers inside each denoising step with the training objective and sampler unchanged. On OpenWebText and FineWeb-Edu, matching the validation NLL of a non-looped model trained on ~0.5T tokens takes up to 2.8× and 3.3× less training compute; at equal function-call counts generation perplexity is up to 9.8% lower. Sampling smaller loop counts more often during training lets downstream tasks use 12% fewer loops without accuracy loss.
3.55
CASIA and Keling propose ALLOT to decide how many blocks a video diffusion Transformer's block-sparse attention keeps for each layer × denoising step: measure each cell's effect on final video error, then jointly solve densities under a measured latency budget without changing the existing block-selection policy or kernels. The authors observe that the most sensitive layer–step pairs are nearly unchanged across inputs. At equal or lower latency with XAttention it beats every compared allocation; on HunyuanVideo PSNR rises 11.8 dB; on four published Wan2.1, Wan2.2, and HunyuanVideo pipelines, all nine settings meet or beat the default allocation.