INFRA · 2026-10-22
Infra — Paper Daily, 2026-10-22
Community search over short user-generated posts, questions and comments. Focus: relevance, CTR, semantic matching, query understanding, retrieval and ranking.
4.40
Grouped weight quantization avoids an explicit codebook search by projecting onto a structured lattice. At each format’s largest feasible batch, decoding reaches 1.97 times BF16 throughput. This comparison includes the batch-size benefit of lower memory use.
Key insight: Treat storage format and feasible batch size as coupled decisions. Arithmetic lattice projection reduces codebook search overhead, while memory savings create additional serving capacity.
4.40
Lafa adapts pretrained autoregressive models into blockwise continuous-flow language models. A single model proposes flow-based drafts and verifies them autoregressively, reaching 2.54 times autoregressive throughput at comparable quality.
Key insight: Drafting and verification can share a model. Retaining autoregressive capability creates a verification path for parallel flow proposals without a separately maintained drafter.
4.15
OptEP selects expert subsets by reconstructing each original layer’s output, then lightly adapts the retained weights. On Qwen3-30B-A3B it improves accuracy by up to 15.05 percentage points over the strongest expert-pruning baseline.
Key insight: Optimize the expert subset as a whole. Reconstruction captures complementary contributions that independent popularity rankings can miss, and retained-weight adaptation helps after aggressive pruning.
4.15
The study jointly tunes vector-codebook representation, calibration data and recovery data for very low-bit compression. Codebook-only recovery retains 89–97% of full-precision QA accuracy in settings where scalar quantization falls to chance.
Key insight: Calibration and recovery data interact. At low bit widths, evaluate the pair together and inspect generative tasks separately, because aggregate QA scores hide uneven capability losses.
4.15
RouterWorld evaluates model routing with partial feedback and hardware-grounded latency labels. Across 13 baselines, routers tend to favor accuracy while neglecting latency; the benchmark includes static cold-start logs and a streaming setting.
Key insight: A router changes the data it will observe next. Evaluate that feedback loop instead of assuming every candidate model’s answer is available for training and scoring.
4.05
MLIPS shares a broad candidate pool across sparse-attention layers while preserving each layer’s own final selection. A vLLM implementation reports 1.14–1.40 times speedup on long-context suites; broader sharing incurs task-dependent quality losses.
Key insight: Reuse candidate discovery rather than forcing identical attention selections. Broad pools can cover different layers’ needs even when their final selections overlap little.
3.95
DScout aligns diffusion-drafter training with contiguous-prefix acceptance and tunes inference using accepted tokens per round divided by round latency. It reports a mean 4.14 times speedup over autoregressive decoding on Qwen3-8B.
Key insight: A better drafter is not necessarily a faster system. Optimize accepted-prefix yield together with round latency, and choose draft depth and block size for the actual concurrency regime.
3.85
Meta Expert shares experts across layers to reduce duplicated parameters. With activation count and expert granularity fixed, the model matches a conventional MoE using 70–80% as many experts.
Key insight: Cross-layer reuse changes how storage is allocated across capabilities. It is an architectural training choice, so compare its adoption cost separately from post-training expert pruning.
3.60
Byteification retrofits pretrained subword models to operate on bytes using learned byte patches around the existing backbone. The conversion uses less than 1% of a typical pretraining budget.
Key insight: Reuse a mature backbone when exploring alternative text representations. Byte-level adaptation can focus training on patch boundaries and fine-grained character behavior instead of rebuilding the entire pretraining pipeline.
3.45
AmazonDynamoDB adds native vector indexing and filtered similarity search beside operational records, removing a separate replication pipeline. AWS claims recall above 99%; the announcement does not provide a full comparative benchmark.
Key insight: Co-locating vectors with operational records reduces synchronization work. This is most relevant when the application already uses DynamoDB; assess filters, update rates and query volume in the cost comparison.