AGENTIC · 2026-10-22
Agentic — Paper Daily, 2026-10-22
Community search over user posts, questions, answers, and comments. Primary metrics: relevance (nDCG) and click-through rate; focus on semantic matching, query understanding, retrieval, ranking, and LLM-assisted search.
4.61
Deep search can reinforce an early mistaken hypothesis. FALCA separates answer proposals from independent counterevidence searches and trains research-state revision with hierarchical credit. Its 4B model reaches 56.0% accuracy on BrowseComp.
Key insight: Give counterevidence checking its own context. A useful falsification report names conflicting evidence and unmet constraints, giving the next search a specific gap to resolve.
4.45
DeclineCraft tests whether models abstain when neither retrieved evidence nor internal knowledge supports an answer. Version-specific Minecraft questions allow controlled knowledge cutoffs. Iterative retrieval reduces correct abstention in 30 of 40 configurations.
Key insight: Evaluate retrieval depth separately from reliability. Track both accuracy on answerable questions and abstention when evidence is missing; a willingness to answer is not evidence coverage.
4.35
A controlled comparison asks whether explicitly constructed memories outperform retrieval over raw conversation history. Across six systems and two benchmarks, gains are localized rather than consistent. Oracle retrieval separates construction losses from access failures.
Key insight: Diagnose preservation and access separately. Oracle retrieval can reveal whether to improve memory construction or retrieval, before paying for more elaborate persistent representations.
4.21
OACR estimates how adjacent knowledge-graph search states change the expected outcome, then reweights token updates without reversing the trajectory advantage. With Qwen2.5-7B, it improves CWQ F1 by 4.6 points over outcome-only GRPO.
Key insight: Use local progress to change update magnitude while leaving final success in charge of its direction. The state evaluator runs only during training, avoiding an extra inference-time judge.
4.15
COMPASS chooses models by consulting performance and cost records through tools, rather than training a dedicated router. It can inspect candidate answers before selection and update its choices as records change. LLMRouterBench accuracy reaches 69.85%.
Key insight: Make model capabilities queryable records. Count the router and candidate inspections in end-to-end cost when assessing whether this approach saves money.
4.15
TURTLE evolves tool-use evaluations with a task proposer, solver, and feedback generator. Solver traces guide tasks toward a difficult but solvable range. On AppWorld, the share of hard tasks increases by 165%.
Key insight: Use execution traces to calibrate task difficulty continuously. Failure locations give a task generator more useful guidance than a single success bit.
4.15
FinHopper evaluates financial search chains whose intermediate entities, dates, and indicators determine later queries. Gold intermediate answers expose where a chain breaks. The study compares 20 model-agent configurations.
Key insight: Use typed intermediate checkpoints. Separate a wrong inherited constraint from missing evidence or a final synthesis error, rather than collapsing every failure into an incorrect final answer.
4.15
Agent S4 evolves tool interfaces and usage knowledge from execution traces, then selects repositories on unseen validation tasks. With the same model, its OSWorld V2 binary score is 3.8 times the fixed-interface baseline.
Key insight: Treat interfaces as an optimization target. Select interface versions on independent tasks and record the underlying model, so improvements remain attributable and reusable.
4.15
RiseHarness compares successful and failed runs of the same task, using internal execution traces to revise shared harness mechanisms without changing model weights. DeepSeek-V4-Flash gains 7.25 percentage points in mean TBH success.
Key insight: Attach executable evidence to completion decisions. Record command dispatch, observations, and completion checks separately to locate reliability failures in the harness.
3.80
Terminal X
OpenAITerminal X encodes firm-specific investment standards and uses a separate evaluation agent to check research outputs. The vendor case study reports 88.6% agreement with senior investor judgment.
Key insight: Make institutional standards explicit and inspectable. This customer case does not publish a complete test set; its transferable contribution is the separation of research generation and firm-specific evaluation.
EMNLP 2026