Contents · 5 sections
1What problem does it actually solve?2Method: add two kinds of knowledge3Results4Limitations5What to borrowSHARE
E-commerce search has to decide whether a product is relevant to the words a user typed, and many teams now use a large language model for that call. This JD.com paper finds that among the hard cases a model keeps getting wrong, more than 60% are not reasoning failures at all: the model simply lacks the domain knowledge. It does not recognise a niche brand's nickname, or it is unsure where "partly relevant" ends and "fully relevant" begins. Further optimising the reasoning path with reinforcement learning cannot fix that. The paper proposes K-CARE (Knowledge-driven Contextual Anchoring and Analogical Prototype Reasoning), which feeds the model two kinds of outside knowledge: knowledge mined from search behaviour (which products a query retrieves, which queries lead to a product) and a library of expert-judged hard cases with written rationales. On a 124k-sample human-labelled test set, three-level Macro F1 rises from 81.18 for the reinforcement-learning baseline to 82.58; online, the share of queries whose top 10 results contain an irrelevant product drops from about 12.10% to 11.39%.
It is worth reading for anyone working on search relevance, especially those trying to bring large models to long-tail queries: it treats "missing knowledge" and "weak reasoning" as separate problems and shows a path from an 8B model to a small production model. The evidence comes from a single platform, JD.com, and every experiment uses one base model, Qwen3-8B.
What problem does it actually solve?
The task is three-way classification: given a query and a product, label the pair Perfect, Passable or Bad. The test set is very uneven: Bad 46.02%, Passable only 11.83%, Perfect 42.14%.
The paper splits the model's knowledge gaps into two kinds:
- Not recognising things: slang, new words, niche brands and long-tail categories are absent from the model's parameters, and no amount of careful reasoning can recover them.
- Unsure boundaries: the meaning is understood, but fine-grained judgement rules such as "how close counts as Passable" are unstable.
Much concurrent work optimises the reasoning process with methods such as DPO and GRPO. The authors argue this does not help with the first kind of gap, because the knowledge to reason over is not in the parameters in the first place.

Method: add two kinds of knowledge
The first kind comes from search behaviour; the authors call this part SCA (Symmetrical Contextual Anchoring). It works in two directions:
- Products explain the query: the query is sent to the live search system and the top-ranked products are given to the model as reference, telling it "this word usually means these things". The authors do not use click logs, because long-tail and new queries have too few clicks; live ranking covers every query, and although it is noisy, a large model can pull useful signal out of noisy references.
- Queries explain the product: seller-written descriptions lean towards marketing, while the way users search reveals what a product really is. The authors train a product-to-query generation model on historical click logs that produces 5 to 6 likely queries for each product, including niche ones.
Putting reference material into the input does not guarantee the model uses it. So the authors first train on four auxiliary tasks (called TGKI, Task-Guided Knowledge Ingestion): given a query and its top products, state the search intent; given a top product, predict likely queries; given a product and the queries that clicked it, write a product profile; given only queries, predict product attributes. Answers for these tasks are generated by a stronger model (GPT-5.2). The model is trained on the four tasks first and then separately on the main task; the two stages keep the auxiliary and main objectives from interfering with each other's gradients.
The second kind of knowledge is expert-judged cases; the authors call this part APR (Analogical Prototype Reasoning). One case holds a query, a product, the correct label and a written rationale; the paper calls such an entry a prototype. The library is built automatically in four steps:
| Step | What it does |
|---|---|
| Mine hard cases | Run a weaker model on human-labelled data and keep the pairs it gets wrong |
| Multi-model proposals | Several different models each judge whether the human label is right and write a rationale |
| Arbitration | A stronger model weighs all proposals and keeps only samples it judges correctly labelled |
| Language check | Another model audits and polishes the rationale so the logic reads cleanly |
At inference the query and product are concatenated, encoded with an embedding model, and the most similar case is retrieved from the library; it goes into the input together with its rationale, so the model can follow the same line of reasoning.
The base model is Qwen3-8B, trained for one epoch on 32 Ascend 910B NPUs, with inference at temperature 0.
Results

Key points:
- Un-tuned Qwen3-8B reaches 92.42% precision on Bad but only 39.72% recall, because it tends to call most pairs relevant. Fine-tuning lifts recall to 90.30%.
- From the reinforcement-learning baseline (GRPO, Macro F1 81.18) to full K-CARE (82.58) is a gain of 1.4 points. When components are added one by one, using search results to explain the query contributes the most on its own, from 81.18 to 82.00; each later step adds only a few tenths, but every step is positive.
- The test set has 124k samples and the comparison is against a baseline already trained with reinforcement learning, so those 1.4 points correspond to a real reduction in errors.
The 8B model is not served online. K-CARE labels a large pool of unlabelled data, and a small BERT model trained on those labels is deployed, with latency an order of magnitude lower than the 8B model. The online test uses 10,000 representative queries routed through three lanes at once: two run the current model and one runs the distilled BERT. The metric is the share of queries with at least one irrelevant product in the top 10 results:
| Lane | Share |
|---|---|
| Current model 1 | 12.11% |
| Current model 2 | 12.09% |
| BERT distilled from K-CARE | 11.39% |
The two control lanes are almost identical, which shows the control is stable; the paper reports a 5.87% relative reduction.
Limitations
- Using live ranking results as reference inevitably brings noise, and the method relies on the large model tolerating it.
- The case library needs human labels plus several cooperating models, which is not cheap.
- The product-to-query model itself needs historical click data, so a new platform with no history cannot use it.
- Experiments use a single base model, Qwen3-8B, and the online test measures one metric, the share of queries with an irrelevant product in the top 10; with smaller models, or with clicks and purchases as the goal, the gain needs to be re-validated on your own data.
What to borrow
The easiest piece to copy is using live search results to explain the query: it needs no new data, just the top products in the input, and on its own it is the largest source of gain. If you add reference material, first teach the model to read it with a few auxiliary tasks, then train the main task. The case library is costlier; start with a small library for the few boundary situations that are most often misjudged and measure the effect. For deployment, follow the paper: label offline with the large model and serve a small model online, rather than putting the large model in production.
How did this read for you?
Hoot made a note. Thanks for the feedback.
One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.
Discussion
Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.
Loading the discussion…