Contents · 10 sections
1What problem is it actually solving?2Overview: three training stages plus a deployment architecture3Stage 1: teach the LLM to read item IDs first4Stage 2: write rationales on a clean history first, then learn to write them amid noise5Stage 3: a reward scored on beam search results6Deployment: think about the direction offline, fill in the details online7What the experiments show8Limitations as I see them9How a recommendation team could try it10Conclusion and assessmentSHARE
Teams doing generative recommendation who are thinking about adding a "think before recommending" ability to a Semantic ID model should read this paper, but should keep the paper's narrative and its evidence apart. The paper, from Kuaishou, tackles this problem: generative recommenders such as OneRec emit item IDs directly, with no readable or controllable reasoning in between. The core idea in one sentence: add item Semantic IDs to an LLM's vocabulary, first teach the LLM to read these IDs, then teach it to write a rationale for "why this user would watch this" before it emits an ID, and finally push recommendation accuracy up with a reinforcement learning reward scored on beam search results. In production it uses a "Think-Ahead" architecture: the reasoning and the first two ID positions are generated offline, and a real-time model only fills in the last position. In a one-week A/B test on 1.29% of Kuaishou traffic, App Stay Time rose 0.159%. I think the two engineering designs most worth borrowing are the reward that gives partial credit to the best candidate in a beam, and the prefix constraint of "set the direction offline, fill in the details online". Keep in mind that the Beauty ablation shows the recall gain coming mainly from item alignment, with reasoning's increment mainly on the ranking metrics; that the "reasoning" on public data is a manually written category template; and that the paper's experiments cannot separate how much of the online gain comes from the reasoning itself.
What problem is it actually solving?
Generative recommendation represents every item as a Semantic ID of a few positions, and the model generates the next item's ID position by position, like writing text; one property used later is that similar items share their leading positions. Following OneRec, each item's Semantic ID is generated from its multimodal content and collaborative information. Kuaishou's OneRec works this way and works well, but it is an "implicit predictor": given a history, it outputs an ID directly, cannot say why, and cannot change direction when a user says "I'm feeling down today, give me something light". LLMs are good at reasoning and dialogue, but they do not know item IDs. The paper wants to merge the two into one model that can generate item IDs position by position, reason in text before generating, and hold a conversation.
One illustrative sample from the paper's appendix runs through this piece: a user's history has 13 videos, 5 of them about cooking (a beginner's guide to sourdough, a knife-skills tutorial, handmade pasta, kitchen organization and meal prep, a stand-mixer comparison), and the other 8 unrelated to cooking. The next one the user watches is "Advanced French pastry techniques and equipment guide", with Semantic ID <a_3672><b_9814><c_2657>. The difficulty: the LLM does not know what <a_3672> is, and more than half the history is noise, so the rationale has to be induced from those 5 videos.
Overview: three training stages plus a deployment architecture

| Stage | Why it is needed | What it does | Output |
|---|---|---|---|
| Itemic alignment | The LLM does not know the newly added ID tokens | First freeze the LLM and train only the new token embeddings, then train on 4 tasks together | An LLM that can read and write item IDs |
| Reasoning activation | The model cannot write recommendation rationales, and long histories are noisy | Have the model write rationales from the 10 most relevant history behaviours, then learn to write the rationale plus the ID from the full history | A model that writes a rationale before the ID |
| Reasoning enhancement | A fluent rationale does not mean an accurate prediction | GRPO, with a reward equal to how many positions the best beam candidate matches against the target | A reasoning model that recommends more accurately |
| Deployment | Writing reasoning token by token is too slow for request latency | Write the reasoning and generate the first two ID positions offline; a real-time online model fills in the third | A deployable recommender |
The industrial backbone is Qwen-8B. The vocabulary gains 24,576 new tokens, i.e. a 3-position Semantic ID with 8,192 values per position, plus two boundary tokens, <|item_begin|> and <|item_end|>. The model is trained incrementally once a day on 80 GPUs, about 20 billion tokens per day.
Stage 1: teach the LLM to read item IDs first
Why is this step needed? The newly added tokens start with random embeddings. Training directly on recommendation data would leave the model memorizing co-occurrences of <a_3672> as a meaningless symbol.
Itemic alignment is done in two steps, so that the new tokens first get a sensible starting point and are then merged with the language ability. The first step is called Token Warm-up: all of the LLM's original parameters are frozen and only the new token embeddings are trained, using 6 billion tokens at learning rate 5e-4 in the industrial setting. The second step is called Multi-Task Integration: training is opened up, with LoRA in the industrial setting and full-parameter fine-tuning on public data. The data mix of the 4 tasks is:
| Task | Content | Share |
|---|---|---|
| Interleaved user persona | User persona text with the IDs of items the user watched embedded in it | 24.30% |
| Sequential preference modeling | Given up to 256 history items, predict the next one, with loss only on the target tokens | 65.73% |
| Item dense captioning | Given an item ID, write the video's content | 4.94% |
| General language | Ordinary text, to keep language ability from degrading | 5.03% |
In terms of the running sample: the captioning task teaches the model to write "a French pastry tutorial" when it sees <a_3672><b_9814><c_2657>, while the sequence task teaches it that "cooking content is often followed by dessert content". The paper's Table 4 evaluates on two internal Kuaishou understanding benchmarks with BertScore. On user understanding, raw Qwen3 scores 0.6588, Warm-up alone drops it to 0.6492, and adding multi-task training raises it to 0.7053; on video understanding the scores are 0.6031, 0.6443 and 0.7300. This shows that training only the embeddings helps the pure-ID video understanding, is slightly worse on text-heavy user personas, and that the multi-task training afterwards is what really matters. Note that the "reference answers" in this benchmark are outputs of Kuaishou's online multimodal large model, so it measures how closely the model writes like another model.
Stage 2: write rationales on a clean history first, then learn to write them amid noise
Why is this step needed? Asking the model to write a rationale directly over the full 13-video history makes it easy to be led astray by the 8 unrelated videos, and the rationale it writes may not match the target.
This is done in two substages. The first substage "prunes" first: using pretrained item embeddings, compute the cosine similarity between each history item and the target item, and keep only the top k=10. When reproducing, you can simply use the item vectors from which the Semantic IDs were generated. In the appendix's illustrative sample, the pruned history lists the 5 cooking videos; the illustrative sample lists only 5, while the setting in the paper's experiments is 10. The pruned history and the target item are then given to the Stage 1 model, which writes a rationale τ explaining why the user would watch the target. In the appendix, the model writes that the user has progressed from basic skills to more time-intensive cooking projects, and that French pastry, combining baking experience with technical precision, is a natural next step.
The second substage switches to the full, unpruned 13-video history and trains the model to generate τ first and then the target ID, with the negative log-likelihoods of the two parts added with equal weight (the paper's Equation 5). What the model learns this way is: faced with a noisy history, find the relevant few items itself, then write the rationale.
There is a design trade-off here: when the rationale is written, the model can see the answer, so τ is a "post-hoc explanation", yet training asks the model to write it before knowing the answer. The paper uses Figure 5 to argue that the reasoning really guides the recommendation, which the results section below covers.
Stage 3: a reward scored on beam search results
Why is this step needed? Stage 2 is only imitation, and a fluent rationale does not mean an accurate prediction. When reinforcement learning directly optimizes accuracy, the most natural reward is "1 point for hitting the target item, otherwise 0". But in recommendation the vast majority of samples miss; if all 16 samples in a group score 0, GRPO, which computes advantages from relative scores within the group, gets all-zero advantages, and the whole group is wasted.
The paper's Rollout-Beam reward changes two things. First, after the reasoning is written, it does not decode just one item but runs a beam search of width K=32 and looks at the best of those 32 candidates. Second, it does not require the whole ID to match; it counts how many positions match (Equation 6). A hand calculation with the running sample: the target is <a_3672><b_9814><c_2657>. After one reasoning path, the best of its 32 candidates is <a_3672><b_9814><c_0001>, matching the first two positions, so the reward is 2. After another path, the best is only <a_3672><b_5555><c_0420>, reward 1. Yet another path matches nothing, reward 0. The same group now has differences of 2, 1 and 0, so the advantages are no longer all zero.
The training setup samples |G|=16 reasoning paths per question and runs a K=32 beam search after each, so each question requires scoring 512 candidates, which means reward computation is itself a sizeable cost. Note also what is being rewarded: per the appendix, a group of samples is 16 reasoning paths, and beam search only scores each path, so what is rewarded is mainly the reasoning itself; a better reasoning path makes the beam search that follows more likely to match the target. The remaining settings are 2 epochs, learning rate 1e-5, KL coefficient 0.001, clip ratio 0.2, on VERL (an open-source reinforcement learning training framework for large models).
One detail of Equation 6 deserves attention: it compares each position independently and sums, without requiring the prefix to agree. The candidate <a_0007><b_9814><c_2657> gets the first position wrong, but its last two positions happen to match, so it also scores 2. In a hierarchical Semantic ID, though, once the first position is wrong, the last two positions denote a subdivision under a different top-level category, and their meaning is already different. So this reward is not fully equivalent to "how close to the target"; when reproducing it, you can change it to count only the consecutive matching positions starting from the first, and compare the two versions.
Deployment: think about the direction offline, fill in the details online
Why is this step needed? Having an 8B model write a few hundred tokens of reasoning token by token, then run beam search, cannot meet recommendation request latency.
Think-Ahead splits inference into an offline part and an online part, in order to move the slow part outside the request. Offline: for each user, OneRec-Think samples T different reasoning paths; after each path it runs beam search, decoding only the first two positions of the Semantic ID and keeping m prefixes; the results of the T paths together give T×m prefixes, stored in a distributed cache. Online: when a request arrives, the prefixes are fetched, and the real-time-updated OneRec runs constrained decoding on the user's current history. Per the paper's Equation 11, it picks only among items whose first two positions fall in the cached prefix set, ranks them by the online model's probability for the whole ID, and takes the top K. The online model does not skip the first two positions: it scores the whole ID as usual, except that the first two positions can only come from the T×m cached prefixes, so the candidate range shrinks from all items to the items under those prefixes; "filling in the last position" refers to that candidate range. Candidates under different prefixes are ranked together by the same online model's probabilities. The computation saved is mainly the generation of the reasoning text and the wide beam search over the first two positions, which move offline. This also requires both models to share the same Semantic ID mapping, and the paper's IDs follow OneRec's generation method in the first place.
An order-of-magnitude calculation: the first two positions have 8,192×8,192, about 67 million, combinations. Assuming T=4 and m=8, the cache holds only 32 prefixes, which amounts to the reasoning model first marking out 32 fine-grained directions for this user, and the online model picking specific videos only within those directions. I could not find the actual values of T and m in either the main text or the appendix.
The implication of this design needs to be clear: which video is finally served online is decided by the real-time OneRec; the reasoning model's role is to constrain the first two positions, that is, the broad direction.
What the experiments show
The public data is Amazon Beauty, Sports and Toys, with the backbone switched to Qwen3-1.7B, a 4-position Semantic ID with 256 values per position, leave-one-out evaluation, and beam width 10. Excerpt from Table 1:
| Dataset | Metric | HSTU | ReaRec | OneRec-Think |
|---|---|---|---|---|
| Beauty | R@5 | 0.0424 | 0.0450 | 0.0563 |
| Beauty | N@10 | 0.0353 | 0.0344 | 0.0471 |
| Sports | R@10 | 0.0343 | 0.0332 | 0.0412 |
| Toys | R@5 | 0.0366 | 0.0523 | 0.0579 |
| Toys | R@10 | 0.0566 | 0.0764 | 0.0797 |
OneRec-Think is first in every cell. Its Beauty R@5 is 25% above ReaRec (0.0450 to 0.0563), while its Toys R@10 is only 4% higher.
Next, the ablation on Beauty (Table 2):
| Configuration | R@5 | R@10 | N@5 | N@10 |
|---|---|---|---|---|
| Base (fine-tuned on item ID sequences only) | 0.0460 | 0.0654 | 0.0314 | 0.0377 |
| + itemic alignment | 0.0532 | 0.0735 | 0.0342 | 0.0402 |
| + itemic alignment + reasoning | 0.0563 | 0.0791 | 0.0398 | 0.0471 |
Read together, the two tables lead to a conclusion that differs from the paper's claim. Base, fine-tuned only on ID sequences, already reaches R@5 0.0460, above the best baseline, ReaRec (0.0450); its N@10 is 0.0377, also above the best, HSTU (0.0353). Splitting the 0.0113 total R@5 gain over ReaRec: Base itself contributes 0.0010, itemic alignment 0.0072, and reasoning 0.0031. This shows that on the recall metric, the largest share comes from itemic alignment; reasoning helps the ranking metric more, lifting N@10 from 0.0402 to 0.0471, which is 73% of the total N@10 gain over Base. The paper attributes its lead "directly to explicit, text-based reasoning ability", and that attribution does not match the ablation numbers.
More importantly, the appendix states that reasoning activation on public data uses "manually constructed category-based CoT", and the limitations section admits that only a "simplified reasoning capacity" is obtained on public data. So a substantial part of the "reasoning" gain in Table 2 may come from feeding category information to the model, rather than from the model learning to reason.

Figure 5 is the paper's main evidence that "the reasoning guides the recommendation": forcing decoding when the reasoning has reached different points, the recommendations follow the reasoning content. This shows the reasoning text really can change the direction of ID generation and is not decoration ignored once written. But it is a single case, with no count of what share of samples shows this consistency.
The online A/B test runs on Kuaishou short video, comparing against the online model on 1.29% of traffic for one week, with App Stay Time as the primary metric because it reflects users' total time spent (Table 3): App Stay Time +0.159%, Watch Time +0.169%, Video View +0.150%, Follow +0.431%, Forward +0.758%, Like +0.019%, Collect +0.098%.
The paper says that in industry 0.1% already counts as a large improvement, and states that multiple experiments at different times all showed significant improvements. This shows the gains are concentrated in social interactions such as forwards and follows, while Like at +0.019% barely moved.
Limitations as I see them
First, the contribution of reasoning is not measured on its own. The ablation merges reasoning activation and reasoning enhancement into one "R", so you cannot see what GRPO and the Rollout-Beam reward are each worth; there is also no comparison against the ordinary "1 point for a hit" reward, so the need for the reward design is argued, not shown with numbers.
Second, the source of the online gain cannot be separated. In Think-Ahead, the reasoning model only constrains the first two positions, and the third is chosen by the real-time OneRec. To show that the reasoning is what works, you would at least need to compare against a prefix constraint from "the same model without reasoning, beam-searching the first two positions directly". Without that control, the +0.159% could come mainly from "a large offline model marking out candidate directions" by itself.
Third, offline prefixes go stale. The prefixes are computed before the request, so a change of interest the user produces at this moment only enters the reasoning at the next offline generation, and the paper does not discuss how the refresh frequency affects results.
Fourth, the costs are not fully accounted for. Industrial training takes 80 GPUs and 20 billion tokens per day; offline, T reasoning paths with beam search also have to be sampled for every active user, and Kuaishou has hundreds of millions of daily active users, yet the main text gives no numbers for this compute and storage cost.
Fifth, the public-data evaluation is weak: only three small Amazon datasets, histories of at most 50 items, reasoning from a manual template, absolute metric values all between 0.02 and 0.08, and no variance or significance reported.
How a recommendation team could try it
- Start with itemic alignment, the step with the largest recall gain on Beauty and also the lowest cost. Add Semantic ID tokens to a small backbone, train only the embeddings first, then add the "write a caption for an ID" and "predict the next ID" tasks, and compare against a baseline fine-tuned only on ID sequences.
- Then test the reward. On your own data, measure the share of GRPO groups whose 16 samples all score 0. If that share is high, switch to the "how many positions the best beam candidate matches" reward, and try both per-position scoring and counting only the consecutive prefix.
- Run controls for the reasoning part. Prepare three groups: no rationale, a category-template rationale, and a model-generated rationale. If the third is not clearly better than the second, the gain comes mainly from category information, and you can drop the cost of having the LLM write rationales.
- Only then consider Think-Ahead. Generate prefixes offline with the large model and complete them online with your existing model, and add a control group whose "prefixes come from the large model without reasoning", to confirm the reasoning is worth the offline compute.
Conclusion and assessment
The key takeaway: for adding reasoning to generative recommendation, what can be borrowed directly is two engineering components. One is the reward that gives partial credit to the best beam candidate, which solves the problem of all-zero group advantages under sparse rewards; the other is constraining online decoding with offline-generated ID prefixes, which moves the slow part outside the request. How much the reasoning itself is worth still has to be answered by your own experiments.
| Dimension | Assessment |
|---|---|
| Problem importance | High. Making generative recommendation explainable and conversational is the natural next step for this line of work |
| Method novelty | Medium. Itemic alignment and rationale distillation have precedents; the Rollout-Beam reward and the Think-Ahead prefix constraint are practical new designs |
| Experimental evidence | Medium to weak. On Beauty, Base fine-tuned only on IDs already beats all baselines on R@5 and both NDCG metrics, and the reasoning is a manual category template; the ablation merges the two reasoning steps; online there is no "remove the reasoning" control |
| Engineering cost | High. Daily updates of an 8B model take 80 GPUs, plus sampling several reasoning paths per user offline and caching prefixes |
| Worth trying? | Itemic alignment and the Rollout-Beam reward are worth trying; reasoning and Think-Ahead should first be confirmed with control experiments |
How did this read for you?
Hoot made a note. Thanks for the feedback.
One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.
Related papersMore from Paper Daily
Generative · 2026-11-04Think Globally, Generate Personally: Multi-View Thinking with Adaptive Generation for Generative RecommendationIn SID generative recommendation, a user's newly emerging interest is often buried under a pile of old behavior, and every user shares one output transformation. This paper patches both weak spots in TIGER-style models. Before generation, the model gets two extra thinking tokens: one reads only the user's full history, the other reads only the last few items. The two resulting vectors sit at the…
Generative · 2026-10-24OneLatent: Latent Reasoning for Efficient Foundation Recommendation ModelsWhen a large language model serves as a recommendation foundation model and first writes a natural-language reasoning trace before outputting the next item's semantic ID, the reasoning text is far longer than the semantic ID and holds back both decoding time and throughput. OneLatent appends 3 fixed special tokens after the user context and trains in several steps. First the model learns to repro…
Generative · 2026-06-05Bridging Short Videos and Live Streams: Reasoning-Guided Multimodal LLMs for Cross-Domain Representation LearningDeep dive: Bridging Short Videos and Live Streams: Reasoning-Guided Multimodal LLMs for Cross-Domain Representation Learning.
Generative · 2026-11-10Rethinking Latent Reasoning for Generative Recommendation: Beyond Teacher-Generated CoT and Recurrent Hidden-State FeedbackLatent reasoning for generative recommendation usually needs a teacher model to write chain-of-thought traces, which is costly and hard to stabilize. This paper skips teacher generation. It builds semantic targets from the text meaning of items already in a user's history, and uses those targets to supervise the model's intermediate tokens that are reserved for reasoning steps, without looking at…
Generative · 2026-10-29Latent Residual Reasoning for Ranking in Generative RecommendationGenerative ranking models usually score every candidate in one forward pass. Refiner, from Meituan and the University of Science and Technology of China, adds a few more steps. Each candidate gets a reasoning token, and each step predicts only a correction added to the original score. The base model stays frozen; only LoRA and a new output layer are trained. On Steam it lifts MTGR's AUC from 0.76…
Generative · 2026-07-24Implicit Reasoning for Large Language Model-based Generative RecommendationDeep dive: Implicit Reasoning for Large Language Model-based Generative Recommendation.
Generative · 2026-11-16From Tokens to Behaviors: Rethinking Generative Recommendation for User Trajectory PredictionMost recommenders guess only the user's next click, while user trajectory prediction guesses the next several steps at once, such as buying running shoes and then sports socks and a water bottle. Generative recommendation does this by generating each item's SID token by token before moving to the next item, so the encoding within an item and the transitions between items are tangled in one chain.…
Generative · 2026-11-06BRIDGE: Bridging the Semantic Quantization Gap for Generative RecommendationGenerative retrieval predicts the next item's Semantic ID (SID) one code at a time from user history, but ordinary next-code training checks only the probability assigned to the correct code. When that probability stays the same, redistributing the remaining probability among incorrect candidates does not change this cross-entropy loss. BRIDGE (the generative retrieval model studied here) also us…
Discussion
Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.
Loading the discussion…