Implicit Reasoning for Large Language Model-based Generative Recommendation

37 minread-scoreDeep dives
Contents · 7 sections1What problem is it actually solving?2Where explicit reasoning gets stuck: four diagnostic experiments3How PauseRec works4What the results show5Limitations as I see them6How a recommendation team could try it7Conclusion and rating

Teams building LLM-based generative recommenders should read this, especially those considering "have the model write a rationale first, then emit the SID". The paper takes the multi-stage explicit-reasoning training pipeline apart and tests each stage. The conclusion is blunt: for recommendation over Semantic IDs (SIDs), CoT SFT on its own does not help and actually does worse than plain next-item SFT. The authors' alternative, PauseRec, is light: insert 5 trainable <pause> tokens between the user history and the target SID, write no rationale text, and train them only through the final SID-prediction loss. On Amazon Toys, PauseRec lifts Hit@5 from next-item SFT's 0.0565 to 0.0615, also above OneRec-Think with RL post-training (0.0579); training drops from 305.62 to 107.34 GPU hours and per-sample inference from 0.2043 s to 0.0586 s. I rate the diagnostic part above the method: the method's absolute gains are small, and its comparison with the SFT baseline mixes in other variables, as discussed below.

What problem is it actually solving?

The standard recipe for LLM generative recommendation: give each item a string of SID tokens (say 4 special tokens), add them to the LLM vocabulary, then have the LLM read the user history and generate the next item's SID token by token. People pick an LLM backbone to tap its pretrained world knowledge, such as "someone who bought shampoo and conditioner may next buy a styling spray". The catch is that an LLM's knowledge is accessed through natural language, while SIDs are tokens it never saw in pretraining. The model has to read user interest from one string of SIDs and generate another, with no step in between that runs on the language it knows.

Existing systems (OneRec-Think and others) try to put the language step back, usually with four training stages:

StageTraining dataWhat is updatedPurpose
CPTText interleaving SIDs with item titles and categoriesOnly the SID token embeddingsTie SIDs to item meaning
Next-item SFTUser history → SID of the next itemFull modelLearn to recommend
CoT SFTUser history → a rationale → SID of the next itemFull modelLearn to write a rationale before recommending
RL post-trainingReward = whether the recommendation hitsFull modelFurther optimize recommendations

One running example (illustrative): a user bought Axe shampoo, Pantene conditioner and a Nexxus styling spray, and the actual next purchase is a Dove root-lift spray (this example comes from the attention-analysis sample in Appendix F.2). An explicit-reasoning pipeline wants the model to first write "this user keeps buying hair-care and styling products, so a styling spray is likely next", then generate the Dove spray's SID from that. The paper asks which of the four stages actually matters, and why.

Where explicit reasoning gets stuck: four diagnostic experiments

The first experiment tests whether the model knows SIDs after CPT. The authors run 2 epochs of CPT with Qwen3-1.7B on Amazon Beauty, then give the model a SID and ask for the item's title and category, scored by exact match. Amazon categories are three-level paths such as "Beauty > Hair Care > Conditioners". Coarse categories are almost always right: level-1 accuracy is above 97.8% on all three datasets. Level-2 drops to 11.85%–40.49%, the full three-level category is below 7.2%, and titles are near 0. This shows that CPT only teaches the model which broad category a SID belongs to, far from knowing the item.

The second experiment tests whether CoT SFT helps at all. After CPT + SFT, the authors run CoT SFT with 7 kinds of rationale text: two templates, such as "The user is likely to buy items in the {target category} category"; free-form rationales written by Gemini 3.1 Flash-Lite and Pro; two rejection-sampling variants that pick one of several rationales; and a restricted format that must cite SIDs. As the table shows, all 7 fall below next-item SFT with no rationale.

Method (Amazon Beauty)Hit@5NDCG@5
Next-item SFT (no rationale)0.05330.0381
Category template0.04920.0343
Extended template0.04250.0282
Gemini 3.1 Flash-Lite free-form0.05200.0361
Gemini 3.1 Pro free-form0.04220.0276
Rationale picked by target-SID logit0.05260.0362
Rationale picked by Gemini Pro scoring0.05240.0367
Restricted-format rationale0.04490.0307

The best one reaches only 0.0526, still below the baseline; the worst, the Gemini Pro version, loses over 20%. Interestingly, rationales from the stronger teacher are worse. So how well a rationale is written and whether it helps the model pick the right SID are two different things. Earlier work that reports CoT gains only sees them after an extra round of RL with verifiable rewards on top of CoT SFT.

The third experiment asks whether the model can still put its knowledge into words after CoT SFT. The authors run the recommendation-CoT-SFT model on four multiple-choice benchmarks, MMLU, HellaSwag, PIQA and ARC-Challenge, scored two ways: one has the model generate the answer letter and checks text match; the other compares the logits of A/B/C/D and takes the largest. On MMLU the base model scores 57.60, text match falls to 0.10, while comparing logits still gives 55.20. The appendix shows why: inside <think> the model usually writes only a short phrase like "The user is likely to answer C" (the letter varies), then emits a string of SIDs right after </think>; in 27,074 of 27,094 samples what follows </think> is SIDs. So the knowledge is still in the logits, but the model no longer states the answer in language; generation always slides into SIDs.

The fourth experiment tests how fragile rationales are. Feeding the CoT SFT model the gold rationale from training at inference gives Hit@5 0.1165, more than double the no-rationale baseline of 0.0533. But that rationale already contains the target category, which hands the model part of the answer. Removing the target category drops Hit@5 to 0.0540; randomly dropping just 5 words cuts Hit@5 by 18.5%; randomly adding 5 noise words cuts NDCG@5 by 18.4%. The model relies on a few keywords in the rationale, so any small drift in the rationale it writes itself at inference costs a lot.

PCA projection of text-token (blue contours) and SID-token (orange dots) embeddings; the four panels are right after adding SIDs, after CPT, after SFT and after CoT SFT. In the three post-CPT panels, some SIDs fall inside the text distribution while another cluster sits bottom-right, away from text, and barely changes afterwards
PCA projection of text-token (blue contours) and SID-token (orange dots) embeddings; the four panels are right after adding SIDs, after CPT, after SFT and after CoT SFT. In the three post-CPT panels, some SIDs fall inside the text distribution while another cluster sits bottom-right, away from text, and barely changes afterwards

The authors also project text-token and SID-token embeddings to 2D with PCA (figure above). After CPT the two kinds of tokens are already clearly apart, and SFT and CoT SFT widen the gap only slightly. A theorem in the appendix explains why this matters: which SID gets the higher logit depends only on the hidden state's component along the "SID-separating" directions. If the hidden-state change caused by rationale text lies mostly in other directions, it can only shift the logit gap between the target SID and a competing SID a little. If the target SID started far behind, the rationale cannot push it ahead. In my view the theorem only turns "if the two spaces are weakly coupled" into an upper bound; it never measures how small the coupling is in the real model, and the PCA figure is just a 2D projection, so the evidence is thin.

How PauseRec works

PauseRec keeps CPT and next-item SFT, drops CoT SFT and RL, and replaces them with one new token and two training steps.

PauseRec pipeline: left is CPT (only the purple SID tokens are trained); top middle is plain next-item SFT, bottom middle is <code><pause></code> pretraining (only <code><pause></code> itself is trained); right merges the two and places <code><pause></code> between <code><think></code> and <code></think></code> for implicit-reasoning SFT; bottom is the inference input
PauseRec pipeline: left is CPT (only the purple SID tokens are trained); top middle is plain next-item SFT, bottom middle is <pause> pretraining (only <pause> itself is trained); right merges the two and places <pause> between <think> and </think> for implicit-reasoning SFT; bottom is the inference input

Why a separate <pause> token? Explicit reasoning asks the model to write its intermediate thinking as text, and the diagnostics above show this step is hard and fragile. <pause> gives the model a few extra positions before it generates the SID: each position passes through all Transformer layers and produces new hidden states, and when the SID is generated, attention can read those hidden states. They never have to be decoded into text, and training does not ask them to look like anything; it only asks that the final SID be right.

Step one is initialization. The authors set the <pause> embedding to the mean of all token embeddings in the vocabulary after CPT, with only a tiny perturbation (variance times the embedding variance). The vocabulary holds both text and SID tokens, so the mean sits between the two groups, a neutral starting point.

Step two is <pause> pretraining. Starting from the CPT result, <pause> is inserted at random positions in the CPT corpus, amounting to 10% of each sequence's length, e.g. "The user has <pause> purchased the following items… its title is "Phyto <pause> <pause> Phytocitrus…"". The training objective is the same as CPT: next-token prediction on this corpus. Only the <pause> embedding is updated and all other parameters are frozen, so the gradient can change it only through whether the text and SIDs after an inserted <pause> are predicted better or worse. The point is to teach <pause> to sit at the boundary between text and SIDs without disturbing the SID embeddings learned in CPT. This step and next-item SFT start in parallel from the same CPT result and do not affect each other.

Step three is implicit-reasoning SFT. The pretrained <pause> embedding is loaded into the next-item SFT result, and training samples become "user history + <think> + 5 <pause> + </think> + target SID", with loss only on the target SID tokens:

Here is the input with 5 <pause> tokens, is the -th token of the target SID, and is the SID length. <pause> positions get no loss, and this step trains all model parameters. The intuition: the model does not imitate any teacher rationale and uses the 5 positions only where they help SID prediction. At inference, the same 5 <pause> tokens are added, then the SID is decoded directly, with no rationale text generated.

In the running example, explicit reasoning first generates dozens of rationale tokens, then the Dove spray's 4 SID tokens; PauseRec's input is always followed by 5 <pause> tokens, which the model processes in one forward pass before generating the SID. Generated tokens go from "dozens of rationale tokens + 4 SID tokens" to "4 SID tokens", which is why inference is much faster.

Attention changes across <code><pause></code> positions in one real sample: red means more attention than the previous <code><pause></code>, blue means less. Early <code><pause></code> tokens attend broadly to the instruction and the history boundary; later ones focus on a few historical items
Attention changes across <pause> positions in one real sample: red means more attention than the previous <pause>, blue means less. Early <pause> tokens attend broadly to the instruction and the history boundary; later ones focus on a few historical items

The authors use this figure to explain what <pause> does. The target of this sample is exactly the Dove root-lift spray, and later <pause> tokens attend noticeably more to a "hair, skin and nails" supplement gummy in the history, which the authors see as sharing the hair-care intent. This is a single-sample visualization: it helps intuition but is not evidence.

What the results show

The main experiments use Amazon Beauty, Sports and Toys, filter out users and items with fewer than 5 interactions, and hold out each user's last interaction for test and the second-to-last for validation. All LLM methods use Qwen3-1.7B, and PauseRec uses 5 <pause> tokens. Baselines include classic and generative models such as SASRec, HSTU and TIGER, the authors' reproduction of next-item SFT, OneRec-Think with explicit reasoning and RL, and ReaRec, another implicit-reasoning method.

MethodBeauty H@5Beauty N@10Sports H@5Sports N@10Toys H@5Toys N@10
HSTU0.04240.03530.02680.02260.03660.0309
TIGER0.04050.03370.02150.01790.03370.0276
ReaRec0.04500.03440.02140.01540.05230.0376
Next-item SFT0.05330.04450.02870.02370.05650.0471
OneRec-Think0.05630.04710.02880.02390.05790.0482
PauseRec0.05680.04670.02940.02450.06150.0509

PauseRec beats next-item SFT on all 12 metrics and OneRec-Think on 10 of them; OneRec-Think is higher on Beauty Hit@10 and NDCG@10. So without rationales or RL, one can match explicit reasoning plus RL. But mind the scale: on Hit@5, PauseRec vs OneRec-Think is 0.0568 vs 0.0563 on Beauty and 0.0294 vs 0.0288 on Sports; the largest gap is on Toys, a 6.22% relative gain. The paper reports no variance over multiple runs, so the table cannot tell whether gaps this small are stable.

On efficiency, on Beauty PauseRec trains in 107.34 GPU hours versus 305.62 for OneRec-Think, about 65% less; per-sample inference is 0.0586 s versus 0.2043 s, about 3.5× faster. The appendix measures latency for different rationale lengths: longer templates and teacher rationales take 0.32–0.42 s, 5.5–7.1× PauseRec. The setup is a single A100, Hugging Face generate, batch 16, no vLLM. This shows the savings come mainly from RL training and rationale generation; the longer the rationale, the bigger PauseRec's advantage.

Two ablations are worth reading together. For initialization, the pretrained <pause> gives Beauty Hit@5 0.0568, text-token-mean initialization 0.0559, SID-token-mean 0.0548, and the default initialization for a new special token 0.0560. For the number of <pause> tokens, Beauty Hit@5 at k = 1, 3, 5, 10 is 0.0562, 0.0559, 0.0568, 0.0555. In other words, whatever the initialization or the number of <pause> tokens, Hit@5 lands between 0.0548 and 0.0568, all above next-item SFT's 0.0533. Pretrained initialization is the best of these, and a single <pause> already gives 0.0562. The existing ablations cannot tell which part the overall gain mainly comes from; the next section explains why.

Limitations as I see them

First, the comparison with next-item SFT differs in more than <pause>. Per the training samples in Appendix F.1, the next-item SFT input lists only the SIDs of past items, while the implicit-reasoning SFT input also carries each item's title and category; and implicit-reasoning SFT trains 5 more epochs on top of the next-item SFT result. The ablation above shows that a single <pause> already gives 0.0562, so the paper's experiments cannot separate how much of the 0.0533 to 0.0568 gain comes from titles, categories and extra training. The missing control: same titles and categories, same 5 extra epochs, no <pause>.

Second, the diagnosis that "CoT SFT weakens language ability" rests mainly on format collapse. MMLU text match falls from 57.60 to 0.10 because the model usually writes only a short phrase like "The user is likely to answer C" (with varying letters) and then emits SIDs. That shows recommendation CoT SFT locks the output format; it only partly supports the claim that the knowledge is "hard to verbalize".

Third, the gains are small and the evaluation is narrow. All three datasets are Amazon review data with 12K–18K items; even on Toys, where the gain is largest, Hit@5 only goes from 0.0565 to 0.0615, with no variance and no online test. Only one backbone size (1.7B) was tried, so whether CoT fails the same way on larger models is left open.

How a recommendation team could try it

  1. If you already run LLM + SID generative recommendation, run the missing control first: add item titles and categories to the next-item SFT input, train a few more epochs, and see how much Hit@5 moves. This costs almost nothing and tells you how much of PauseRec's gain comes from <pause> itself.
  2. On top of that, add 1–5 <pause> tokens with the paper's initialization and <pause>-only pretraining. If offline metrics move only about a thousandth beyond step 1, it is not worth changing the serving path.
  3. If the team is evaluating an explicit CoT + RL design, first run a CoT SFT control as in Table 2. When CoT SFT alone cannot beat next-item SFT, account for the RL cost: in the paper OneRec-Think trains in 305.62 GPU hours, PauseRec in 107.34.
  4. Before launch, re-measure latency with your own inference stack and batch settings; the paper's 3.5× was measured with Hugging Face generate.

Conclusion and rating

The key takeaway: in SID-based recommendation, making the model write a rationale in language does not automatically unlock its world knowledge; what the model needs more is a few extra computation steps before generating the SID.

DimensionRating
Problem importanceHigh. CoT and RL are being moved into generative recommendation at scale, and this is the first systematic test of whether each stage is needed
NoveltyMedium-low. Pause tokens and implicit reasoning have precedents in LLMs; the addition here is a pretraining step that bridges text and SIDs
EvidenceMedium. The diagnostics are clearly designed; main-result gaps are small with no variance, and the SFT comparison mixes in input format and training epochs
Engineering costLow. One extra token and two short training steps; inference adds only 5 positions of forward computation, and no RL
Worth trying?Worth running the cheap controls first. Teams on explicit CoT + RL designs especially should reproduce Table 2 first

How did this read for you?

One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.

Related papersMore from Paper Daily

  1. Generative · 2026-11-10Rethinking Latent Reasoning for Generative Recommendation: Beyond Teacher-Generated CoT and Recurrent Hidden-State FeedbackLatent reasoning for generative recommendation usually needs a teacher model to write chain-of-thought traces, which is costly and hard to stabilize. This paper skips teacher generation. It builds semantic targets from the text meaning of items already in a user's history, and uses those targets to supervise the model's intermediate tokens that are reserved for reasoning steps, without looking at…
  2. Generative · 2026-10-29Latent Residual Reasoning for Ranking in Generative RecommendationGenerative ranking models usually score every candidate in one forward pass. Refiner, from Meituan and the University of Science and Technology of China, adds a few more steps. Each candidate gets a reasoning token, and each step predicts only a correction added to the original score. The base model stays frozen; only LoRA and a new output layer are trained. On Steam it lifts MTGR's AUC from 0.76…
  3. Generative · 2026-07-13OneRec-Think: In-Text Reasoning for Generative RecommendationDeep dive: OneRec-Think: In-Text Reasoning for Generative Recommendation.
  4. Generative · 2026-07-31MLPs are Efficient Distilled Generative RecommendersDeep dive: MLPs are Efficient Distilled Generative Recommenders.

Discussion

Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.

Loading the discussion…