Rethinking Latent Reasoning for Generative Recommendation: Beyond Teacher-Generated CoT and Recurrent Hidden-State Feedback

Yiwen Chen, Fuwei Zhang, Zehao Chen, Deqing Wang, Hehan Li, Peizhi Xu, Hanmeng Liu, Shuanglong Li, Xin Pei, Fuzhen Zhuang, Zhao Zhang
24 minread4.00scoreNo. 211-10 Generative issue
Contents · 5 sections1Which step of a teacher CoT actually helps predict the SID2Keeping only history analysis shortens the supervision text and still improves the metrics3No teacher traces, and no recurrent hidden state4How to read the main table and the ablations5What to take away
Rel 4Nov 4Prac 4Evid 4Time 4

Generative recommendation predicts the user's next item by encoding items as short Semantic IDs (SIDs), then autoregressively generating that ID sequence from the interaction history and mapping it back to an item. Adding a "reason first, then generate" stage usually takes one of two expensive routes: have a large model write a CoT for every training example and compress the whole trace into hidden states, or, as in Coconut, feed the last-layer hidden state back into the next step — which makes the serving engine open a non-standard loop. LASAR, a representative method, does both: it aligns recurrent hidden states with teacher CoT representations, then uses reinforcement learning to set reasoning depth. PHiT changes where the supervision comes from and how the intermediate state is computed: the supervision is the semantics of items the user has already interacted with, and the intermediate computation is an ordinary prefill of thought tokens.

Which step of a teacher CoT actually helps predict the SID

Following the three-step template from GREAM (a generative-recommendation template that writes the chain of thought as history, preference, and intent) and LASAR, the authors split each teacher trace into history analysis (U1), preference construction (U2), and intent inference (U3). On the Beauty training set, a Stage-1 student trained only on the recommendation objective is used to measure two things. The answer contribution is the target-SID log-likelihood with that step kept minus the same likelihood after that step is deleted from the trace, so a positive number means the step helps. The coherence contribution is the same before/after difference on the next unit of text. For answer contribution, the other trace units stay in the input and only the deleted step is removed; for coherence, the comparison is between prefixes with and without that step when predicting the next unit of text. Answer contribution is normalized by the token length of the target SID; coherence contribution is normalized by the token length of the next unit of text, U_{i+1}. U3 is the last step and has no successor, so coherence is not computed for it.

Step-wise diagnosis of recommendation CoT: history analysis is the only step whose mean answer and coherence contributions are both positive; preference construction and intent inference have negative mean answer contributions.
Step-wise diagnosis of recommendation CoT: history analysis is the only step whose mean answer and coherence contributions are both positive; preference construction and intent inference have negative mean answer contributions.

On averages, only U1 is positive on both sides (answer +0.0183, coherence +0.0344). U2 is negative on both (−0.0627 and −0.0766); U3's answer contribution is also negative (−0.0307). Some medians are sharper: the median answer contribution of U1 is only +0.0006, almost on zero; U3's answer median is more negative than its mean. Because this is a likelihood diagnosis rather than a final verdict on an entire family of latent-reasoning methods, what it unsettles is the default that "the whole teacher trace should be distilled," not a universal claim that preference or intent stories are always harmful.

Keeping only history analysis shortens the supervision text and still improves the metrics

Instruments gives a more direct comparison: in the same LASAR framework, SFT only, keeping U1 versus the full CoT cuts mean supervision text from 2377 characters to 921, more than 60% less. All six recommendation metrics improve, though the margins are modest. Full CoT is also not uniformly better than recommendation-only SFT with no CoT at all — some hit-rate cutoffs are worse. That shows, in this comparison, that historical evidence is the more effective intermediate supervision; treating "preference and intent steps are always harmful" as a general law would go beyond what this table is willing to carry.

On Instruments, keeping only the history-analysis step cuts supervision text by more than 60% and improves all six metrics relative to the full CoT.
On Instruments, keeping only the history-analysis step cuts supervision text by more than 60% and improves all six metrics relative to the full CoT.

The backbone is Qwen3-0.6B, and evaluation is leave-one-out on the last interaction. Beauty, Instruments, and Sports use the GREAM splits; Steam uses the LASAR setup. The main-table comparisons below should be read inside that protocol.

No teacher traces, and no recurrent hidden state

Rethinking the supervision source and the compute interface: historical semantic targets replace teacher CoT; fixed thought tokens form history-dependent states in a single prefill.
Rethinking the supervision source and the compute interface: historical semantic targets replace teacher CoT; fixed thought tokens form history-dependent states in a single prefill.

PHiT (Prefilled, History-Informed Thought Tokens) does two concrete things.

First, the supervision target comes from items the user has already clicked. A frozen sparse encoder (SPLADEv2) reads each item's title, brand, category, and description, computes a lexical weight vector once, and caches it; the user's history item vectors are summed, and the K keywords with the highest total weight (K=16 in the paper) are set to 1 and the rest to 0, yielding a multi-hot target. Construction never sees the next-item label, so that intermediate supervision stops at evidence already observed rather than peeking at the answer. A recommendation loss alone need not force the model to summarize the semantics of history items in those intermediate positions; using the keywords as an auxiliary target is meant to make the thought tokens write a checkable history digest before the SID is generated. Building all 131,413 Beauty training targets takes about 113 seconds on one GPU plus CPU; writing teacher CoT for the same set with Qwen3.6-35B-A3B takes about 145 minutes on eight A800s — different hardware, but the order of magnitude is still clear. The encoder stays frozen while the historical targets are built, and catalog vectors are computed once and cached, so that cost does not have to be paid again every training epoch. Stage-2 training time is almost the same as a Base run that has thought tokens but no history loss.

Second, N fixed thought tokens are added to the vocabulary and inserted between the history prompt and the target SID. The IDs are the same for every user, but after causal attention over the history, the final-layer states become user-specific. Training mean-pools those states and fits the historical keywords with a light linear head (class-balanced binary cross-entropy), jointly with the autoregressive SID loss; the history prompt and the thought block themselves are not language-model labels. At inference the sparse encoder and the auxiliary head are dropped; the same thought-token IDs are appended to the prompt, one standard causal prefill is run, and the SID is decoded.

Hoot saysThe same thought IDs, two histories, and how the states still get a training signal (illustrative)Close ✕
The IDs are shared; the states are not. Thought positions are not language-model labels; training signal comes from the keyword auxiliary loss and the later SID loss.
Two illustrative users share the same thought IDs: different histories yield different states; keyword loss and SID loss supply training signal; thought positions are not LM labelsUser A (illustrative)history: polish, lampUser B (illustrative)history: earphones, chargerThe same thought IDsshared by every userA's final-layer statedepends on A's historyB's final-layer statedepends on B's history
Two illustrative users share the same thought IDs: different histories yield different states; keyword loss and SID loss supply training signal; thought positions are not LM labelsUser A (illustrative)history: polish, lampUser B (illustrative)history: earphones, chargerThe same thought IDsshared by every userA's final-layer statedepends on A's historyB's final-layer statedepends on B's history
  1. Training: mean-pool the states, fit historical keywords, plus the SID loss. Thought positions themselves are not language-model labels.

Two illustrative users are given the same thought-token IDs. Different histories yield different final-layer states after causal attention; those states are mean-pooled to fit each user's historical keywords, jointly with the SID loss. The paper does not use these two users; the words are illustrative.

SourcePaper §4: fixed thought-token IDs are the same for every user, and final-layer states become user-specific after causal attention; training mean-pools them and fits historical keywords with a linear head, jointly with the SID loss; the history prompt and thought block are not language-model labels. The two users are added here.
PHiT: all thought tokens are ordinary input IDs in a single prefill, with no recurrent feedback of the final-layer hidden state.
PHiT: all thought tokens are ordinary input IDs in a single prefill, with no recurrent feedback of the final-layer hidden state.

There is no recurrent hidden-state feedback, and the KV-cache interface does not need to change; the cost is N extra positions in the sequence and the cache (plus boundary tokens). That is a deliberate trade for a simpler interface. N is chosen among {1, 2, 4, 8} on validation data, and Beauty selects N=8. On the test set, a recommendation-only Base scores 0.02271, 0.02263, 0.02218, and 0.02356 NDCG@5; the curve is not monotonic in N, but N=8 is highest on both the validation and test splits. The thought tokens need not be readable as keywords: the sparse encoder and auxiliary head are already gone at inference, so it is enough that training wrote the signal into those positions.

How to read the main table and the ablations

On four benchmarks (Beauty, Instruments, Sports, Steam), PHiT uses SFT only and is highest on all 24 main-table metrics, 22 of them significant against the strongest baseline on the same dataset and metric. The significance threshold is a lenient p<0.1; the NDCG@5 cells on Sports and Beauty are the two that are not significant. Baseline scores are copied from the LASAR manuscript and were not re-run; LASAR includes a reinforcement-learning stage, while PHiT is SFT only. The comparison asks whether the five-seed mean is higher than a published point estimate, not whether PHiT would win a rerun under an identical training recipe.

DatasetMetricLASARPHiT
BeautyNDCG@5 / @10 / @200.0239 / 0.0303 / 0.03660.0242 / 0.0311 / 0.0386
BeautyHit@5 / @10 / @200.0365 / 0.0563 / 0.08130.0375 / 0.0591 / 0.0890*
InstrumentsNDCG@5 / @10 / @200.0612 / 0.0667 / 0.07300.0621 / 0.0685 / 0.0749*
InstrumentsHit@5 / @10 / @200.0763 / 0.0937 / 0.11840.0780 / 0.0980 / 0.1233*
SportsHit@20 / NDCG@200.0425 / 0.01880.0517 / 0.0211
SteamHit@50.05050.0561*

Stars mark cells the original main table reports as significant versus the strongest baseline. On Sports, PHiT is about +21.7% Hit@20 and +12.2% NDCG@20 versus LASAR; on Steam, Hit@5 is about +11.2%. Those relative lifts show PHiT sitting at the top of the main table on all four sets, but they should not be read as an absolute advantage once you leave the "baselines not re-run, one side has RL" setup.

Five-seed ablations on Beauty split the credit more clearly. Relative to MiniOneRec (SFT) with no thought tokens — a single run in that ablation — adding eight thought tokens and still supervising only the recommendation objective (Base) already contributes most of the NDCG@5 gain relative to MiniOneRec; PHiT adds about another +2.5% NDCG@5 on top of Base. Versus MiniOneRec, PHiT is about +20.6% / +10.3% / +8.6% on NDCG@1 / @5 / @10. Shuffled targets permute the historical-keyword targets across examples (keeping the keyword marginals but breaking the user–target match); PHiT is significantly better (NDCG@5, p=0.013). A control that aligns to teacher CoT representations is weaker than PHiT (lower in all five matched runs, p=0.018 on NDCG@5). A control that predicts keywords extracted from the teacher CoT, on the same thought tokens, has higher means for PHiT at every cutoff, but the edge is less stable (p about 0.083 on NDCG@5; the paper does not call this a five-run sweep). In other words, the extra token positions themselves are valuable; once the positions are fixed, matched historical targets help more than shuffled or teacher-derived ones, but the increment is controlled, and "+20% NDCG@1 versus a no-thought baseline" should not be read as the whole credit of the supervision recipe.

The diagnosis and the U1-only retrain sit on Beauty and Instruments respectively, and the authors do not write them as a law that holds in every domain. Beyond the main table, the inference side only argues interface compatibility (one standard prefill, no KV-interface change) and does not report production latency. The auxiliary-loss weight is λ=1.0, and decoding uses beam 50. Main-table gains should be read with those settings attached.

What to take away

Intermediate computation in generative recommendation need not be tied to "first let a teacher write preferences and intents." In the Beauty diagnosis, history analysis is the only step whose mean answer and coherence contributions are both positive (and the margins are modest); preference and intent steps have negative mean answer contributions. PHiT writes the semantics of observed items into a cacheable supervision signal, places it in ordinary thought tokens, and finishes in one prefill. What it drops is the teacher trace and the custom hidden-state loop; the main-table gains should be read together with "a few extra token positions," not as a single percentage against LASAR.

How did this read for you?

One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.

Related papersMore from Paper Daily

  1. Generative · 2026-10-29Latent Residual Reasoning for Ranking in Generative RecommendationGenerative ranking models usually score every candidate in one forward pass. Refiner, from Meituan and the University of Science and Technology of China, adds a few more steps. Each candidate gets a reasoning token, and each step predicts only a correction added to the original score. The base model stays frozen; only LoRA and a new output layer are trained. On Steam it lifts MTGR's AUC from 0.76…
  2. Generative · 2026-10-24OneLatent: Latent Reasoning for Efficient Foundation Recommendation ModelsWhen a large language model serves as a recommendation foundation model and first writes a natural-language reasoning trace before outputting the next item's semantic ID, the reasoning text is far longer than the semantic ID and holds back both decoding time and throughput. OneLatent appends 3 fixed special tokens after the user context and trains in several steps. First the model learns to repro…
  3. Generative · 2026-11-04Diverse-Teacher On-Policy Distillation for Generative RecommendationOn-policy distillation (OPD), where the student generates on its own and a teacher gives a full probability distribution at each position, makes generative recommenders train faster, but the student drifts further and further toward popular items. The bias comes from two places: the teacher's probabilities already favor the SIDs of popular items, and the student rarely generates the SIDs of unpop…
  4. Generative · 2026-11-17RecMem: Recurrent Memory Compression for Long-Sequence Generative RecommendationOnce a generative recommender feeds a user's long history straight into the model, compute and memory grow with history length, and long interaction sequences quickly become slow and expensive. The authors compress history segment by segment into a fixed number of memory slots, and the decoder uses a fixed-length context that no longer expands as the history grows. On a long-sequence benchmark fr…
  5. Generative · 2026-10-28LoopGR: Parameter-Efficient Scaling for Generative Recommendation via Recurrent DepthMaking a generative recommender stronger usually means adding layers, which means adding parameters and memory. Tencent's LoopGR (a generative recommendation backbone that reuses one decoder in a loop) runs the same decoder several times. Between passes, the decoder's BOS hidden state (its current judgement of the target) is added to every history position and a tiny GRU rewrites the history alon…
  6. Generative · 2026-11-16ReSID: Rethinking Long-Tail Retrieval in Generative Recommendation through Item ReachabilityGenerative recommendation encodes each item as a string of Semantic ID tokens and generates them one position at a time with beam search, and cold long-tail items are hard to retrieve this way. The authors take apart the whole chain from encoding through decoding to matching and find that tail items are mostly lost early in decoding: on Amazon Industrial, 62.5% of tail targets have already fallen…
  7. Generative · 2026-11-16From Tokens to Behaviors: Rethinking Generative Recommendation for User Trajectory PredictionMost recommenders guess only the user's next click, while user trajectory prediction guesses the next several steps at once, such as buying running shoes and then sports socks and a water bottle. Generative recommendation does this by generating each item's SID token by token before moving to the next item, so the encoding within an item and the transitions between items are tangled in one chain.…
  8. Generative · 2026-10-25A Cue's Hidden Price: Auditing Seller Influence on AI ShoppersTwo headphone listings are identical in price, star rating and specs, except that one adds the line “Our best-selling model, chosen by over 10,000 customers.” Across six such pairs, GPT-4o picked the listing with that line in all 144 trials. Raise that listing's price step by step, and it has to cost about 13% more before the model starts to waver. With a plain product description instead, the ad…

Discussion

Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.

Loading the discussion…