OneLatent: Latent Reasoning for Efficient Foundation Recommendation Models

When a large language model is used directly as a recommender, one approach is to have the model first write a natural-language reasoning trace (CoT) and then output the three-level Semantic ID (SID) of the next item. The reasoning text is far longer than a three-level SID, so it holds back both decoding time and throughput; moreover, earlier work on OneReason already reported that reasoning first and then answering does not consistently beat answering directly with the same model. Kuaishou's OneLatent appends 3 fixed special tokens after the user context and trains in several steps: first the model learns to reproduce a teacher LLM's reasoning text after these 3 tokens, then it is switched to outputting the target SID directly after them; at serving time only these 3 tokens are appended and no reasoning text is generated. On Kuaishou's local-services industrial data, OneLatent reaches SID@64 of 0.1993, 17.44% higher than OneReason-CoT in reason-then-answer mode, and its inference throughput is 17.49 times that of explicit reasoning, essentially the same as direct prediction without reasoning. Against the strongest non-OneLatent result, a direct-answer model trained on evaluation-view reasoning data, OneLatent is only 1.58% higher on the industrial data; against the direct-answer model trained on the same merged three-view data (0.1851), it is 7.67% higher. These numbers show the full training pipeline works, but they do not split the credit between the teacher reasoning data and the latent tokens. An online A/B test on 5% of traffic estimates a 9.6% revenue lift, and the model is now serving all traffic.

Why explicit reasoning works poorly in recommendation

The paper calls such a model a foundation recommendation model (FRM). Its input is the user profile $P_u$, the chronologically ordered interaction history $H_u$ and the request-time context ${\mathcal{C}}_u$ (time, location, search terms and so on); the output is the SID of the next item the user interacts with. Making the model write reasoning first causes two problems. One is cost: every request must decode the whole reasoning text token by token. The other is that accuracy does not necessarily rise: an extra generation path sits between the reasoning text and the SID, and if the reasoning goes off by one step, the SID that follows goes off with it. In the paper's Table 2, OneReason-CoT in reason-first mode reaches SID@64 of 0.1697 on the industrial data, below the 0.1823 of the same checkpoint answering directly.

The authors' idea is to use the teacher's reasoning only as a training signal and keep the reasoning process in the model's internal hidden states, with no text output at inference.

Method: 3 latent tokens plus three-view teacher reasoning

What the latent tokens are

The authors add 3 special tokens $⟨L_1⟩⟨L_2⟩⟨L_3⟩$ to the vocabulary, and all samples share this same set of token IDs. When the Transformer reaches these 3 positions it produces hidden states that vary with the context, and the SID tokens that follow read those hidden states through ordinary causal attention. There is no extra encoder, projection layer or decoder; the only added compute at inference is the fixed 3 tokens. The authors also state explicitly that later tokens can still see the raw context directly, so there is no guarantee that all the reasoning information is compressed into these 3 positions. Hoot says▾

Hoot saysOne set of token IDs: how it carries different users' reasoningClose ✕

Same token IDs, different hidden states: the 3 latent tokens are identical for everyone, but at those positions attention reads each user's own context, and the SID reads those hidden states.

1. Not a bottleneck: SID positions still see the raw context, so the reasoning need not live only in these 3 positions; the paper says explicitly it makes no such claim.

The two users are illustrative. During training the prefix also holds a view marker (exploration / evaluation / attribution); at inference only the 3 latent tokens follow the context. A plain causal mask throughout. Source: paper §3.1, §3.3 (prefix [P;H;⟨v⟩;L]; Stages I/II reproduce the teacher rationale after the latent tokens, Stage III swaps in the next item's SID; standard causal mask, no strict attention bottleneck)

Teacher reasoning: three views, length adjusted to difficulty

The teacher model is Qwen3-235B-A22B-Instruct-2507, which writes reasoning for each sample from three views:

• Exploration: given only the user profile and history, the teacher infers the user's likely consumption intents and outputs reasoning plus a list of intents.
• Evaluation: additionally told the item the user actually interacted with, the teacher judges how well that item matches the user's evidence and outputs reasoning plus a score from 1 to 5.
• Attribution: told that this item is a positive sample, the teacher explains which historical signals led to the interaction.

The evaluation and attribution views use the true item, but this information goes only to the teacher and never enters the student model's input, so the answer does not leak. The prompt also asks the teacher to first judge how complex the evidence is: samples with a clear signal converge quickly, and samples with conflicting signals get a multi-dimensional comparison. The resulting reasoning drops to 65% of its original average length (taking the non-adaptive average length as 100), and on the longest tenth of samples the semantic similarity between the two kinds of reasoning is about 0.92.

Three-stage alignment and post-training

The student model is initialized from Qwen3.5-0.8B and first pretrained on the recommendation domain so that SIDs and item text descriptions can generate each other:
$${\mathcal{L}}_{\mathrm{sem}}=-\log p_{\varphi }({\mathcal{S}}_v\mid C_v)-\log p_{\varphi }(C_v\mid {\mathcal{S}}_v)$$
Here ${\mathcal{S}}_v$ is the three-level SID of item $v$ and $C_v$ is its text description. The latent tokens are then trained in three stages:

• Stage I: the input is "profile, history, view marker, 3 latent tokens", and the target is to reproduce one view's teacher reasoning.
• Stage II: all of the user's available views are used. Each view is a separate training example: the input carries that view's marker, and the target is that view's reasoning. The loss is averaged over tokens within each reasoning, then over the user's views, so that long reasoning does not take a larger share of the loss.
• Stage III: the input becomes "profile, history, request context, 3 latent tokens"; the view marker is dropped from here on, and the target changes from reasoning text to the SID of the next item. Serving uses this same input without a view marker.

Post-training first runs curriculum fine-tuning from high-activity to low-activity users, then trains on a 1:1 mix of "append latent tokens, then predict the SID" and "predict the SID directly without latent tokens", so that both modes remain usable.

Experiments

Main results

This experiment compares OneLatent with three groups of baselines: item-ID-based sequential recommenders (GRU4Rec, BERT4Rec, HSTU), SID-based generative recommenders (TIGER, OneRec), and reasoning-enhanced models based on OneReason. There are two datasets: 7 days of industrial logs from Kuaishou local services (test set of 20,393 instances), and Kuaishou's public LLM-Rec benchmark (test set of 10,000). The metric Hit rate is reported two ways: SID@64 checks whether the target item's semantic ID is among the top 64 semantic IDs produced by beam search; ID@64 first decodes the generated semantic IDs back to item IDs and checks whether the target item is among the top 64. ID-based baselines generate no semantic IDs, so they only have the ID columns. The table below is taken from the paper's Table 2; NoThink and Think are the direct-answer and reason-first modes of the same checkpoint, and the parentheses give the reasoning data used for training:

OneLatent is highest on every metric. But the source of the gain must be read from two separate comparisons. Against Think mode, industrial SID@64 is 17.44% higher. When trained on the same three-view data, Think mode reaches only 0.1505 while OneLatent reaches 0.1993, which shows that for the same reasoning data, OneLatent's full training pipeline beats having the model write the reasoning out before answering. The authors' explanation is that small errors in explicit reasoning carry over into the final recommendation; but the two training pipelines differ as a whole, so this comparison alone cannot show the gap comes from writing the reasoning out. Against the strongest non-OneLatent result in each column, industrial SID@64 is only 1.58% higher and ID@64 only 0.45% higher; on the public data the gains are 12.21% and 19.10%. The strongest baseline on the industrial data is the model trained on evaluation-view reasoning that answers directly, and it already captures most of the gain.

On the 9B backbone, OneLatent reaches SID@64 of 0.2008, 8.13% above the best same-size non-latent result of 0.1857 (paper Table 3).

Inference efficiency

This experiment checks the inference overhead of the latent tokens. The authors compare three inference modes on 872 test instances (paper Table 7):

OneLatent's throughput is 17.49 times that of explicit reasoning and nearly identical to direct prediction, because each request processes only 3 extra fixed tokens and does not need to decode reasoning text token by token. The paper's conclusion says "more than 10 times", which disagrees with the 17.49 times in the body; the number in the table is used here.

Quality and role of the teacher reasoning

This experiment checks whether the three-view reasoning is better than the original reasoning. The authors ask GPT-5.5 to score 100 samples of each kind of reasoning from 1 to 5 on nine dimensions such as factuality, evidence use and causal logic. All three new views score above 4, the evaluation view highest (4.33), while OneReason's original reasoning gets only 3.14, which shows the new reasoning really is of higher quality (paper Table 4). Higher quality also brings better recommendations: training OneLatent on different reasoning data raises public SID@64 from 0.0092 with the original reasoning to 0.0239 with the three views merged, and a model trained on the evaluation view alone reaches 0.0233 (paper Table 5). The authors read this as the evaluation view being strong on its own, while the views' ranking varies by dataset and metric and they complement each other; since each single-view model is trained separately, these numbers do not split the merged model's gain by view.

As the number of latent tokens goes from 1 to 3, the hit rates of all three SID levels rise; with 4 or 5 tokens they no longer improve consistently, so 3 are used in the end.

To see what is stored in the latent tokens, the authors feed the profile, view marker and 3 latent tokens to the trained model, deliberately leaving out the interaction history, have it generate reasoning text, and compare its semantic similarity with the teacher's reasoning; the cosine similarities of the three views are between 0.79 and 0.80 (paper Table 8). The authors themselves note that the profile is still visible, so this similarity only shows an association and cannot prove that the reasoning information is all stored in the latent tokens.

Deployment and A/B test

Even without reasoning text, calling the model synchronously for every request still exceeds the latency and throughput budget. OneLatent is therefore deployed as a separate retrieval channel: new user behavior triggers asynchronous inference, refresh priority is set by user value and behavior strength, the generated candidates are written to Redis, and online requests take candidates from the cache, where they compete with the existing retrieval results in fine ranking. The A/B test ran in a sub-scenario of Kuaishou's local-services advertising, with 5% of traffic each for the treatment and control groups, from September 3 to 10, 2026. The treatment group's impressions rose 2.566%, ad spend rose 5.157%. These two are directly observed; revenue is not, and the platform's global posterior estimator puts its lift at 9.6%, without the paper saying how that estimator works.

My view of its limitations

The paper has no separate limitations section; in the body the authors acknowledge two points: the latent tokens are not a strict information bottleneck, and the reasoning information may be spread across the whole model; and the gain in Table 6 comes from the complete training pipeline and cannot be attributed to the latent tokens alone.

Two further points deserve attention. First, on the industrial data OneLatent is only 1.58% above the strongest non-OneLatent result and 7.67% above the direct-answer model trained on the same merged data, and the paper reports no variance across runs, so it is impossible to judge whether this gap is stable. Second, the online deployment uses an asynchronous cache, with candidates generated after the user's last behavior; the paper does not separate how much of the A/B gain comes from latent reasoning and how much from adding one more retrieval channel.

How a recommendation or search team could try it

1. Set up the right baselines first. On an existing LLM recommender, evaluate both the direct-answer and reason-first modes of the same checkpoint, and add a model trained on teacher reasoning that still answers directly. On the industrial data in this paper, most of the gain already appears in that last baseline; comparing only against explicit reasoning overstates what the latent tokens do.
2. Use teacher reasoning as training data, not as online output. Have a large model write exploration, evaluation and attribution rationales offline; the true item is shown only to the teacher and never enters the student's input. The evaluation view alone is a good start: on the public data it reaches 0.0233 by itself, against 0.0239 for all three views merged.
3. Then add the latent tokens. Add 3 special tokens shared by all samples, train the model first to reproduce the rationale after them and then to predict the next item's SID directly; in post-training, mix samples with and without the latent tokens 1:1 so both modes keep working.
4. Ship it first as an asynchronous retrieval channel: new user behavior triggers a refresh, results go to a cache, online requests only read the cache, and the candidates compete with existing channels in ranking. Add an A/B arm that adds the same new channel without latent tokens, so the gain from a new channel and the gain from latent reasoning can be separated.
5. Repeat the offline comparison several times. On the industrial data the margin over the strongest baseline is only 1.58%, small enough that repeated runs and their variance are needed to confirm it.

Overall assessment