Contents · 5 sections
1If you only score the final hit, the sentences in the middle are never measured2Walkthrough: how one sentence's prediction gain is computed, and where the case differs3How much it gains over the strongest baseline on three datasets4What this means for people building generative recommenders5LimitationsSHARE
The reasoning correctly wrote "likes Disney animation," but the target still missed the top 10: training generative-recommendation reasoning with prediction gain
A user has recently watched Disney and family-oriented animation, and the next item is actually related to "adventure." A recommender that first writes reasoning and then generates the product SID puts "Disney-related" and "family-friendly animation" into the reasoning, but misidentifies titles in the history, and even treats the target to be predicted as already watched. In this reasoning, only 3 sentences actually raise the likelihood of the target SID, and the target does not make the top 10.
Another reasoning that also starts from the user history has 6 positively contributing sentences, stays closer to the history, and places the target in the top 3. This is the comparison of OneRec-Think and GainReason in the paper's Figure 8. Both look as if they "reason," but whether each sentence helps or hurts the prediction of the next item is very different.
This paper is from Gaoling School of Artificial Intelligence at Renmin University of China, the University of International Business and Economics, and Alibaba International Digital Commerce. In existing work, generative recommenders such as TIGER generate the next item's SID directly. OneRec-Think, SIDReasoner, Evo-Rec and similar methods add explicit reasoning and train it with Group Relative Policy Optimization (GRPO): the reward is usually decided by the final recommendation, for example whether the SID is a hit, and is then assigned to the whole reasoning trajectory. SAPO, which assigns this kind of matching reward to reasoning blocks, still does not measure each sentence's contribution to the prediction. This paper proposes GainReason: it looks at how much the log-likelihood of the target SID changes after each extra sentence of reasoning, calls that difference the prediction gain (PG), and uses it to optimize the reasoning. On three public datasets, the average gains over the strongest baseline are 6.2%, 2.7%, and 6.1%.
Scoring each reasoning sentence by how it changes the target SID's likelihood is more discriminative than only asking whether the final recommendation was right; the numbers come from full-catalog offline evaluation on three public datasets.
If you only score the final hit, the sentences in the middle are never measured
Generative recommendation writes the next product as a string of SID tokens. This paper encodes metadata such as title, brand, and category with Sentence-T5, then quantizes the vector with RQ-KMeans into 3 discrete tokens. Explicit reasoning means that, before generating the SID, the model first writes a natural-language judgment of the user's interests, hoping to use world knowledge inside the large model.
The problem is that the model has often already been trained on recommendation data, so a correct final recommendation does not mean these reasoning sentences helped. A rule-based reward looks at the recommendation after reasoning is finished; which middle sentence raised the probability of the target SID, and which one pulled it down, does not appear directly in the training signal. During training the authors saw that positively and negatively contributing sentences were close to half and half, yet rule-based methods do not reflect that difference.
Another problem appears inside a group of rollouts, the trajectories sampled together. Hitting the exact target SID among a huge catalog is already hard, different reasoning trajectories contribute differently to the prediction, and a hit-or-miss rule often gives them the same reward. The group-relative advantage then becomes 0, and the policy update cannot move. The authors also observed that in a substantial share of groups, every reward is identical.
A simpler alternative is to ask another model to score the intermediate steps, but judging "whether this sentence helped the recommendation" is both expensive and easy to bias. GainReason instead looks directly at the change in the target SID's likelihood.
Walkthrough: how one sentence's prediction gain is computed, and where the case differs
The definition of prediction gain comes from the paper's Equations 7 and 8. The target SID has 3 tokens. Given the user history and the reasoning sentences already written, the model assigns an average log-likelihood to these three tokens. The prediction gain of sentence j is the likelihood after this sentence is added, minus the likelihood before it was added. Step 0 uses only the history, with no reasoning yet. The walkthrough first settles the arithmetic with a set of illustrative numbers, then returns to the real comparison in Figure 8.
Step 1. Look at the history only. In the illustration, the average log-likelihood of the target SID is -2.00.
Step 2. The model writes the next sentence, "the user likes Disney and family animation." The likelihood becomes -1.70, and this sentence's prediction gain is -1.70 minus -2.00, which equals +0.30, a positively contributing sentence.
Step 3. If the next sentence mistakenly treats the target as already watched, the likelihood drops to -1.85, and the gain is -1.85 minus -1.70, which equals -0.15, a negatively contributing sentence. The trajectory's net gain is the sum of the sentences, here +0.15.
If a rule-based reward only looks at whether the final SID is a hit, the whole trajectory receives a single score. Sentence-level prediction gain can still separate +0.30 and -0.15 inside the same trajectory, which means positively and negatively contributing sentences can be measured separately.
Step 4. At training time, GainReason standardizes the prediction gains of all sentences in a group of rollouts, then updates the reasoning-writing policy in the GRPO style; this part is called PGRO. The prediction gain is not back-propagated during the update. The authors also use a custom causal attention pattern to compute the likelihood under each reasoning prefix in one pass, so the recommendation itself does not have to be decoded again just to assign a reward.
Step 5. A net gain near zero has two cases: the reasoning itself provided no new information, or the information was written down but not used at prediction time. PGRU is a reasoning-utilization task. It is applied only on trajectories whose net gain is near zero. It strengthens predicting the target SID with that reasoning in place. A large positive net gain means the reasoning already helped; a large negative value is treated as misleading reasoning, and further supervision would only write the error more firmly, so those trajectories are not preferred for this auxiliary task.
The paper's Figure 8 is a comparison on a real user record, and no longer uses the illustrative numbers above. OneRec-Think and GainReason both write "Disney-related" and "family-friendly animation," and both infer "adventure," which is related to the target. But OneRec-Think also misidentifies items in the history, writes the target into the already-watched list, and has more vague, ungrounded sentences. The result is 3 positively contributing sentences versus 6; GainReason ranks the target in the top 3, and OneRec-Think does not make the top 10. The authors also note that correct reasoning does not always bring a positive gain, especially on OneRec-Think. If the information has already been written down but is not used at prediction time, that is the layer PGRU is meant to fill.

How much it gains over the strongest baseline on three datasets
The data are Amazon Industrial & Scientific, Movies & TV, and Yelp business reviews. Preprocessing is 5-core filtering, then an 8:1:1 split after sorting all interactions by absolute time, to avoid leakage. User history uses a sliding window of at most 10 items. Evaluation is over the full item set, with no sampled candidates. To compare the reinforcement-learning stage fairly, every explicit-reasoning method starts from the same checkpoint after the preparation stage.
| Method | Industrial NDCG@5 | Movies NDCG@5 | Yelp NDCG@5 |
|---|---|---|---|
| SASRec | 0.0727 | 0.0118 | 0.0157 |
| OneRec-Think | 0.0915 | 0.0415 | 0.0191 |
| Evo-Rec | 0.0902 | 0.0422 | 0.0188 |
| GainReason | 0.0985 | 0.0435 | 0.0204 |
(Numbers from the paper's Table 2.)
Relative to the strongest baseline on each dataset, the average relative gains across four metrics are 6.2%, 2.7%, and 6.1%. On Industrial, NDCG@5 rises 7.7% over OneRec-Think, and HR@5 and HR@10 rise 6.8% and 4.2% over SIDReasoner. On Movies, the gains on the four metrics sit between 2.1% and 3.1%. On Yelp, NDCG@5 rises 6.8%. OneRec-Think, SIDReasoner, and Evo-Rec are close to one another, which means that changing only the design of the final reward does not make a large difference.

More important is whether the reasoning itself helped. After reasoning is turned on, rule-based methods usually improve on Industrial, often get worse on Movies, and gain only modestly on Yelp. After GainReason turns reasoning on, the average relative gains on the three datasets are 5.8%, 2.3%, and 7.4%, and they remain positive on Movies. On Movies, without reasoning, GainReason's HR@5 is 0.0525, slightly below Evo-Rec's 0.0528; after reasoning is turned on it becomes 0.0541, ahead of Evo-Rec's 0.0526. On Yelp it is also reasoning that widens the lead. That means a large part of its advantage comes from "the reasoning actually improved the recommendation," not from already being stronger at direct generation in the preparation stage.
After PGRO is removed, Industrial NDCG@5 falls from 0.0985 to 0.0892, close to the 0.0881 of the preparation stage alone. Keeping only PGRO and removing PGRU gives 0.0954; changing PGRU's adaptive weighting to uniform supervision over all trajectories gives 0.0963. The drop is larger when PGRO is removed, and PGRU still adds a further contribution.

What this means for people building generative recommenders
If you already use GRPO to train a generative recommender with reasoning, first check whether the reward only looks at whether the final SID is a hit, and whether a group of rollouts often receives completely identical scores.
A useful check is to compute, in the training logs, whether the log-likelihood of the target SID rose or fell after each reasoning sentence. If positively and negatively contributing sentences are each about half, and the group advantage is often 0, sentence-level prediction gain lands directly on the sentences that should be rewarded or penalized. It is also worth checking whether the likelihood actually rises after the reasoning is written correctly; if correct sentences often add no gain, the reasoning may have provided no new information, or the prediction head may not yet be using the reasoning. The second case is the place PGRU is meant to fill.
Limitations
The experiments are offline evaluations on three public datasets, with history at most 10 items and 3-token SIDs, and there is no online comparison. On Movies, the average gain over the strongest baseline is only 2.7%, and the average gain after reasoning is turned on is 2.3%, clearly smaller than on the other two datasets.
Prediction gain needs the likelihood of the target SID at training time, and therefore depends on this supervision signal. Figure 8 is a qualitative case.
How did this read for you?
Hoot made a note. Thanks for the feedback.
One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.
Discussion
Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.
Loading the discussion…