Controllable Generative Recommendation via Guided Token Refinement

Generative recommenders first encode each item as a discrete token string (Semantic ID), then generate the next item's tokens from the user history one by one. Businesses often need temporary controls—for example mid-sale, covering more categories—when retraining is too slow, and forcibly rewriting a token during left-to-right decoding often produces IDs that do not exist in the catalog. Xi'an Jiaotong-Liverpool's ReGen changes the decoding style: the model predicts all 4 tokens of an item ID at once; at inference it adds a "bias toward categories the user has not seen" score only on the coarsest few tokens, remasks low-confidence tokens and repredicts, and uses a prefix tree of all legal catalog IDs to drop tokens that cannot complete a real item. Control never changes model parameters. On Amazon 2023 Instruments and Scientific, raising top-10 category coverage (Diversity@10) from 0.41 to 0.59 drops Instruments Recall@10 only from 0.0427 to 0.0394, while TIGER-MMR falls from 0.0433 to 0.0319. Pure accuracy is not best: without a diversity requirement, Recall@10 trails Paragon, and on Video Games it also trails SASRec, RPG, and LLaDA-Rec.

1. Why left-to-right generation is hard to control at inference

Items are encoded with RQ-VAE: Sentence-T5 embeds title and text metadata, then 4 residual codebooks of 256 codes each quantize them into tokens $(c_1,c_2,c_3,c_4)$. Residual quantization makes position 1 set the coarse direction and later positions fill detail, so the first few tokens roughly correspond to category-level semantics.

TIGER-style models generate $c_1\to c_2\to c_3\to c_4$; once position 1 is chosen, every later token must continue under it. The authors call this decision locking: to raise diversity one can only rewrite a mid-generation token, but later tokens were trained conditioned on the original prefix and often form catalog-missing combinations. The other two existing families also have issues: baking the control objective into parameters at training (e.g. Paragon) requires retraining when the objective changes; post-generation reranking (e.g. MMR) can only pick among already generated candidates and fails when the candidate set itself lacks diversity.

2. Method: three changes

2.1 Predict all tokens at once

ReGen uses a BERT-style bidirectional Transformer encoder. Input is the user's history of semantic IDs plus a 4-position ID to fill, with some positions known and the rest [MASK]. The model outputs a 256-d score vector at each [MASK], softmaxed into per-code probabilities. Bidirectional attention lets predicting $c_3$ see $c_1$, $c_2$, and $c_4$ together.

Training masks coarse positions less often (${\alpha }_1<\cdots <{\alpha }_L$), so the model more often sees "coarse known, fine to fill"—matching inference where control fixes the coarse positions and the model fills the rest.

2.2 Control scores only on coarse tokens

For diversity, the goal is to land recommendations in categories rare in the user history. Count how often each code appears in the user's history on the first $K$ coarse positions, give frequent codes a negative score $s_y^{(l)}$, and at inference add $\lambda s_y^{(l)}$ to the model scores on those first $K$ positions while leaving later positions untouched:
$${\tilde{Z}}^{(l)}=Z^{(l)}+\lambda s_y^{(l)}(l\le K),{\tilde{Z}}^{(l)}=Z^{(l)}(l>K)$$
Larger $\lambda$ pushes toward large categories the user rarely saw. Fine positions get no bias and are left for the model to refill under the new coarse class.

2.3 Low-confidence reprediction + prefix-tree constraint

After coarse positions are pushed to a new class, previously filled fine tokens may no longer match. Each round takes every position's max softmax probability as confidence; positions below a threshold are remasked and repredicted, while high-confidence positions stay as known context. Default $T=4$ rounds.

Parallel prediction also risks selecting four individually top codes that form no real item. The authors build a prefix tree of all catalog IDs and allow at each position only codes that can complete at least one full legal path with already fixed positions; other codes get score $-\infty$.

Inference keeps a beam of 50 candidates with 100 legal expansions per step. Candidate IDs are mapped back to items, deduplicated, and items already in the user history removed; top-10 is built greedily by picking the item that maximizes "controlled generation score + $\mu$ × newly covered categories."

3. Experiments

Data: Amazon 2023 Instruments (57,439 users, 24,587 items), Scientific (50,985 / 25,848), Video Games (94,762 / 25,612); leave-one-out; last interaction for test. Encoder: 4 layers on Instruments/Scientific, 6 on Video Games; embedding dim 256; trained from scratch.

3.1 Accuracy without control

This checks whether parallel fill-in decoding loses basic recommendation quality. ReGen's config is chosen by validation accuracy and compared to 9 baselines on Recall and NDCG. Recall@10 only:

ReGen ranks second on the first two categories; Paragon is clearly higher; on Video Games it trails SASRec, RPG, and LLaDA-Rec. The authors' own conclusion: this structure provides a usable accuracy floor but is not designed for pure accuracy.

3.2 At matched diversity, whose accuracy drops less

This is the main experiment for inference-time control. Each method tunes its own diversity knob (ReGen tunes $\lambda$) and plots Diversity@10 against Recall@10 / NDCG@10—upper-right is better.

Format is Recall@10 / NDCG@10. On Instruments, raising diversity from 0.41 to 0.59 drops ReGen Recall@10 from 0.0427 to 0.0394 and NDCG@10 from 0.0205 to 0.0188; TIGER-MMR in the same range drops Recall@10 from 0.0433 to 0.0319. ReGen is almost flat on this segment. On Scientific near diversity 0.59, ReGen still has the highest Recall@10 at 0.0296.

3.3 What each of the three parts does

Ablations under a shared "bias toward diversity" inference setting, removing one part at a time:

Three rows make three points. Dropping the control score slightly raises accuracy and lowers diversity—diversity mainly comes from the control score. Dropping reprediction pushes diversity to 0.73 but drops Instruments Recall@10 from 0.0339 to 0.0130: after coarse positions move to a new class, fine positions are not adjusted, so the system surfaces category-diverse but user-irrelevant items—what the authors call "fake diversity." Dropping the prefix tree leaves under 30% of generated IDs mapping to real items—most of the 4-token combination space is empty.

3.4 Parameters and speed

$\lambda$ is stable between 4 and 12; past 20, reprediction cannot recover and accuracy falls sharply. Reprediction rounds $T$ converge near 4; more rounds barely help while eval time keeps rising—on Instruments, $T$ from 1 to 8 grows full-eval time from 41 to 114 minutes. Only full-eval wall time is reported, not per-request latency.

4. Limitations

Only category diversity is tested as a control objective; fairness, freshness, and exposure balance are left to future work. Diversity is top-10 category coverage—the paper itself calls it a simplified proxy. Control depends on RQ-VAE's first positions truly corresponding to category; a poorly learned codebook may not turn coarse-position scores into category change. All three datasets have only ~20K items; prefix-tree and 50-beam cost at tens of millions of items is unmeasured.

5. Takeaways

If you already use semantic IDs for generative recall, this offers a no-retrain way to add operational objectives: put control scores only on the first few ID positions, let the model refill the rest, and use a prefix tree for legality. Remember the ablation row where dropping reprediction raises diversity but collapses recall—diversity alone can mislead. Before shipping, measure per-request latency (4 reprediction rounds + 50 beams) and verify that coarse positions align with category on your own codebook.