Keep it Simple! A Multiple-Choice Task can Benefit Generative Recommendation

Wei-Hsiang Huang, Jyun-Yu Jiang, Pu-Jen Cheng
27 minread4.40scoreNo. 111-18 Generative issue
Contents · 6 sections1Generative recommenders skip a comparison during training2Three tokens plug the multiple-choice task into TIGER3Walkthrough: the history is two items, and the next item is chosen from three candidates4How much this gains over TIGER on four datasets5What this means for people building generative recommenders6Limitations
Rel 5Nov 4Prac 4Evid 4Time 5

You bought a piano and headphones, but the system first recommended the bench you'd already bought: how a multiple-choice task changes generative recommendation

Someone shopping in the musical-instruments category bought, in order, a Yamaha portable piano, a matching bench, a double-X stand, and a pair of studio monitor headphones. The next item he actually bought was a Blue Yeti microphone: a natural next step from playing the piano to recording and going live.

The original generative recommender, TIGER, looked at this history and returned a top three of the bench he had already bought, plus two keyboard accessories. Semantically, they all cling to "piano," but they never reach "this person is putting together a recording setup." After a multiple-choice auxiliary task was added, the same model ranked the Blue Yeti first, with a microphone pop filter second.

This paper, from National Taiwan University and Coupang, is about the generative setup in sequential recommendation. In the usual approach, TIGER uses an RQ-VAE to assign each item a SID. RQ-VAE was originally a way to turn a continuous vector into a sequence of discrete tokens, layer by layer; here it writes an item's text semantics as a short token string, and an encoder-decoder then generates the SID of the next item. Later methods include LETTER, which adds collaborative signals to the SID and controls diversity, and ETEGRec, which trains the tokenizer and generative retrieval end to end. Both add more alignment tasks or constraints. This paper changes the training objective instead: it adds a multiple-choice auxiliary task to TIGER, so that at training time the model must pick the next item out of a candidate set, while at inference it still generates directly and is given no candidates. On four public datasets, the NDCG@10 gain over TIGER ranges from 4.9% to 79%.

A multiple-choice task added only at training time can supply the missing positive-versus-negative comparison signal for generative recommendation; the relative gains are versus TIGER, and on Yelp TIGER itself lags SASRec by a wide margin.

Generative recommenders skip a comparison during training

TIGER's main task is direct: given a user's history, maximize the probability of generating each SID token of the next item. Training never places "the next item" next to "not the next item" and asks the model to compare them. That is a different idea from BPR, which requires a positive item to rank above a negative one.

There are two consequences. First, the model tends to memorize combinations it has seen in training, and weakens on combinations it has not. The paper gives an example: people who like guitars more often buy guitar accessories, and rarely buy orchestral items at the same time; such combinations barely appear in training, so the model is unstable at test time. Second, when two items are close in semantics but different in behavior, a pre-assigned semantic SID pulls them together, and the main task has no negatives to pull them apart.

The multiple-choice task is concrete. Besides the history, the model is shown a candidate set that contains one positive item and several negatives drawn uniformly from items the user has not seen, and it must generate the positive item's SID. Candidates are re-sampled every epoch and shuffled.

A simpler alternative is to add negatives only in the loss, without putting candidates into the input. Some earlier work, such as P5, also used a multiple-choice setup, but items often kept their original IDs, and the method relied on pretrained language-model weights. This paper concatenates the candidates' SIDs directly onto the history, so that in a single forward pass the model sees both the positive and the negatives, then generates the correct answer. The generation itself does not depend on pretrained language weights.

Three tokens plug the multiple-choice task into TIGER

To keep the two tasks distinct, the authors add three learnable tokens. p1 marks the main task, p2 marks the multiple-choice task, and p3 separates the history from the candidates; each has its own embedding. The main-task input is p1 plus the history SIDs, with no candidates; the history part is still organized the same way as in the original TIGER. The multiple-choice input is p2, plus the history SIDs, then p3, then the shuffled candidate SIDs.

Both tasks decode the same target: the SID of the next item. The aim is to reduce target conflict. The total loss is the sum of the two negative log-likelihoods. In each epoch, the same sequence goes through the main task once and the multiple-choice task once, and batches mix the two kinds of examples.

The candidate count is written nc: one positive plus nc-1 negatives. When nc=1, the input contains no negatives and simply places the correct answer in the input, which is equivalent to writing down the answer. The paper still sees gains on several datasets, usually smaller. Even just writing "this history corresponds to this item" back into the input helps at test time.

Hoot saysWhen only the correct answer is given, what is the auxiliary task still training?Close ✕

The article already walks Figure 1 at nc=3: the history is item 1 and item 3, the shuffled multiple-choice candidates are 10, 6, 2, and serving uses no candidates. This walkthrough only pursues a boundary that walk did not unpack: after the candidate count drops to 1 and the wrong options are gone, what the auxiliary task is still training. The same sequence and the same positive item, item 6, are used below.

at nc=3 the input has 2 negatives, so the correct SID sits next to wrong SIDs → at nc=1 there are 0 negatives and the comparison disappears → the decoding target is still the SID of item 6, so this is still an auxiliary training pass → several datasets still improve at nc=1, which does not prove the cause

  1. nc=3 multiple-choice input → 2 wrong SIDs sit beside the correct answerIn the article's walkthrough, nc=3 means one positive plus 2 negatives, and after shuffling the SID of item 6 sits next to the SIDs of item 10 and item 2. This step is still a comparison.nc − 1 = 3 − 1 = 2 negatives
  2. candidate count drops to nc=1 → no negatives remain, and the comparison disappearsWhen nc=1, the number of negatives is 1 − 1 = 0, and after p3 there is only the SID of item 6. The input has no wrong options, so a positive is no longer placed next to negatives to be compared.nc − 1 = 1 − 1 = 0 negatives
  3. the decoding target is still the SID of item 6 → this is still an auxiliary training passThe decoding target does not change: it is still the SID of the next item, item 6, the same as in the main task. So even though nc=1 is no longer a positive-versus-negative comparison, it is still an extra pass each epoch that writes “this history corresponds to this item” into the input.target = SID of item 6, same as the main task
  4. several datasets still improve at nc=1 → that does not prove the causeThe paper still sees gains on several datasets, usually smaller. That only shows test numbers can improve after this extra auxiliary pass; it does not prove the gain comes from a comparison signal, or from writing the answer into the input.a gain at nc=1 ≠ a proven cause

Figure 2's axis includes nc=0 (the original TIGER) and nc=1. It is used below only to mark this boundary, not to retell the trend at nc=10 and nc=20.

Paper Figure 2: Hit@10 and NDCG@10 versus the candidate count nc on four datasets, with the axis running from nc=0 to nc=20, including nc=1.
Paper Figure 2: nc=0 is the original TIGER, and the axis also includes nc=1. The walk above only contrasts the nc=3 and nc=1 inputs; it does not retell the rest of the curve, and it does not treat a gain at nc=1 as proof of a particular mechanism.
SourceThe article and paper §3.2 / Figure 1: nc equals one positive plus nc-1 negatives; the Figure 1 example uses nc=3, with item 6 as the positive and shuffled candidates 10, 6, 2. §4.2 and the article: at nc=1 the input has no negatives, and several datasets still improve, usually by less. Figure 2's axis includes nc=0 (original TIGER) and nc=1.

Inference and validation use only the main task. At serving time there are no candidates, and SIDs are generated with constrained beam search of size 20. Early stopping also watches only the main-task validation loss.

Figure 1. The main-task input has no candidates and is used in both training and evaluation; the multiple-choice input appends the shuffled candidates after p3 and is used only in training. Both targets are the SID of item 6. Source: paper Figure 1
Figure 1. The main-task input has no candidates and is used in both training and evaluation; the multiple-choice input appends the shuffled candidates after p3 and is used only in training. Both targets are the SID of item 6. Source: paper Figure 1

Walkthrough: the history is two items, and the next item is chosen from three candidates

The walkthrough follows the example in the paper's Figure 1, with candidate count nc=3. The item IDs come from that figure. The piano-and-microphone record at the start comes from the paper's Table 6, and is used later to show how the ranking changes after training.

Step 1. Look at the input and the label. The history is item 1 and item 3, and the next item to predict is item 6. RQ-VAE encodes each item as 4 tokens; item 6 is written a5, b2, c3, d1.

Step 2. Run the main task. The input is p1, plus the SIDs of item 1 and item 3. The decoder must write out the 4 tokens of item 6 from scratch. This is the same format used at serving time, and the input contains no candidates.

Step 3. Run the multiple-choice task. Two negatives are drawn uniformly from items the user has not seen, here item 2 and item 10, then shuffled with item 6 to get the candidate order 10, 6, 2. The input becomes p2, plus the history SIDs, then the separator p3, then the SIDs of these three candidates. The decoding target is still the four tokens of item 6. The answer is therefore in the input, sitting next to two incorrect SIDs.

Step 4. Add the two tasks' negative log-likelihoods and update the same encoder-decoder. In one epoch, this sequence goes through the main task once and the multiple-choice task once.

Step 5. At test time, only step 2 is used: no candidates, and constrained beam search of size 20. In the paper's Table 6, TIGER trained with the main task only ranked the already-purchased bench first; after multiple-choice training as above, first place became the Blue Yeti microphone, the true next item.

This arrangement shows that the ranking can change when only training is changed and the inference interface stays the same. It does not show what would happen if candidates were also given at serving time, because the paper's default is to use no candidates at inference. The authors ran a separate check: if candidates are actually constructed for test examples, the model almost always places its first-ranked item inside the candidate set, with accuracy close to 1.0.

How much this gains over TIGER on four datasets

The experiments use four public datasets: Amazon Scientific and Instruments, Yelp, and the denser MovieLens 1M. Preprocessing is 5-core filtering and leave-one-out, with history truncated at length 20. SIDs are built from Sentence-T5 text embeddings, then a 4-layer RQ-VAE with codebook size 256 at each layer.

The table below reports only NDCG@10. TIGER+nc=10 means each multiple-choice example has 10 candidates at training time. SASRec is a conventional self-attention sequential model, included to show how strong the generative base itself is.

DatasetSASRecTIGERTIGER+nc=10
Scientific0.01930.02130.0235
Instruments0.02720.02930.0323
Yelp0.03830.02050.0367
MovieLens0.11640.15200.1586

(Numbers from the paper's Table 3; the metric is NDCG@10.)

Relative to TIGER, the NDCG@10 gains on the four datasets are 10.3%, 10.2%, 79.0%, and 4.9%. That last 4.9% comes from TIGER+nc=2 at 0.1594, because on MovieLens nc=2 is slightly better than nc=10. Table 3 notes that these gains over TIGER are significant under a paired t-test on per-user metrics (p<0.05). On Scientific, TIGER+nc=10 is the best number in the full table. On Yelp, TIGER's NDCG@10 is 0.0205, SASRec is 0.0383, and TIGER+nc=10 is 0.0367; in the same table, Hit@10 for TIGER+nc=10 is 0.0621, and for SASRec is 0.0613.

Figure 2. Hit@10 and NDCG@10 versus the candidate count nc, where nc=0 is the original TIGER. Instruments and Yelp peak near nc=10 and drop at nc=20. Source: paper Figure 2
Figure 2. Hit@10 and NDCG@10 versus the candidate count nc, where nc=0 is the original TIGER. Instruments and Yelp peak near nc=10 and drop at nc=20. Source: paper Figure 2

As the candidate count grows, performance first rises, then saturates, and on some datasets it falls back. The authors' explanation is that the model has already learned enough compositional patterns, extra negatives become redundant, and a larger candidate set is more likely to include false negatives: items the user has not seen that still fit the current intent. At nc=1 there are no negatives, and several datasets still improve, usually by less.

Figure 3. Log-likelihood of the ground-truth next item under teacher forcing, computed step by step from the true prefix. The median and the mean rise with nc and then saturate, following the same trend as Figure 2. Source: paper Figure 3
Figure 3. Log-likelihood of the ground-truth next item under teacher forcing, computed step by step from the true prefix. The median and the mean rise with nc and then saturate, following the same trend as Figure 2. Source: paper Figure 3

Split by how popular the next item is in the training set, the auxiliary task improves steadily on the popular subset, and does not clearly hurt the medium or long-tail subsets. On Yelp, the medium and long-tail slices rise a lot: Hit@10 goes from 0.0070 to 0.0354, and from 0.0038 to 0.0432. For long-tail next items on Scientific and Instruments, Hit@10 is still almost zero: 0.0004 to 0.0004, and 0.0007 to 0.0009. Long-tail next items remain hard on those two datasets.

In the default setup, the two tasks are each used once, in equal proportion. If instead only one of the two tasks is sampled by probability, the results drop on three datasets when the main-task share is too high; they also drop on Scientific and Yelp when that share is too low. The authors' conclusion is that keeping the two tasks balanced is still the most effective choice when moving to a new dataset. For negative sampling, uniform sampling is already enough. Switching to long-tail, non-long-tail (Warm-set), or a dynamic schedule that gradually shifts from long-tail to non-long-tail brings no clear extra gain over uniform sampling.

Here is another record, from the paper's Table 5. The user's recent movies are mostly Drama, but one of them is the Children's film Babe. TIGER's top three are all Drama. After the multiple-choice task is added, first place becomes E.T., which is both Children's and Drama, and that is the true next item. The authors put Table 5 and Table 6 together to argue that the model starts to capture a higher-level behavioral intent, not only semantic closeness in item metadata.

What this means for people building generative recommenders

If you already use a SID generative recommender such as TIGER, you can start by adding a multiple-choice task that appears only during training, without first taking on a more complex alignment task.

The inference interface does not have to change: the model still generates SIDs directly. A practical starting point is uniform negative sampling, 10 candidates, and one pass through each of the two tasks in every epoch. Also check whether the TIGER base on your data already lags a conventional sequential model such as SASRec. If the base is weak, relative gains will look larger.

Limitations

The auxiliary task was fully experimented only on top of TIGER. Moving it to other generative architectures is future work stated in the paper.

On Scientific and Instruments, when the next item is long-tail, Hit@10 remains close to 0: TIGER scores 0.0004 and 0.0007, and with the multiple-choice task the scores are 0.0004 and 0.0009.

On MovieLens, TIGER+nc=10 has Hit@10 of 0.2770, slightly below ETEGRec's 0.2790.

The experiments use four public datasets, 5-core filtering, leave-one-out, and a history cap of 20, with no online comparison. When the candidate count is raised to 20, Instruments and Yelp already fall back.

How did this read for you?

One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.

Discussion

Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.

Loading the discussion…