Towards Reliable, Scalable, and Value-Aligned Generative Recommendation for Large-Scale Advertising

Teams running generative retrieval, especially for ads, should read this one. It pins down the mismatch that is hardest to avoid once generative retrieval goes live, and shows that it exists at every level of the SID.Generative retrieval for ads has a basic mismatch: the model picks ads by generation probability, while the platform cares about revenue. Kuaishou measured it. The top ten ads by generation probability and the top ten by eCPM barely overlap, with a hit rate of only about 0.02. GR4AD-X changes the deployed GR4AD (Kuaishou's production generative ad retrieval system) in three places. When building SIDs, the first level keeps the few nearest cluster centers, so an ad has several paths and is not lost when one prefix is pruned. Capacity comes not from a bigger backbone but from more input signals, a larger encoder, a larger embedding for the SID vocabulary and so on. Each SID prefix takes the mean revenue of the ads beneath it as its value, and a value head at each level is trained to predict it. Pruning first keeps twice as many candidates, then uses RRF to merge the probability ranking with the value ranking before cutting. The changes were stacked one at a time in online tests, and together they raise ad revenue by 3.07% over GR4AD.The value-alignment group contributes 1.07 points of that.

What problem is it solving?

Generative retrieval encodes each ad as a 3-code SID and runs beam search level by level: the first code picks a coarse region, the second refines it, and the third is a hash of the ad's own features that separates ads within the same fine cluster. If a high-eCPM ad sits under a first-level prefix with low generation probability, that prefix is pruned at level 1 and the ad never comes back. The paper quantifies the mismatch with HitRate@K, the overlap between the top K ads by generation score and the top K by eCPM. For the base model, HR@10 is 0.017, 0.019 and 0.021 when scoring with the first level, the first two levels, or all three. The three numbers are nearly the same, so the problem is not one level. GR4AD already had sequence-level revenue optimization, yet alignment stays weak at every level.

Two more issues make it harder. Many ads sit almost equally far from two first-level centroids, so hard assignment to one of them is close to arbitrary. And serving is tight: in Figure 1, scaling the model from 0.03B to 0.50B drops QPS to 18% while the revenue gain flattens after 0.16B.

Two SID paths per ad

Standard RQ-KMeans picks only the nearest centroid at each level. Over 7.556M ad embeddings and 16,384 first-level centroids, 29.31% of ads have a second-nearest centroid less than about 5% farther than the nearest. Multi-Path SID changes only the first level: it keeps the m nearest centroids (m=2 in production), runs a normal residual quantization from each, and shares the final hash code across paths. Each ad gets 2 full SIDs rather than a tree of paths.

Training uses both paths as targets, weighting their next-token losses. Both are packed into one forward pass with a block mask so each path sees only the user context and its own earlier codes, and the user context is computed once. At inference, results are deduplicated per ad, keeping the higher-scoring path. First-level top-1 hit rate goes from 14.38% to 17.58%, and online revenue rises 0.58% with the same beam width and latency.

Capacity without a bigger backbone

The authors add a little capacity in five directions, each chosen for low serving cost. Input: 20+ profile and context features from the ranking system, with sequence length unchanged. Encoder: key embeddings widened from 16 to 64 dimensions and a SwiGLU FFN. Vocabulary (PCGM): each decoder layer gets its own SID memory table, plus memory entries for 1-, 2- and 3-code prefixes located by hashing; the first code is predicted without PCGM, the second uses the first code's memory, the third uses the memory of the first two, and the full SID's memory is used in the rerank step. The looked-up vector replaces the FFN gate projection. This step takes parameters from 0.20B to 0.46B while QPS (relative to the 0.16B multi-path model) recovers from 90% to 95%, because part of the matrix math becomes a table lookup. Residual streams: the hidden state is copied into 4 parallel streams that are mixed by learned scalar gates, with attention and FFN still computed once. First-code reasoning: a small learned network maps the BOS hidden state to a continuous vector that is fed to the decoder between BOS and the first code. The decoder's state at that slot predicts the first code, needs no target ad at inference, is shared by both SID paths, and is supervised to predict which of 256 coarse clusters the ad belongs to.

Teaching generation about revenue

A sequence-level reward is not enough, because the reward lives only at the leaves: a wrong third code penalizes a good first-level prefix too. PV-RSPO pushes rewards up the tree. For a user, the model rolls out a batch of ads, each rewarded with the production system's revenue signal (eCPM-style values), and a prefix's value is the mean reward of the sampled ads under it. A light value head per level takes the prefix's own lookup embedding, which does not depend on the user, cross-attends from it to the backbone's user memory, and is trained with GR4AD's listwise RSPO loss (a ranking loss that makes the predicted values of a group of prefixes order the same way as their true values).

PV-BeamSearch gives that head a second vote during decoding. From level 2 on, it keeps 2k candidates by probability (k=512), ranks them by value, merges the two rankings with RRF, and keeps the top k. Ranks are merged instead of scores because log probability and value are on different scales. Level 1 is left alone: in the authors' experiments, fusing there let the value head pick coarse codes with few valid ads beneath them.

Finally, the discriminative rerank builds a rank token from each candidate's SID embedding and ad features, including eCPM signals, and runs one extra step of the same network, reusing the user representation. One head learns eCPM order with a LambdaLoss, another models how deep the conversion chain goes, and RRF merges the two.

Results

Each variant got 5% of traffic for 7 days, with changes stacked cumulatively against GR4AD.

The scaling group alone takes parameters from 0.16B to 0.49B for +1.41% revenue at 85% QPS, while enlarging the backbone to 0.50B gives +0.53% at 33% QPS. Full-depth HR@10 rises from 0.021 to 0.298. Level-1 HR@10 also rises from 0.017 to 0.145 even though beam fusion skips level 1, so that part comes from PV-RSPO training. Training-time value supervision and decode-time value pruning cover different stages and do not replace each other.

Limitations

The value group keeps 75% of the scaled model's QPS, and the scaled model itself runs at 85% of the 0.16B baseline; multiplying the two gives a rough end-to-end figure of a bit over 60%. Every gain is measured on top of all earlier changes, so the table cannot say what PV-BeamSearch adds without PV-RSPO. Rewards are defined at the eCPM and revenue level; content feeds that optimize watch time would need a new prefix value definition.

How a team could try it

First measure your own boundary problem: the share of items whose distance ratio to the two nearest first-level centroids is close to 1. If it is high, Multi-Path SID is cheap and leaves the model unchanged. Then measure the top-K overlap between generation score and business value in your logs. If it is low, start with the discriminative rerank, which is local and gave the largest single gain here, and add prefix value heads and beam fusion later, leaving level 1 untouched.

Bottom line

To align generative retrieval with business value, a final rerank is not enough: every prefix level needs a value estimate, while the first level's coarse choice must stay protected from the value head.