SetMIR: Multi-Interest Retrieval as Set Prediction

32 minread-scoreDeep dives
Contents · 8 sections1What problem is it actually solving?2The big picture3Taking it apart: assigning targets at training time4Taking it apart: deciding online how many searches to send5Does it actually work better?6Limitations as I see them7How a recommendation/search team could try it8One-sentence conclusion

Retrieval teams should read this carefully, especially those already running multi-interest retrieval whose interest vectors keep drifting toward each other. The change is small, one matching rule at training time and one output head, yet it handles two old problems at once, one in training and one in serving. It comes from Snap's Dynamic Product Ads (DPA) team and is already live as a new retrieval source. The core idea in one sentence: use K learnable interest queries to "predict a set of interests", let queries that get no real target learn to say "not present this time", and use that to decide how many ANN searches to actually issue online. The recipe is borrowed from DETR (an object detector that predicts a set of boxes directly from learnable queries): at training time, Hungarian matching assigns real targets to queries one-to-one. A few numbers: replacing the usual argmax assignment with Hungarian matching makes a 36.5% R@10 difference (the argmax version reaches only 63.5% of the full model); online, each request issues on average 6.70 ANN queries, 33% fewer than a fixed 10; launched as a new retrieval source, it lifts overall CVR by 3.11%.

What problem is it actually solving?

Start with an illustrative scenario. A user this week looked at shirts, pants, dresses, sandals and earrings. A single-vector two-tower model averages these into one vector, which tends to lean toward the most-clicked category. The paper's Figure 3 shows a real case of exactly this: the top 10 single-vector retrievals are all pants.

Multi-interest retrieval (MIND, ComiRec and the like) outputs K vectors per user, each issuing its own ANN search, then merges results. But these methods have two problems.

The first is interest collapse. The usual training rule lets each target item pick its closest interest vector (argmax) and updates only that one. Look at an illustrative similarity table:

Interest queryTarget 1: shirtTarget 2: dress
q10.80.6
q20.30.5
q30.10.2

Under argmax both targets pick q1 (0.8 and 0.6 are each column's maximum), and q2 and q3 get no gradient at all. As training continues q1 handles everything, while the other queries either stay put or drift, leaving only one or two effective interests. In the paper's ablation, the argmax-trained model has on average only 2.16 queries judged "present".

The second is static dispatch. Online, no matter how many interests the user has today, the system always issues K ANN searches with N/K candidate slots each. If 4 of the 10 vectors are near-duplicates, 4 searches fetch the same items again, wasting candidate budget and call cost.

SetMIR's starting point: a user's interests form a set of unknown size, and the model should predict that set rather than force out K independent vectors.

The big picture

SetMIR's three stages. On the left is the input embedder: history items go through a frozen pretrained item content model, plus a learned positional embedding and an event-type embedding (view, cart, purchase). In the middle, a J-layer transformer encoder reads the history to give H. On the right is the M-layer interest decoder: K learnable interest queries first self-attend to each other, then cross-attend to H, and each query finally outputs a retrieval embedding e_k and a presence score p_k.
SetMIR's three stages. On the left is the input embedder: history items go through a frozen pretrained item content model, plus a learned positional embedding and an event-type embedding (view, cart, purchase). In the middle, a J-layer transformer encoder reads the history to give H. On the right is the M-layer interest decoder: K learnable interest queries first self-attend to each other, then cross-attend to H, and each query finally outputs a retrieval embedding e_k and a presence score p_k.
StageWhat it doesWhy it is needed
Inputlast 40 events, each item as a frozen content vector plus position and event typethe item tower is frozen, so the existing single-vector ANN index is reused online
Encode2-layer transformer over the historycontextualized history representations
Decode10 queries, 6 decoder layers; queries look at each other first, then read the historylets queries "divide the work" before reading history instead of chasing the same dominant signal
Outputper query, one normalized retrieval vector and one presence scorethe presence score decides whether that search is issued online
TrainingHungarian matching + InfoNCE + presence loss + diversity lossassigns targets one-to-one; queries left out learn to say "absent"
Servingpresence threshold → query dedup → per-query ANN → max-score mergedecides per request how many searches to issue

The model is small: hidden size 128, about 3M learnable parameters. Supervision comes from high-intent events in a future 3-day window: purchases, add-to-carts and swipe-ups (the action of opening a product landing page from a Snap ad). Targets are deduplicated by product, keeping the strongest event per product, and capped at 15; the most recent 40 events before the window are the input.

Taking it apart: assigning targets at training time

What is Hungarian matching, and why does it prevent collapse? Hungarian matching is a classic algorithm: given a "query × target" cost table, it finds the one-to-one pairing with the minimum total cost, with no query or target used twice. SetMIR's cost is "negative similarity" plus a "negative presence confidence" term:

The two weights are 1 and 0.5, and the second term warms up linearly over the first 10% of training steps so early matching is driven mainly by vector geometry.

Back to the illustrative table, looking at similarity only. Among the one-to-one pairings, q1→shirt, q2→dress totals 0.8+0.5=1.3; q1→dress, q2→shirt totals 0.6+0.3=0.9; q1→shirt, q3→dress totals 0.8+0.2=1.0. The first is best. So q2 is made responsible for dresses and gets a gradient toward them; q3 gets no target and receives a "present = 0" label. This shows Hungarian matching guarantees at the training-objective level that each interest is owned by one query, with no extra regularizer needed to force queries apart.

How do matched pairs learn? Each (query, target) pair is trained with InfoNCE: the positive is the assigned target, and the negatives are all target vectors gathered across GPUs in the same batch, at temperature 0.05. Unmatched queries do not take part in InfoNCE and are not pulled in any direction.

How is the presence score learned, and why is it needed? Each query's presence head outputs p_k, whose label is simply "was it matched this time", trained with binary cross-entropy. This is the piece argmax training lacks: under argmax an unchosen query gets no signal at all, while in SetMIR it is told explicitly "absent this time". Note the paper's own caveat: being unmatched only means there are no more high-intent targets in the future 3-day window, not that the user never has this interest. So the presence score is a scheduling signal for "should this request issue it", not a user-profile label.

What does the diversity loss do? It computes pairwise cosine similarity only among queries with presence above 0.5, and penalizes only the part above 0.3. Illustrative: two active queries at cosine 0.5 are penalized 0.5−0.3=0.2; at cosine 0.2, no penalty. Queries that are already apart are left alone. The three loss weights are 1.0, 0.5 and 0.1.

Taking it apart: deciding online how many searches to send

Serving does one forward pass to get 10 pairs (e_k, p_k), then three steps:

  1. Presence gating: keep only queries with p_k > 0.3; if none pass, keep the highest-scoring one.
  2. Query dedup: sort by p_k from high to low, and drop a query if its cosine to a higher-ranked, already kept query exceeds 0.9. The paper calls this query-level NMS (non-maximum suppression, the detection trick for removing duplicate boxes).
  3. Each remaining query retrieves ⌈N/K̃⌉ items, where K̃ is the number of remaining queries; an item retrieved by several queries keeps its highest score, and the top N are taken.

A hand-computed illustrative example: 5 queries have presence 0.95, 0.90, 0.70, 0.25, 0.10. Gating keeps the first 3. The second has cosine 0.93 with the first, above 0.9, so it is dropped. That leaves K̃=2; with a candidate budget N=1000, each query fetches 500. This shows total retrieval depth stays about the same; what is saved is the number of ANN calls and the slots wasted on duplicate searches.

Does it actually work better?

The offline data are Snap DPA user-product engagement logs, with a catalog of hundreds of millions of products; the test set has about 1M users and 5.3M target products. Per Snap's policy only relative values are reported, and every column below is normalized so SetMIR is 100%. All methods share the same frozen item vectors, K=10, the same optimizer and the same retrieval budget:

MethodAvg ANN queriesR@1R@10R@100MRR
MIND1045.7%52.9%61.7%44.4%
ComiRec-SA1067.4%89.8%99.9%80.0%
DCM1026.0%37.4%49.3%29.1%
KuaiFormer1043.0%52.0%62.8%42.2%
SetMIR (argmax)2.1662.1%63.5%69.1%56.4%
SetMIR6.70100%100%100%100%

The strongest baseline, ComiRec-SA, nearly matches at R@100 (99.9%) but reaches only 67.4% at R@1 and 80.0% MRR. This shows it recovers a similar candidate pool but orders it much worse, and the top of the list is exactly what the downstream ranker cares about most. The most convincing row is SetMIR (argmax): the identical architecture with Hungarian matching swapped for argmax drops to 63.5% R@10, and active queries fall from 6.70 to 2.16. KuaiFormer, the closest architecture to SetMIR (also a transformer with learnable queries), reaches only 52.0% R@10. This shows the gap comes mainly from how queries are supervised, not from the network architecture.

In the single-component ablations, removing the presence loss drops R@10 to 91.4%, and gating lets through 8.23 queries instead of 6.70. This shows an unsupervised presence score carries no information about whether a query should be issued, so the result is more calls and worse recall. Removing the diversity loss drops R@10 to 93.2%, with queries becoming more similar to each other. Of the three, the assignment rule matters most.

The K sweep is informative too: with K=1 as 100%, R@10 already reaches 137.5% at K=3, 155.0% at K=10 and 155.1% at K=15, essentially saturated. More interesting is the active count: even at K=15 only 7.59 queries on average are judged present. This shows the number of effective interests is set by the data; give the model more queries and it will not use them all.

The serving ablation uses one model under three dispatch policies:

Dispatch policyAvg ANN callsR@1R@10MRR
Always issue 1010.00100%100%100%
+ presence gating (threshold 0.3)7.73101.9%99.4%100.7%
+ query dedup (threshold 0.9)6.70101.8%99.2%100.7%

Gating removes 23% of calls at a cost of only 0.6% R@10, and R@1 even rises slightly, because candidates from low-confidence queries would otherwise dilute the merged list.

Retrieval comparison for one user. At the top is the user's history, spanning shirts and tops, pants, dresses, shoes, skirts, handbags and accessories. The yellow band is single-vector retrieval, whose top 10 are all pants. The green part shows SetMIR's active queries: head 3 covers shirts, head 9 dresses and skirts, head 5 pants, head 0 accessories and head 7 shoes; head 1 is also shirts and is removed by dedup (greyed) for being too similar to head 3.
Retrieval comparison for one user. At the top is the user's history, spanning shirts and tops, pants, dresses, shoes, skirts, handbags and accessories. The yellow band is single-vector retrieval, whose top 10 are all pants. The green part shows SetMIR's active queries: head 3 covers shirts, head 9 dresses and skirts, head 5 pants, head 0 accessories and head 7 shoes; head 1 is also shirts and is removed by dedup (greyed) for being too similar to head 3.

The online A/B ran for about a week with the downstream ranker unchanged; SetMIR was added as a new retrieval source with the total retrieval quota unchanged. Overall CTR rose 0.21% and CVR 3.11%. Separately, attributing engagement by source within the treatment arm, SetMIR beat the parallel I2I source (which sends the user's recent high-intent items directly as ANN seeds) by 44% CTR and 51% CVR; the two share the same item vectors, index and per-source quota and differ only in how query vectors are produced.

Limitations as I see them

First, the offline comparison is not quite fair to the baselines. SetMIR uses a frozen item tower, while MIND, DCM (a multi-interest model that builds interest vectors by differentiable clustering) and others were designed to train jointly with the item tower. DCM and KuaiFormer have no public code and were reimplemented by the authors from their papers. Training lengths also differ: baselines converge in 60K to 100K steps, while SetMIR trains to 160K. The authors themselves say the MIND and DCM numbers should be read only as "how these mechanisms behave in a frozen item space". So large gaps like 37.4% and 52.9% should not be read directly as which method is better.

Second, the online evidence is incomplete. The paper does not report traffic share, sample size or significance; overall CTR is only +0.21% while CVR is +3.11%, a 15-fold difference the paper does not explain. The 44% and 51% numbers come from per-source attribution within the treatment arm, not a randomized comparison, and are affected by the ranker's preferences and by sources crowding each other out.

Third, the presence label depends on the supervision window. "Unmatched" only means no high-intent events in the next 3 days, so changing the window changes what the presence score means. Also, targets are capped at 15 while there are only 10 queries, which means users with more than 10 targets always have targets that get no query. The paper has no sensitivity study on window length or the target cap.

How a recommendation/search team could try it

  1. If you already run multi-interest retrieval, first measure offline the average pairwise cosine among the K interest vectors and how many targets each query gets on average. A very skewed distribution (one or two queries taking most targets) indicates collapse and makes this worth trying.
  2. The smallest change is swapping only the training-time assignment: argmax for Hungarian matching. At K=10 each user is just a small 10×W problem, so the algorithmic cost is negligible. Watch R@10 and MRR.
  3. Then add the presence head without changing online dispatch at first; replay offline with the paper's 0.3 gating threshold and 0.9 dedup threshold, and compute how many ANN calls are saved and how much recall is lost.
  4. In the online A/B, look first at overall treatment-vs-control metrics; treat per-source attribution only as a reference so crowd-out effects are not mistaken for gains.
  5. Train one version per supervision window length (e.g. 1, 3 and 7 days) and check whether presence scores and the active count are stable, to confirm the setting fits your own business rhythm.

One-sentence conclusion

Multi-interest retrieval's problem is not all in the model architecture; it is more in how queries are assigned targets during training. SetMIR merges collapse prevention and per-request dynamic dispatch into one thing via one-to-one matching plus a "present this time?" score, with a small change backed by both offline ablations and online results.

DimensionAssessment
Importance of the problemHigh. Interest collapse and fixed dispatch are common problems in multi-interest retrieval
Novelty of methodMedium-high. A direct transfer of DETR-style set prediction to recommendation retrieval; using the presence head for dispatch is new
Experimental evidenceMedium-high. Clean ablations and an online A/B, but the baseline comparison is not fully fair and the online results lack significance
Engineering costLow. About 3M parameters; no change on the item side or the ANN index
Worth trying?Yes, especially for teams with existing multi-interest retrieval; start by swapping the assignment rule
Where it fitsE-commerce and ad retrieval with scattered user interests across many categories

How did this read for you?

One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.

Related papersMore from Paper Daily

  1. Personalization · 2026-10-27iFusion: Integrating Dynamic Interest Streams via Diffusion Model for Click-Through Rate PredictionCTR models usually encode long-term and short-term behavior separately, then concatenate or gate them into one interest vector, so one-off noise in recent behavior, such as stocking up during a sale or buying for someone else, gets written straight into long-term preference. This JD.com ICLR 2026 paper treats long-term interest as a draft: it adds noise, then denoises step by step conditioned on…
  2. Personalization · 2026-10-30Multi-Branch Cooperation Networks for Enhanced Click-Through Rate Prediction in Large-Scale E-Commerce SearchRanking teams should read this one, because it points out an often overlooked waste in multi-branch models: the branches are good at different samples, yet never learn from each other in training. The Pailitao team at Taobao (search by photo) makes three branches correct each other during training. On a given sample, if one branch gets the click right and another gets it wrong, the right branch's…
  3. Personalization · 2026-07-27CASE: Cadence-Aware Set Encoding for Large-Scale Next Basket Repurchase RecommendationDeep dive: CASE: Cadence-Aware Set Encoding for Large-Scale Next Basket Repurchase Recommendation.
  4. Personalization · 2026-09-02CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads RetrievalDeep dive: CAMIE: Co-Engagement-Aware Multimodal Item Embeddings for Snap Dynamic Product Ads Retrieval.

Discussion

Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.

Loading the discussion…