Perplexity releases two multi-vector retrieval models, pplx-embed-v2-late at 0.6B and 9B, that encode text, images, and document page screenshots. They are late-interaction retrievers: every document token stores a 128-d vector, every query token gets a vector, and scoring sums MaxSim—each query vector takes its max inner product against document vectors. The two models are not trained separately with contrastive learning; they both distill token-by-token from the same 18B teacher's output vectors, so they live in one vector space: the corpus can be indexed offline with 9B and queried online with 0.6B. On the authors' 72 domain retrieval tasks, 9B averages nDCG@10 81.3%, 1.6 points above next-best nemotron-3-embed-8b; a 0.6B query with a 9B index scores 79.6%, 1.6 points above both ends at 0.6B, with unchanged query-side cost. Note that the full list of the 72 tasks is not public, so the scores cannot be independently reproduced.
1. Why late interaction, why distill
Retrievers usually trade off between single-vector ANN (fast, loses detail) and cross-encoders (strong, one pass per candidate, only for reranking). Late interaction sits in between: document token vectors are precomputed; at query time only query vectors are computed, then MaxSim. Storage is the cost, so this work compresses each token vector to 128-d.
The authors' other judgment: small models trained with contrastive learning alone are not strong enough; fitting a large model's token vectors works better. A side effect is that differently sized students distilled from the same teacher are interchangeable.
2. Training
2.1 18B teacher: two-stage contrastive learning
The teacher is Qwen3.5-27B with the last 20 layers dropped (18B) and causal masks removed so every token sees both sides. Training has two stages.
Stage 1 uses query–document pairs from 28 datasets, further split by language and domain into 981 subsets, 186M pairs total (88.3% text–text, 8.3% text–image, 3.4% text–screenshot); sampling weights change the mix to 56.5% / 30.9% / 12.6%. Each batch comes from one subset so in-batch negatives are harder. To hit target mix and even spacing, subsets are sampled with a golden-ratio quasi-random sequence rather than independent draws. Contrastive batch size is 16,384 pairs, LoRA rank 128; MaxSim is a Triton kernel to avoid materializing the token-level similarity matrix.
Stage 2 merges LoRA, adds a new rank-32 adapter, freezes vision, and trains only on text triples: each query with one positive and up to 4 hard negatives. Some hard negatives are synthesized by editing key fields in highly templated documents.
2.2 Students: token-wise fit to teacher vectors
Student loss is direct: for every valid query and document token, Euclidean distance between L2-normalized student and teacher vectors, averaged on the query side and document side then summed. The teacher is frozen.
Because the loss needs no in-batch negatives, a batch can mix subsets and gradients are stabler. The 9B student is based on Qwen3.5-9B; LoRA alone underfit, so the last 8 text layers are fully fine-tuned and the rest use LoRA. The 0.6B student is pruned from Qwen3.5-0.8B by dropping 2 of 3 Gated DeltaNet linear-attention layers per block (594M total; 254M embedding, 100M vision; ~240M active on text) and is fully fine-tuned.
3. Experiments
3.1 Text retrieval
This checks whether the two distilled models beat same-size single-vector models on text. The domain suite has 72 public tasks (finance 18, law 20, medical 8, dialogue 5, tech 4, multilingual 17), nDCG@10, equal-weight average over six domains:
9B leads every domain except medical; 0.6B leads every domain among sub-1B models.
Q2D-Web is closer to Perplexity's online setting: ~70K agent-rewritten online search queries over 190M web pages, Recall@1000. Relevance merges three sources: documents the agent actually cited, documents ranked high by the online retriever, and LLM-judged relevant docs. On that merge, 9B and 0.6B score 74.8% and 73.6% versus 69.3% for the strongest baseline nemotron-3-embed-8b. This benchmark comes from Perplexity traffic; other models were not tuned for it.
3.2 Document screenshots and images
On images it is not fully dominant. On ViDoRe-V3, vision-specialized EVIE-8B (66.4%) and EVIE-4.5B (65.9%) are higher, but they store 4,096 and 2,048 dims per token, while pplx-embed-v2-late stores only 128-d. On MIRACL-Vision (Wikipedia screenshots) and PPLX-Q2I (Perplexity online image-search logs: 10K queries, 100K images), gemini-embedding-2 is higher.
3.3 Small-model query + large-model index
This checks whether the shared vector space is usable in deployment. Index documents with 9B; encode queries with 9B or 0.6B. 0.6B query + 9B docs beats both-ends-0.6B in all six domains, averaging 79.6% versus 78.0% (both-ends-9B is 81.3%). Query encoding cost matches pure 0.6B; document index cost is amortized over the index lifetime.
3.4 As the retriever for a search agent
BrowseComp+ lets GPT-OSS-120B act as an agent that repeatedly searches a fixed corpus with the tested retriever and answers complex questions, scored by another LLM. 9B accuracy is 64.0%, 4.9 points above another late-interaction model Reason-ModernColBERT and 8.7 above the best single-vector voyage-4-large; mean search count is also lower than every baseline, and recall is highest.
4. Limitations
No storage or latency numbers are given. Multi-vector indexes store a 128-d vector per token; how much more storage and MaxSim cost that is versus single-vector at web scale is unstated. The domain-task list is private, so scores cannot be reproduced. On image retrieval, specialized vision models and gemini-embedding-2 still win on some benchmarks. The claim that distillation beats pure contrastive learning—especially for small models—has no contrastive numbers in the body.
5. Takeaways
For retrieval teams, the most directly usable piece is "large model indexes, small model queries": document compute is paid once, query latency is 0.6B, and quality beats a pure small model by 1.6 points. The premise is that both ends are distilled from the same teacher; independently trained models cannot be mixed this way. The 0.6B text side has only ~240M active parameters; the authors argue it can run on-device. If you are evaluating multi-vector retrieval, measure storage on your own corpus first—that is the unanswered question in the report.