Meta Andromeda: Powering the Next Generation of Ads Retrieval Through Model-System Co-Design

Meta's next-generation ads retrieval engine, Andromeda, brings a model close to ranking quality into retrieval and scores every eligible ad. The new version replaces the single user tower with several specialized user towers acting as experts, encodes each ad together with the live request context, and uses an attention-based gate in the interaction layer to activate only the few experts most relevant to each ad. Inference compute is about 300x lower than the 2024 version, with quality close to a ranking model. The bottleneck moved from compute to memory bandwidth and network transfer, so they fused the per user-ad steps of gathering embeddings, running the interaction model and scoring into one GPU kernel that keeps intermediate results in registers and shared memory instead of writing them back to GPU memory. They also put targeting-rule matching, scoring and top-K selection on the same GPU machine, which returns only the final results. This lets the system score more ads within the same latency; since 2025 it has lifted conversions on Facebook by 2.4%.

Retrieval teams should read this, and the reason is not a single new model. It reframes the question of how complex a retrieval model can be: first work out how many bytes have to move for each ad scored, then decide how large the model may grow. Several numbers can be compared directly with your own system. On a single GH200 (NVIDIA's Grace Hopper superchip, a CPU and GPU in one package), the interaction network scores more than 10 billion ads per second and already uses about 85% of peak memory bandwidth. Targeting match runs at more than 100 billion ads per second, an order of magnitude faster than scoring. Moving the whole retrieval path onto one GPU machine cut network traffic by about 100x. Keep in mind that this is an engineering blog post. Quality is described only as "close to ranking level" plus one cumulative conversion number spanning two years, and model details stop at the level of an architecture diagram, so reproducing it means filling in the gaps yourself.

What problem does it solve?

Retrieval has to pick a few hundred or a few thousand ads out of everything eligible within milliseconds, so its model has always been much simpler than ranking's. This split works when the candidate pool is modest. The trouble starts when the pool explodes. Generative AI creative tools let advertisers produce hundreds of millions of new ads, and the pool retrieval has to filter grew a lot.

A small example shows where the pressure comes from (illustrative). A user opens the feed and 500,000 ads pass targeting:

Retrieval quality has an outsized effect on ads revenue: however good ranking is, it can only choose among the candidates retrieval hands over, and a good ad lost at retrieval never comes back. Meta's goal is therefore simple: make the retrieval model closer to the ranking model without adding GPUs in proportion.

Overall structure

The model: MoE comes to retrieval

The old retrieval model paired one user tower with one ad tower, so a user's interests were squeezed into one vector. Someone who is browsing camping gear and also comparing credit cards is hard to express with one vector, and dot-product scoring can only compare against that compressed vector.

The new version splits the user side into a set of experts. Each expert reads one kind of signal and learns one view of the user's interests. At scoring time the router first looks at which ad is being scored and then decides which experts to listen to. Take the same user (illustrative), with 6 user experts and a router that picks 2 each time:

Each ad computes 2 experts instead of 6, about two thirds less work, yet every ad sees the part of the user's interests that matters for it. Compared with MoE in large language models, the difference is that a language model routes per token while this model routes per ad, so the same user is represented by different experts when facing different ads.

The post reports that inference compute of the new model is about 300x lower than the 2024 retrieval model, quality is close to the ranking model of a typical retrieval-then-ranking system, and a single GH200 scores more than 10 billion ads per second. A stronger model that needs less compute sounds odd; the reason is sparse activation in MoE, since most experts do no work at all for a given ad.

The system: why the bottleneck became bandwidth

Once the model became light, multiply-adds no longer limited throughput; moving data did. The 2024 version evaluated tens of millions of ads per second per card, while the new requirement is billions per card, two orders of magnitude more.

A quick calculation shows the problem (illustrative). If each ad leaves only 256 bytes of intermediate results between two kernels, written to GPU memory and read back, 10 billion ads per second means 2.56 TB written plus 2.56 TB read every second. That is in the same range as the GPU's memory bandwidth itself, so the compute units spend most of their time waiting for data. The post says that in multi-kernel pipelines, intermediate tensors create terabytes per second of memory traffic and become the main bottleneck.

The first change is a fused kernel. The 2024 version fused only the operations inside the model, while the system logic around it ran as separate kernels. The new version puts embedding gathering, the interaction network and scoring all into one kernel. Intermediate results stay in registers and thread-block shared memory and are never written back to GPU memory. This is possible precisely because model compute fell by about 300x: only a small enough model fits the register and shared-memory budget of one kernel. Here model and system help each other, since the model was designed with its execution in mind.

The second change is to put the whole path on one machine. Measured alone, the fused kernel already handles billions of ads per second. But targeting match and top-K selection used to run on local CPU hosts, and candidate lists were shipped over the network to remote GPUs for scoring. At billions per second, those lists alone saturated the network. The new approach reverses this: the request carries only user features and the targeting query to one GPU server, which runs targeting match, scoring and top-K selection locally and returns only the final candidates. Network traffic drops by about 100x and the system is again limited by GPU compute.

The hardest part of this step is targeting match. It evaluates complex advertiser-defined rules across tens of feature categories and must support real-time index updates as ads are created, changed or retired all day. The original index structure was designed for CPUs; moving it to GPUs required redesigning it for parallel execution without losing any production targeting semantics.

Together, the two changes let the production pipeline process more than one billion ads per second on a single GPU server while scoring every eligible ad in the pool, about 100x the 2024 version.

How far from the hardware limit

After throughput passed one billion per second, Meta and NVIDIA started a collaboration in early 2026 to push each part of the pipeline toward the limits of current hardware and answer how much room is left for larger models.

The first row matters most. At more than 10 billion per second, the gating network takes as long as the simplest dot-product scoring kernel on the same hardware. In a bandwidth-bound regime, extra model complexity is almost free: the compute units are waiting for data anyway, so a few more multiply-adds do not slow them down. This is the most direct evidence for co-design.

The third row shows targeting match will not become the bottleneck: it is an order of magnitude faster than scoring and can be rerun across several retrieval paths. The fourth row shows there is still room end to end: data movement between parts took a sizable share of end-to-end latency, and after the data layouts were changed, benchmark throughput reached several times production.

Limitations as I see them

First, the quality evidence is thin. The only business number in the post is a 2.4% conversion lift on Facebook since 2025, a cumulative result across several versions that cannot separate the MoE design, the fused kernel and full scoring. "Close to ranking level" is also qualitative; offline metrics, baselines and the remaining gap to the ranking model are left for readers to estimate.

Second, the 300x compute reduction needs context. It is measured against the 2024 retrieval model, and the number of experts, how many are active each time and how large each one is appear only as boxes in a diagram. Other teams cannot tell from this how much they would save by switching their own retrieval model to MoE; they have to run the experiment.

Third, the conclusions are tied to specific hardware. "Extra complexity is almost free" holds when memory bandwidth is the limit, which is true on GH200. The post itself notes that on Grace Blackwell the CPU-to-GPU compute ratio strongly affects end-to-end performance, so the findings must be rechecked. For teams without this kind of hardware, or whose retrieval runs at tens of millions per second, the bottleneck is likely somewhere else.

How a recommendation or ads team could try it

1. Measure where you are stuck. Use a profiler to check memory-bandwidth and compute utilization in the retrieval scoring stage; if bandwidth is near saturation while compute idles, adding model complexity costs less than you think.
2. Do a byte budget per candidate: how many intermediate bytes each score reads and writes, and how much data crosses the network. The large items are where fusion or co-location should start.
3. In offline evaluation, replace the single user tower with a few user experts split by signal type and add a gate that picks the top-K experts per ad; compare recall and compute for K of 1, 2 and 4.
4. If candidate lists cross machines to reach the scoring service, evaluate moving targeting filters and top-K selection onto the scoring machine and returning only final candidates; first check how much network traffic falls.
5. Only then consider writing a fused kernel and compare INT8 against higher precision. This step costs the most engineering, and its payoff depends most on the earlier steps confirming that bandwidth really is the bottleneck.

Bottom line

The most reusable idea is to count the cost of moving data first and only then decide model size: when scoring is bound by memory bandwidth, a more complex interaction model adds almost no time, which opens room to narrow the quality gap between retrieval and ranking.