How Language Models Learn Taste: Brand Preference in LLMs

Ask a language model to "name a running shoe brand" and it will most likely say Nike; ask it to "recommend a running shoe brand" and it is more likely to say Brooks. This paper measures that effect across 102 models and 42 categories. When a model is first asked for recommendations and then asked to pick one, the "panel answer" (the brand all models give most often) differs from the "name one" answer in 26 categories. And the models do not each drift somewhere different; they move together to the same brand: Delta becomes Singapore Airlines, Chrome becomes Firefox, Google becomes DuckDuckGo. The brands named off the cuff are mostly market leaders; the brands picked after some thought are mostly niche or specialist ones. The author then asks at which training step this "taste" appears. Using three open models that publish their weights after every training stage, they find the key step is preference training: in the step where the model learns which of two answers is better, the pick moves by 11 to 24 percentage points, even though the data used in that step hardly mentions brands.

1. The problem: where do the brands a model recommends come from?

More and more people simply ask an AI assistant to "recommend one". Existing audits look at which brands production assistants recommend and whether the answer holds up when the question is reworded; few ask which training step put those preferences there. The paper starts from a simple contrast: "name one" asks what the model thinks of first, while "recommend one" asks it to make a judgment for the user. If the two answers differ, the model is not judging by "who is most famous" but by a different set of associations.

2. How the experiment works: four ways of asking about the same category, put to 102 models

The author picked 42 categories, from soda to cloud providers, and asked about each one in several ways to see whether the answer changes. The first three are a single question that tells the model to reply with one name only; below we call them the one-name questions:

• "Name": name a running shoe brand.
• "Choose": choose a running shoe brand.
• "Recommend": can you recommend a running shoe brand?

The fourth is the two-turn question. In one conversation the model is first asked "Can you recommend a running shoe brand?" and usually lists three to five; it is then asked "Which one would you pick?" and has to choose one from its own list. The brand from this turn is called the Picked brand below. Going from "Name" to the two-turn question, the request moves step by step from "what comes to mind first" to "make the call for me", and the author wants to see how the answer changes along that line.

All 102 models are called through OpenRouter, a service that reaches many vendors' models through one interface. The "temperature" is set to 1. Temperature controls how random the answers are: at 0, a model always gives the answer it rates most likely, so asking again changes nothing; at 1, it draws an answer according to its own probabilities. If it gives Nike 70% and Brooks 30% (illustrative), ten asks yield roughly seven Nikes and three Brookses. What gets measured is therefore each model's whole spread of preferences, not only its single most common answer. Because answers are drawn at random, each question is repeated: 8 times per model and category for the one-name questions and 4 times for the two-turn question.

Twenty-five models can either "write some internal thinking before answering" or "answer directly"; the author also ran each of them with thinking switched off, so both modes are in the data. Perplexity's Sonar is left out because it searches the web before answering, so it would measure search results, not the model's own preference. Data were collected between 30 September and 10 October 2026, and since vendors keep updating their models, the conclusions describe the versions served in that window.

3. How a reply is mapped to a brand

The one-name questions ask for a single name, so only a clean single name counts. "Brooks" is counted; "I'd recommend Brooks because…", which adds a lead-in, or a list of several brands is not.

In the two-turn question the second reply is usually a paragraph. The author extracts the brand the model finally says it would choose: "These are all good, but if I had to pick one I'd go with Brooks" is recorded as Brooks. Sometimes the second reply is still just a list, or asks the user back, "Do you mostly run on roads or trails?", or declines to choose for the user; all of these count as no pick. No pick happens in 17% of two-turn replies, and later shares are computed only over replies that do pick a brand. Variant spellings and product names of one brand are merged into that brand.

The mapping is done by a program. The author hand-checked three samples, and the human reading agreed with the program on 87% to 97% of replies. In other words, the program occasionally gets one wrong, but rarely enough that it does not change the conclusions.

4. Finding one: the name that comes to mind is the leader, the considered pick is someone else

Start with the overall trend in Table 1. The more the question looks like "make the call for the user", the more categories change their panel answer: out of 42, "Choose" changes 6, "Recommend" changes 16, and the two-turn question changes 26, more than half. Individual models behave the same way. For each model, take the brand it most often gives when asked to "name" one; on average, 89% of a model's answers to "name" are that brand, which shows it is very stable when answering off the cuff; in the two-turn question, only 31% of its picks are still that brand. Once the question changes, the models do not scatter; they move together to a new consensus.

The brands named off the cuff are market leaders such as Google, Walmart, McDonald's, Facebook and Verizon. The picks are mostly smaller brands with a good reputation among insiders, such as DuckDuckGo, Aldi, Sierra Nevada, Reddit, Signal and Brooks; a few are more upmarket, such as Hermès over Chanel and Hyatt over Marriott.

The models do not throw out the brand they named; they just move it down. In the two-turn question, that brand appears in the model's own recommendation list 83% of the time, Among only the lists that include it, 60% put it first, yet just 37% end up picking it (counting the cases where it is not listed, the pick rate is 31%). Lists vary in length, with a median of 4 brands; drawing one at random from each list at its actual length would pick it about 30% of the time on average. Being listed first therefore gives it almost no advantage: the model votes for another brand on the list, and different models vote for the same one. Nor is this one lab's quirk: in each of the 12 labs with at least 3 models tested, the two-turn question changes that lab's consensus brand in 55% to 74% of categories.

Two side observations. First, labs favour their own AI assistant: asked to "recommend an AI assistant", Anthropic's models answer Claude 92% of the time, while other labs' models answer Claude 15% of the time. The favouritism does not extend to other products; Google's models recommend Google search and Chrome less often than other labs' models do. Second, the author spot-checked Claude Opus 5.5 with web search switched on. It answered the two-turn question without searching once, and its picks largely matched the panel, which suggests that production assistants also rely mainly on the model's own preferences for this kind of recommendation.

5. Finding two: taste forms in the step that learns "which answer is better"

To find the step where taste appears, one needs the same model's version after each training stage. A chat model typically goes through pretraining (giving the base model), SFT, DPO and RL, in that order.

OLMo 3 7B, OLMo 3.1 32B and Tulu 3 8B are open models that publish their weights after every step, while Nemotron 3.5 Lightning publishes only a base and a final version. For this part the author stops drawing repeated samples and reads directly the probability the model assigns to each brand name. One read tells how much the model wants to say that brand, so every stage can be measured on every category.

The first step looks at pretraining. A base model cannot yet answer "please recommend one", but it can finish half a sentence. Given the half-sentence "If you want a good web browser, I recommend", Tulu's base model continues with Firefox in 13 of 20 draws; given "A well-known web browser is", it continues with Chrome in 10. Across the 16 categories where the recommended brand differs from the named one, switching to advice phrasing raises the recommended brand's share in 7 to 10 categories and lowers it in at most 1. This shows that the brand picked later does not come from nowhere: in the text seen during pretraining, it already tends to appear next to words like "recommend".

The second step compares SFT with DPO. The author gives every stage the same multiple-choice question: a list of three brands, namely the named brand, the picked brand and another frequently recommended brand, followed by "Which one would you pick?". To read probabilities directly, the reply is made to start with "I'd pick", and the author checks which brand the model is most likely to put next. Base models tend to favour the first item, so each list is shown in forward and reverse order and the two results are averaged.

The table looks only at the probabilities of the named and the picked brand: the picked brand's share of their sum (the third brand is left out), averaged first over the two list orders and then over the 26 categories where the two differ. 50% would be a tie. After SFT all three models are still near their base level; after DPO all three rise clearly, and RL mostly keeps what DPO set. The brackets in the last column are a 90% interval, i.e. the range the rise moves within when the author resamples the categories many times. All three intervals are above zero, which shows the rise is not driven by a handful of categories. Nemotron has no intermediate versions and rises from 58% at base to 79% in its final version.

Which brands exactly does DPO lift? Across all 42 categories, the author compares two things: how likely the base model is to name a brand off the cuff, and how much DPO raises that brand when the model recommends. The more readily a brand is named off the cuff, the less DPO lifts it, in all three models. So DPO lifts the brands the base model found less obvious. But DPO does not push down famous brands across the board: in the 16 categories where the panel already recommends the famous brand, that brand does not fall after DPO. Since a three-item multiple-choice question is an artificial setup, the author re-ran the test on the SFT model's own recommendation lists, and the DPO step still adds 9 to 14 percentage points.

6. Finding three: the training data hardly mentions brands; the taste may come from the models that grade answers

The training data of OLMo 3 and Tulu 3 are public. The author searched them for requests that mention one of the 26 categories together with words like "recommend", "best" or "which". OLMo 3's DPO data has 260,000 pairs; the better answer names the picked brand only 3 times and the worse answer once, and Tulu 3 is just as sparse. So DPO did not learn these picks from brand examples, although indirect associations elsewhere in the data cannot be ruled out.

The same change appears outside brands. The author used a set of everyday categories, such as a tree, a smell or an animal, and compared "name one" with "choose one". Base models barely tell the two questions apart; after DPO, all three models lean toward the panel's answer when asked to "choose one". The clearest case is sauce: asked to choose a sauce, OLMo 3 7B picks pesto 5% of the time at base, 27% after SFT and 67% after DPO, while ketchup falls from 72% to 6%.

So who decides which answer is better? Half of OLMo 3's DPO data is graded by a large model acting as judge; the other half simply treats Qwen 3 32B's answer as the better one and a much smaller model's answer as the worse one. Tulu 3's pairs are graded by GPT-4o. The author therefore suspects the taste is passed on from these judge models. Tulu 3 has a single judge, GPT-4o, which allows a test: take the categories where GPT-4o recommends a different brand from the Tulu base model's favourite, put both brands in one list (asked in both orders), and ask the model before and after DPO which it picks. After DPO, the share picking GPT-4o's brand is 8 percentage points higher than before DPO, with a 90% interval of 2 to 14. This is only an approximate test, because the author queried today's GPT-4o, not the version that graded Tulu's data back then.

7. Limitations and what it means in practice

The author lists three main limitations. The questions are short and carry no personal context, so preferences measured on them may not reflect real use. The three models that can be inspected stage by stage come from the same organization and use similar training recipes, so the "DPO brings taste" path is shown only on them; the 102-model results, however, show the effect itself is not limited to these three. For OLMo 3 7B, if picks are sampled rather than read as probabilities, the change appears at RL instead of DPO. In addition, "judge models pass on taste" is the author's conjecture, and the 8 points on Tulu are only partial evidence. The author also notes that Claude Opus 5.5 was used to run probes, do the analysis and draft text, while Claude models are themselves among the subjects.

For post-training teams, the paper's warning is that preference training changes the model's judgments in areas its training data never covered, and the direction may come from the judge model. If a product has a recommendation or choice step, such as a shopping assistant, local services or travel planning, it is worth running a set of "name one vs. recommend one" questions at every training stage, watching the change around DPO, and checking separately which brands the judge model itself favours. For brands, the author's conclusion is that whether a model recommends you depends more on whether you fit the model's implicit "taste" than on how much the model knows about you. How a brand can build that association is left to future work.