HARP: Training-Free Dual-Profile Agentic Communication for LLM-Based Recommendation

Zhen Tao, Xun Zhou, Xuhui Chen, Ziyue Qiao, Qingqiang Sun
27 minread4.35scoreNo. 411-05 Generative issue
Contents · 9 sections1What problem is it solving?2The overall flow3Item profiles: filled by rules, no LLM calls4Discounting neighbours by profile match5One reflection pass at warmup6Experiments: which part does the work?7Limitations as I see them8How a recommendation team could try it9Verdict
Rel 4Nov 4Prac 5Evid 4Time 5

Teams using LLMs for recommendation may want to read this: without training anything, it changes only how profiles are built, where they are used and when they are corrected, and gets clear gains. LLM-based recommenders often give the user a written profile, but items get no matching description; the user profile only goes into the reranker prompt, where it competes for length; and once written, the profile is never corrected. HARP (the paper's method) trains no parameters and fills all three gaps. It gives each item a profile with the same four fields as the user profile (preference, quality, context and audience), filled from metadata and reviews by rules. Before reranking, the existing method picks users and items with similar behaviour as collaborative neighbours and puts their text records into the reranker prompt. HARP discounts these neighbours' scores by how closely their profiles match the user's: the worse the match, the bigger the discount, and the less likely they are picked. During warmup, it runs one recommendation on the user's most recent past interaction, records whether it hit, and has the LLM reflect on that result to write or rewrite the profile. On four public datasets, Hit@1 beats the strongest published baseline by 6.9% to 24.0%, and ablations show that weighting neighbours by profile match contributes the most.

My take: the most useful part is not the "dual profile" label but a concrete engineering finding: putting the profile into the prompt barely helps, while using it to pick neighbours does. The reason is that the authors' ablation tests the two routes separately, and the gap is clear. The caveat is that the main table's baseline numbers are copied from the original papers rather than rerun in one codebase.

What problem is it solving?

The paper targets a family of LLM-agent recommenders. Each user and each item has an agent, and each agent keeps a free-text record, maintained by the LLM, describing that user or item; the paper calls it memory. Recommendation has two steps. The first picks collaborative neighbours: each user has a neighbour pool of nearby users and items, each with a behavioural score, and the top k are taken by score. The pool and the behavioural scores come from the existing method, and the paper takes them as given inputs. The second puts these neighbours' memories into a prompt for a frozen LLM, which reranks a candidate list. Full HARP also writes the user profile into the reranker prompt; item profiles are used only to compute the match and filter neighbours. The candidate list is given by the evaluation protocol; HARP does not retrieve candidates.

Here is an illustration of how behaviourally similar neighbours can mislead. Suppose the user asks for "somewhere to eat tonight":

Neighbour (illustrative)Behavioural scoreMatches the user profile?
A Sichuan restaurant0.8Yes: the user often goes for Sichuan food and late-night meals
A wedding photo studio0.8No: same shopping district, clicked by the same crowd

On behaviour alone, both neighbours score the same, and both send their memories into the prompt. The studio's memory does nothing for "where to eat tonight", and it uses up prompt length. The authors call these cross-category misleading neighbours.

The authors see three gaps in current methods. First, only users have profiles; items have no matching description, so there is no way to compare how alike the two sides are. Second, the user profile only goes into the reranking prompt, and prompt length is limited: every extra bit of profile means less neighbour memory. Third, the profile is written at warmup and never corrected from recommendation results.

The overall flow

The HARP pipeline. The upper half follows one user's request through the whole process; the lower half is the technical view: building profiles, scoring neighbours by how closely their profiles match, sending neighbour memories to the reranker, and one reflection pass at warmup
The HARP pipeline. The upper half follows one user's request through the whole process; the lower half is the technical view: building profiles, scoring neighbours by how closely their profiles match, sending neighbour memories to the reranker, and one reflection pass at warmup

HARP trains no parameters. It adds three parts around a frozen LLM, one for each gap: give items a profile too; rescore neighbours by how closely their profiles match; and reflect once at warmup to rewrite the user profile. Each is covered below.

Item profiles: filled by rules, no LLM calls

The user profile is written by one LLM call over the history. In the main experiments, that call is the warmup reflection described below. Recent history keeps more detail; older history is compressed more. The profile has seven fields, and HARP uses only the four that line up with items, called channels: preference, quality, context and audience.

Item profiles use the same four channels but make no LLM calls at all. Rules pull phrases from metadata and reviews, using one phrase dictionary per dataset. For a restaurant, the preference channel says what it serves, and the quality channel gives it a quality label based on stars and review count. This way the user's "likes Sichuan food" and the restaurant's "serves Sichuan food" sit in the same channel and can be compared.

Discounting neighbours by profile match

Once both sides have profiles, HARP computes a match score, which the paper calls homophily. The preference and audience channels are phrase lists, so for them it measures what share of the user's phrases are mentioned by the item's phrases. The quality channel checks whether the item's quality label meets the user's written threshold, such as a minimum star rating. The context channel checks whether location and setting phrases match: 1 on a match, 0 otherwise. For example, if the user's preference channel has three phrases (Sichuan food, late-night meals, small portions) and the restaurant mentions Sichuan food and late-night meals, that channel scores 2/3.

The share is taken over the user's phrases, so it asks: of what the user cares about, how much does the item mention? The four channels are summed with weights into an overall match H. The preference channel usually takes half the weight or more, and the other three split the rest evenly; these weights are fixed hyperparameters, not tuned on any evaluation metric. Between two users, the four channels are not used; only how much their preference-channel phrases overlap is compared.

Then, before the top k are taken, the match is used to discount the behavioural score from the existing method: new score = behavioural score × (α + (1 − α) × H). α is a floor, 0.4 between users and items and 0.3 between users, so the behavioural signal is never wiped out.

Back to the example. Both neighbours have a behavioural score of 0.8. The restaurant has H = 0.9, so its new score is 0.8 × (0.4 + 0.6 × 0.9) = 0.752. The studio has H = 0.2, so its new score is 0.8 × (0.4 + 0.6 × 0.2) = 0.416. Once the scores separate, a different set of neighbours from the pool makes the top k, and the reranker sees different memories.

Hoot saysSame behaviour score, but the mismatched neighbour is pushed outClose ✕
Two neighbours with the same behaviour score get different discounts by profile match.
Two rows: the Sichuan restaurant has behaviour score 0.8, profile match 0.9, discount factor 0.94, new score 0.752, picked; the photo studio has behaviour score 0.8, profile match 0.2, discount factor 0.52, new score 0.416, droppedSichuan placebehaviour 0.8Profile matchH = 0.9Discount factor0.94New score 0.752makes top kPhoto studiobehaviour 0.8Profile matchH = 0.2Discount factor0.52New score 0.416dropped
Two rows: the Sichuan restaurant has behaviour score 0.8, profile match 0.9, discount factor 0.94, new score 0.752, picked; the photo studio has behaviour score 0.8, profile match 0.2, discount factor 0.52, new score 0.416, droppedSichuan placebehaviour 0.8Profile matchH = 0.9Discount factor0.94New score 0.752makes top kPhoto studiobehaviour 0.8Profile matchH = 0.2Discount factor0.52New score 0.416dropped
  1. Illustrative: restaurant factor = 0.4 + 0.6 × 0.9 = 0.94, new score = 0.8 × 0.94 = 0.752
  2. Illustrative: studio factor = 0.4 + 0.6 × 0.2 = 0.52, new score = 0.8 × 0.52 = 0.416

Both neighbours have the same behaviour score. Comparing profiles, the Sichuan restaurant matches and the photo studio does not. The discount factor starts at 0.4 and gets closer to 1 the better the match. The studio's score drops by almost half, so it misses the top k. Its memory then never reaches the prompt. The numbers below are illustrative.

Sourcepaper §3.3 Eq. 7: new score = behaviour score × (α + (1 − α) × H), α = 0.4 between users and items

The authors explain why the profile should be used to pick neighbours rather than written into the prompt. First, prompt length has a cap; once it is full, every word of profile is one less word of neighbour memory. Second, the reranker is frozen and cannot learn to balance the two kinds of information. Third, if the wrong neighbours were picked, even a well-written prompt can hardly push the right answer up.

One reflection pass at warmup

Each user gets one warmup before testing. The system treats the interaction just before the test item as a practice question, runs the full recommendation once to see whether it is hit, and records the result. In the main experiments, the user has no profile yet at this point. By the paper's rule, when one side's profile is empty the match counts as a neutral 0.5, so profiles cannot tell neighbours apart in this pass. That interaction is already part of the user's history; it is not the one the model must guess at test time. Then one LLM call reflects. It gets the compressed history, any existing profile, and this trial's candidate list, the recommendation and the item the user actually chose, and from these writes or rewrites the profile.

The whole process happens once per user; the profile is not updated after every interaction.

Experiments: which part does the work?

The experiments use four public datasets: Amazon Books, Goodreads, MovieTV and Yelp. The reranker is a frozen gpt-4o-mini, with 16 neighbours each time. Against MemRec, the strongest published baseline, Hit@1 is 6.9% higher (relative) on Books, 17.2% higher on Goodreads and 24.0% higher on Yelp. These are relative gains. The gains are largest on Yelp and Goodreads; the authors think behavioural signal alone is weakest there, so picking neighbours by profile helps most. But this does not fit their own ablation: on Goodreads, the neighbour-picking step gains the least of the four datasets.

On Yelp, the gain shrinks as more top positions are counted: 24.0% at Hit@1, 14.3% at Hit@3 and 9.4% at Hit@5. This suggests HARP mainly moves the right answer from the top few up to first place.

The ablation says more. On 1,000 Yelp users, the authors compare five configurations:

ConfigurationItem profileProfile picks neighboursProfile in promptWarmup reflectionHit@1
Memory (baseline approach)NoNoNoNo.580
MonologueNoNoUser profileNo.578
ConsultYesNoBoth profilesNo.580
DialogueYesYesNoNo.635
HARPYesYesYesYes.643

Writing profiles into the prompt alone leaves Hit@1 flat: Monologue and Consult stay at .578 to .580, the same as Memory. Once the profile is used to pick neighbours, Dialogue jumps to .635, 5.7 points above Monologue, and a paired test finds this significant. The cleaner contrast is Consult versus Dialogue: both have both profiles and differ only in whether the profile goes into the prompt or picks neighbours, and Hit@1 differs by 5.5 points, also significant. Prompt injection and warmup reflection together add only 0.8 more points.

The five configurations on four datasets; red numbers are total gains over Memory. On Yelp, picking neighbours by profile (teal) gives the clearest jump; on the other datasets the gains are much smaller, and on Books full HARP is slightly below Dialogue
The five configurations on four datasets; red numbers are total gains over Memory. On Yelp, picking neighbours by profile (teal) gives the clearest jump; on the other datasets the gains are much smaller, and on Books full HARP is slightly below Dialogue

On the other datasets the gains are much smaller. Total Hit@1 gains over Memory are 6.3 points on Yelp, 3.4 on MovieTV, and 1.9 on both Books and Goodreads. Goodreads is the exception: 1.3 of its 1.9 points come from prompt injection plus reflection, and the neighbour-picking step gains little. The prompt injection plus reflection part is −0.4 points on Books, a small loss.

Limitations as I see them

First, all baseline numbers in the main table are copied from their own papers, not rerun with the same code and the same LLM. Part of the 6.9% to 24.0% gain may come from implementation differences. In the appendix the authors rerun MemRec in the same code with three other LLMs on 1,000-user subsets, and HARP still leads, but that is not the full test set of the main table.

Second, profile matching relies on substring matching and a hand-written phrase dictionary per dataset. Goodreads item metadata is mostly free text, so the rule-filled context and audience channels are sparse, which is one reason neighbour picking helps least there. A new domain needs a new dictionary.

Third, the ablations use only 1,000 users per dataset, and outside Yelp the gains over Memory are only 1.9 to 3.4 points. Prompt injection and reflection together are still negative on Books. The ablation does not isolate reflection, so it is unclear whether it is worth its LLM call.

How a recommendation team could try it

  1. If you already have an LLM reranker that puts neighbour memories into the prompt, first measure offline how many of the chosen neighbours clearly do not match the user profile.
  2. Build item profiles with the same channels as the user profile using rules, with no LLM calls; this is cheap.
  3. Add the α + (1 − α) × H discount at the neighbour retrieval step, starting with α between 0.3 and 0.4, and compare Hit@1.
  4. Add warmup reflection last, and confirm on a small slice of traffic that it actually helps.

Verdict

In one line: do not just stuff the profile into the prompt; use it to decide which neighbours get into the prompt, and the gain is larger.

DimensionAssessment
Problem importanceMedium-high. Prompt length is limited in LLM recommenders, so neighbour choice directly affects results
NoveltyMedium. The three parts are known ideas; the value is in cleanly testing where the profile should be used
EvidenceMedium. The ablation is clear, but baselines were not rerun and gains outside Yelp are small
Engineering costLow. No training, rule-based item profiles, one LLM reflection call per user at warmup
Worth trying?Yes. Discounting neighbours by profile match costs almost nothing; try that step first

How did this read for you?

One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.

Discussion

Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.

Loading the discussion…