RecSoup: Efficient Model Merging Approach for Controllable Multi-Objective Recommendation

RecSoup makes recommendation trade-offs adjustable by combining models trained for different objectives. It first prunes their parameter updates, then learns how a requested preference should determine merge coefficients. The goal is to avoid retraining or generating a large parameter set whenever the desired balance changes.

1. Problem and motivation

Recommendation systems balance objectives such as accuracy and diversity. A fixed compromise may be unsuitable across users, product surfaces or changing operating requirements.

Reranking can adjust a list, but only within the available candidates. Hypernetworks and diffusion-based approaches can generate preference-conditioned parameters, at the cost of auxiliary models and additional construction work. RecSoup asks whether existing objective-specialized checkpoints can provide the necessary building blocks.

Naive averaging is unreliable when objectives change overlapping parameters in competing ways. In addition, objective weights need not equal useful parameter-mixing weights. RecSoup separates choosing which updates to retain from learning how strongly to combine them.

2. Method: prune the basis, learn the coefficients

Each specialist starts from a common base model. Its difference from that base is a task vector: the parameter update associated with an objective. Merging adds a weighted combination of these updates back to the base, requiring aligned parameter structures and a compatible starting point.

The pruning stage combines two importance signals. Objective-specific importance measures a coordinate's squared update relative to the vector's total squared norm. Cross-objective relevance measures the magnitude of its interaction with updates at the same coordinate in other objectives.

The cross-objective term uses an absolute value deliberately. Opposite update directions can encode an important trade-off, so sign disagreement alone is not sufficient reason to discard them. The procedure seeks useful support rather than simply enforcing agreement.

Coordinates are ranked by the combined score. The method retains enough of them to preserve a chosen fraction of objective-specific update energy. Experiments use a norm-retention threshold of 0.9–0.95; this is not the fraction of parameter coordinates retained.

The second stage trains a lightweight gate from objective preferences to merge coefficients. The coefficients need not numerically match the preference vector. Only the gate is updated; the base and pruned task vectors remain frozen. A preference-weighted loss trains the mapping on a subset of data.

Pruning and adaptive merging solve distinct problems. The former identifies parameter support, while the latter selects a combination for the requested operating point.

3. Contributions and the scope of the argument

The contribution is not model merging in isolation. It is a recommendation-specific way to turn existing specialists into a controllable family of models, rather than one fixed merged checkpoint.

The authors examine accuracy- and diversity-specialized task vectors using PCA and find that updates are not concentrated in a small shared subspace. This motivates tailored merging, although PCA geometry alone does not establish all claims about nonlinear model behavior.

The theoretical comparison also has explicit assumptions. It treats the convex hull of two experts as the Pareto-optimal set and considers a parameter generator with a restricted Lipschitz constant. The result establishes a coverage limitation under those conditions; it does not show that arbitrary hypernetworks are universally inferior.

The empirical contribution is more directly actionable: pruning rules, coefficient mappings and recommendation backbones are evaluated separately through ablations.

4. Experiments: parameter construction is one stage

Experiments use MovieLens-1M and Amazon food and toy datasets, with three sequential-recommendation backbones. Accuracy–diversity is the main setting, supplemented by fairness and three-objective experiments. Accuracy specialists use pairwise ranking loss; diversity specialists encourage category novelty and reduce redundancy.

The table reports mean inference-time parameter construction latency in milliseconds. Standard deviations are omitted here for readability.

These measurements concern constructing parameters. They exclude the full feature, retrieval and ranking pipeline, and do not establish the same speedup over every generation method. Serving impact depends on how often preferences change and how long constructed parameters can be reused.

For quality, hypervolume summarizes the objective-space coverage of the trade-off frontier. Controllability checks whether changing a preference moves its corresponding metric in the intended direction. On MovieLens with SASRec, RecSoup achieves hypervolume 0.7090 versus 0.6536 for simple merging.

The table does not show universal dominance. In that same setting, an unpruned ablation has higher hypervolume, and some retrained operating points remain stronger. Pruning is an empirical trade-off whose value should be assessed across both quality and controllability.

Training and evaluation use sampled negatives. Full-catalog ranking may change the effects of popularity, candidate scale and parameter construction costs. Released code provides a basis for checking these conditions.

5. Limitations

The paper's complexity analysis acknowledges that full-parameter merging can be dominated by a large item-embedding table. Generating a few coefficients is cheap, but reading and combining many parameters still consumes bandwidth.

The evidence concerns offline trade-off quality and parameter construction efficiency. It does not include production CTR or revenue experiments. Specialist training is also an existing cost, even when only a small gate is trained in the final stage.

A serving system must decide whether to merge per request or cache a finite number of preference settings. Highly individualized, rapidly changing preferences have different concurrency and memory implications from shared operating points. The latency table does not settle that deployment choice.

6. Practical implications

Teams with aligned accuracy and diversity checkpoints can compare simple averaging, pruning plus averaging, and learned coefficients under identical backbones and candidates. This isolates whether each stage adds value.

Community recommendation can begin with a small set of interpretable exploration settings and cache their merged models. Per-request merging is a separate experiment that should include parameter bandwidth and tail latency.

Report accuracy, diversity, preference responsiveness and end-to-end cost independently. A model can improve average quality while responding inconsistently to control inputs. RecSoup's reusable parameter interface is valuable precisely because these properties can be tested separately.