Contents · 6 sections
1Where existing methods get stuck2Build an original fine view and a generated coarse view, then fuse during propagation3Walkthrough: how this baby bouncer becomes two views and then gets fused4What the results and ablations show on four datasets5What this means for people building graph multimodal recommenders6LimitationsSHARE
The listing says “massage 20 seconds, rest 5 seconds” — sparse clicks may never learn that: how DIGCN keeps the original features and generates one coarser view
In the Baby dataset there is a Fisher-Price Soothing Massage Bouncer. The fine-grained listing is thorough: it can give an infant a back massage and play soothing music, it has 3 music modes, the battery massage unit runs in 8-minute cycles, massaging for 20 seconds and resting for 5, plus a detachable massage toy and teether, a machine-washable pad, and a 25-pound weight limit. The photo is a real product shot, with texture and print still visible.
A recommender has to decide which users will like this item. Interactions are scarce: after 5-core filtering, four Amazon datasets are still 99.55% to 99.95% sparse. From so few clicks, the model has to learn both category cues such as “this is a baby bouncer that massages and is machine-washable,” and enough detail to tell two similar-looking bouncers apart. That is not easy.
This paper is from the University of Hong Kong, the Hong Kong Polytechnic University, the University of British Columbia, the Hong Kong University of Science and Technology, and the University of Luxembourg. It proposes DIGCN (Dual-granularity raw feature empowered Intermediate fusion Graph Convolutional Network). It targets two problems in multimodal recommendation: multi-granularity representations are hard to train from sparse interactions, and fusion that only aligns modalities can squeeze out modality-specific information. Among existing methods, MambaRec builds multi-scale representations during training to capture fine-grained semantics; MMSSL (a recommender that uses contrastive learning to align modalities with collaborative signals) and BM3 (another contrastive model of modality relations) pull different modalities closer. Most recent work also fuses after graph convolution — each modality is propagated on its own, then merged at the end, which is late fusion. DIGCN takes a step back: before training, a large model generates one coarse-grained text and image as a complement to the original fine-grained features; an intermediate-fusion graph convolutional network (GCN) then, during propagation, uses the ID as an anchor and merges modalities with dynamic weights. On four standard Amazon datasets, Recall@20 rises 5.17% to 8.81% relative to the strongest baseline, MambaRec, on Sports, Pet, and Office. On Baby, Table 2 gives 0.1052 versus 0.1013, but Section 4.2’s 3.54% relative gain does not match a recomputation from the table, so Baby is not included in that range. The advantage also holds for sparser users and for cold-start items.
Keep the original fine-grained content, generate one extra coarse-grained view with a large model before training, then do intermediate fusion with dynamic weight correction during graph convolution, and you can improve both multi-granularity representation and modality fusion in sparse recommendation; this conclusion so far rests only on four 5-core Amazon datasets.
Where existing methods get stuck
If multi-granularity has to be learned in training, the already sparse user–item interactions must also supervise a fine-versus-coarse distinction. The signal is thin, so the representations are hard to train stably. If fusion only does alignment, the objective is to pull modalities closer, and useful information that is not shared across modalities gets squeezed out with it.
The timing of fusion has a cost as well. Early fusion knits modalities together before they enter the GCN, which can wash out structure inside each modality. Late fusion waits until graph convolution is finished, and then misses cross-modal signals that only appear during propagation. DIGCN’s proposal is to prepare granularity on the raw features first, and to do fusion in the middle of message passing at each layer.
Build an original fine view and a generated coarse view, then fuse during propagation
The coarse view is not meant to replace the original listing and photo. For text, the prompt is “Please generate a coarse-grained version for this text”; for images, the prompt asks for a coarser visual reference from the original image and the item title. In the implementation, text uses GPT-4o and images use DALL-E-3. The authors generate only one extra coarse view and pair it with the original fine view, both to limit cost and to reduce fluctuation from large-model randomness. The original listing and photo, and the generated coarse listing and image, first go through a pretrained extractor and a linear map into the same dimension, and only then enter graph propagation. The behavioral ID embedding shares one initialization across the coarse and fine branches, but each branch keeps its own anchor state.
Fusion happens inside the intermediate fusion GCN. At each layer, the ID and each modality are first propagated from neighbors, producing a provisional ID anchor and the text and image representations that have arrived. A dynamic correction weight is then computed: the weight is 1 minus the cosine similarity between that modality and this provisional anchor. Cosine lies in [-1, 1], so the weight lies in [0, 2]. The modalities are added back to the ID anchor with these weights, and the denominator is 1 plus the sum of the weights, so no single modality can take over.
Illustration: in one layer, the text representation has cosine 0.8 with the provisional ID anchor, and the image has 0.2. The text weight is 1-0.8=0.2, and the image weight is 1-0.2=0.8. The denominator is 1+0.2+0.8=2. A modality already close to the ID contributes less; a modality that differs more contributes more. From layer to layer, who should speak more can change with the representations; there is no need to fix a set of modality weights in advance.
Finally the representations from all layers and both granularities are read out and combined with a learned weight. Beside this sits a single item–item semantic graph: cosine similarities of the raw features are summed across modalities, the nearest neighbors are kept, and the graph stays frozen during training, with only one-hop aggregation. Most prior work builds one graph per modality and aggregates over several hops; keeping one graph and one hop also costs less. The main loss is BPR. Contrastive alignment is not a required component, but it can be stacked on when needed.

Walkthrough: how this baby bouncer becomes two views and then gets fused
The walkthrough uses this Baby item from the paper’s Figure 6. The listing text comes from the paper’s case; the weight calculation is a labeled illustration.
Step 1, keep the fine view. The fine-grained listing is the full product description, including “gentle back massage,” “soothing music,” 3 music modes, and the exact interval “massage 20 seconds, rest 5 seconds” inside an 8-minute cycle, plus the detachable toy, the machine-washable pad, and the 25-pound limit. The original image keeps texture and print. These details later help separate items that look alike.
Step 2, generate the coarse view. The large model shortens the same listing, keeps category-level cues such as “Gentle back massage,” “soothing music,” and “machine-washable pad,” drops the exact “massage 20 seconds, rest 5 seconds,” and turns the text into a denser short description. The coarse image weakens texture and emphasizes color, shape, and structure. The item’s identity is still there; only low-level detail has been stripped.
Step 3, the two branches propagate separately. The fine view and the coarse view each go through the intermediate fusion GCN, with the ID as the anchor. User-side modality representations are the average of the features of items that are one-hop training neighbors. At this point the bouncer’s text, image, and ID have not been prematurely tied into a single vector.
Step 4, fuse dynamically at each layer. Using the same illustrative numbers: when text is already close to the ID it contributes only 0.2, and when the image is farther away it contributes 0.8, then both are normalized by the denominator 2. Cross-modal signals are written into the ID anchor during propagation, while the text and image representations themselves still update by modality and are not first pulled into the same direction by an alignment objective.
Step 5, read out and score. The fused ID anchors from all layers are averaged first; a learnable Sigmoid weight (initialized so the two views each start at one half) then combines the coarse and fine views into the final user and item vectors, and an inner product is used for BPR ranking. An item can also pick up information from similar neighbors on that one-hop semantic graph. This bouncer therefore carries both coarse cues such as “can massage, can be machine-washed,” and fine cues such as the 20-second / 5-second interval and the specific toys.

If only the fine view is kept, the model still has semantics from the original text and image, but it lacks the explicitly summarized perspective that the generated coarse view provides. If only the generated coarse view is used to replace the original text and image, the ability to tell similar items apart is lost. The paper later checks this with control settings.
What the results and ablations show on four datasets
The experiments use four 5-core Amazon review sets — Baby, Sports, Pet, and Office — with interactions split 8:1:1 at random. The metrics are Recall and NDCG, reported as the mean over five random seeds. DIGCN and all baselines are implemented in the MMRec framework, with an initial dimension of 64. The table below looks only at Recall@20, against the strongest baseline, MambaRec.
| Dataset | MambaRec | DIGCN | Relative gain |
|---|---|---|---|
| Baby | 0.1013 | 0.1052 | — |
| Sports | 0.1084 | 0.1140 | 5.17% |
| Pet | 0.1589 | 0.1729 | 8.81% |
| Office | 0.1449 | 0.1543 | 6.49% |
(The numbers come from Table 2 and Section 4.2; DIGCN is p<0.05 against every baseline on every dataset and metric.) The Baby relative-gain cell is left blank because Section 4.2 writes 3.54%, which does not match the about 3.85% recomputed from Table 2’s four-decimal figures.
This shows that dual-granularity raw features and ID-anchored intermediate fusion bring a statistically significant ranking gain on all four sets. The relative gains are larger on Pet and Office, which the authors link to the longer item descriptions in those two sets: coarse-grained abstraction is more likely to keep the core semantics. DIGCN uses no alignment loss, and already outperforms contrastive baselines such as MMSSL, BM3, and MambaRec.
The ablation compares four variants: drop the coarse-grained features, replace intermediate fusion with late fusion, drop the item–item graph, and keep the fusion structure but use fixed uniform weights. On all four datasets the full model is best, and removing any piece causes a clear drop. Dual granularity is especially useful on Pet and Office, which matches the observation above about longer descriptions. The uniform-weight variant shows that dynamic correction itself contributes, not just the skeleton of “fuse during propagation.”
Alignment is not required. After adding three InfoNCE variants, the DIGCN base still rises a little: aligning modalities only, or aligning the two granularities only, both help, and doing both together (+MG-Align) is best. On Baby Recall@20, for example, the number goes from 0.1052 to 0.1063. So these tasks are suitable as optional add-ons, not as a precondition for DIGCN to work.
When users are grouped by the number of training interactions, DIGCN is best in every bin, and the margin is larger in the sparsest groups, (0-5] and (5-10]. For item cold start, 20% of items have all of their training interactions removed: DIGCN’s Recall@20 on the four sets is 0.0633, 0.0669, 0.0552, and 0.0544, while MambaRec in the same setting is 0.0611, 0.0633, 0.0497, and 0.0486. This shows that the pre-constructed content semantics still help when the collaborative signal is weaker, and even when an item has no training interactions at all.
A key control is this: if other strong baselines use only the generated features to replace the original text and image (-C), results usually get worse; using generated and original features together (-D) is what shows that the gain comes from combining the two views. So DIGCN’s benefit is not that the generated content is better than the original on its own.

Switching among GPT-4o, Qwen3-Max, Gemini-2.5-Pro, DeepSeek-R1, and different image generators changes Recall@20 only slightly, and the standard deviation over five rewritten prompts is low. Open-source combinations are competitive as well. The default remains two views: going to 3 or 4 views raises the mean a little more, but the standard deviation grows, so the authors keep two views as the default. Intermediate fusion works best at L=3 on every set. Training itself is not heavier: on Baby, DIGCN takes 2.99 seconds per epoch and 2.40 GB of memory, versus 9.01 seconds and 4.88 GB for MambaRec. Large-model generation is offline preprocessing: each modality is called once per item, and the features can be reused across training runs.
What this means for people building graph multimodal recommenders
Multi-granularity can be built as raw features first, and fusion can happen in the middle of graph propagation; you do not have to train fine and coarse representations on sparse interactions from the start, and you do not have to treat alignment as the only way to fuse.
If item listings are long and interactions are few, you can first ask a large model for a shorter coarse description and a coarser image that emphasizes structure, and treat them explicitly as a complement, not a replacement. At fusion time you can look at how close each modality is to the ID anchor in that layer, then decide how much to add, without fixing modality weights in advance. If you already have a contrastive alignment loss, you can add it as an optional plugin on top of DIGCN, and first check whether the base without it is already enough. Before serving, the generated features still need a check for factual errors and harmful content, because they inherit the generator’s own biases.
Limitations
All main results are on four 5-core Amazon review datasets, with a random 8:1:1 split, and there is no validation in another domain or in an online A/B test. The Recall@20 gain over MambaRec runs from 5.17% to 8.81% on Sports, Pet, and Office, and varies by dataset, so it should not be read as a fixed universal lift. Baby’s relative gain is not included in that range, because Table 2 and Section 4.2 do not agree.
The ablations show that removing each piece causes a drop, and the complete model is best on all four datasets. The extra gain from stacking alignment is also small. Generated coarse views inherit the underlying generator’s biases and factual errors, so the fine view must be kept; the paper also notes that factual consistency, harmful content, and domain fairness should be checked before deployment.
How did this read for you?
Hoot made a note. Thanks for the feedback.
One tap, no sign-in; tap again to undo. Only anonymous counts are kept, never who you are.
Related papersMore from Paper Daily
Personalization · 2026-08-19UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale RecommendationDeep dive: UniDot: A Unified Network for Sequence Modeling and Feature Interaction in Large-scale Recommendation.Personalization · 2026-10-16Bridging the Topological Divide: Dual-Branch Graph Distillation with Semantic Priors for Robust Cold-Start RecommendationDualGCN gives interaction-free items a representation through semantic-anchored graph distillation. A separate student learns directions, while inference restores vector magnitude. On Sports cold items, Recall@20 exceeds DropoutNet by 20.8%.
Personalization · 2026-10-29Rethinking Positional Encoding and Interaction Granularity for CTR PredictionIn unified Transformer CTR models like OneTrans, the authors tried six positional encodings on Kuaishou's KuaiRand data, and none beat using no positional encoding at all. Building on this, the Wuhan University and Tencent paper proposes TRIP (the authors' name for the whole method). Each behavior encodes both its rank in the sequence and how long before this prediction it happened. Behaviors are…
Discussion
Add experiment details, reproduction notes or a different take. Requires a GitHub sign-in.
Loading the discussion…