NGCF: Neural Graph
Collaborative Filtering

Chapter 1: Embedding Propagation Layer

NGCF's key idea is embedding propagation: instead of learning embeddings in isolation, refine them by passing messages along the interaction graph. Each propagation layer aggregates information from one additional hop, so after K layers, each user's embedding encodes information from their K-hop neighborhood in the interaction graph.

The propagation is bidirectional on the bipartite graph: users aggregate from items (what items do my neighbors like?), items aggregate from users (what kind of users buy me?). One layer of propagation:

There are three distinct terms here. First, W₁e_u: the user's own current embedding, projected. Second, W₁e_i summed over neighbors: what each neighboring item says about the user, projected by W₁. Third, W₂(e_i ∘ e_u): an interaction-aware message — the element-wise product of item and user embeddings, projected by W₂. This third term is NGCF's key novelty.

Chapter 2: Message Construction — Showcase

Let's dissect exactly what message item i sends to user u in NGCF. The message is:

Three components combined into one message:

1. Normalization 1/√(|N_u||N_i|): Symmetric normalization by degree — prevents high-degree nodes from dominating. Same as LightGCN's normalization. Has a clean spectral interpretation: it's the symmetric normalized Laplacian applied to the bipartite graph.
2. W₁e_i: The item's embedding projected to the aggregation space. This is the item's "intrinsic message" — what it says about itself, independent of who u is. This is what standard GCN uses.
3. W₂(e_i ∘ e_u): The interaction signal. The element-wise product highlights dimensions where both i and u have large magnitude. W₂ then projects this agreement signal. This term only exists in NGCF and is zero in LightGCN.

Chapter 3: Multi-Layer Propagation

NGCF stacks L propagation layers. Each layer aggregates one additional hop. After L layers, each user's embedding encodes the collaborative signal from all users and items within L hops in the interaction graph. The receptive field grows exponentially with layers.

After L layers, concatenate all layer embeddings to form the final representation:

The concatenation is important: it preserves all granularities of collaborative signal. Layer 0 is the intrinsic user representation. Layer L is the richest contextual representation. Using all layers together in the final prediction is consistently better than using only the last layer.

python
import torch
import torch.nn as nn

class NGCFLayer(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.W1 = nn.Linear(dim, dim, bias=False)
        self.W2 = nn.Linear(dim, dim, bias=False)
        self.act = nn.LeakyReLU(0.2)

    def forward(self, eu, ei_neighbors, norm_weights):
        # eu: [dim], ei_neighbors: [K, dim], norm_weights: [K]
        # W1 self-term
        self_msg = self.W1(eu)
        # Neighbor messages
        neighbor_sum = torch.zeros_like(eu)
        for k, (ei, w) in enumerate(zip(ei_neighbors, norm_weights)):
            msg = self.W1(ei) + self.W2(ei * eu)  # interaction-aware
            neighbor_sum += w * msg
        return self.act(self_msg + neighbor_sum)

Chapter 4: BPR Training

NGCF uses the same BPR (Bayesian Personalized Ranking) loss as LightGCN. Given user u, positive item i (interacted), and negative item j (random non-interaction), train the model to rank i above j in u's preferences:

Here ŷ_ui = e_u*ᵀ · e_i* is the predicted score using the final concatenated embeddings. λ regularizes all learnable parameters Θ = {E⁽⁰⁾, W₁⁽¹⁾, W₂⁽¹⁾, ..., W₁⁽ᴸ⁾, W₂⁽ᴸ⁾} — crucially, this includes the weight matrices at every layer.

The key training challenge for NGCF vs LightGCN: NGCF must regularize 2L weight matrices plus the initial embeddings. LightGCN only regularizes the initial embeddings. With L=3 layers, NGCF has 6 weight matrices (each d×d) plus embeddings. For d=64 and 100K users/items, this is about 6×64² = 25K parameters in weight matrices vs potentially millions in embeddings. The relative regularization between these groups is crucial.

Chapter 5: Results

NGCF is evaluated on three datasets: Gowalla (location check-ins, 29K users, 40K items, 1.3M interactions), Yelp-2018 (business reviews, 32K users, 38K items, 1.6M interactions), and Amazon-Book (13K users, 76K items, 294K interactions — sparser). Metrics: Recall@20 and NDCG@20.

NGCF outperforms all GCN-based baselines on Gowalla and Amazon-Book. The improvement over MF-BPR on Amazon-Book (+37.6% Recall) is particularly striking — Amazon-Book is sparser, where higher-order signals matter more. When data is dense, 1-hop MF works reasonably well. When sparse, multi-hop signals are essential.

NGCF uses embedding dimension d=64, L=3 layers, batch size 1024, and node dropout (randomly zeroing entire node embeddings during training) plus message dropout (randomly zeroing individual messages). These two dropout variants are important for regularizing the complex model — without them, NGCF with L=3 tends to overfit on smaller datasets.

Chapter 6: NGCF vs LightGCN

NGCF (2019) and LightGCN (2020) are natural counterparts — same task, same framework, diametrically opposite design philosophies. NGCF adds: feature transformation matrices W₁, W₂; nonlinear activation (LeakyReLU); interaction-aware messages. LightGCN removes: all weight matrices; all nonlinearities; self-connections. The performance reversal is sharp.

LightGCN outperforms NGCF by ~17% on Gowalla. But this doesn't mean NGCF's components are universally wrong — just wrong for pure collaborative filtering with ID embeddings. If you have side information (item text, user demographics), the weight matrices make sense: they project the rich features into embedding space. NGCF's architecture would be appropriate for a setting where nodes have meaningful content features.

Chapter 7: Connections

NGCF's lasting contribution isn't the specific architecture — LightGCN showed that simpler works better in the pure CF setting. It's the framework: model the interaction graph as a GNN, propagate collaborative signals through message passing, concatenate multi-layer representations. This framework is the foundation of every subsequent GNN-based recommender system.

The open question that followed NGCF/LightGCN: what about nodes with few interactions? Sparse users and long-tail items are the hardest recommendation cases — and the 2021-2022 work adds contrastive self-supervised objectives (SGL, SimGCL, NCL) to provide extra training signal for sparse nodes, addressing the limitation that pure BPR on sparse data doesn't generalize well to cold-start or long-tail scenarios.

"We develop a new framework NGCF for CF that explicitly encodes the collaborative signal in the form of high-order connectivities."
— Wang et al. (2019)