Graph neural networks

In short: Graph neural networks (GNN) process data made of nodes and connections: social networks, molecules, roads, knowledge graphs. Each node gathers information from its neighbours.

In more detail: Images have a grid, texts a row, but much data is networks without a fixed shape. GNNs work on them without the order of neighbours mattering.

In detail

• Message passing: in each layer every node sends its vector to neighbours, each node aggregates what it received (sum, mean, maximum) and computes a new vector.
• Reach: after k layers a node knows its k-hop neighbourhood.
• Tasks: node classification (fraudster?), edge prediction (friendship, recommendation), graph classification (molecule toxic?).
• Variants: GCN, GraphSAGE, GAT (with attention).
• Applications: drug discovery, traffic forecasting (Google Maps), recommendations, fraud detection.
• Related: transformers are at their core GNNs on a complete graph in which every token sees every other.
• Limits: too many layers smooth all nodes into indistinguishability (“over-smoothing”).

See also: Self-attention in detail, Recommender systems