05-gnn-extensions.md

Heterogeneous Graphs

• So far we have assumed that graphs are homogenous, i.e., have only one kind of node and relation, i.e. G = (V, E)
• Graphs in real world are often heterogeneous, i.e, they can have multiple entity types and relation types, i.e. G = (V, E, R, T)
• Examples of heterogeneous grophs -- Knowledge Graphs, Social Graphs, etc.
• Relational GCN (torch_geometric.nn.RGCN) -- network learns individual sets of weight matrices for each relation type in graph instead of one set of weights.

• Heterogeneous GCN (torch_geometric.nn.HeteroConv) -- network learns different weights for each unique (h, r, t) triple by type, using individual GNN layers.

hetero_conv = HeteroConv({
    ('paper', 'cites', 'paper'): GCNConv(-1, 64),
    ('author', 'writes', 'paper'): SAGEConv(-1, 64),
    ('paper', 'written_by', 'author'): GATConv(-1, 64),
}, aggr='sum')

---

Custom Layers

• Pytorch Geometric provides a comprehensive list of different kinds of Graph layers in torch_geometric.nn
• Operations in a graph layer can be thought of as a scatter-gather (or message-aggregation) operation.

• Custom graph layers possible by extending torch_geometric.nn.MessagePassing (see Creating Message Passing Networks for example code)
• Exercise: Create custom Graph Attention (GAT) layer.

---

Custom Datasets

• Pytorch Geometric and other platforms offer large array of useful graph datasets
  • PyG datasets
  • OGB datasets
• Dataset and DataLoader abstraction useful for working with PyG networks easy.
• Creating DataLoader
  • From Dataset -- DataLoader(dataset, ...)
  • From Data -- DataLoader([Data, ...], ...)
• Creating custom Dataset
  • In Memory Dataset -- extend torch_geometric.data.InMemoryDataset
  • On Disk Dataset -- extend torch_geometric.data.Dataset
• Example code to create custom dataset
• Video: Creating a Custom Dataset in Pytorch Geometric by DeepFindr
• Exercise: Create custom dataset

---