Skip to content

GitLab

Unverified Commit 1a6dd735 authored by Minjie Wang's avatar Minjie Wang Committed by GitHub
Browse files

[Doc] other colab tutorials (#5189) 

* Created using Colaboratory

* Created using Colaboratory

* Created using Colaboratory

* add link

* Created using Colaboratory

* add link
parent 2c143daf
Hide whitespace changes
Inline Side-by-side
Showing with 689 additions and 11 deletions
docs/source/notebooks/sparse/gcn.nblink 0 → 100644
View file @ 1a6dd735
{
"path": "../../../../notebooks/sparse/gcn.ipynb"
}
docs/source/notebooks/sparse/hgnn.nblink 0 → 100644
View file @ 1a6dd735
{
"path": "../../../../notebooks/sparse/hgnn.ipynb"
}
docs/source/notebooks/sparse/index.rst
View file @ 1a6dd735
...@@ -8,5 +8,7 @@ TODO(minjie): intro for the new library. ...@@ -8,5 +8,7 @@ TODO(minjie): intro for the new library.
:titlesonly: :titlesonly:
quickstart.nblink quickstart.nblink
gcn.nblink
graph_diffusion.nblink graph_diffusion.nblink
hgnn.nblink
graph_transformer.nblink graph_transformer.nblink
notebooks/sparse/gcn.ipynb 0 → 100644
View file @ 1a6dd735
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"source": [
"# Building a Graph Convolutional Network Using Sparse Matrices\n",
"\n",
"This tutorial illustrates step-by-step how to write and train a Graph Convolutional Network ([Kipf et al. (2017)](https://arxiv.org/abs/1609.02907)) using DGL's sparse matrix APIs.\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dmlc/dgl/blob/master/notebooks/sparse/gcn.ipynb) [![GitHub](https://img.shields.io/badge/-View%20on%20GitHub-181717?logo=github&logoColor=ffffff)](https://github.com/dmlc/dgl/blob/master/notebooks/sparse/gcn.ipynb)"
],
"metadata": {
"id": "_iqWrPwxtZr6"
}
},
{
"cell_type": "code",
"source": [
"# Install required packages.\n",
"import os\n",
"import torch\n",
"os.environ['TORCH'] = torch.__version__\n",
"os.environ['DGLBACKEND'] = \"pytorch\"\n",
"\n",
"# TODO(Steve): change to stable version.\n",
"# Uncomment below to install required packages.\n",
"#!pip install --pre dgl -f https://data.dgl.ai/wheels-test/repo.html > /dev/null\n",
"\n",
"try:\n",
" import dgl\n",
" installed = True\n",
"except ImportError:\n",
" installed = False\n",
"print(\"DGL installed!\" if installed else \"Failed to install DGL!\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "FTqB360eRvya",
"outputId": "f5cfb27c-82ba-43af-fb58-3fdc62cec193"
},
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"DGL installed!\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Graph Convolutional Layer\n",
"\n",
"Mathematically, the graph convolutional layer is defined as:\n",
"$$f(X^{(l)}, A) = \\sigma(\\hat{D}^{-\\frac{1}{2}}\\hat{A}\\hat{D}^{-\\frac{1}{2}}X^{(l)}W^{(l)})$$\n",
"with $\\hat{A} = A + I$, where $A$ denotes the adjacency matrix and $I$ denotes the identity matrix, $\\hat{D}$ refers to the diagonal node degree matrix of $\\hat{A}$ and $W^{(l)}$ denotes a trainable weight matrix. $\\sigma$ refers to a non-linear activation (e.g. relu).\n",
"\n",
"The code below shows how to implement it using the `dgl.sparse` package. The core operations are:\n",
"* `dgl.sparse.identity` creates the identity matrix $I$.\n",
"* The augmented adjacency matrix $\\hat{A}$ is then computed by adding the identity matrix to the adjacency matrix $A$.\n",
"* `A_hat.sum(0)` aggregates the augmented adjacency matrix $\\hat{A}$ along the first dimension which gives the degree vector of the augmented graph.\n",
"* `dgl.sparse.diag` creates the diagonal degree matrix $\\hat{D}$ from the degree vector.\n",
"* `D_hat @ A_hat @_hat` computes the convolution matrix which is then multiplied by the linearly transformed node features."
],
"metadata": {
"id": "r3qB1atg_ld0"
}
},
{
"cell_type": "code",
"source": [
"import torch\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"\n",
"import dgl.sparse as dglsp\n",
"\n",
"class GCNLayer(nn.Module):\n",
" def __init__(self, in_size, out_size):\n",
" super(GCNLayer, self).__init__()\n",
" self.W = nn.Linear(in_size, out_size)\n",
"\n",
" def forward(self, A, X):\n",
" ########################################################################\n",
" # (HIGHLIGHT) Compute the symmetrically normalized adjacency matrix with\n",
" # Sparse Matrix API\n",
" ########################################################################\n",
" A_hat = A + dglsp.identity(A.shape)\n",
" D_hat = dglsp.diag(A_hat.sum(0)) ** -0.5\n",
" return D_hat @ A_hat @ D_hat @ self.W(X)"
],
"metadata": {
"id": "Y4I4EhHQ_kKb"
},
"execution_count": 2,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"A Graph Convolutional Network is then defined by stacking this layer."
],
"metadata": {
"id": "bvP7O2IwV_c7"
}
},
{
"cell_type": "code",
"source": [
"# Create a GCN with the GCN layer.\n",
"class GCN(nn.Module):\n",
" def __init__(self, in_size, out_size, hidden_size):\n",
" super(GCN, self).__init__()\n",
" self.conv1 = GCNLayer(in_size, hidden_size)\n",
" self.conv2 = GCNLayer(hidden_size, out_size)\n",
"\n",
" def forward(self, A, X):\n",
" X = self.conv1(A, X)\n",
" X = F.relu(X)\n",
" return self.conv2(A, X)"
],
"metadata": {
"id": "BHX3vRjDWJTO"
},
"execution_count": 3,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Training the GCN\n",
"\n",
"We then train the GCN model on the Cora dataset for node classification. Note that since the model expects an adjacency matrix as the first argument, we first construct the adjacency matrix from the graph using the `dgl.sparse.from_coo` API which returns a DGL `SparseMatrix` object."
],
"metadata": {
"id": "2Qw7fTdGNnEp"
}
},
{
"cell_type": "code",
"source": [
"def evaluate(g, pred):\n",
" label = g.ndata[\"label\"]\n",
" val_mask = g.ndata[\"val_mask\"]\n",
" test_mask = g.ndata[\"test_mask\"]\n",
"\n",
" # Compute accuracy on validation/test set.\n",
" val_acc = (pred[val_mask] == label[val_mask]).float().mean()\n",
" test_acc = (pred[test_mask] == label[test_mask]).float().mean()\n",
" return val_acc, test_acc\n",
"\n",
"def train(model, g):\n",
" features = g.ndata[\"feat\"]\n",
" label = g.ndata[\"label\"]\n",
" train_mask = g.ndata[\"train_mask\"]\n",
" optimizer = torch.optim.Adam(model.parameters(), lr=1e-2, weight_decay=5e-4)\n",
" loss_fcn = nn.CrossEntropyLoss()\n",
"\n",
" # Preprocess to get the adjacency matrix of the graph.\n",
" src, dst = g.edges()\n",
" N = g.num_nodes()\n",
" A = dglsp.from_coo(dst, src, shape=(N, N))\n",
"\n",
" for epoch in range(100):\n",
" model.train()\n",
"\n",
" # Forward.\n",
" logits = model(A, features)\n",
"\n",
" # Compute loss with nodes in the training set.\n",
" loss = loss_fcn(logits[train_mask], label[train_mask])\n",
"\n",
" # Backward.\n",
" optimizer.zero_grad()\n",
" loss.backward()\n",
" optimizer.step()\n",
"\n",
" # Compute prediction.\n",
" pred = logits.argmax(dim=1)\n",
"\n",
" # Evaluate the prediction.\n",
" val_acc, test_acc = evaluate(g, pred)\n",
" if epoch % 5 == 0:\n",
" print(\n",
" f\"In epoch {epoch}, loss: {loss:.3f}, val acc: {val_acc:.3f}\"\n",
" f\", test acc: {test_acc:.3f}\"\n",
" )\n",
"\n",
"\n",
"# Load graph from the existing dataset.\n",
"dataset = dgl.data.CoraGraphDataset()\n",
"g = dataset[0]\n",
"\n",
"# Create model.\n",
"feature = g.ndata['feat']\n",
"in_size = feature.shape[1]\n",
"out_size = dataset.num_classes\n",
"gcn_model = GCN(in_size, out_size, 16)\n",
"\n",
"# Kick off training.\n",
"train(gcn_model, g)"
],
"metadata": {
"id": "5Sp1B1_QHgC2",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "8ea64434-1b03-4c4e-8a07-752b438c9603"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Downloading /root/.dgl/cora_v2.zip from https://data.dgl.ai/dataset/cora_v2.zip...\n",
"Extracting file to /root/.dgl/cora_v2\n",
"Finished data loading and preprocessing.\n",
" NumNodes: 2708\n",
" NumEdges: 10556\n",
" NumFeats: 1433\n",
" NumClasses: 7\n",
" NumTrainingSamples: 140\n",
" NumValidationSamples: 500\n",
" NumTestSamples: 1000\n",
"Done saving data into cached files.\n",
"In epoch 0, loss: 1.957, val acc: 0.122, test acc: 0.130\n",
"In epoch 5, loss: 1.932, val acc: 0.200, test acc: 0.210\n",
"In epoch 10, loss: 1.897, val acc: 0.386, test acc: 0.433\n",
"In epoch 15, loss: 1.851, val acc: 0.518, test acc: 0.571\n",
"In epoch 20, loss: 1.788, val acc: 0.542, test acc: 0.569\n",
"In epoch 25, loss: 1.706, val acc: 0.710, test acc: 0.729\n",
"In epoch 30, loss: 1.606, val acc: 0.746, test acc: 0.780\n",
"In epoch 35, loss: 1.491, val acc: 0.756, test acc: 0.787\n",
"In epoch 40, loss: 1.366, val acc: 0.770, test acc: 0.789\n",
"In epoch 45, loss: 1.237, val acc: 0.768, test acc: 0.789\n",
"In epoch 50, loss: 1.111, val acc: 0.772, test acc: 0.795\n",
"In epoch 55, loss: 0.995, val acc: 0.770, test acc: 0.796\n",
"In epoch 60, loss: 0.891, val acc: 0.772, test acc: 0.801\n",
"In epoch 65, loss: 0.801, val acc: 0.776, test acc: 0.806\n",
"In epoch 70, loss: 0.723, val acc: 0.774, test acc: 0.807\n",
"In epoch 75, loss: 0.657, val acc: 0.780, test acc: 0.810\n",
"In epoch 80, loss: 0.600, val acc: 0.782, test acc: 0.811\n",
"In epoch 85, loss: 0.551, val acc: 0.788, test acc: 0.811\n",
"In epoch 90, loss: 0.510, val acc: 0.788, test acc: 0.814\n",
"In epoch 95, loss: 0.475, val acc: 0.788, test acc: 0.819\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Check out the full example script [here](https://github.com/dmlc/dgl/blob/master/examples/sparse/gcn.py)."
],
"metadata": {
"id": "yQnJZvE9ZduM"
}
}
]
}
\ No newline at end of file
notebooks/sparse/graph_diffusion.ipynb
View file @ 1a6dd735
...@@ -31,7 +31,7 @@ ...@@ -31,7 +31,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": null,
"metadata": { "metadata": {
"id": "F6eQWmWn7lqh", "id": "F6eQWmWn7lqh",
"colab": { "colab": {
...@@ -113,7 +113,7 @@ ...@@ -113,7 +113,7 @@
"id": "_TnCECJmBKJE", "id": "_TnCECJmBKJE",
"outputId": "740780d6-cdf2-40b2-a2f3-ffc83af974d2" "outputId": "740780d6-cdf2-40b2-a2f3-ffc83af974d2"
}, },
"execution_count": 2, "execution_count": null,
"outputs": [ "outputs": [
{ {
"output_type": "stream", "output_type": "stream",
...@@ -160,7 +160,7 @@ ...@@ -160,7 +160,7 @@
"id": "JyzctBGaC_O5", "id": "JyzctBGaC_O5",
"outputId": "854e5803-5437-4503-930b-96275ef604d5" "outputId": "854e5803-5437-4503-930b-96275ef604d5"
}, },
"execution_count": 3, "execution_count": null,
"outputs": [ "outputs": [
{ {
"output_type": "stream", "output_type": "stream",
...@@ -205,7 +205,7 @@ ...@@ -205,7 +205,7 @@
"metadata": { "metadata": {
"id": "DXb0uKqXDZKb" "id": "DXb0uKqXDZKb"
}, },
"execution_count": 4, "execution_count": null,
"outputs": [] "outputs": []
}, },
{ {
...@@ -250,7 +250,7 @@ ...@@ -250,7 +250,7 @@
}, },
"outputId": "c79e9be0-afbf-4766-e8cb-da365a86c38c" "outputId": "c79e9be0-afbf-4766-e8cb-da365a86c38c"
}, },
"execution_count": 5, "execution_count": null,
"outputs": [ "outputs": [
{ {
"output_type": "execute_result", "output_type": "execute_result",
...@@ -5784,7 +5784,7 @@ ...@@ -5784,7 +5784,7 @@
"metadata": { "metadata": {
"id": "__U3Hsp_S0SR" "id": "__U3Hsp_S0SR"
}, },
"execution_count": 6, "execution_count": null,
"outputs": [] "outputs": []
}, },
{ {
...@@ -5891,7 +5891,7 @@ ...@@ -5891,7 +5891,7 @@
}, },
"outputId": "4c0bddc5-643c-4b67-99d3-a79a9c6a695f" "outputId": "4c0bddc5-643c-4b67-99d3-a79a9c6a695f"
}, },
"execution_count": 7, "execution_count": null,
"outputs": [ "outputs": [
{ {
"output_type": "stream", "output_type": "stream",
...@@ -5925,8 +5925,7 @@ ...@@ -5925,8 +5925,7 @@
"Check out the full example script [here](https://github.com/dmlc/dgl/blob/master/examples/sparse/sign.py). Learn more about how graph diffusion is used in other GNN models:\n", "Check out the full example script [here](https://github.com/dmlc/dgl/blob/master/examples/sparse/sign.py). Learn more about how graph diffusion is used in other GNN models:\n",
"\n", "\n",
"* *Predict then Propagate: Graph Neural Networks meet Personalized PageRank* [paper](https://arxiv.org/abs/1810.05997) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/appnp.py)\n", "* *Predict then Propagate: Graph Neural Networks meet Personalized PageRank* [paper](https://arxiv.org/abs/1810.05997) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/appnp.py)\n",
"* *Combining Label Propagation and Simple Models Out-performs\n", "* *Combining Label Propagation and Simple Models Out-performs Graph Neural Networks* [paper](https://arxiv.org/abs/2010.13993) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/c_and_s.py)\n",
"Graph Neural Networks* [paper](https://arxiv.org/abs/2010.13993) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/c_and_s.py)\n",
"* *Simplifying Graph Convolutional Networks* [paper](https://arxiv.org/abs/1902.07153) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/sgc.py)\n", "* *Simplifying Graph Convolutional Networks* [paper](https://arxiv.org/abs/1902.07153) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/sgc.py)\n",
"* *Graph Neural Networks Inspired by Classical Iterative Algorithms* [paper](https://arxiv.org/pdf/2103.06064.pdf) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/twirls.py)" "* *Graph Neural Networks Inspired by Classical Iterative Algorithms* [paper](https://arxiv.org/pdf/2103.06064.pdf) [code](https://github.com/dmlc/dgl/blob/master/examples/sparse/twirls.py)"
], ],
...@@ -5935,4 +5934,4 @@ ...@@ -5935,4 +5934,4 @@
} }
} }
] ]
} }
\ No newline at end of file
notebooks/sparse/graph_transformer.ipynb
View file @ 1a6dd735
...@@ -398,6 +398,15 @@ ...@@ -398,6 +398,15 @@
"# Kick off training.\n", "# Kick off training.\n",
"train(model, dataset, evaluator, dev)" "train(model, dataset, evaluator, dev)"
] ]
},
{
"cell_type": "markdown",
"source": [
"*Check out the full example script [here](https://github.com/dmlc/dgl/blob/master/examples/sparse/graph_transformer.py).*"
],
"metadata": {
"id": "mifdq1Ftc-Nz"
}
} }
], ],
"metadata": { "metadata": {
...@@ -418,4 +427,4 @@ ...@@ -418,4 +427,4 @@
}, },
"nbformat": 4, "nbformat": 4,
"nbformat_minor": 0 "nbformat_minor": 0
} }
\ No newline at end of file
notebooks/sparse/hgnn.ipynb 0 → 100644
View file @ 1a6dd735
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"gpuClass": "standard"
},
"cells": [
{
"cell_type": "markdown",
"source": [
"# Hypergraph Neural Networks\n",
"\n",
"This tutorial illustrates what is hypergraph and how to build a Hypergraph Neural Network using DGL's sparse matrix APIs.\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dmlc/dgl/blob/master/notebooks/sparse/hgnn.ipynb) [![GitHub](https://img.shields.io/badge/-View%20on%20GitHub-181717?logo=github&logoColor=ffffff)](https://github.com/dmlc/dgl/blob/master/notebooks/sparse/hgnn.ipynb)"
],
"metadata": {
"id": "eiDu3XgReCt4"
}
},
{
"cell_type": "code",
"source": [
"# Install required packages.\n",
"import os\n",
"import torch\n",
"os.environ['TORCH'] = torch.__version__\n",
"os.environ['DGLBACKEND'] = \"pytorch\"\n",
"\n",
"# TODO(Steve): change to stable version.\n",
"# Uncomment below to install required packages.\n",
"#!pip install --pre dgl -f https://data.dgl.ai/wheels-test/repo.html > /dev/null\n",
"#!pip install torchmetrics > /dev/null\n",
"\n",
"try:\n",
" import dgl\n",
" installed = True\n",
"except ImportError:\n",
" installed = False\n",
"print(\"DGL installed!\" if installed else \"Failed to install DGL!\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "__2tKqL0eaB0",
"outputId": "bfd8f958-511b-47a5-b44f-8823421a62e4"
},
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting torchmetrics\n",
" Downloading torchmetrics-0.11.0-py3-none-any.whl (512 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m512.4/512.4 KB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torchmetrics) (4.4.0)\n",
"Requirement already satisfied: numpy>=1.17.2 in /usr/local/lib/python3.8/dist-packages (from torchmetrics) (1.21.6)\n",
"Requirement already satisfied: torch>=1.8.1 in /usr/local/lib/python3.8/dist-packages (from torchmetrics) (1.13.0+cu116)\n",
"Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from torchmetrics) (21.3)\n",
"Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->torchmetrics) (3.0.9)\n",
"Installing collected packages: torchmetrics\n",
"Successfully installed torchmetrics-0.11.0\n",
"DGL installed!\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Hypergraphs\n",
"\n",
"A [**hypergraph**](https://en.wikipedia.org/wiki/Hypergraph) consists of *nodes* and *hyperedges*. Contrary to edges in graphs, a *hyperedge* can connect arbitrary number of nodes. For instance, the following figure shows a hypergraph with 11 nodes and 5 hyperedges drawn in different colors.\n",
"![](https://data.dgl.ai/tutorial/img/hgnn/hypergraph4.PNG)\n",
"\n",
"A hypergraph is usually characterized by its *incidence matrix* $H$, whose rows represent nodes and columns represent hyperedges. An entry $H_{ij}$ is 1 if hyperedge $j$ includes node $i$, or 0 otherwise. For example, the hypergraph in the figure above can be characterized by a $11 \\times 5$ matrix as follows:\n",
"\n",
"$$\n",
"H = \\begin{bmatrix}\n",
"1 & 0 & 0 & 0 & 0 \\\\\n",
"1 & 0 & 0 & 0 & 0 \\\\\n",
"1 & 1 & 0 & 1 & 1 \\\\\n",
"0 & 0 & 1 & 0 & 0 \\\\\n",
"0 & 1 & 0 & 0 & 0 \\\\\n",
"1 & 0 & 1 & 1 & 1 \\\\\n",
"0 & 0 & 1 & 0 & 0 \\\\\n",
"0 & 1 & 0 & 1 & 0 \\\\\n",
"0 & 1 & 0 & 1 & 0 \\\\\n",
"0 & 0 & 1 & 0 & 1 \\\\\n",
"0 & 0 & 0 & 0 & 1 \\\\\n",
"\\end{bmatrix}\n",
"$$\n",
"\n",
"One can construct the hypergraph incidence matrix by specifying two tensors `nodes` and `hyperedges`, where the node ID `nodes[i]` belongs to the hyperedge ID `hyperedges[i]` for all `i`. In the case above, the incidence matrix can be constructed below.\n"
],
"metadata": {
"id": "unL_mAj-TqC6"
}
},
{
"cell_type": "code",
"source": [
"import dgl.sparse as dglsp\n",
"import torch\n",
"\n",
"H = dglsp.from_coo(\n",
" torch.LongTensor([0, 1, 2, 2, 2, 2, 3, 4, 5, 5, 5, 5, 6, 7, 7, 8, 8, 9, 9, 10]),\n",
" torch.LongTensor([0, 0, 0, 1, 3, 4, 2, 1, 0, 2, 3, 4, 2, 1, 3, 1, 3, 2, 4, 4])\n",
")\n",
"\n",
"print(H.to_dense())"
],
"metadata": {
"id": "I_cExvtIJD1F",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "4cc9fa37-7726-42e5-f01a-018c409044be"
},
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"tensor([[1., 0., 0., 0., 0.],\n",
" [1., 0., 0., 0., 0.],\n",
" [1., 1., 0., 1., 1.],\n",
" [0., 0., 1., 0., 0.],\n",
" [0., 1., 0., 0., 0.],\n",
" [1., 0., 1., 1., 1.],\n",
" [0., 0., 1., 0., 0.],\n",
" [0., 1., 0., 1., 0.],\n",
" [0., 1., 0., 1., 0.],\n",
" [0., 0., 1., 0., 1.],\n",
" [0., 0., 0., 0., 1.]])\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"The degree of a node in a hypergraph is defined as the number of hyperedges including the node. Similarly, the degree of a hyperedge in a hypergraph is defined as the number of nodes included by the hyperedge. In the example above, the hyperedge degrees can be computed by the sum of row vectors (i.e. all 4), while the node degree can be computed by the sum of column vectors."
],
"metadata": {
"id": "p-shCPQPHvBB"
}
},
{
"cell_type": "code",
"source": [
"node_degrees = H.sum(1)\n",
"print(\"Node degrees\", node_degrees)\n",
"\n",
"hyperedge_degrees = H.sum(0)\n",
"print(\"Hyperedge degrees\", hyperedge_degrees)"
],
"metadata": {
"id": "wjKm9gkTOnU9",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "bb6538aa-26f8-42ee-8d54-a2ea5a4fbd51"
},
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Node degrees tensor([1., 1., 4., 1., 1., 4., 1., 2., 2., 2., 1.])\n",
"Hyperedge degrees tensor([4., 4., 4., 4., 4.])\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"\n",
"## Hypergraph Neural Network (HGNN) Layer\n",
"\n",
"The **[HGNN layer](https://arxiv.org/pdf/1809.09401.pdf)** is defined as:\n",
"$$\n",
"f(X^{(l)}, H; W^{(l)}) = \\sigma(L X^{(l)} W^{(l)}) \\\\\n",
"L = D_v^{-1/2} H B D_e^{-1} H^\\top D_v^{-1/2}\n",
"$$\n",
"where\n",
"* $H \\in \\mathbb{R}^{N \\times M}$ is the incidence matrix of hypergraph with $N$ nodes and $M$ hyperedges.\n",
"* $D_v \\in \\mathbb{R}^{N \\times N}$ is a diagonal matrix representing node degrees, whose $i$-th diagonal element is $\\sum_{j=1}^M H_{ij}$.\n",
"* $D_e \\in \\mathbb{R}^{M \\times M}$ is a diagonal matrix representing hyperedge degrees, whose $j$-th diagonal element is $\\sum_{i=1}^N H_{ij}$.\n",
"* $B \\in \\mathbb{R}^{M \\times M}$ is a diagonal matrix representing the hyperedge weights, whose $j$-th diagonal element is the weight of $j$-th hyperedge. In our example, $B$ is an identity matrix."
],
"metadata": {
"id": "7kxrINkVHrAi"
}
},
{
"cell_type": "code",
"source": [
"\"\"\"\n",
"Hypergraph Neural Networks (https://arxiv.org/pdf/1809.09401.pdf)\n",
"\"\"\"\n",
"import dgl.sparse as dglsp\n",
"import torch\n",
"import torch.nn as nn\n",
"import torch.nn.functional as F\n",
"import tqdm\n",
"from dgl.data import CoraGraphDataset\n",
"from torchmetrics.functional import accuracy\n",
"\n",
"\n",
"class HGNN(nn.Module):\n",
" def __init__(self, H, in_size, out_size, hidden_dims=16):\n",
" super().__init__()\n",
"\n",
" self.Theta1 = nn.Linear(in_size, hidden_dims)\n",
" self.Theta2 = nn.Linear(hidden_dims, out_size)\n",
" self.dropout = nn.Dropout(0.5)\n",
"\n",
" ###########################################################\n",
" # (HIGHLIGHT) Compute the Laplacian with Sparse Matrix API\n",
" ###########################################################\n",
" d_V = H.sum(1) # node degree\n",
" d_E = H.sum(0) # edge degree\n",
" n_edges = d_E.shape[0]\n",
" D_V_invsqrt = dglsp.diag(d_V**-0.5) # D_V ** (-1/2)\n",
" D_E_inv = dglsp.diag(d_E**-1) # D_E ** (-1)\n",
" W = dglsp.identity((n_edges, n_edges))\n",
" self.laplacian = D_V_invsqrt @ H @ W @ D_E_inv @ H.T @ D_V_invsqrt\n",
"\n",
" def forward(self, X):\n",
" X = self.laplacian @ self.Theta1(self.dropout(X))\n",
" X = F.relu(X)\n",
" X = self.laplacian @ self.Theta2(self.dropout(X))\n",
" return X\n",
"\n",
"\n",
"def train(model, optimizer, X, Y, train_mask):\n",
" model.train()\n",
" Y_hat = model(X)\n",
" loss = F.cross_entropy(Y_hat[train_mask], Y[train_mask])\n",
" optimizer.zero_grad()\n",
" loss.backward()\n",
" optimizer.step()\n",
"\n",
"\n",
"def evaluate(model, X, Y, val_mask, test_mask, num_classes):\n",
" model.eval()\n",
" Y_hat = model(X)\n",
" val_acc = accuracy(\n",
" Y_hat[val_mask], Y[val_mask], task=\"multiclass\", num_classes=num_classes\n",
" )\n",
" test_acc = accuracy(\n",
" Y_hat[test_mask],\n",
" Y[test_mask],\n",
" task=\"multiclass\",\n",
" num_classes=num_classes,\n",
" )\n",
" return val_acc, test_acc\n",
"\n",
"\n",
"def load_data():\n",
" dataset = CoraGraphDataset()\n",
"\n",
" graph = dataset[0]\n",
" # The paper created a hypergraph from the original graph. For each node in\n",
" # the original graph, a hyperedge in the hypergraph is created to connect\n",
" # its neighbors and itself. In this case, the incidence matrix of the\n",
" # hypergraph is the same as the adjacency matrix of the original graph (with\n",
" # self-loops).\n",
" # We follow the paper and assume that the rows of the incidence matrix\n",
" # are for nodes and the columns are for edges.\n",
" src, dst = graph.edges()\n",
" H = dglsp.from_coo(dst, src)\n",
" H = H + dglsp.identity(H.shape)\n",
"\n",
" X = graph.ndata[\"feat\"]\n",
" Y = graph.ndata[\"label\"]\n",
" train_mask = graph.ndata[\"train_mask\"]\n",
" val_mask = graph.ndata[\"val_mask\"]\n",
" test_mask = graph.ndata[\"test_mask\"]\n",
" return H, X, Y, dataset.num_classes, train_mask, val_mask, test_mask\n",
"\n",
"\n",
"def main():\n",
" H, X, Y, num_classes, train_mask, val_mask, test_mask = load_data()\n",
" model = HGNN(H, X.shape[1], num_classes)\n",
" optimizer = torch.optim.Adam(model.parameters(), lr=0.001)\n",
"\n",
" with tqdm.trange(500) as tq:\n",
" for epoch in tq:\n",
" train(model, optimizer, X, Y, train_mask)\n",
" val_acc, test_acc = evaluate(\n",
" model, X, Y, val_mask, test_mask, num_classes\n",
" )\n",
" tq.set_postfix(\n",
" {\n",
" \"Val acc\": f\"{val_acc:.5f}\",\n",
" \"Test acc\": f\"{test_acc:.5f}\",\n",
" },\n",
" refresh=False,\n",
" )\n",
"\n",
" print(f\"Test acc: {test_acc:.3f}\")\n",
"\n",
"\n",
"if __name__ == \"__main__\":\n",
" main()"
],
"metadata": {
"id": "58WnPtPvT2mx",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "e27529b5-a097-4fe2-e1d6-7ae89484dd23"
},
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" NumNodes: 2708\n",
" NumEdges: 10556\n",
" NumFeats: 1433\n",
" NumClasses: 7\n",
" NumTrainingSamples: 140\n",
" NumValidationSamples: 500\n",
" NumTestSamples: 1000\n",
"Done loading data from cached files.\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"100%|██████████| 500/500 [00:48<00:00, 10.24it/s, Val acc=0.76600, Test acc=0.76000]"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Test acc: 0.760\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"For the complete example of HGNN, please refer to https://github.com/dmlc/dgl/blob/master/examples/sparse/hgnn.py."
],
"metadata": {
"id": "59pCzjpBOyEW"
}
}
]
}
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment