Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "29e666152390c272e0115ce8455da1adb5fcacb1"
Unverified Commit 0868f179 authored by Minjie Wang's avatar Minjie Wang Committed by GitHub
Browse files

[Doc] sparse lib tutorial and colab links (#5181) 

* Created using Colaboratory

* nbsphinx & nblink

* Created using Colaboratory

* update notebook

* minor update
parent 0c5d6f1b
Hide whitespace changes
Inline Side-by-side
Showing with 1205 additions and 1 deletion
docs/source/conf.py
View file @ 0868f179
......@@ -48,6 +48,8 @@ extensions = [
'sphinx.ext.graphviz',
'sphinx_gallery.gen_gallery',
'sphinx_copybutton',
'nbsphinx',
'nbsphinx_link',
]
# Add any paths that contain templates here, relative to this directory.
......
docs/source/index.rst
View file @ 0868f179
......@@ -25,6 +25,7 @@ Welcome to Deep Graph Library Tutorials and Documentation
guide/index
guide_cn/index
guide_ko/index
notebooks/sparse/index
tutorials/large/index
tutorials/cpu/index
tutorials/multi/index
......@@ -51,10 +52,10 @@ Welcome to Deep Graph Library Tutorials and Documentation
api/python/dgl.ops
api/python/dgl.optim
api/python/dgl.sampling
api/python/dgl.sparse_v0
api/python/dgl.multiprocessing
api/python/transforms
api/python/udf
api/python/dgl.sparse_v0
.. toctree::
:maxdepth: 1
......
docs/source/notebooks/sparse/index.rst 0 → 100644
View file @ 0868f179
🆕 Tutorials: dgl.sparse
========================
TODO(minjie): intro for the new library.
.. toctree::
:maxdepth: 2
:titlesonly:
quickstart.nblink
docs/source/notebooks/sparse/quickstart.nblink 0 → 100644
View file @ 0868f179
{
"path": "../../../../notebooks/sparse/quickstart.ipynb"
}
notebooks/sparse/quickstart.ipynb 0 → 100644
View file @ 0868f179
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"private_outputs": true,
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"gpuClass": "standard",
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"source": [
"# Quickstart\n",
"\n",
"The tutorial provides a quick walkthrough of the classes and operators provided by the `dgl.sparse` package.\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dmlc/dgl/blob/master/notebooks/sparse/quickstart.ipynb) [![GitHub](https://img.shields.io/badge/-View%20on%20GitHub-181717?logo=github&logoColor=ffffff)](https://github.com/dmlc/dgl/blob/master/notebooks/sparse/quickstart.ipynb)"
],
"metadata": {
"id": "E0DAKDMuWz7I"
}
},
{
"cell_type": "code",
"source": [
"# Uncomment below to install the required packages.\n",
"\n",
"import os\n",
"import torch\n",
"os.environ['TORCH'] = torch.__version__\n",
"os.environ['DGLBACKEND'] = \"pytorch\"\n",
"\n",
"# TODO(Steve): change to stable version.\n",
"!pip install --pre dgl -I https://data.dgl.ai/wheels-test/repo.html > /dev/null\n",
"\n",
"try:\n",
" import dgl.sparse as dglsp\n",
" installed = True\n",
"except ImportError:\n",
" installed = False\n",
"print(\"DGL installed!\" if installed else \"Failed to install DGL!\")"
],
"metadata": {
"id": "19UZd7wyWzpT"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Sparse Matrix\n",
"\n",
"The core abstraction of DGL's sparse package is the `SparseMatrix` class. Compared with other sparse matrix libraries (such as `scipy.sparse` and `torch.sparse`), DGL's `SparseMatrix` is specialized for the deep learning workloads on structure data (e.g., Graph Neural Networks), with the following features:\n",
"\n",
"* **Auto sparse format.** Don't bother choosing between different sparse formats. There is only one `SparseMatrix` and it will select the best format for the operation to be performed.\n",
"* **Non-zero elements can be scalar or vector.** Easy for modeling relations (e.g., edges) by vector representation.\n",
"* **Fully PyTorch compatible.** The package is built upon PyTorch and is natively compatible with other tools in the PyTorch ecosystem.\n"
],
"metadata": {
"id": "GsWoAGC4RpHw"
}
},
{
"cell_type": "markdown",
"source": [
"### Initializing a DGL Sparse Matrix\n",
"\n",
"A DGL Sparse Matrix can be initialized in various ways.\n",
"\n",
"* `from_coo()`\n",
"* `from_csr()`\n",
"* `from_csc()`\n",
"\n",
"Take a look at the following examples:\n",
"\n",
"**From [COO format](https://en.wikipedia.org/wiki/Sparse_matrix#Coordinate_list_(COO))**"
],
"metadata": {
"id": "_q4HYodcWenB"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "h-ryVEs1PuIP"
},
"outputs": [],
"source": [
"import torch\n",
"import dgl.sparse as dglsp\n",
"\n",
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"A = dglsp.from_coo(row, col) # 1.0 is default value for nnz elements.\n",
"print(A)\n",
"print(\"\")\n",
"print(\"In dense format:\")\n",
"print(A.to_dense())"
]
},
{
"cell_type": "markdown",
"source": [
"*Compare implicit shape vs explicit shape*"
],
"metadata": {
"id": "W1JJg-eZ7K3t"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 0, 1])\n",
"col = torch.tensor([0, 2, 0])\n",
"\n",
"A1 = dglsp.from_coo(row, col)\n",
"print(f\"Implicit Shape: {A1.shape}\")\n",
"print(A1.to_dense())\n",
"print(\"\")\n",
"\n",
"A2 = dglsp.from_coo(row, col, shape=(3, 3))\n",
"print(f\"Explicit Shape: {A2.shape}\")\n",
"print(A2.to_dense())"
],
"metadata": {
"id": "80NNSQfd7L5V"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Both scalar values and vector values can be set for nnz elements in Sparse Matrix."
],
"metadata": {
"id": "zdNgUf0ShfCe"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"# The length of the value should match the nnz elements represented by the\n",
"# sparse matrix format.\n",
"scalar_val = torch.tensor([1., 2., 3.])\n",
"vector_val = torch.tensor([[1., 1.], [2., 2.], [3., 3.]])\n",
"\n",
"print(\"-----Scalar Values-----\")\n",
"A = dglsp.from_coo(row, col, scalar_val)\n",
"print(A)\n",
"print(\"\")\n",
"print(\"In dense format:\")\n",
"print(A.to_dense())\n",
"print(\"\")\n",
"\n",
"print(\"-----Vector Values-----\")\n",
"A = dglsp.from_coo(row, col, vector_val)\n",
"print(A)\n",
"print(\"\")\n",
"print(\"In dense format:\")\n",
"print(A.to_dense())"
],
"metadata": {
"id": "buE9ZkKvhp1f"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"*Duplicated indices*"
],
"metadata": {
"id": "7ufTCDAVsrmP"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 0, 0, 1])\n",
"col = torch.tensor([0, 2, 2, 0])\n",
"val = torch.tensor([1., 2., 3., 4])\n",
"A = dglsp.from_coo(row, col, val)\n",
"print(A)\n",
"print(f\"Whether A contains duplicate indices: {A.has_duplicate()}\")\n",
"print(\"\")\n",
"\n",
"B = A.coalesce()\n",
"print(B)\n",
"print(f\"Whether B contains duplicate indices: {B.has_duplicate()}\")"
],
"metadata": {
"id": "ilSAlFLOs0o8"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**From [CSR format](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_row_(CSR,_CRS_or_Yale_format)) and [CSC format](https://en.wikipedia.org/wiki/Sparse_matrix#Compressed_sparse_column_(CSC_or_CCS))**"
],
"metadata": {
"id": "XfnW7kGnd1lF"
}
},
{
"cell_type": "code",
"source": [
"indptr = torch.tensor([0, 1, 2, 5])\n",
"indices = torch.tensor([1, 2, 0, 1, 2])\n",
"\n",
"print(\"-----Create from CSR format-----\")\n",
"A = dglsp.from_csr(indptr, indices)\n",
"print(A)\n",
"print(\"\")\n",
"print(\"In dense format:\")\n",
"print(A.to_dense())\n",
"print(\"\")\n",
"\n",
"print(\"-----Create from CSC format-----\")\n",
"B = dglsp.from_csc(indptr, indices)\n",
"print(B)\n",
"print(\"\")\n",
"print(\"In dense format:\")\n",
"print(B.to_dense())"
],
"metadata": {
"id": "3puXyMFsvdlj"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**val_like**\n",
"\n",
"Similar to pytorch, we can create a Sparse Matrix with new values, the same nonzero indices as the given sparse matrix."
],
"metadata": {
"id": "ZJ09qM5NaxuI"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A = dglsp.from_coo(row, col, val)\n",
"\n",
"new_val = torch.tensor([4., 5., 6.])\n",
"B = dglsp.val_like(A, new_val)\n",
"print(B)"
],
"metadata": {
"id": "UB3lKJVBbsUD"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Attributes and methods of a DGL Sparse Matrix"
],
"metadata": {
"id": "nd4hJ9ysd4St"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 1, 1, 2])\n",
"col = torch.tensor([1, 0, 2, 0])\n",
"val = torch.tensor([1., 2., 3., 4.])\n",
"A = dglsp.from_coo(row, col, val)\n",
"\n",
"print(f\"Shape of sparse matrix: {A.shape}\")\n",
"print(f\"The number of nonzero elements of sparse matrix: {A.nnz}\")\n",
"print(f\"Datatype of sparse matrix: {A.dtype}\")\n",
"print(f\"Device sparse matrix is stored on: {A.device}\")\n",
"print(f\"Get the values of the nonzero elements: {A.val}\")\n",
"print(f\"Get the row indices of the nonzero elements: {A.row}\")\n",
"print(f\"Get the column indices of the nonzero elements: {A.col}\")\n",
"print(f\"Get the coordinate (COO) representation: {A.coo()}\")\n",
"print(f\"Get the compressed sparse row (CSR) representation: {A.csr()}\")\n",
"print(f\"Get the compressed sparse column (CSC) representation: {A.csc()}\")"
],
"metadata": {
"id": "OKbFiWKIzZVe"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**dtype and/or device conversion**"
],
"metadata": {
"id": "VzosM7i3yQPK"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 1, 1, 2])\n",
"col = torch.tensor([1, 0, 2, 0])\n",
"val = torch.tensor([1., 2., 3., 4.])\n",
"A = dglsp.from_coo(row, col, val)\n",
"\n",
"B = A.to(device='cuda:0', dtype=torch.int32)\n",
"print(f\"Device sparse matrix is stored on: {B.device}\")\n",
"print(f\"Datatype of sparse matrix: {B.dtype}\")"
],
"metadata": {
"id": "y_RJihw-ypXp"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Similar to pytorch, we also provide various fine-grained APIs ([Doc](https://docs.dgl.ai/en/latest/api/python/dgl.sparse_v0.html)) for dtype and/or device conversion."
],
"metadata": {
"id": "U26arLlJzfkN"
}
},
{
"cell_type": "markdown",
"source": [
"## Diagonal Matrix\n",
"\n",
"Diagonal Matrix is a special type of Sparse Matrix, in which the entries outside the main diagonal are all zero.\n",
"\n",
"\n"
],
"metadata": {
"id": "EFe9ABRuWHqf"
}
},
{
"cell_type": "markdown",
"source": [
"### Initializing a DGL Diagonal Matrix\n",
"A DGL Diagonal Matrix can be initiate by `dglsp.diag()`.\n",
"\n",
"Identity Matrix is a special type of Diagonal Matrix, in which all the value on the diagonal are 1.0. Use `dglsp.identity()` to initiate a Diagonal Matrix."
],
"metadata": {
"id": "1CeCoE2Fgl_x"
}
},
{
"cell_type": "code",
"source": [
"val = torch.tensor([1., 2., 3., 4.])\n",
"D = dglsp.diag(val)\n",
"print(D)\n",
"\n",
"I = dglsp.identity(shape=(3, 3))\n",
"print(I)"
],
"metadata": {
"id": "9wzJNApahXAR"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### Attributes and methods of a DGL Diagonal Matrix"
],
"metadata": {
"id": "s-JpSHGLhWlm"
}
},
{
"cell_type": "code",
"source": [
"val = torch.tensor([1., 2., 3., 4.])\n",
"D = dglsp.diag(val)\n",
"\n",
"print(f\"Shape of sparse matrix: {D.shape}\")\n",
"print(f\"The number of nonzero elements of sparse matrix: {D.nnz}\")\n",
"print(f\"Datatype of sparse matrix: {D.dtype}\")\n",
"print(f\"Device sparse matrix is stored on: {D.device}\")\n",
"print(f\"Get the values of the nonzero elements: {D.val}\")"
],
"metadata": {
"id": "QMV0u-kQWsWd"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Operations on Sparse Matrix and Diagonal Matrix\n",
"* Elementwise operations\n",
" * `A + B`\n",
" * `A - B`\n",
" * `A * B`\n",
" * `A / B`\n",
" * `A ** scalar`\n",
"* Reduce operations\n",
" * `reduce()`\n",
" * `sum()`\n",
" * `smax()`\n",
" * `smin()`\n",
" * `smean()`\n",
"* Matrix transformations\n",
" * `SparseMatrix.transpose()` or `SparseMatrix.T`\n",
" * `SparseMatrix.neg()`\n",
" * `DiagMatrix.transpose()` or `DiagMatrix.T`\n",
" * `DiagMatrix.neg()`\n",
" * `DiagMatrix.inv()`\n",
"* Matrix multiplication\n",
" * `matmul()`\n",
" * `sddmm()`\n",
"\n",
"\n",
"*We are using dense format to print sparse matrix in this tutorial since it is more intuitive to read.*"
],
"metadata": {
"id": "Tjsapqp6zSFR"
}
},
{
"cell_type": "markdown",
"source": [
"### *Elementwise operations*"
],
"metadata": {
"id": "psvGwcIqYvC2"
}
},
{
"cell_type": "markdown",
"source": [
"**add(A, B), equivalent to A + B**\n",
"\n",
"The supported combinations are shown as follows.\n",
"\n",
"A \\ B | **DiagMatrix**|**SparseMatrix**|**scalar**\n",
"----------------|---------------|----------------|----------\n",
"**DiagMatrix** |✅ |✅ |🚫\n",
"**SparseMatrix**|✅ |✅ |🚫\n",
"**scalar** |🚫 |🚫 |🚫"
],
"metadata": {
"id": "39YJitpW-K9v"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A1 = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A1:\")\n",
"print(A1.to_dense())\n",
"\n",
"row = torch.tensor([0, 1, 2])\n",
"col = torch.tensor([0, 2, 1])\n",
"val = torch.tensor([4., 5., 6.])\n",
"A2 = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A2:\")\n",
"print(A2.to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D1 = dglsp.diag(val)\n",
"print(\"D1:\")\n",
"print(D1.to_dense())\n",
"\n",
"val = torch.tensor([-4., -5., -6.])\n",
"D2 = dglsp.diag(val)\n",
"print(\"D2:\")\n",
"print(D2.to_dense())\n",
"\n",
"print(\"A1 + A2:\")\n",
"print((A1 + A2).to_dense())\n",
"\n",
"print(\"A1 + D1:\")\n",
"print((A1 + D1).to_dense())\n",
"\n",
"print(\"D1 + D2:\")\n",
"print((D1 + D2).to_dense())"
],
"metadata": {
"id": "pj3Ckx41-BSu"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**sub(A, B) equivalent to A - B**\n",
"\n",
"The supported combinations are shown as follows.\n",
"\n",
"A \\ B | **DiagMatrix**|**SparseMatrix**|**scalar**\n",
"----------------|---------------|----------------|----------\n",
"**DiagMatrix** |✅ |🚫 |🚫\n",
"**SparseMatrix**|🚫 |🚫 |🚫\n",
"**scalar** |🚫 |🚫 |🚫"
],
"metadata": {
"id": "i25N0JHUTUX9"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A1 = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A1:\")\n",
"print(A1.to_dense())\n",
"\n",
"row = torch.tensor([0, 1, 2])\n",
"col = torch.tensor([0, 2, 1])\n",
"val = torch.tensor([4., 5., 6.])\n",
"A2 = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A2:\")\n",
"print(A2.to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D1 = dglsp.diag(val)\n",
"print(\"D1:\")\n",
"print(D1.to_dense())\n",
"\n",
"val = torch.tensor([-4., -5., -6.])\n",
"D2 = dglsp.diag(val)\n",
"print(\"D2:\")\n",
"print(D2.to_dense())\n",
"\n",
"print(\"A1 - A2:\")\n",
"print((A1 - A2).to_dense())\n",
"\n",
"print(\"A1 - D1:\")\n",
"print((A1 - D1).to_dense())\n",
"\n",
"print(\"D1 - A1:\")\n",
"print((D1 - A1).to_dense())\n",
"\n",
"print(\"D1 - D2:\")\n",
"print((D1 - D2).to_dense())"
],
"metadata": {
"id": "GMxfz-cyT129"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**mul(A, B), equivalent to A * B**\n",
"\n",
"The supported combinations are shown as follows.\n",
"\n",
"A \\ B | **DiagMatrix**|**SparseMatrix**|**scalar**\n",
"----------------|---------------|----------------|----------\n",
"**DiagMatrix** |✅ |🚫 |✅\n",
"**SparseMatrix**|🚫 |🚫 |✅\n",
"**scalar** |✅ |✅ |🚫"
],
"metadata": {
"id": "bg45jnq8T9EJ"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"print(\"A * 3:\")\n",
"print((A * 3).to_dense())\n",
"print(\"3 * A:\")\n",
"print((3 * A).to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D1 = dglsp.diag(val)\n",
"print(\"D1:\")\n",
"print(D1.to_dense())\n",
"\n",
"val = torch.tensor([-4., -5., -6.])\n",
"D2 = dglsp.diag(val)\n",
"print(\"D2:\")\n",
"print(D2.to_dense())\n",
"\n",
"print(\"D1 * -2:\")\n",
"print((D1 * -2).to_dense())\n",
"print(\"-2 * D1:\")\n",
"print((-2 * D1).to_dense())\n",
"\n",
"print(\"D1 * D2:\")\n",
"print((D1 * D2).to_dense())"
],
"metadata": {
"id": "4PAITJqHUB8J"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**div(A, B), equivalent to A / B**\n",
"\n",
"The supported combinations are shown as follows.\n",
"\n",
"A \\ B | **DiagMatrix**|**SparseMatrix**|**scalar**\n",
"----------------|---------------|----------------|----------\n",
"**DiagMatrix** |✅ |🚫 |✅\n",
"**SparseMatrix**|🚫 |🚫 |✅\n",
"**scalar** |🚫 |🚫 |🚫"
],
"metadata": {
"id": "Xb2RU6H4UBCs"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"print(\"A / 2:\")\n",
"print((A / 2).to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D1 = dglsp.diag(val)\n",
"print(\"D1:\")\n",
"print(D1.to_dense())\n",
"\n",
"val = torch.tensor([-4., -5., -6.])\n",
"D2 = dglsp.diag(val)\n",
"print(\"D2:\")\n",
"print(D2.to_dense())\n",
"\n",
"print(\"D1 / D2:\")\n",
"print((D1 / D2).to_dense())\n",
"\n",
"print(\"D1 / 2:\")\n",
"print((D1 / 2).to_dense())"
],
"metadata": {
"id": "TFB_UcmEUdr3"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**power(A, B), equivalent to A \\*\\* B**\n",
"\n",
"The supported combinations are shown as follows.\n",
"\n",
"A \\ B | **DiagMatrix**|**SparseMatrix**|**scalar**\n",
"----------------|---------------|----------------|----------\n",
"**DiagMatrix** |🚫 |🚫 |✅\n",
"**SparseMatrix**|🚫 |🚫 |✅\n",
"**scalar** |🚫 |🚫 |🚫"
],
"metadata": {
"id": "2lZbyTYUUgSi"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"print(\"A ** 3:\")\n",
"print((A ** 3).to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D = dglsp.diag(val)\n",
"print(\"D:\")\n",
"print(D.to_dense())\n",
"\n",
"print(\"D1 ** 2:\")\n",
"print((D1 ** 2).to_dense())"
],
"metadata": {
"id": "ox-XxCnuUqAy"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### *Reduce operations*\n",
"\n",
"All DGL sparse reduce operations only consider non-zero elements. To distinguish them from dense PyTorch reduce operations that consider zero elements, we use name `smax`, `smin` and `smean` (`s` stands for sparse)."
],
"metadata": {
"id": "TQJJlctZjYPv"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 1, 1, 2])\n",
"col = torch.tensor([1, 0, 2, 0])\n",
"val = torch.tensor([1., 2., 3., 4.])\n",
"A = dglsp.from_coo(row, col, val)\n",
"print(A.T.to_dense())\n",
"print(\"\")\n",
"\n",
"# O1, O2 will have the same value.\n",
"O1 = A.reduce(0, 'sum')\n",
"O2 = A.sum(0)\n",
"print(\"Reduce with reducer:sum along dim = 0:\")\n",
"print(O1)\n",
"print(\"\")\n",
"\n",
"# O3, O4 will have the same value.\n",
"O3 = A.reduce(0, 'smax')\n",
"O4 = A.smax(0)\n",
"print(\"Reduce with reducer:max along dim = 0:\")\n",
"print(O3)\n",
"print(\"\")\n",
"\n",
"# O5, O6 will have the same value.\n",
"O5 = A.reduce(0, 'smin')\n",
"O6 = A.smin(0)\n",
"print(\"Reduce with reducer:min along dim = 0:\")\n",
"print(O5)\n",
"print(\"\")\n",
"\n",
"# O7, O8 will have the same value.\n",
"O7 = A.reduce(0, 'smean')\n",
"O8 = A.smean(0)\n",
"print(\"Reduce with reducer:smean along dim = 0:\")\n",
"print(O7)\n",
"print(\"\")"
],
"metadata": {
"id": "GhS49Js1jW4b"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### *Matrix transformations*"
],
"metadata": {
"id": "kanwnB7LOQui"
}
},
{
"cell_type": "markdown",
"source": [
"*Sparse Matrix*"
],
"metadata": {
"id": "NiiXso9elM2p"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 1, 1, 2])\n",
"col = torch.tensor([1, 0, 2, 0])\n",
"val = torch.tensor([1., 2., 3., 4.])\n",
"A = dglsp.from_coo(row, col, val)\n",
"print(A.to_dense())\n",
"print(\"\")\n",
"\n",
"print(\"Get transpose of sparse matrix.\")\n",
"print(A.T.to_dense())\n",
"# Alias\n",
"# A.transpose()\n",
"# A.t()\n",
"print(\"\")\n",
"\n",
"print(\"Get a sparse matrix with the negation of the original nonzero values.\")\n",
"print(A.neg().to_dense())\n",
"print(\"\")"
],
"metadata": {
"id": "qJcmZHmf-oTY"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"*Diagonal Matrix*"
],
"metadata": {
"id": "iE3ANjFolIJu"
}
},
{
"cell_type": "code",
"source": [
"val = torch.tensor([1., 2., 3., 4.])\n",
"D = dglsp.diag(val)\n",
"print(D.to_dense())\n",
"print(\"\")\n",
"\n",
"print(\"Get inverse of diagonal matrix:\")\n",
"print(D.inv().to_dense())\n",
"print(\"\")\n",
"\n",
"print(\"Get a diagonal matrix with the negation of the original nonzero values.\")\n",
"print(D.neg().to_dense())\n",
"print(\"\")"
],
"metadata": {
"id": "j9kjY9RdlGXx"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"### *Matrix multiplication*"
],
"metadata": {
"id": "4uQlDFb0Uzto"
}
},
{
"cell_type": "markdown",
"source": [
"**matmul(A, B), equivalent to A @ B**\n",
"\n",
"The supported combinations are shown as follows.\n",
"\n",
"A \\ B | **Tensor**|**DiagMatrix**|**SparseMatrix**\n",
"----------------|-----------|--------------|----------\n",
"**Tensor** |✅ |🚫 |🚫\n",
"**DiagMatrix**. |✅ |✅ |✅\n",
"**SparseMatrix**|✅ |✅ |✅"
],
"metadata": {
"id": "THWE30v6WpAk"
}
},
{
"cell_type": "markdown",
"source": [
"**Union[DiagMatrix, SparseMatrix] @ Union[DiagMatrix, SparseMatrix] -> Union[SparseMatrix, DiagMatrix]:**\n",
"\n",
"For a $L \\times M$ sparse matrix A and a $M \\times N$ sparse matrix B, the shape of `A @ B` will be $L \\times N$ sparse matrix."
],
"metadata": {
"id": "VxyykR-vX7lF"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A1 = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A1:\")\n",
"print(A1.to_dense())\n",
"\n",
"row = torch.tensor([0, 1, 2])\n",
"col = torch.tensor([0, 2, 1])\n",
"val = torch.tensor([4., 5., 6.])\n",
"A2 = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A2:\")\n",
"print(A2.to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D1 = dglsp.diag(val)\n",
"print(\"D1:\")\n",
"print(D1.to_dense())\n",
"\n",
"val = torch.tensor([-4., -5., -6.])\n",
"D2 = dglsp.diag(val)\n",
"print(\"D2:\")\n",
"print(D2.to_dense())\n",
"\n",
"print(\"A1 @ A2:\")\n",
"print((A1 @ A2).to_dense())\n",
"\n",
"print(\"A1 @ D1:\")\n",
"print((A1 @ D1).to_dense())\n",
"\n",
"print(\"D1 @ A1:\")\n",
"print((D1 @ A1).to_dense())\n",
"\n",
"print(\"D1 @ D2:\")\n",
"print((D1 @ D2).to_dense())"
],
"metadata": {
"id": "XRDFC2rOYQM4"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**Union[DiagMatrix, SparseMatrix] @ Tensor -> Tensor:**\n",
"\n",
"For a $L \\times M$ sparse matrix A and a $M \\times N$ dense matrix B, the shape of `A @ B` will be $L \\times N$ dense matrix."
],
"metadata": {
"id": "g13fG8nvaVOt"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"val = torch.tensor([-1., -2., -3.])\n",
"D = dglsp.diag(val)\n",
"print(\"D:\")\n",
"print(D.to_dense())\n",
"\n",
"X = torch.tensor([[11., 22.], [33., 44.], [55., 66.]])\n",
"print(\"X:\")\n",
"print(X)\n",
"\n",
"print(\"A @ X:\")\n",
"print(A @ X)\n",
"\n",
"print(\"D @ X:\")\n",
"print(D @ X)"
],
"metadata": {
"id": "FcQ-CnqdlgWF"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"This operator also supports batched sparse-dense matrix multiplication. The sparse matrix A should have shape $L \\times M$, where the non-zero values are vectors of length $K$. The dense matrix B should have shape $M \\times N \\times K$. The output is a dense matrix of shape $L \\times N \\times K$."
],
"metadata": {
"id": "_KZiULLbmEZE"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([0, 2, 0])\n",
"val = torch.tensor([[1., 1.], [2., 2.], [3., 3.]])\n",
"A = dglsp.from_coo(row, col, val, shape=(3, 3))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"X = torch.tensor([[[1., 1.], [1., 2.]],\n",
" [[1., 3.], [1., 4.]],\n",
" [[1., 5.], [1., 6.]]])\n",
"print(\"X:\")\n",
"print(X)\n",
"\n",
"print(\"A @ X:\")\n",
"print(A @ X)"
],
"metadata": {
"id": "ZUzXQk7Ab2wG"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"**Sampled-Dense-Dense Matrix Multiplication**\n",
"\n",
"``sddmm`` matrix-multiplies two dense matrices X1 and X2, then elementwise-multiplies the result with sparse matrix A at the nonzero locations. This is designed for sparse matrix with scalar values.\n",
"\n",
"$$out = (X_1 @ X_2) * A$$\n",
"\n",
"For a $L \\times N$ sparse matrix A, a $L \\times M$ dense matrix X1 and a $M \\times N$ dense matrix X2, `sddmm(A, X1, X2)` will be a $L \\times N$ sparse matrix."
],
"metadata": {
"id": "qO_8f_vhPKtf"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([2, 3, 3])\n",
"val = torch.tensor([1., 2., 3.])\n",
"A = dglsp.from_coo(row, col, val, (3, 4))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"X1 = torch.randn(3, 5)\n",
"X2 = torch.randn(5, 4)\n",
"print(\"X1:\")\n",
"print(X1)\n",
"print(\"X2:\")\n",
"print(X2)\n",
"\n",
"O = dglsp.sddmm(A, X1, X2)\n",
"print(\"dglsp.sddmm(A, X1, X2):\")\n",
"print(O.to_dense())"
],
"metadata": {
"id": "3ZIFV0TgPhwH"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"This operator also supports batched sampled-dense-dense matrix multiplication. For a $L \\times N$ sparse matrix A with non-zero vector values of length $𝐾$, a $L \\times M \\times K$ dense matrix X1 and a $M \\times N \\times K$ dense matrix X2, `sddmm(A, X1, X2)` will be a $L \\times N \\times K$ sparse matrix."
],
"metadata": {
"id": "RmNmXU_ZqyF7"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([1, 1, 2])\n",
"col = torch.tensor([2, 3, 3])\n",
"val = torch.tensor([[1., 1.], [2., 2.], [3., 3.]])\n",
"A = dglsp.from_coo(row, col, val, (3, 4))\n",
"print(\"A:\")\n",
"print(A.to_dense())\n",
"\n",
"X1 = torch.randn(3, 5, 2)\n",
"X2 = torch.randn(5, 4, 2)\n",
"print(\"X1:\")\n",
"print(X1)\n",
"print(\"X2:\")\n",
"print(X2)\n",
"\n",
"O = dglsp.sddmm(A, X1, X2)\n",
"print(\"dglsp.sddmm(A, X1, X2):\")\n",
"print(O.to_dense())"
],
"metadata": {
"id": "DuSAjamyrIO_"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Non-linear activation functions\n",
"\n",
"* `softmax()`"
],
"metadata": {
"id": "XuaNdFO7XG2r"
}
},
{
"cell_type": "markdown",
"source": [
"**Softmax**\n",
"\n",
"Apply row-wise softmax to the nonzero entries of the sparse matrix."
],
"metadata": {
"id": "y8OQZReVXpo3"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 1, 1, 2])\n",
"col = torch.tensor([1, 0, 2, 0])\n",
"val = torch.tensor([1., 2., 3., 4.])\n",
"A = dglsp.from_coo(row, col, val)\n",
"\n",
"print(A.softmax())\n",
"print(\"In dense format:\")\n",
"print(A.softmax().to_dense())\n",
"print(\"\\n\")"
],
"metadata": {
"id": "CQaKgzCJULjt"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Exercise"
],
"metadata": {
"id": "1iBNlJVYz3zi"
}
},
{
"cell_type": "markdown",
"source": [
"*Let's test what you've learned. Feel free to [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dmlc/dgl/blob/master/notebooks/sparse/quickstart.ipynb).*\n",
"\n",
"Given a sparse symmetrical adjacency matrix $A$, calculate its symmetrically normalized adjacency matrix: $$norm = \\hat{D}^{-\\frac{1}{2}}\\hat{A}\\hat{D}^{-\\frac{1}{2}}$$\n",
"\n",
"Where $\\hat{A} = A + I$, $I$ is the identity matrix, and $\\hat{D}$ is the diagonal node degree matrix of $\\hat{A}$."
],
"metadata": {
"id": "yDQ4Kmr_08St"
}
},
{
"cell_type": "code",
"source": [
"row = torch.tensor([0, 0, 1, 1, 2, 2, 3])\n",
"col = torch.tensor([1, 3, 2, 5, 3, 5, 4])\n",
"asym_A = dglsp.from_coo(row, col, shape=(6, 6))\n",
"# Step 1: create symmetrical adjacency matrix A from asym_A.\n",
"# A =\n",
"\n",
"# Step 2: calculate A_hat from A.\n",
"# A_hat =\n",
"\n",
"# Step 3: diagonal node degree matrix of A_hat\n",
"# D_hat =\n",
"\n",
"# Step 4: calculate the norm from D_hat and A_hat.\n",
"# norm = "
],
"metadata": {
"id": "0dDhfbJo0ByV"
},
"execution_count": null,
"outputs": []
}
]
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment