Commit 8bc01c63 authored by Zihao Ye's avatar Zihao Ye Committed by Minjie Wang
Browse files

[Tutorial] Tutorial fix (SSE link & Transformer) (#285)

* upd tutorial

* upd tutorial
parent 322bd713
...@@ -32,8 +32,8 @@ Graph Neural Network and its variant ...@@ -32,8 +32,8 @@ Graph Neural Network and its variant
operation, sparse-matrix tensor operations, along with message-passing with operation, sparse-matrix tensor operations, along with message-passing with
DGL. DGL.
* **SSE** `[paper] <http://proceedings.mlr.press/v80/dai18a/dai18a.pdf>`__ * **SSE** `[paper] <http://proceedings.mlr.press/v80/dai18a/dai18a.pdf>`__ `[tutorial]
`[tutorial <1_gnn/8_sse_mx.html>]` `[code] <1_gnn/8_sse_mx.html>`__ `[code]
<https://github.com/jermainewang/dgl/blob/master/examples/mxnet/sse>`__: <https://github.com/jermainewang/dgl/blob/master/examples/mxnet/sse>`__:
the emphasize here is *giant* graph that cannot fit comfortably on one GPU the emphasize here is *giant* graph that cannot fit comfortably on one GPU
card. SSE is an example to illustrate the co-design of both algorithm and card. SSE is an example to illustrate the co-design of both algorithm and
......
...@@ -44,7 +44,7 @@ Transformer Tutorial ...@@ -44,7 +44,7 @@ Transformer Tutorial
# v_i = W_v\cdot x_i\\ # v_i = W_v\cdot x_i\\
# \textrm{score} = q_j^T k_i # \textrm{score} = q_j^T k_i
# #
# where :math:`W_q, W_k, W_v \in \mathbb{R}^{n\times n}` map the # where :math:`W_q, W_k, W_v \in \mathbb{R}^{n\times d_k}` map the
# representations :math:`x` to “query”, “key”, and “value” space # representations :math:`x` to “query”, “key”, and “value” space
# respectively. # respectively.
# #
...@@ -472,8 +472,8 @@ Transformer Tutorial ...@@ -472,8 +472,8 @@ Transformer Tutorial
# #
# By calling ``update_graph`` function, we can “DIY our own # By calling ``update_graph`` function, we can “DIY our own
# Transformer” on any subgraphs with nearly the same code. This # Transformer” on any subgraphs with nearly the same code. This
# flexibility enables us to discover new, sparse structures (c.f. # flexibility enables us to discover new, sparse structures (c.f. local attention
# `here <https://arxiv.org/pdf/1508.04025.pdf>`__). Note in our # mentioned `here <https://arxiv.org/pdf/1508.04025.pdf>`__). Note in our
# implementation we does not use mask or padding, which makes the logic # implementation we does not use mask or padding, which makes the logic
# more clear and saves memory. The trade-off is that the implementation is # more clear and saves memory. The trade-off is that the implementation is
# slower; we will improve with future DGL optimizations. # slower; we will improve with future DGL optimizations.
...@@ -874,3 +874,8 @@ Transformer Tutorial ...@@ -874,3 +874,8 @@ Transformer Tutorial
# .. |image9| image:: https://s1.ax1x.com/2018/12/06/F1sGod.png # .. |image9| image:: https://s1.ax1x.com/2018/12/06/F1sGod.png
# .. |image10| image:: https://s1.ax1x.com/2018/12/06/F1r8Cq.gif # .. |image10| image:: https://s1.ax1x.com/2018/12/06/F1r8Cq.gif
# #
# .. note::
# We apologize that this notebook itself is not runnable due to many dependencies,
# please download the `7_transformer.py <https://s3.us-east-2.amazonaws.com/dgl.ai/tutorial/7_transformer.py>`__,
# and copy the python script to directory ``examples/pytorch/transformer``
# then run ``python 7_transformer.py`` to see how it works.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment