Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
dgl
Commits
8bc01c63
Commit
8bc01c63
authored
Dec 11, 2018
by
Zihao Ye
Committed by
Minjie Wang
Dec 10, 2018
Browse files
[Tutorial] Tutorial fix (SSE link & Transformer) (#285)
* upd tutorial * upd tutorial
parent
322bd713
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
10 additions
and
5 deletions
+10
-5
tutorials/models/1_gnn/README.txt
tutorials/models/1_gnn/README.txt
+2
-2
tutorials/models/4_old_wines/7_transformer.py
tutorials/models/4_old_wines/7_transformer.py
+8
-3
No files found.
tutorials/models/1_gnn/README.txt
View file @
8bc01c63
...
@@ -32,8 +32,8 @@ Graph Neural Network and its variant
...
@@ -32,8 +32,8 @@ Graph Neural Network and its variant
operation, sparse-matrix tensor operations, along with message-passing with
operation, sparse-matrix tensor operations, along with message-passing with
DGL.
DGL.
* **SSE** `[paper] <http://proceedings.mlr.press/v80/dai18a/dai18a.pdf>`__
* **SSE** `[paper] <http://proceedings.mlr.press/v80/dai18a/dai18a.pdf>`__
`[tutorial]
`[tutorial
<1_gnn/8_sse_mx.html>
]
` `[code]
<1_gnn/8_sse_mx.html>`
__
`[code]
<https://github.com/jermainewang/dgl/blob/master/examples/mxnet/sse>`__:
<https://github.com/jermainewang/dgl/blob/master/examples/mxnet/sse>`__:
the emphasize here is *giant* graph that cannot fit comfortably on one GPU
the emphasize here is *giant* graph that cannot fit comfortably on one GPU
card. SSE is an example to illustrate the co-design of both algorithm and
card. SSE is an example to illustrate the co-design of both algorithm and
...
...
tutorials/models/4_old_wines/7_transformer.py
View file @
8bc01c63
...
@@ -44,7 +44,7 @@ Transformer Tutorial
...
@@ -44,7 +44,7 @@ Transformer Tutorial
# v_i = W_v\cdot x_i\\
# v_i = W_v\cdot x_i\\
# \textrm{score} = q_j^T k_i
# \textrm{score} = q_j^T k_i
#
#
# where :math:`W_q, W_k, W_v \in \mathbb{R}^{n\times
n
}` map the
# where :math:`W_q, W_k, W_v \in \mathbb{R}^{n\times
d_k
}` map the
# representations :math:`x` to “query”, “key”, and “value” space
# representations :math:`x` to “query”, “key”, and “value” space
# respectively.
# respectively.
#
#
...
@@ -472,8 +472,8 @@ Transformer Tutorial
...
@@ -472,8 +472,8 @@ Transformer Tutorial
#
#
# By calling ``update_graph`` function, we can “DIY our own
# By calling ``update_graph`` function, we can “DIY our own
# Transformer” on any subgraphs with nearly the same code. This
# Transformer” on any subgraphs with nearly the same code. This
# flexibility enables us to discover new, sparse structures (c.f.
# flexibility enables us to discover new, sparse structures (c.f.
local attention
# `here <https://arxiv.org/pdf/1508.04025.pdf>`__). Note in our
#
mentioned
`here <https://arxiv.org/pdf/1508.04025.pdf>`__). Note in our
# implementation we does not use mask or padding, which makes the logic
# implementation we does not use mask or padding, which makes the logic
# more clear and saves memory. The trade-off is that the implementation is
# more clear and saves memory. The trade-off is that the implementation is
# slower; we will improve with future DGL optimizations.
# slower; we will improve with future DGL optimizations.
...
@@ -874,3 +874,8 @@ Transformer Tutorial
...
@@ -874,3 +874,8 @@ Transformer Tutorial
# .. |image9| image:: https://s1.ax1x.com/2018/12/06/F1sGod.png
# .. |image9| image:: https://s1.ax1x.com/2018/12/06/F1sGod.png
# .. |image10| image:: https://s1.ax1x.com/2018/12/06/F1r8Cq.gif
# .. |image10| image:: https://s1.ax1x.com/2018/12/06/F1r8Cq.gif
#
#
# .. note::
# We apologize that this notebook itself is not runnable due to many dependencies,
# please download the `7_transformer.py <https://s3.us-east-2.amazonaws.com/dgl.ai/tutorial/7_transformer.py>`__,
# and copy the python script to directory ``examples/pytorch/transformer``
# then run ``python 7_transformer.py`` to see how it works.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment