[Doc] main doc page update and contribute guide (#336)

* reorg the doc mainpage

* contribute guide

CONTRIBUTING.md → CONTRIBUTORS.md

@@ -2,21 +2,9 @@
 
 Contribution is always welcomed. A good starting place is the roadmap issue, where
 you can find our current milestones. All contributions must go through pull requests
-and be reviewed by the committors.
+and be reviewed by the committors. See our [contribution guide](https://docs.dgl.ai/contribute.html) for more details.
 
-For document improvement, simply PR the change and prepend the title with `[Doc]`.
-
-For new features, we suggest first create an issue using the feature request template.
-Follow the template to describe the features you want to implement and your plans.
-We also suggest pick the features from the roadmap issue because they are more likely
-to be incoporated in the next release.
-
-For bug fix, we suggest first create an issue using the bug report template if the
-bug has not been reported yet. Please reply the issue that you'd like to help. Once
-the task is assigned, make the change in your fork and PR the codes. Remember to
-also refer to the issue where the bug is reported.
-
-Once your PR is merged, congratulation, you are now an contributor to the DGL project.
+Once your contribution is accepted and merged, congratulation, you are now an contributor to the DGL project.
 We will put your name in the list below and also on our [website](https://www.dgl.ai/ack).
 
 Contributors

README.md

@@ -156,7 +156,7 @@ Check out the open source book [*Dive into Deep Learning*](http://en.diveintodee
 Please let us know if you encounter a bug or have any suggestions by [filing an issue](https://github.com/dmlc/dgl/issues).
 
 We welcome all contributions from bug fixes to new features and extensions.
-We expect all contributions discussed in the issue tracker and going through PRs.  Please refer to the PR guide.
+We expect all contributions discussed in the issue tracker and going through PRs.  Please refer to our [contribution guide](https://docs.dgl.ai/contribute.html).
 
 
 ## The Team

docs/README.md

 DGL document and tutorial folder
 ================================
 
+Requirements
+------------
+* sphinx
+* sphinx-gallery
+* Both pytorch and mxnet installed.
+
 Build documents
 ---------------
 First, clean up existing files:

docs/source/conf.py

@@ -211,4 +211,5 @@ sphinx_gallery_conf = {
     'gallery_dirs' : gallery_dirs,
     'within_subsection_order' : FileNameSortKey,
     'filename_pattern' : '.py',
+    'download_all_examples' : False,
 }

docs/source/contribute.rst

+Contribute to DGL
+=================
+
+Any contribution to DGL is welcome. This guide covers everything
+about how to contribute to DGL.
+
+General development process
+---------------------------
+
+A non-inclusive list of types of contribution is as follows:
+
+* New features and enhancements (`example <https://github.com/dmlc/dgl/pull/331>`__).
+* Bugfix (`example <https://github.com/dmlc/dgl/pull/247>`__).
+* Document improvement (`example <https://github.com/dmlc/dgl/pull/263>`__).
+* New models and examples (`example <https://github.com/dmlc/dgl/pull/279>`__).
+
+For features and bugfix, we recommend first raise an `issue <https://github.com/dmlc/dgl/issues>`__
+using the corresponding issue template, so that the change could be fully discussed with
+the community before implementation. For document improvement and new models, we suggest
+post a thread in our `discussion forum <https://discuss.dgl.ai>`__.
+
+Before development, please first read the following sections about coding styles and testing.
+All the changes need to be reviewed in the form of `pull request <https://github.com/dmlc/dgl/pulls>`__.
+Our `committors <https://github.com/orgs/dmlc/teams/dgl-team/members>`__
+(who have write permission on the repository) will review the codes and suggest the necessary
+changes. The PR could be merged once the reviewers approve the changes.
+
+Git setup (for developers)
+--------------------------
+
+First, fork the DGL github repository. Suppose the forked repo is ``https://github.com/username/dgl``.
+
+Clone your forked repository locally:
+
+.. code-block:: bash
+
+   git clone --recursive https://github.com/username/dgl.git
+
+
+Setup the upstream to the DGL official repository:
+
+.. code-block:: bash
+
+   git remote add upstream https://github.com/dmlc/dgl.git
+
+You could verify the remote setting by typing ``git remove -v``:
+
+.. code-block:: bash
+
+   origin  https://github.com/username/dgl.git (fetch)
+   origin  https://github.com/username/dgl.git (push)
+   upstream        https://github.com/dmlc/dgl.git (fetch)
+   upstream        https://github.com/dmlc/dgl.git (push)
+
+During developing, we suggest work on another branch than the master.
+
+.. code-block:: bash
+
+   git branch working-branch
+   git checkout working-branch
+
+Once the changes are done, `create a pull request <https://help.github.com/articles/creating-a-pull-request/>`__
+so we could review your codes.
+
+Once the pull request is merged, update your forked repository and delete your working branch:
+
+.. code-block:: bash
+
+   git checkout master
+   git pull upstream master
+   git push origin master  # update your forked repo
+   git branch -D working-branch  # the local branch could be deleted
+
+Coding styles
+-------------
+
+For python codes, we generally follow the `PEP8 style guide <https://www.python.org/dev/peps/pep-0008/>`__.
+The python comments follow `NumPy style python docstrings <https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html>`__.
+
+For C++ codes, we generally follow the `Google C++ style guide <https://google.github.io/styleguide/cppguide.html>`__.
+The C++ comments should be `Doxygen compatible <http://www.doxygen.nl/manual/docblocks.html#cppblock>`__.
+
+Coding styles check is mandatory for every pull requests. To ease the development, please check it
+locally first (require cpplint and pylint to be installed first):
+
+.. code-block:: bash
+
+   bash tests/scripts/task_lint.sh
+
+The python code style configure file is ``tests/scripts/pylintrc``. We tweak it a little bit from
+the standard. For example, following variable names are accepted:
+
+* ``i,j,k``: for loop variables
+* ``u,v``: for representing nodes
+* ``e``: for representing edges
+* ``g``: for representing graph
+* ``fn``: for representing functions
+* ``n,m``: for representing sizes
+* ``w,x,y``: for representing weight, input, output tensors
+* ``_``: for unused variables
+
+Building and Testing
+--------------------
+
+To build DGL locally, follow the steps described in :ref:`Install from source <install-from-source>`.
+However, to ease the development, we suggest NOT install DGL but directly working in the source tree.
+To achieve this, export following environment variables:
+
+.. code-block:: bash
+
+   export DGL_HOME=/path/to/your/dgl/clone
+   export DGL_LIBRARY_PATH=$DGL_HOME/build
+   export PYTHONPATH=$PYTHONPATH:$DGL_HOME/python
+
+You could test the build by running the following command and see the path of your local clone.
+
+.. code-block:: bash
+
+   python -c 'import dgl; print(dgl.__path__)'
+
+TBD by Quan about how to run and write unittests.
+
+Building documents
+------------------
+
+If the change is about document improvement, we suggest build the document and render it locally
+before pull request. See instructions `here <https://github.com/dmlc/dgl/tree/master/docs>`__.
+
+Data hosting
+------------
+
+If the change is about new models or applications, it is very common to have some data files. Data
+files are not allowed to be uploaded to our repository. Instead, they should be hosted on the
+cloud storage service (e.g. dropbox, Amazon S3) and downloaded on-the-fly. See our :ref:`dataset APIs <apidata>`
+for more details. All the dataset of current DGL models are hosted on Amazon S3. If you want your
+dataset to be hosted as well, please post in our `discussion forum <https://discuss.dgl.ai>`__.

docs/source/index.rst

@@ -37,36 +37,113 @@ frameworks. It provides a backend adapter interface that allows easy porting
 to other tensor-based, autograd-enabled frameworks. Currently, our prototype
 works with MXNet/Gluon and PyTorch.
 
-Free software
--------------
-DGL is free software; you can redistribute it and/or modify it under the terms
-of the Apache License 2.0. We welcome contributions.
-Join us on `GitHub <https://github.com/dmlc/dgl>`_ and check out our `contribution guidelines <https://github.com/dmlc/dgl/blob/master/CONTRIBUTING.md>`_.
-
-History
--------
-Prototype of DGL started in early Spring, 2018, at NYU Shanghai by Prof. Zheng
-Zhang and Quan Gan. Serious development began when Minjie, Lingfan and Prof
-Jinyang Li from NYU's system group joined, flanked by a team of student
-volunteers at NYU Shanghai, Fudan and other universities (Yu, Zihao, Murphy,
-Allen, Qipeng, Qi, Hao), as well as early adopters at the CILVR lab (Jake
-Zhao). Development accelerated when AWS MXNet Science team joined force, with
-Da Zheng, Alex Smola, Haibin Lin, Chao Ma and a number of others. For full
-credit, see `here <https://www.dgl.ai/ack>`_.
+Get Started
+-----------
 
 .. toctree::
    :maxdepth: 1
    :caption: Get Started
+   :hidden:
    :glob:
 
    install/index
 
+Follow the :doc:`instructions<install/index>` to install DGL. The :doc:`DGL at a glance<tutorials/basics/1_first>`
+is the most common place to get started with. Each tutorial is accompanied with a runnable
+python script and jupyter notebook that can be downloaded.
+
+.. ================================================================================================
+   (start) MANUALLY INCLUDE THE GENERATED TUTORIALS/BASIC/INDEX.RST HERE TO EMBED THE EXAMPLES
+   ================================================================================================
+
+.. raw:: html
+
+    <div class="sphx-glr-thumbcontainer">
+
+.. only:: html
+
+    .. figure:: /tutorials/basics/images/thumb/sphx_glr_1_first_thumb.png
+
+        :ref:`sphx_glr_tutorials_basics_1_first.py`
+
+.. raw:: html
+
+    </div>
+
+
 .. toctree::
-   :maxdepth: 2
-   :caption: Tutorials
+   :hidden:
+
+   /tutorials/basics/1_first
+
+.. raw:: html
+
+    <div class="sphx-glr-thumbcontainer">
+
+.. only:: html
+
+    .. figure:: /tutorials/basics/images/thumb/sphx_glr_2_basics_thumb.png
+
+        :ref:`sphx_glr_tutorials_basics_2_basics.py`
+
+.. raw:: html
+
+    </div>
+
+
+.. toctree::
+   :hidden:
+
+   /tutorials/basics/2_basics
+
+.. raw:: html
+
+    <div class="sphx-glr-thumbcontainer">
+
+.. only:: html
+
+    .. figure:: /tutorials/basics/images/thumb/sphx_glr_3_pagerank_thumb.png
+
+        :ref:`sphx_glr_tutorials_basics_3_pagerank.py`
+
+.. raw:: html
+
+    </div>
+
+.. toctree::
+   :hidden:
+
+   /tutorials/basics/3_pagerank
+
+.. raw:: html
+
+    <div style='clear:both'></div>
+
+.. ================================================================================================
+   (end) MANUALLY INCLUDE THE GENERATED TUTORIALS/BASIC/INDEX.RST HERE TO EMBED THE EXAMPLES
+   ================================================================================================
+
+Learning DGL through examples
+-----------------------------
+
+The model tutorials are categorized based on the way they utilize DGL APIs.
+
+* :ref:`Graph Neural Network and its variant <tutorials1-index>`: Learn how to use DGL to train
+  popular **GNN models** on one input graph.
+* :ref:`Dealing with many small graphs <tutorials2-index>`: Learn how to **batch** many
+  graph samples for max efficiency.
+* :ref:`Generative models <tutorials3-index>`: Learn how to deal with **dynamically-changing graphs**.
+* :ref:`Old (new) wines in new bottle <tutorials4-index>`: Learn how to combine DGL with tensor-based
+  DGL framework in a flexible way. Explore new perspective on traditional models by graphs.
+
+Or go through all of them :doc:`here <tutorials/models/index>`.
+
+.. toctree::
+   :maxdepth: 3
+   :caption: Model Tutorials
+   :hidden:
    :glob:
 
-   tutorials/basics/index
    tutorials/models/index
 
 .. toctree::
@@ -78,10 +155,35 @@ credit, see `here <https://www.dgl.ai/ack>`_.
 
 .. toctree::
    :maxdepth: 1
+   :caption: Notes
+   :hidden:
    :glob:
 
+   contribute
    faq
    env_var
+   resources
+
+Free software
+-------------
+DGL is free software; you can redistribute it and/or modify it under the terms
+of the Apache License 2.0. We welcome contributions.
+Join us on `GitHub <https://github.com/dmlc/dgl>`_ and check out our
+`contribution guidelines <https://github.com/dmlc/dgl/blob/master/CONTRIBUTING.md>`_.
+
+History
+-------
+Prototype of DGL started in early Spring, 2018, at NYU Shanghai by Prof. `Zheng
+Zhang <https://shanghai.nyu.edu/academics/faculty/directory/zheng-zhang>`_ and
+Quan Gan. Serious development began when `Minjie
+<https://jermainewang.github.io/>`_, `Lingfan <https://cs.nyu.edu/~lingfan/>`_
+and Prof. `Jinyang Li <http://www.news.cs.nyu.edu/~jinyang/>`_ from NYU's
+system group joined, flanked by a team of student volunteers at NYU Shanghai,
+Fudan and other universities (Yu, Zihao, Murphy, Allen, Qipeng, Qi, Hao), as
+well as early adopters at the CILVR lab (Jake Zhao). Development accelerated
+when AWS MXNet Science team joined force, with Da Zheng, Alex Smola, Haibin
+Lin, Chao Ma and a number of others. For full credit, see `here
+<https://www.dgl.ai/ack>`_.
 
 Index
 -----

docs/source/install/index.rst

@@ -68,6 +68,8 @@ The backend is controlled by ``DGLBACKEND`` environment variable, which defaults
 | numpy   | NumPy   | Does not support gradient computation            |
 +---------+---------+--------------------------------------------------+
 
+.. _install-from-source:
+
 Install from source
 -------------------
 First, download the source files from GitHub:

docs/source/resources.rst

+Resources
+=========
+* If you are new to deep learning, `Dive into Deep Learning <http://diveintodeeplearning.org>`__
+  is a nice book to start with.
+* `Pytorch tutorials <https://pytorch.org/tutorials/>`__
+* Thomas Kipf's `blog on Graph Convolutional Networks <https://tkipf.github.io/graph-convolutional-networks/>`__

tutorials/basics/3_pagerank.py

@@ -241,5 +241,17 @@ print(g.ndata['pv'])
 ###############################################################################
 # Next steps
 # ----------
-# Check out :doc:`GCN <../models/1_gnn/1_gcn>` and :doc:`Capsule <../models/4_old_wines/2_capsule>`
-# for more model implemenetations in DGL.
+# 
+# It is time to move on to some real models in DGL.
+#
+# * Check out the :doc:`overview page<../models/index>`
+#   of all the model tutorials.
+# * Would like to know more about Graph Neural Networks? Start with the
+#   :doc:`GCN tutorial<../models/1_gnn/1_gcn>`.
+# * Would like to know how DGL batches multiple graphs? Start with the
+#   :doc:`TreeLSTM tutorial<../models/2_small_graph/3_tree-lstm>`.
+# * Would like to play with some graph generative models? Start with our tutorial
+#   on the :doc:`Deep Generative Model of Graphs<../models/3_generative_model/5_dgmg>`.
+# * Would like to see how traditional models are interpreted in a view of graph?
+#   Check out our tutorials on :doc:`CapsuleNet<../models/4_old_wines/2_capsule>` and
+#   :doc:`Transformer<../models/4_old_wines/7_transformer>`.

tutorials/basics/README.txt

-Basic Tutorials
-===============
-
-These tutorials cover the basics of DGL. If you are new to deep learning, please refer to the open source book `Dive into Deep Learning <http://diveintodeeplearning.org>`__.

tutorials/models/1_gnn/README.txt

 .. _tutorials1-index:
 
 Graph Neural Network and its variant
-------------------------------------
+====================================
 
 * **GCN** `[paper] <https://arxiv.org/abs/1609.02907>`__ `[tutorial]
   <1_gnn/1_gcn.html>`__ `[Pytorch code]

tutorials/models/2_small_graph/README.txt

 .. _tutorials2-index:
 
-
 Dealing with many small graphs
-------------------------------
+==============================
 
 * **Tree-LSTM** `[paper] <https://arxiv.org/abs/1503.00075>`__ `[tutorial]
   <2_small_graph/3_tree-lstm.html>`__ `[code]

tutorials/models/3_generative_model/README.txt

 .. _tutorials3-index:
 
 Generative models
-------------------------------
+==================
 
 * **DGMG** `[paper] <https://arxiv.org/abs/1803.03324>`__ `[tutorial]
   <3_generative_model/5_dgmg.html>`__ `[code]

tutorials/models/4_old_wines/README.txt

@@ -2,7 +2,8 @@
 
 
 Old (new) wines in new bottle
------------------------------
+=============================
+
 * **Capsule** `[paper] <https://arxiv.org/abs/1710.09829>`__ `[tutorial]
   <4_old_wines/2_capsule.html>`__ `[code]
   <https://github.com/dmlc/dgl/tree/master/examples/pytorch/capsule>`__:

tutorials/models/README.txt

-Graph-based Neural Network Models
-=================================
+Model overview
+==============
 
 We developed DGL with a broad range of applications in mind. Building
 state-of-art models forces us to think hard on the most common and useful APIs,
@@ -14,4 +14,3 @@ directions.
 We categorize the models below, providing links to the original code and
 tutorial when appropriate. As will become apparent, these models stress the use
 of different DGL APIs.
-