1. 20 May, 2019 1 commit
    • Da Zheng's avatar
      [BUGFix] Improve multi-processing training (#526) · cdfca992
      Da Zheng authored
      * fix.
      
      * add comment.
      
      * remove.
      
      * temp fix.
      
      * initialize for shared memory.
      
      * fix graphsage.
      
      * fix gcn.
      
      * add more unit tests.
      
      * add more tests.
      
      * avoid creating shared-memory exclusively.
      
      * redefine remote initializer.
      
      * improve initializer.
      
      * fix unit test.
      
      * fix lint.
      
      * fix lint.
      
      * initialize data in the graph store server properly.
      
      * fix test.
      
      * fix test.
      
      * fix test.
      
      * small fix.
      
      * add comments.
      
      * cleanup server.
      
      * test graph store with a random port.
      
      * print.
      
      * print to stderr.
      
      * test1
      
      * test2
      
      * remove comment.
      
      * adjust the initializer signature.
      cdfca992
  2. 07 May, 2019 1 commit
    • Da Zheng's avatar
      [Model] add multiprocessing training with sampling. (#484) · 3a1392e6
      Da Zheng authored
      * reorganize sampling code.
      
      * add multi-process training.
      
      * speed up gcn_cv
      
      * fix graphsage_cv.
      
      * add new API in graph store.
      
      * update barrier impl.
      
      * support both local and distributed training.
      
      * fix multiprocess train.
      
      * fix.
      
      * fix barrier.
      
      * add script for loading data.
      
      * multiprocessing sampling.
      
      * accel training.
      
      * replace pull with spmv for speedup.
      
      * nodeflow copy from parent with context.
      
      * enable GPU.
      
      * fix a bug in graph store.
      
      * enable multi-GPU training.
      
      * fix lint.
      
      * add comments.
      
      * rename to run_store_server.py
      
      * fix gcn_cv.
      
      * fix a minor bug in sampler.
      
      * handle error better in graph store.
      
      * improve graphsage_cv for distributed mode.
      
      * update README.
      
      * fix.
      
      * update.
      3a1392e6
  3. 16 Apr, 2019 1 commit
  4. 08 Apr, 2019 1 commit
    • Da Zheng's avatar
      [Feature] Create shared memory graph store. (#468) · bfdd1eaa
      Da Zheng authored
      * accelerate gcn_ns.
      
      * add timing.
      
      * run infer with whole graph.
      
      * distributed gcn_ns.
      
      * reconstruct gcn_ns.
      
      * minor fix.
      
      * change graphsage_cv for numa.
      
      * fix #OMP threads.
      
      * accelerate graphsage_cv.
      
      * fix a weird bug.
      
      * add profiler in graphsage_cv.
      
      * accelerate graphsage_cv.
      
      manually aggregate neighbors' embeddings with pull.
      
      * load csr directly in gcn_ns_sc.
      
      * parallel sort for graph index.
      
      * Revert "parallel sort for graph index."
      
      This reverts commit 86fe2c7117fe5e56b0d481b39849c258b166945b.
      
      * run gcn_ns_sc on GPUs.
      
      * acc gcn_cv_sc.
      
      * change gcn_cv for numa.
      
      * fix gcn_cv to use numa and gpu.
      
      * improve graphsage_cv to use numa and gpu.
      
      * improve gcn_ns.
      
      * improve graphsage_cv.
      
      * init shared memory graph store.
      
      * fix.
      
      * enable init ndata.
      
      * improve tests.
      
      * add bidirectional communication.
      
      * link to rt.
      
      * fix compilation error.
      
      * fix shared memory init.
      
      * use MessageQueue for inter-process communication.
      
      * reconstruct immutable graph csr.
      
      * fix gcn.
      
      * load csr to shared memory.
      
      * fix minor bugs.
      
      * add comments.
      
      * refactor SharedMemory.
      
      * fix bugs in ImmutableGraph.
      
      * create CSR graph from shared memory.
      
      * add more test for loading a csr graph.
      
      * terminate graph store properly.
      
      * allow initializing ndata in the graph store server.
      
      * use RPC for inter-process communication.
      
      * a script for loading a graph.
      
      * allow customizing port.
      
      * list all ndata and edata.
      
      * support dtype.
      
      * reorganize SharedMemoryGraphStore.
      
      * fix ndata shape.
      
      * reconstruct gcn_ns.
      
      * print info.
      
      * set omp in gcn_ns.
      
      * reset sampling examples.
      
      * fix lint.
      
      * fix lint.
      
      * reset gcn.
      
      * disable shared memory in windows.
      
      * fix.
      
      * fix.
      
      * reset changes.
      
      * revert nodeflow changes.
      
      * fix cmake.
      
      * fix test.
      
      * fix test.
      
      * fix test.
      
      * fix test.
      
      * add comments.
      
      * fix test.
      
      * move vector out.
      
      * fix lint.
      
      * fix lint.
      
      * move SharedMemory.
      
      * update cmake.
      
      * update comment.
      
      * fix comments.
      
      * Revert "update cmake."
      
      This reverts commit 592445e37077f70a6e3f2e5245f9a3d086b04f3b.
      
      * update cmake.
      
      * add comments.
      
      * rename.
      
      * change the comment.
      
      * fix a bug.
      
      * rename.
      
      * add comments.
      
      * add comments.
      
      * add init_edata.
      
      * rewrite memory alloc.
      
      * move vector to CSR.
      
      * fix.
      
      * init data.
      
      * Revert "init data."
      
      This reverts commit 2b217b9553911b7dd84a9f1d9b68430b5aa18e23.
      
      * init data.
      
      * init new columns correctly.
      bfdd1eaa
  5. 25 Feb, 2019 1 commit
    • Minjie Wang's avatar
      [WIP] [NN] Refactor NN package (#406) · 565f0c88
      Minjie Wang authored
      * refactor graph conv
      
      * docs & tests
      
      * fix lint
      
      * fix lint
      
      * fix lint
      
      * fix lint script
      
      * fix lint
      
      * Update
      
      * Style fix
      
      * Fix style
      
      * Fix style
      
      * Fix gpu case
      
      * Fix for gpu case
      
      * Hotfix edgesoftmax docs
      
      * Handle repeated features
      
      * Add docstring
      
      * Set default arguments
      
      * Remove dropout from nn.conv
      
      * Fix
      
      * add util fn for renaming
      
      * revert gcn_spmv.py
      
      * mx folder
      
      * fix wierd bug
      
      * fix mx
      
      * fix lint
      565f0c88
  6. 17 Jan, 2019 1 commit
    • Da Zheng's avatar
      [Feature] Reimplement Immutable graph index in DGL (#342) · 929742b5
      Da Zheng authored
      * init.
      
      * it's compiled.
      
      * add immutable graph constructor.
      
      * add immutable graph API.
      
      * fix.
      
      * impl get adjacency matrix.
      
      * fix.
      
      * fix graph_index from scipy matrix.
      
      * add neighbor sampling.
      
      * remap vertex ids.
      
      * fix.
      
      * move sampler test.
      
      * fix tests.
      
      * add comments
      
      * remove mxnet-specific immutable graph.
      
      * fix.
      
      * fix lint.
      
      * fix.
      
      * try to fix windows compile error.
      
      * fix.
      
      * fix.
      
      * add test.
      
      * unify Graph and ImmutableGraph.
      
      * fix bugs.
      
      * fix compile.
      
      * move immutable graph.
      
      * fix.
      
      * remove print.
      
      * fix lint.
      
      * fix
      
      * fix lint.
      
      * fix lint.
      
      * fix test.
      
      * fix comments.
      
      * merge GraphIndex and ImmutableGraphIndex.
      
      * temp fix.
      
      * impl GetAdj.
      
      * fix lint
      
      * fix.
      
      * fix.
      
      * fix.
      
      * fix.
      
      * fix.
      
      * use csr only for readonly graph.
      
      * Revert "use csr only for readonly graph."
      
      This reverts commit 8e24bb033af8504531b22849de5b7567b168e0d5.
      
      * remove code.
      
      * fix.
      
      * fix.
      
      * fix.
      
      * fix.
      
      * fix.
      
      * fix.
      
      * address comments.
      
      * fix for comments.
      
      * fix comments.
      
      * revert.
      
      * move test_graph_index to compute.
      
      * fix.
      
      * fix.
      
      * impl GetAdj for coo.
      
      * fix.
      
      * fix tests.
      
      * address comments.
      
      * address comments.
      
      * fix comment.
      
      * address comments.
      
      * use lambda.
      
      * other comments.
      
      * address comments.
      
      * modify the semantics of edges.
      
      * fix order.
      
      * use DGLIdIter
      
      * fix.
      
      * remove NotImplemented.
      
      * revert some code.
      929742b5
  7. 06 Jan, 2019 1 commit
    • Quan (Andy) Gan's avatar
      [Test] Unify tests for different backends (#333) · e19cd62e
      Quan (Andy) Gan authored
      * test basics
      
      * batched graph & filter, mxnet filter fix
      
      * frame and function; bugfix
      
      * test graph adj and inc matrices
      
      * fixing start = 0 for mxnet
      
      * test index
      
      * inplace update & line graph
      
      * multi send recv
      
      * more tests
      
      * oops
      
      * more tests
      
      * removing old test files; readonly graphs for mxnet still kept
      
      * modifying test scripts
      
      * adding a placeholder for pytorch to reserve directory
      
      * torch 0.4.1 compat fixes
      
      * moving backend out of compute to avoid nose detection
      
      * tests guide
      
      * mx sparse-to-dense/sparse-to-numpy is buggy
      
      * oops
      
      * contribution guide for unit tests
      
      * printing incmat
      
      * printing dlpack
      
      * small push
      
      * typo
      
      * fixing duplicate entries that causes undefined behavior
      
      * move equal comparison to backend
      e19cd62e
  8. 01 Jan, 2019 1 commit
    • Minjie Wang's avatar
      [Lint] Pylint (#330) · 4bd4d6e3
      Minjie Wang authored
      * fix lint for graph_index.py
      
      * pylint for base.py
      
      * pylint for batched_graph.py
      
      * pylint for frame.py; simplify and fix bugs in frame when index is slice type
      
      * pylint for graph.py
      
      * pylint for immutable_graph_index.py
      
      * pylint for init.py
      
      * pylint for rest files in root package
      
      * pylint for _ffi package
      
      * pylint for function package
      
      * pylint for runtime package
      
      * pylint for runtime.ir package
      
      * add pylint to ci
      
      * fix mx tests
      
      * fix lint errors
      
      * fix ci
      
      * fix as requested
      
      * fix lint
      4bd4d6e3
  9. 17 Dec, 2018 2 commits
  10. 16 Dec, 2018 2 commits
  11. 07 Dec, 2018 1 commit
  12. 06 Dec, 2018 1 commit
  13. 05 Dec, 2018 5 commits
  14. 03 Dec, 2018 2 commits
  15. 26 Nov, 2018 1 commit
    • Minjie Wang's avatar
      [Bugfix][Runtime] Zero degree behaviors (#177) · b1eeb934
      Minjie Wang authored
      * fix recv nodes are all 0deg; fix hybriddict does not through keyerror properly
      
      * fallback to apply_nodes when all nodes are 0deg; WIP on pull spmv 0deg
      
      * new 0deg behavior
      
      * new 0deg behavior
      
      * update mx utest for pull-0deg
      
      * fix mx
      
      * fix mx
      
      * get rid of unnecessary sort-n-unique
      b1eeb934
  16. 22 Nov, 2018 1 commit
    • Lingfan Yu's avatar
      [Runtime] Scheduler and Executor (#140) · deb653f8
      Lingfan Yu authored
      * executor api
      
      * draft executor interface
      
      * WIP
      
      * revert changes to avoid conflict with api change
      
      * core scheduling logic
      
      * WIP: build graph adj
      
      * incidence matrix for in edges
      
      * support incidence matrix for partial recv nodes
      
      * improve
      
      * build adjmat in scheduler
      
      * graph store
      
      * get degree bucketing schedule
      
      * connect to c++ degree bucketing
      
      * conceptual executor creation code
      
      * executor comments
      
      * fix
      
      * more executor comments
      
      * WIP: full send_and_recv schedule
      
      * most schedulers
      
      * simplify scheduler
      
      * executors
      
      * runtime
      
      * builtin function base class
      
      * adj indices and shape
      
      * completely refactor scheduler
      
      * rename and move bundled out to function.py
      
      * use_edge_feature in msg func
      
      * rewrite scheduler
      
      * node edge executor
      
      * connect with graph api
      
      * handle zero degree
      
      * misc
      
      * fix test cases
      
      * fix a good many bugs...
      
      * remove old scheduler
      
      * push and pull
      
      * fix send recv
      
      * c++ lint
      
      * fix batched send recv
      
      * hot fix for mxnet
      
      * typo
      
      * write back executor
      
      * apply node edge
      
      * clean up, doc string
      
      * fix as requested
      
      * refactor
      
      * fix
      
      * WIP
      
      * WIP
      
      * ir draft
      
      * more on ir
      
      * WIP: spmv schedule
      
      * WIP
      
      * recv schedule
      
      * refactor
      
      * WIP
      
      * snr degree bucketing
      
      * snr scheduler
      
      * move prog to graph.py; rename
      
      * unittest for send/recv
      
      * remove some legacy codes
      
      * WIP: update_all
      
      * pass test_basics
      
      * passed all current utests
      
      * more utests; fix mx utest
      
      * WIP: fixing zero deg initial value
      
      * some tests
      
      * fix 0deg problem
      
      * fix mx
      
      * fix mx
      
      * some notes
      
      * fix as requested
      deb653f8
  17. 21 Nov, 2018 1 commit
  18. 14 Nov, 2018 2 commits
    • Da Zheng's avatar
      [API] Subgraph sampling API. (#138) · 2389df81
      Da Zheng authored
      * add neighbor sampler.
      
      * fix sampler.
      
      * handle non-existing nodes differently in MapParentIdToSubgraphId.
      
      * fix sampling tests.
      
      * parallelize sampling.
      
      * add more tests.
      
      * fix a bug in subgraph loader.
      
      * get number of sampled nodes directly.
      
      * load mxnet CSR to DGLGraph directly.
      
      * create graph from mxnet csr array directly.
      
      * use subgraph loader in SSE.
      
      * fix a bug in sampling.
      
      * output framework tensor.
      
      * fix.
      
      * remove unnecessary code.
      
      * fix tests.
      
      * add SSE README.
      
      * move to contrib.
      
      * address comments.
      
      * add tests.
      2389df81
    • Minjie Wang's avatar
      [GraphIndex] refactor graph caching (#150) · 048f6d7a
      Minjie Wang authored
      * refactor graph caching
      
      * fix mx test
      
      * fix typo
      048f6d7a
  19. 08 Nov, 2018 1 commit
    • Da Zheng's avatar
      [MXNet][API] move to the new API (#123) · bd0e4fa0
      Da Zheng authored
      * move gat to the new api.
      
      * fix gcn.
      
      * update sse.
      
      * fix dgl core.
      
      * update sse.
      
      * fix small bugs in dgl core.
      
      * fix mxnet tests.
      
      * retrigger
      
      * address comments and fix more bugs.
      
      * fix
      
      * fix tests.
      bd0e4fa0
  20. 05 Nov, 2018 1 commit
    • Minjie Wang's avatar
      [Backend] backend interface (#109) · 7241a9c0
      Minjie Wang authored
      * backend interface
      
      * small fix
      
      * more comments to the data type dict
      
      * WIP
      
      * convert_to and narrow
      
      * WIP
      
      * pytorch and numpy backend; WIP on mxnet backend
      
      * mxnet backend
      
      * narrow
      
      * Fix all usages
      
      * fix for mx
      
      * fix for mx
      
      * fix mx
      
      * fix mx
      
      * fix mx
      
      * fix mx
      
      * fix mx
      
      * fix mx
      
      * fix mx
      
      * revert jenkins
      
      * add sparse_matrix api
      
      * sparse matrix api
      
      * some fixme
      
      * Fix as requested
      7241a9c0
  21. 02 Nov, 2018 1 commit
    • Minjie Wang's avatar
      [API][Doc] API change & basic tutorials (#113) · 68ec6247
      Minjie Wang authored
      * Add SH tutorials
      
      * setup sphinx-gallery; work on graph tutorial
      
      * draft dglgraph tutorial
      
      * update readme to include document url
      
      * rm obsolete file
      
      * Draft the message passing tutorial
      
      * Capsule code (#102)
      
      * add capsule example
      
      * clean code
      
      * better naming
      
      * better naming
      
      * [GCN]tutorial scaffold
      
      * fix capsule example code
      
      * remove previous capsule example code
      
      * graph struc edit
      
      * modified:   2_graph.py
      
      * update doc of capsule
      
      * update capsule docs
      
      * update capsule docs
      
      * add msg passing prime
      
      * GCN-GAT tutorial Section 1 and 2
      
      * comment for API improvement
      
      * section 3
      
      * Tutorial API change (#115)
      
      * change the API as discusses; toy example
      
      * enable the new set/get syntax
      
      * fixed pytorch utest
      
      * fixed gcn example
      
      * fixed gat example
      
      * fixed mx utests
      
      * fix mx utest
      
      * delete apply edges; add utest for update_edges
      
      * small change on toy example
      
      * fix utest
      
      * fix out in degrees bug
      
      * update pagerank example and add it to CI
      
      * add delitem for dataview
      
      * make edges() return form that is compatible with send/update_edges etc
      
      * fix index bug when the given data is one-int-tensor
      
      * fix doc
      68ec6247
  22. 26 Oct, 2018 2 commits
    • Da Zheng's avatar
      [GraphIndex] Immutable graph index using MXNet CSRArray (#86) · 9e9a9488
      Da Zheng authored
      * add immutable graph index.
      
      * update ImmutableGraphIndex.
      
      * add benchmarks of subgraph generation.
      
      * fix node_subgraphs.
      
      * use scipy synthetic data in immutable graph.
      
      * create immutable_graph.
      
      * use spmv in gcn.
      
      * fix immutable graph index.
      
      * update graph index.
      
      * create ImmutableSubgraphIndex.
      
      * return subgraph node and edge mapping.
      
      * fix benchmark.
      
      * add mapping to subgraph nid.
      
      * test in_edges and out_edges.
      
      * update benchmark.
      
      * Fix immutable subgraph.
      
      * complete in_edges and out_edges.
      
      * fix benchmarks.
      
      * add test on node_subgraphs
      
      * add sse.
      
      * construct subgraphs in parallel.
      
      * add in/out degree(s).
      
      * make immutable graph index backend-specific.
      
      * implement has_edge(s) and edge_id(s).
      
      * Revert "use spmv in gcn."
      
      This reverts commit 9cfed5f5fb3dd2ed9b98745348b1c0e9731ed7f7.
      
      * implement node_subgraphs in GraphIndex.
      
      * fix sse.
      
      * address comments.
      
      * address comments.
      
      * Update mxnet docker.
      9e9a9488
    • Da Zheng's avatar
      [CI] Enable mxnet CI. (#105) · 0d6cd30b
      Da Zheng authored
      * fix tests.
      
      * enable mxnet CI.
      
      * add mxnet tests.
      
      * update jenkins.
      
      * update mxnet docker image.
      0d6cd30b
  23. 18 Oct, 2018 1 commit
  24. 14 Oct, 2018 1 commit
    • Da Zheng's avatar
      [BACKEND] Add MXNet backend. (#77) · 5567f4a4
      Da Zheng authored
      * support mxnet.
      
      * add mxnet version of GCN.
      
      * rename mxnet.nd as F.
      
      * add mxnet GAT.
      
      * enable GPU for GCN.
      
      * fix MXNet GCN train.
      
      * Use adam to optimize GAT
      
      * support more operators.
      
      * support sparse arrays.
      
      * update mxnet backend.
      
      * support index_copy.
      
      * remove NN.
      
      * update mxnet backend.
      
      * temp check in.
      
      * fix data conversion.
      
      * add test.
      
      * clean up mxnet backend.
      
      * update mxnet examples.
      
      * Revert "remove NN."
      
      This reverts commit d815d9a0ec619f9ce9099c48cd35db9d8e947483.
      
      * temp disable MXNet version of NN.
      5567f4a4