1. 05 Jul, 2020 1 commit
  2. 04 Jul, 2020 1 commit
  3. 03 Jul, 2020 1 commit
  4. 02 Jul, 2020 1 commit
    • Quan (Andy) Gan's avatar
      [Sampling] NodeDataLoader for node classification (#1635) · 168a88e5
      Quan (Andy) Gan authored
      
      
      * neighbor sampler data loader first commit
      
      * more commit
      
      * nodedataloader
      
      * fix
      
      * update RGCN example
      
      * update OGB
      
      * fixes
      
      * fix minibatch RGCN crashing with self loop
      
      * reverting gatconv test code
      
      * fix
      
      * change to new solution that doesn't require tf dataloader
      
      * fix
      
      * lint
      
      * fix
      
      * fixes
      
      * change doc
      
      * fix docstring
      
      * docstring fixes
      
      * return seeds and input nodes from data loader
      
      * fixes
      
      * fix test
      
      * fix windows build problem
      
      * add pytorch wrapper
      
      * fixes
      
      * add pytorch wrapper
      
      * add unit test
      
      * add -1 support to sample_neighbors & fix docstrings
      
      * docstring fix
      
      * lint
      
      * add minibatch rgcn evaluations
      Co-authored-by: default avatarxiang song(charlie.song) <classicxsong@gmail.com>
      Co-authored-by: default avatarTong He <hetong007@gmail.com>
      168a88e5
  5. 30 Jun, 2020 2 commits
  6. 29 Jun, 2020 1 commit
  7. 28 Jun, 2020 4 commits
  8. 23 Jun, 2020 1 commit
  9. 22 Jun, 2020 2 commits
    • Zihao Ye's avatar
      [kernel] New SpMM & SDDMM kernel on CPU and CUDA (#1644) · 071cba1f
      Zihao Ye authored
      * udp
      
      * simplify
      
      * sddmm dot cpu
      
      * upd
      
      * format
      
      * upd
      
      * compatible with MJ's PR
      
      * lint
      
      * upd
      
      * upd
      
      * upd
      
      * python end
      
      * upd
      
      * upd
      
      * lint
      
      * lint
      
      * upd
      
      * upd
      
      * upd
      
      * upd
      
      * upd
      
      * lint
      
      * fix mxnet
      
      * upd
      
      * lint
      
      * use minjie's ptr
      
      * macro
      
      * upd
      
      * reorg
      
      * lint
      
      * fix corner cases
      
      * upd
      
      * enrich cpu docs
      
      * upd
      
      * upd
      
      * lint
      
      * lint
      
      * pylint
      
      * sx review
      
      * improve docstring
      
      * python doc
      
      * upd
      
      * restructure
      
      * lint
      
      * upd test
      
      * upd
      
      * pylint
      
      * fix corner cases and test
      071cba1f
    • Tong He's avatar
      [Model] PointNet and PointNet++ for point cloud (#1510) · 558673e1
      Tong He authored
      
      
      * commit patch
      
      * commit patch
      
      * pointnet basic
      
      * fix data
      
      * reorg
      
      * reorg
      
      * temp status
      
      * remove validate set
      
      * add partseg data and model
      
      * partseg miou
      
      * clean up
      
      * fix loss
      
      * network definition match paper
      
      * fix
      
      * fix miou
      
      * update data format
      
      * fix
      
      * fix
      
      * working pointnet ssg cls
      
      * avoid some pytorch bug
      
      * fix script
      
      * update hyperparams
      
      * add msg module
      
      * try different dataset
      
      * update new dataset info
      
      * quick fix to subgraph
      
      * fix speed
      
      * update training
      
      * update
      
      * fix bs
      
      * update docstring
      
      * update
      
      * update
      
      * remove parallel reduction in fps
      
      * switch to kernel fps, training is 30% faster
      Co-authored-by: default avatarUbuntu <ubuntu@ip-172-31-20-181.us-west-2.compute.internal>
      558673e1
  10. 21 Jun, 2020 3 commits
  11. 19 Jun, 2020 2 commits
  12. 18 Jun, 2020 2 commits
  13. 17 Jun, 2020 2 commits
  14. 15 Jun, 2020 2 commits
    • Da Zheng's avatar
      [Feature] Range partition (#1522) · 41349dce
      Da Zheng authored
      
      
      * add reorder immutable graph.
      
      * add python API.
      
      * add reorder for csr.
      
      * remove gk version.
      
      * fix
      
      * add cpp test.
      
      * bug fixes
      
      * fix tests.
      
      * fix bugs and add check
      
      * fix test.
      
      * add omp.
      
      * add comments.
      
      * add coo reorder.
      
      * fix a bug.
      
      * handle reorder for different graph structues.
      
      * fix lint.
      
      * fix.
      
      * add original ids.
      
      * reshuffle nodes before metis partition.
      
      * inner nodes are in contiguous Id range.
      
      * reshuffle nodes/edges when partitioning.
      
      * load partition return graph partition book.
      
      * use inner_node/inner_edges
      
      * add and test range partition book.
      
      * count inner_edge correctly.
      
      * fix lint.
      
      * fix lint.
      
      * fix lint.
      
      * fix errors.
      
      * fix errors.
      
      * fix for TF.
      
      * fix.
      
      * fix.
      
      * change docstring.
      
      * support logical and.
      
      * add comments.
      
      * avoid copy.
      
      * fix
      
      * update docstring.
      
      * fix a bug.
      
      * add range search.
      
      * fix
      
      * fix a bug.
      
      * add more tests.
      
      * load graph partition book.
      
      * support shared memory for range partition book.
      
      * fix a bug.
      
      * fix.
      
      * fix lint.
      
      * remove check
      
      * fix test.
      
      * remove num_nodes and num_edges
      
      * fix lint.
      
      * fix graph partition book.
      
      * address comments.
      
      * use makedirs.
      
      * fix compile
      Co-authored-by: default avatarxiang song(charlie.song) <classicxsong@gmail.com>
      Co-authored-by: default avatarChao Ma <mctt90@gmail.com>
      41349dce
    • Minjie Wang's avatar
      [Kernel] CUDA CSR2COO COOSort COO2CSR (#1620) · d6d517bb
      Minjie Wang authored
      
      
      * add cuda source
      
      * moving codes from kernel2 branch
      
      * operator overloading
      
      * Better error message for unsupported device
      
      * fix c tests
      
      * coo sort using cusparse
      
      * move test_rpc to distributed
      
      * lint
      
      * address comments and add utests
      Co-authored-by: default avatarJinjing Zhou <VoVAllen@users.noreply.github.com>
      Co-authored-by: default avatarChao Ma <mctt90@gmail.com>
      Co-authored-by: default avatarxiang song(charlie.song) <classicxsong@gmail.com>
      d6d517bb
  15. 11 Jun, 2020 2 commits
  16. 04 Jun, 2020 1 commit
    • Chao Ma's avatar
      [KVStore] Re-write kvstore using DGL RPC infrastructure (#1569) · 64f49703
      Chao Ma authored
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update init_data
      
      * update server_state
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * debug init_data
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * test get_meta_data
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * debug push
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * use F.reverse_data_type_dict
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * fix lint
      
      * update
      
      * fix lint
      
      * update
      
      * fix lint
      
      * update
      
      * update
      
      * update
      
      * update
      
      * fix test
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * set random seed
      
      * update
      64f49703
  17. 27 May, 2020 1 commit
  18. 26 May, 2020 1 commit
    • Jinjing Zhou's avatar
      [Fix] New StreamWithBuffer interface (#1557) · 7639b5e7
      Jinjing Zhou authored
      
      
      * WIP: rpc components
      
      * client & server
      
      * move network package to rpc
      
      * fix include
      
      * fix compile
      
      * c api
      
      * wip: test
      
      * add basic tests
      
      * missing file
      
      * [RPC] Zero copy serializer (#1517)
      
      * zerocopy serialization
      
      * add test for HeteroGraph
      
      * fix lint
      
      * remove unnecessary codes
      
      * add comment
      
      * lint
      
      * lint
      
      * disable pylint for now
      
      * add include for win
      
      * windows guard
      
      * lint
      
      * lint
      
      * skip test on windows
      
      * refactor
      
      * add comment
      
      * fix
      
      * comment
      
      * 1111
      
      * fix
      
      * Update Jenkinsfile
      
      * [RPC] Implementation of RPC infra (#1544)
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * remove client.cc and server.cc
      
      * fix lint
      
      * update
      
      * update
      
      * fix linr
      
      * update
      
      * fix lint
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update test
      
      * update
      
      * update test
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update comment
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * fix lint
      
      * fix lint
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * Refactor StreamWithBuffer (#1550)
      
      * refactor
      
      * fix with new interface
      
      * remove copy
      
      * fix
      
      * remove comment
      Co-authored-by: default avatarMinjie Wang <wmjlyjemaine@gmail.com>
      Co-authored-by: default avatarChao Ma <mctt90@gmail.com>
      7639b5e7
  19. 25 May, 2020 1 commit
  20. 22 May, 2020 1 commit
    • Chao Ma's avatar
      [RPC] New RPC infrastructure. (#1549) · 3e696922
      Chao Ma authored
      
      
      * WIP: rpc components
      
      * client & server
      
      * move network package to rpc
      
      * fix include
      
      * fix compile
      
      * c api
      
      * wip: test
      
      * add basic tests
      
      * missing file
      
      * [RPC] Zero copy serializer (#1517)
      
      * zerocopy serialization
      
      * add test for HeteroGraph
      
      * fix lint
      
      * remove unnecessary codes
      
      * add comment
      
      * lint
      
      * lint
      
      * disable pylint for now
      
      * add include for win
      
      * windows guard
      
      * lint
      
      * lint
      
      * skip test on windows
      
      * refactor
      
      * add comment
      
      * fix
      
      * comment
      
      * 1111
      
      * fix
      
      * Update Jenkinsfile
      
      * [RPC] Implementation of RPC infra (#1544)
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * remove client.cc and server.cc
      
      * fix lint
      
      * update
      
      * update
      
      * fix linr
      
      * update
      
      * fix lint
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update test
      
      * update
      
      * update test
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update comment
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * fix lint
      
      * fix lint
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      Co-authored-by: default avatarMinjie Wang <wmjlyjemaine@gmail.com>
      Co-authored-by: default avatarJinjing Zhou <VoVAllen@users.noreply.github.com>
      3e696922
  21. 18 May, 2020 1 commit
  22. 15 May, 2020 1 commit
  23. 14 May, 2020 1 commit
  24. 08 May, 2020 1 commit
    • Jinjing Zhou's avatar
      [Refactor] Explicit dtype for HeteroGraph (#1467) · dc8ca88e
      Jinjing Zhou authored
      
      
      * 111
      
      * 111
      
      * lint
      
      * lint
      
      * lint
      
      * lint
      
      * fix
      
      * lint
      
      * try
      
      * fix
      
      * lint
      
      * lint
      
      * test
      
      * fix
      
      * ttt
      
      * test
      
      * fix
      
      * fix
      
      * fix
      
      * mxnet
      
      * 111
      
      * fix 64bits computation
      
      * pylint
      
      * roll back
      
      * fix
      
      * lint
      
      * fix hetero_from_relations
      
      * remove index_dtype in to_homo and to_hetero
      
      * fix
      
      * fix
      
      * fix
      
      * fix
      
      * remove default
      
      * fix
      
      * lint
      
      * fix
      
      * fix error message
      
      * fix error
      
      * lint
      
      * macro dispatch
      
      * try
      
      * lint
      
      * remove nbits
      
      * error message
      
      * fix
      
      * fix
      
      * lint
      
      * lint
      
      * lint
      
      * fix
      
      * lint
      
      * fix
      
      * fix random walk
      
      * lint
      
      * lint
      
      * fix
      
      * fix
      
      * fix
      
      * lint
      
      * fix
      
      * lint
      Co-authored-by: default avatarMinjie Wang <wmjlyjemaine@gmail.com>
      dc8ca88e
  25. 03 May, 2020 1 commit
    • Da Zheng's avatar
      [Feature] Distributed graph store (#1383) · 2190c39d
      Da Zheng authored
      * initial version from distributed training.
      
      This is copied from multiprocessing training.
      
      * modify for distributed training.
      
      * it's runnable now.
      
      * measure time in neighbor sampling.
      
      * simplify neighbor sampling.
      
      * fix a bug in distributed neighbor sampling.
      
      * allow single-machine training.
      
      * fix a bug.
      
      * fix a bug.
      
      * fix openmp.
      
      * make some improvement.
      
      * fix.
      
      * add prepare in the sampler.
      
      * prepare nodeflow async.
      
      * fix a bug.
      
      * get id.
      
      * simplify the code.
      
      * improve.
      
      * fix partition.py
      
      * fix the example.
      
      * add more features.
      
      * fix the example.
      
      * allow one partition
      
      * use distributed kvstore.
      
      * do g2l map manually.
      
      * fix commandline.
      
      * a temp script to save reddit.
      
      * fix pull_handler.
      
      * add pytorch version.
      
      * estimate the time for copying data.
      
      * delete unused code.
      
      * fix a bug.
      
      * print id.
      
      * fix a bug
      
      * fix a bug
      
      * fix a bug.
      
      * remove redundent code.
      
      * revert modify in sampler.
      
      * fix temp script.
      
      * remove pytorch version.
      
      * fix.
      
      * distributed training with pytorch.
      
      * add distributed graph store.
      
      * fix.
      
      * add metis_partition_assignment.
      
      * fix a few bugs in distributed graph store.
      
      * fix test.
      
      * fix bugs in distributed graph store.
      
      * fix tests.
      
      * remove code of defining DistGraphStore.
      
      * fix partition.
      
      * fix example.
      
      * update run.sh.
      
      * only read necessary node data.
      
      * batching data fetch of multiple NodeFlows.
      
      * simplify gcn.
      
      * remove unnecessary code.
      
      * use the new copy_from_kvstore.
      
      * update training script.
      
      * print time in graphsage.
      
      * make distributed training runnable.
      
      * use val_nid.
      
      * fix train_sampling.
      
      * add distributed training.
      
      * add run.sh
      
      * add more timing.
      
      * fix a bug.
      
      * save graph metadata when partition.
      
      * create ndata and edata in distributed graph store.
      
      * add timing in minibatch training of GraphSage.
      
      * use pytorch distributed.
      
      * add checks.
      
      * fix a bug in global vs. local ids.
      
      * remove fast pull
      
      * fix a compile error.
      
      * update and add new APIs.
      
      * implement more methods in DistGraphStore.
      
      * update more APIs.
      
      * rename it to DistGraph.
      
      * rename to DistTensor
      
      * remove some unnecessary API.
      
      * remove unnecessary files.
      
      * revert changes in sampler.
      
      * Revert "simplify gcn."
      
      This reverts commit 0ed3a34ca714203a5b45240af71555d4227ce452.
      
      * Revert "simplify neighbor sampling."
      
      This reverts commit 551c72d20f05a029360ba97f312c7a7a578aacec.
      
      * Revert "measure time in neighbor sampling."
      
      This reverts commit 63ae80c7b402bb626e24acbbc8fdfe9fffd0bc64.
      
      * Revert "add timing in minibatch training of GraphSage."
      
      This reverts commit e59dc8957a414c7df5c316f51d78bce822bdef5e.
      
      * Revert "fix train_sampling."
      
      This reverts commit ea6aea9a4aabb8ba0ff63070aa51e7ca81536ad9.
      
      * fix lint.
      
      * add comments and small update.
      
      * add more comments.
      
      * add more unit tests and fix bugs.
      
      * check the existence of shared-mem graph index.
      
      * use new partitioned graph storage.
      
      * fix bugs.
      
      * print error in fast pull.
      
      * fix lint
      
      * fix a compile error.
      
      * save absolute path after partitioning.
      
      * small fixes in the example
      
      * Revert "[kvstore] support any data type for init_data() (#1465)"
      
      This reverts commit 87b6997b
      
      .
      
      * fix a bug.
      
      * disable evaluation.
      
      * Revert "Revert "[kvstore] support any data type for init_data() (#1465)""
      
      This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.
      
      * support set and init data.
      
      * support set and init data.
      
      * Revert "Revert "[kvstore] support any data type for init_data() (#1465)""
      
      This reverts commit f5b8039c6326eb73bad8287db3d30d93175e5bee.
      
      * fix bugs.
      
      * fix unit test.
      
      * move to dgl.distributed.
      
      * fix lint.
      
      * fix lint.
      
      * remove local_nids.
      
      * fix lint.
      
      * fix test.
      
      * remove train_dist.
      
      * revert train_sampling.
      
      * rename funcs.
      
      * address comments.
      
      * address comments.
      
      Use NodeDataView/EdgeDataView to keep track of data.
      
      * address comments.
      
      * address comments.
      
      * revert.
      
      * save data with DGL serializer.
      
      * use the right way of getting shape.
      
      * fix lint.
      
      * address comments.
      
      * address comments.
      
      * fix an error in mxnet.
      
      * address comments.
      
      * add edge_map.
      
      * add more test and fix bugs.
      Co-authored-by: default avatarZheng <dzzhen@186590dc80ff.ant.amazon.com>
      Co-authored-by: default avatarUbuntu <ubuntu@ip-172-31-6-131.us-east-2.compute.internal>
      Co-authored-by: default avatarUbuntu <ubuntu@ip-172-31-26-167.us-east-2.compute.internal>
      Co-authored-by: default avatarUbuntu <ubuntu@ip-172-31-16-150.us-west-2.compute.internal>
      Co-authored-by: default avatarUbuntu <ubuntu@ip-172-31-16-250.us-west-2.compute.internal>
      Co-authored-by: default avatarUbuntu <ubuntu@ip-172-31-30-135.us-west-2.compute.internal>
      2190c39d
  26. 02 May, 2020 2 commits
  27. 28 Apr, 2020 1 commit