Commits · 729ff2ef385f302af562c8305b1006d067d2067f · OpenDAS / dgl

10 Aug, 2020 1 commit

Fix the performance issue of graph partitioning in new DGLGraph (#1934) · 729ff2ef

Da Zheng authored Aug 09, 2020



* fix perf.

* fix.

* accelerate metis.

* fix lint.

* use gklib.

* fix perf.

* fix.

* update metis.

* update launch script

* handle synchronized API.

* fix.

* fix example.

* fix dataloader.

* temp fix.

* temp fix omp.

* distinguish roles.

* initialize iterator of DistDataloader correctly.

* check the correctness of launch script.

* move feature copy to sampler.

* measure mem/network copy time.

* remove

* Revert "measure mem/network copy time."

This reverts commit 86cefdc14b7815fcf5aad6496af912dba48e4aa6.

* fix.

* fix

* fix.

* fix cmake.

* disable metis in windows.

* disable metis tests in windows.

* remove test for multigraph.

* fix test.

* fix.

* fix cmake.

* fix.

* revert.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-115.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>

729ff2ef

29 Jul, 2020 1 commit

[Distributed] Refactor distributed training to new DGLGraph. (#1874) · 3e2f94ed

Da Zheng authored Jul 28, 2020

* fix tests in partition.

* fix DistGraph.

* fix without shared memory.

* fix sampling.

* enable distributed test.

* fix tests.

* fix a bug in shared-mem heterograph.

* print better error messages.

* fix.

* don't specify formats.

* fix.

* fix

* small fix.

3e2f94ed

22 Jul, 2020 1 commit
- [HeteroGraph] Metis partition for hetero (#1843) · 33abd275
  Jinjing Zhou authored Jul 22, 2020
```
* metis for hetero

* fix partition
```
  33abd275
14 Jul, 2020 1 commit

[Distributed] Run distributed graph server inside DGL (#1801) · a1472bcf

Da Zheng authored Jul 14, 2020

* run dist server in dgl.

* fix bugs.

* fix example.

* check environment variables and fix lint.

* fix lint

a1472bcf

26 Jun, 2020 1 commit

[BUGFIX] Fix bugs in range partitioning. (#1703) · 4b96addc

Da Zheng authored Jun 26, 2020

* Fix bugs in partitioning.

* add comment.

* fix lint.

* fix test.

* add more tests.

* simplify the fix

* fix.

4b96addc

24 Jun, 2020 1 commit
- [BUGFIX] fix a bug in partition book. (#1698) · fa1478a4
  Da Zheng authored Jun 24, 2020
```
* fix a bug.

* add msg for assert
```
  fa1478a4
18 Jun, 2020 2 commits

[RPC] Sampling over RPC (#1616) · fb248b67

Jinjing Zhou authored Jun 19, 2020



* fix

* test1111

* 111

* 111

* fff

* lint

* 111

* lint

* lint

* 111

* fijx

* 111

* fix

* 111

* commit

* 111

* 111

* lint

* fix typo

* fix

* lint

* fix

* 111

* support mxnet

* support mxnet

* lint

* remove print

* fix

* fix test.

* fix test.

* fix test.

* try to fix an undetermistic error.
Co-authored-by: Chao Ma <mctt90@gmail.com>
Co-authored-by: Da Zheng <zhengda1936@gmail.com>

fb248b67

[BUGFIX] reshuffle node features in range partitioning. (#1659) · b9ec3e36
Da Zheng authored Jun 17, 2020
```
* fix a bug.

* fix test.

* simplify.

* fix test.

* test edge features.

* fix.
```
b9ec3e36

15 Jun, 2020 1 commit

[Feature] Range partition (#1522) · 41349dce

Da Zheng authored Jun 15, 2020



* add reorder immutable graph.

* add python API.

* add reorder for csr.

* remove gk version.

* fix

* add cpp test.

* bug fixes

* fix tests.

* fix bugs and add check

* fix test.

* add omp.

* add comments.

* add coo reorder.

* fix a bug.

* handle reorder for different graph structues.

* fix lint.

* fix.

* add original ids.

* reshuffle nodes before metis partition.

* inner nodes are in contiguous Id range.

* reshuffle nodes/edges when partitioning.

* load partition return graph partition book.

* use inner_node/inner_edges

* add and test range partition book.

* count inner_edge correctly.

* fix lint.

* fix lint.

* fix lint.

* fix errors.

* fix errors.

* fix for TF.

* fix.

* fix.

* change docstring.

* support logical and.

* add comments.

* avoid copy.

* fix

* update docstring.

* fix a bug.

* add range search.

* fix

* fix a bug.

* add more tests.

* load graph partition book.

* support shared memory for range partition book.

* fix a bug.

* fix.

* fix lint.

* remove check

* fix test.

* remove num_nodes and num_edges

* fix lint.

* fix graph partition book.

* address comments.

* use makedirs.

* fix compile
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>
Co-authored-by: Chao Ma <mctt90@gmail.com>

41349dce

11 Jun, 2020 1 commit

[CI] Use spawn in unittest instead of fork (#1627) · 77006db2

Jinjing Zhou authored Jun 11, 2020



* update

* update

* update

* update

* update

* update

* fix ci using spawn
Co-authored-by: aksnzhy <mctt90@gmail.com>

77006db2

04 Jun, 2020 1 commit

[KVStore] Re-write kvstore using DGL RPC infrastructure (#1569) · 64f49703

Chao Ma authored Jun 04, 2020

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update init_data

* update server_state

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* debug init_data

* update

* update

* update

* update

* update

* update

* test get_meta_data

* update

* update

* update

* update

* update

* debug push

* update

* update

* update

* update

* update

* update

* update

* update

* update

* use F.reverse_data_type_dict

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* fix lint

* update

* fix lint

* update

* update

* update

* update

* fix test

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* set random seed

* update

64f49703

08 May, 2020 1 commit

[Distributed] Add GraphPartitionBook (#1496) · ef7e4750

Chao Ma authored May 08, 2020

* Add graph-partition-book

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* udpate

* fix lint

* update

* update

* update

* update

* update

* update

* update

* skip test on GPU

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* update

* fix unittest

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

ef7e4750

06 May, 2020 1 commit

[Test] Add tests for TensorFlow (#1501) · 16561a2e

Da Zheng authored May 06, 2020



* add test.

* move test code.

* remvoe unnecessary test.

* fix.

* turn on tests for TF.

* Revert "move test code."

This reverts commit e7b4f36395b2121a7be030bd4364a704d0e357bf.

* fix.

* fix.

* skip test for tensorflow.
Co-authored-by: Chao Ma <mctt90@gmail.com>

16561a2e

03 May, 2020 1 commit

[Feature] Distributed graph store (#1383) · 2190c39d

Da Zheng authored May 02, 2020

* initial version from distributed training.

This is copied from multiprocessing training.

* modify for distributed training.

* it's runnable now.

* measure time in neighbor sampling.

* simplify neighbor sampling.

* fix a bug in distributed neighbor sampling.

* allow single-machine training.

* fix a bug.

* fix a bug.

* fix openmp.

* make some improvement.

* fix.

* add prepare in the sampler.

* prepare nodeflow async.

* fix a bug.

* get id.

* simplify the code.

* improve.

* fix partition.py

* fix the example.

* add more features.

* fix the example.

* allow one partition

* use distributed kvstore.

* do g2l map manually.

* fix commandline.

* a temp script to save reddit.

* fix pull_handler.

* add pytorch version.

* estimate the time for copying data.

* delete unused code.

* fix a bug.

* print id.

* fix a bug

* fix a bug

* fix a bug.

* remove redundent code....

2190c39d