GitLab

Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>

* auto-format-test

* more

* remove

---------
Co-authored-by: Ubuntu <ubuntu@ip-172-31-28-63.ap-northeast-1.compute.internal>

Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>

* [Dist] instantiate NodeDataView in lazy mode

* fix test failure

* init node/edge data store at the very beginning

* fix test failures

* refine comment

* add more tests

* [Dist] fix is_node check according to policy

* add more tests

* [Dist] enable access DistGraph.edges via canonical etype

* refine code

* refine test

* refine code

* [Dist] deprecate etype and always use canonical etype for partition and load

* enable canonical etypes in dist part pipeline

* resolve rebase conflicts

* fix lint

* fix test failure

* throw exception if outdated part config is loaded

* refine

* refine

* revert unnecessary change

* fix typo

* add save/load for distributed  optimizer
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-19.ap-northeast-1.compute.internal>

* sample neighbors with masks

* oops

* refactor again

* remove

* remove debug code

* rename macro

* address comments

* more stuff

* remove

* fix

* try fix unit test

* oops

* fix test

* oops

* change name

* rename a lot of stuff

* oops

* ugh

* misc fixes

* lint

* address a lot of comments

* lint

* lint

* fix

* that was silly

* fix

* fix

* fix

* oops

* [Dist] reduce startup overhead: enable to save in specified formats

* [Dist] reduce startup overhead: sort partitions when generating

* sort csc/csr only whenmultiple etypes

* refine

* [Dist] Reduce peak memory in DistDGL: avoid validation, release memory once loaded

* remove orig_id from ndata/edata for partition_graph()

* delete orig_id from ndata/edata in dist part pipeline

* reduce dtype size and format before saving graphs

* fix lint

* ETYPE requires to be int32/64 for CSRSortByTag

* fix test failure

* refine

* [Dist] enable iterate multiple dist dataloaders simultaneously

* format file

* add support for any number of dataloaders

* fix lint

* refine code

Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal>

* first commit

* add test

* fixes

* ah this is how you skip setup

* fix

* ugh

* address comments

* i like black

* [Feature] enable graph partition book support canonical etypes

* fix lint

* fix lint

* add todo

* refine according to review comments

* fix lint

* refine naming

* revert PartitionPolicy __init__

* refine docstring

* fix doc string

* [DistTest] fix incorrect shell if statement

* fix incorrect use of dist.initialize()

* [Dist] enable to specify sort_etype for sample_etype_neighbours

* fix lint

* pass argument instead of env

* fix lint and doc string

* refine args

* remove unnecessary lines

* debug only

* debug add sort time log

* change interface

* fix typo
Co-authored-by: Xin Yao <xiny@nvidia.com>

* [Dist] format dtypes when loading graph in server

* add test

* refine

* add comments

* [Dist] defer to load node/edge feats

* fix lint

* Update python/dgl/distributed/partition.py
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>

* Update python/dgl/distributed/partition.py
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>

* fix lint
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>

* [Dist] set socket as default backend for RPC

* add tests both for socket and tensorpipe

* [ist] enable time out when fetching msg

* fix lint error

* minor refinements

* improve minor log

* fix dist test

* fix timeout issue in tensorpipe

* [Dist][BugFix] enable sampling on bipartite

* add comments for tests

* [Dist] Enable maximum try times for socket backend via DGL_DIST_MAX_TRY_TIMES

* reset env before/after test

* print log for info when trying to connect

* fix

* print log in python instead of cpp

* [Feature] enable socket net_type for rpc

* fix lint

* fix lint

* fix build issue on windows

* fix test failure on windows

* fix test failure

* fix cpp unit test failure

* net_type blocking max_try_times

* fix other comments

* fix lint

* fix comment

* fix lint

* fix cpp

Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>

* [BugFix] pass ntype/etype into partition book when node/edge_split

* fix test failure

* fix test failue on mxnet

* fix test failure

* [Fix] sleep for a while when launching clients which will connect to multiple servers

* pre-allocate more ports

* no multiple partitions on single machine

* initial update

* more

* more

* multi-gpu example

* cluster gcn, finalize homogeneous

* more explanation

* fix

* bunch of fixes

* fix

* RGAT example and more fixes

* shadow-gnn sampler and some changes in unit test

* fix

* wth

* more fixes

* remove shadow+node/edge dataloader tests for possible ux changes

* lints

* add legacy dataloading import just in case

* fix

* update pylint for f-strings

* fix

* lint

* lint

* lint again

* cherry-picking commit fa9f494

* oops

* fix

* add sample_neighbors in dist_graph

* fix

* lint

* fix

* fix

* fix

* fix tutorial

* fix

* fix

* fix

* fix warning

* remove debug

* add get_foo_storage apis

* lint

* migrate to pylint 2.6.0

* fix

* fix?

* ???

* oops

* [Feature] long live server for multiple client groups

* generate globally unique name for DistTensor within DGL automatically

* [Fix] reduce error msg, refine fetch logic of available ports

* un-initialize client before sending shutdown request

* fix import error

* print connect failure log only in debug mode

* enable DMLC_LOG_DEBUG=1 in CI