Commits · 02e4cd8b56514e5ab2c6330dc9782bebc573794a · OpenDAS / dgl

26 Jan, 2022 1 commit

[Feature] long live server for multiple client groups (#3645) · 02e4cd8b

Rhett Ying authored Jan 26, 2022

* [Feature] long live server for multiple client groups

* generate globally unique name for DistTensor within DGL automatically

02e4cd8b

19 Jan, 2022 1 commit

[Fix] reduce error msg, refine fetch logic of available ports (#3658) · e4cb4a37

Rhett Ying authored Jan 19, 2022

* [Fix] reduce error msg, refine fetch logic of available ports

* un-initialize client before sending shutdown request

* fix import error

* print connect failure log only in debug mode

* enable DMLC_LOG_DEBUG=1 in CI

e4cb4a37

11 Jan, 2022 1 commit

[Feature][Dist] change TP::Receiver/TP::Sender for multiple connections (#3574) · 37467e25

Rhett Ying authored Jan 11, 2022



* [Feature] enable TP::Receiver wait for any numbers of senders

* fix random unit test failure

* avoid endless future wait

* fix unit test failure

* fix seg fault when finalize wait in receiver

* [Feature] refactor sender connect logic and remove unnecessary sleeps in unit tests

* fix lint

* release RPCContext resources before process exits

* [Debug] TPReceiver wait start log

* [Debug] add log in get port

* [Debug] add log

* [ReDebug] revert time sleep in unit tests

* [Debug] remove sleep for test_distri,test_mp

* [debug] add more log

* [debug] add listen_booted_ flag

* [debug] restore commented code for queue

* [debug] sleep more in rpc_client

* restore change in tests

* Revert "restore change in tests"

This reverts commit 41a18926d181ec2517069389bfc41de2cc949280.

* Revert "[debug] sleep more in rpc_client"

This reverts commit a908e758eabca0a6ce62eb2e59baea02a840ac67.

* Revert "[debug] restore commented code for queue"

This reverts commit d3f993b3746e6bb6e2cc2f90204dd7e9461c6301.

* Revert "[debug] add listen_booted_ flag"

This reverts commit 244b2167d94942ff2a0acec8823b974975e52580.

* Revert "[debug] add more log"

This reverts commit 4b78447b0a575a824821dc7e25cca2246e6e30e2.

* Revert "[Debug] remove sleep for test_distri,test_mp"

This reverts commit e1df1aadcc8b1c2a0013ed77322ac391a8807612.

* remove debug code

* revert unnecessary change

* revert unnecessary changes

* always reset RPCContext when get started and reset all data

* remove time.sleep in dist tests

* fix lint

* reset envs before each dist test

* reset env properly

* add time sleep when start each server

* sleep for a while when boot server

* replace wait_thread with callback

* fix lint

* add dglconnect handshake check
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>

37467e25

06 Dec, 2021 1 commit

[RPC] Use tensorpipe for rpc communication (#3335) · a3ce780d

Jinjing Zhou authored Dec 06, 2021

* doesn't know whether works

* add change

* fix

* fix

* fix

* remove

* revert

* lint

* lint

* fix

* revert

* lint

* fix

* only build rpc on linux

* lint

* lint

* fix build on windows

* fix windows

* remove old test

* fix cmake

* Revert "remove old test"

This reverts commit f1ea75c777c34cdc1f08c0589676ba6aee1feb29.

* fix windows

* fix

* fix

* fix indent

* fix indent

* address comment

* fix

* fix

* fix

* fix

* fix

* lint

* fix indent

* fix lint

* add introduction

* fix

* lint

* lint

* add more logs

* fix

* update xbyak for C++14 with gcc5

* Remove channels

* fix

* add test script

* fix

* remove unused file

* fix lint

* add timeout

a3ce780d

28 Sep, 2021 1 commit
- [Feature] Implement one thread multiple socket (#3200) · 5cf48fc6
  Jingcheng Yu authored Sep 28, 2021
```
Co-authored-by: JingchengYu94 <jingchengyu94@gmail.com>
```
  5cf48fc6
02 Sep, 2021 1 commit

[Performance, CPU] Rewriting OpenMP pragmas into parallel_for (#3171) · f5183820

Tomasz Patejko authored Sep 02, 2021

* [CPU, Parallel] Rewriting omp pragmas with parallel_for

* [CPU, Parallel] Decrease number of calls to task function

* c[CPU, Parallel] Modify calls to new interface of parallel_for

f5183820

25 Jul, 2021 1 commit

[Bugfix] fix potential starving in socket receiver (#3176) · c8d4d6fb

Jingcheng Yu authored Jul 25, 2021


Co-authored-by: JingchengYu94 <jingchengyu94@gmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

c8d4d6fb

22 Mar, 2021 1 commit
- Print error messages when using TCP socket (#2763) · 74c38a1f
  Da Zheng authored Mar 21, 2021
```
* print error messages.

* fix.
```
  74c38a1f
26 Dec, 2020 1 commit

delete shared memory when receiving signals. (#2419) · 3d1f2e87

Da Zheng authored Dec 26, 2020



* delete shared memory when receive signal.

* rename.

* fix lint.

* fix lint.

* fix compile.

* Fix.

* we need to report error if the shared memory exist.

* disable tensorflow test for shared memory.

* revert.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>

3d1f2e87

15 Dec, 2020 1 commit
- [RPC] Make the port reusable after the socket is closed. (#2418) · 8ff47980
  Da Zheng authored Dec 14, 2020
```
* reuse.

* fix compile.
Co-authored-by: Ubuntu <ubuntu@ip-172-31-2-202.us-west-1.compute.internal>
```
  8ff47980
09 Dec, 2020 1 commit
- change msg_seq from int to int_64 (#2410) · 27b05584
  Chao Ma authored Dec 09, 2020
  
  27b05584
25 Sep, 2020 1 commit
- update (#2236) · 36daf66f
  Chao Ma authored Sep 25, 2020
  
  36daf66f
19 Aug, 2020 1 commit
- [BugFix] Fix a small bug on kvstore (#2058) · a4a7e02f
  Chao Ma authored Aug 19, 2020
```
* small fix

* update
```
  a4a7e02f
14 Aug, 2020 1 commit

Fix hang when sending empty ndarray in rpc request (#2022) · 1f5f31ce

Jinjing Zhou authored Aug 14, 2020

* aaa

* revert

* 111

* fix

* fff

* 111

* try

* debug

* fff

* 111

* 1111

* fix

* try

* 111

* 111

* print

* f

* 111

* revert

* fix

* revert

* add timeout

* Revert "add timeout"

This reverts commit fb34f67f11ad6556824daa6efa29a6ec2c7c1d3e.

1f5f31ce

13 Aug, 2020 1 commit
- [Distributed] Fix memory leak in kvstore (#2016) · 15e3ff87
  Chao Ma authored Aug 13, 2020
```
* Fix memory leak

* update

* update

* update

* update

* update

* update

* update
```
  15e3ff87
05 Aug, 2020 1 commit

[Distributed] DistDataloader (#1901) · 4f499c7f

Jinjing Zhou authored Aug 05, 2020



* 111

* 111

* fix

* 111

* fix

* 11

* fix

* lint

* Update __init__.py

* lint

* fix

* lint

* fix

* fix

* fix

* fix

* fix

* try fix

* try fix

* fix

* Revert "fix"

This reverts commit a0b954fd4e99b7df92b53db8334dcb583d6e1551.

* fixes.

* fix.

* fix test.

* fix exit.

* fix.

* fix

* fix

* lint

* lint

* lint

* fix

* Update .gitignore

* 111

* fix

* 111

* 111

* fff

* 1111

* 111

* 1325315

* ffff

* f???

* fff

* 1111

* 111

* fix

* 111

* asda

* 1111

* 11

* 123

* 啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊啊

* spawn

* 1231231

* up

* 111

* fix

* fix

* Revert "fix"

This reverts commit 7373f95312fdcaa36d2fc330bf242339e89c045d.

* fix

* fix

* 1111

* fix

* fix tests

* start kvclient as early as possible.

* lint

* fix test

* lint

* 1111

* fix

* fix

* 111

* fix

* fix

* 1

* fix

* fix

* lint

* fix

* lint

* lint

* remove quit

* fix

* lint

* fix

* fix several

* lint

* fix minor

* fix

* lint
Co-authored-by: Da Zheng <zhengda1936@gmail.com>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

4f499c7f

30 Jul, 2020 1 commit

[KVStore] Support group barrier (#1880) · 5b515cf6

Chao Ma authored Jul 30, 2020



* udpate

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* update

* udpate

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
Co-authored-by: Da Zheng <zhengda1936@gmail.com>

5b515cf6

23 Jul, 2020 1 commit
- [RPC] Remove socket->SetTimeOut() (#1851) · b885c82f
  Chao Ma authored Jul 23, 2020
```
* update

* update

* update

* update

* udpate
```
  b885c82f
22 Jul, 2020 1 commit

[Distributed] add distributed evaluation. (#1810) · 562871e7

Da Zheng authored Jul 22, 2020



* add eval.

* extend DistTensor.

* fix.

* add barrier.

* add more print.

* add more checks in kvstore.

* fix lint.

* get all neighbors for eval.

* reorganize.

* fix.

* fix.

* fix.

* fix test.

* add reuse_if_exist.

* add test for reuse_if_exist.

* fix lint.

* fix bugs.

* fix.

* print errors of tcp socket.

* support delete tensors.

* fix lint.

* fix

* fix example
Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-1.us-west-2.compute.internal>

562871e7

21 Jul, 2020 1 commit
- Increase seconds for showing the tcp re-connect message (#1834) · 54b71412
  Chao Ma authored Jul 21, 2020
  
  54b71412
09 Jul, 2020 1 commit

Handle EINTR in TCP layer (#1766) · 83f71466

mszarma authored Jul 09, 2020



Added handling EINTR signal by
retrying when returned value and
errno are related to EINTR interrupt
error which my appear during tcp
function execution.
Added information about EAGAIN timeout
failure in accept stage.
Added printing out errno in logs.
Co-authored-by: Chao Ma <mctt90@gmail.com>

83f71466

07 Jul, 2020 1 commit
- [RPC] Client will retry connection until the server starts. (#1758) · bdc1e649
  Chao Ma authored Jul 07, 2020
```
* update

* update

* update
```
  bdc1e649
03 Jul, 2020 1 commit

Use timeval to set up SO_RCVTIMEO (#1734) · f80fc9bd

mszarma authored Jul 03, 2020



Refactor code to use timeval struct
for setting up the SO_RCVTIMEO variable
timeout value for sockets in case of Linux.
Changed SetTimeout API timeout input to seconds.
Co-authored-by: Chao Ma <mctt90@gmail.com>

f80fc9bd

29 Jun, 2020 1 commit

[rpc] Add num_clients and sort client ID by machine-id (#1713) · dd8d5289

Chao Ma authored Jun 29, 2020



* add num_clients to kvstore

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* update

* update

* add test

* update

* update

* update

* update

* fix test

* update

* update

* update

* update

* update

* update

* update
Co-authored-by: Da Zheng <zhengda1936@gmail.com>

dd8d5289

28 Jun, 2020 1 commit

[CUDA][Kernel] More CUDA kernels; Standardize the behavior for sorted COO/CSR (#1704) · 870da747

Minjie Wang authored Jun 28, 2020

* add cub; array cumsum

* CSRSliceRows

* fix warning

* operator << for ndarray; CSRSliceRows

* add CSRIsSorted

* add csr_sort

* inplace coosort and outplace csrsort

* WIP: coo is sorted

* mv cuda_utils

* add AllTrue utility

* csr sort

* coo sort

* coo2csr for sorted coo arrays

* CSRToCOO from sorted

* pass tests for the new kernel changes

* cannot use inplace sort

* lint

* try fix msvc error

* Fix g.copy_to and g.asnumbits; ToBlock no longer uses CSC

* stash

* revert some hack

* revert some changes

* address comments

* fix

* fix to_block unittest

* add todo note

870da747

17 Jun, 2020 2 commits

[RPC] HandleCtrlC Register for handling Ctrl+C event. (#1652) · 8fccdaf6
Chao Ma authored Jun 17, 2020
```
* update

* update

* update

* update

* fix lint
```
8fccdaf6

[KVStore] Add fast-pull for kvstore (#1647) · b372b3c7

Chao Ma authored Jun 17, 2020

* add fast-pull

* update

* add fast-pull

* update

* update

* update

* update test

* update

* update

* update

* update

* update

* update

* update

* update

* add omp

* update

* update

b372b3c7

11 Jun, 2020 1 commit
- [RPC] Add send_request_to_machine() and remote_call_to_machine() (#1619) · a67ba946
  Chao Ma authored Jun 11, 2020
```
* add send_request_to_machine()

* update

* update

* update

* update

* update

* update

* fix lint

* update
```
  a67ba946
04 Jun, 2020 1 commit

[KVStore] Re-write kvstore using DGL RPC infrastructure (#1569) · 64f49703

Chao Ma authored Jun 04, 2020

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update init_data

* update server_state

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* debug init_data

* update

* update

* update

* update

* update

* update

* test get_meta_data

* update

* update

* update

* update

* update

* debug push

* update

* update

* update

* update

* update

* update

* update

* update

* update

* use F.reverse_data_type_dict

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* update

* fix lint

* update

* fix lint

* update

* update

* update

* update

* fix test

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* set random seed

* update

64f49703

26 May, 2020 1 commit

[Fix] New StreamWithBuffer interface (#1557) · 7639b5e7

Jinjing Zhou authored May 27, 2020



* WIP: rpc components

* client & server

* move network package to rpc

* fix include

* fix compile

* c api

* wip: test

* add basic tests

* missing file

* [RPC] Zero copy serializer (#1517)

* zerocopy serialization

* add test for HeteroGraph

* fix lint

* remove unnecessary codes

* add comment

* lint

* lint

* disable pylint for now

* add include for win

* windows guard

* lint

* lint

* skip test on windows

* refactor

* add comment

* fix

* comment

* 1111

* fix

* Update Jenkinsfile

* [RPC] Implementation of RPC infra (#1544)

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* remove client.cc and server.cc

* fix lint

* update

* update

* fix linr

* update

* fix lint

* update

* update

* update

* update

* update

* update

* update test

* update

* update test

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update comment

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Refactor StreamWithBuffer (#1550)

* refactor

* fix with new interface

* remove copy

* fix

* remove comment
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Chao Ma <mctt90@gmail.com>

7639b5e7

25 May, 2020 1 commit
- [RPC] Init server_state during get_server_state() (#1559) · 6a363378
  Chao Ma authored May 25, 2020
```
* init server_state

* update

* update
```
  6a363378
22 May, 2020 1 commit

[RPC] New RPC infrastructure. (#1549) · 3e696922

Chao Ma authored May 22, 2020



* WIP: rpc components

* client & server

* move network package to rpc

* fix include

* fix compile

* c api

* wip: test

* add basic tests

* missing file

* [RPC] Zero copy serializer (#1517)

* zerocopy serialization

* add test for HeteroGraph

* fix lint

* remove unnecessary codes

* add comment

* lint

* lint

* disable pylint for now

* add include for win

* windows guard

* lint

* lint

* skip test on windows

* refactor

* add comment

* fix

* comment

* 1111

* fix

* Update Jenkinsfile

* [RPC] Implementation of RPC infra (#1544)

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* remove client.cc and server.cc

* fix lint

* update

* update

* fix linr

* update

* fix lint

* update

* update

* update

* update

* update

* update

* update test

* update

* update test

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update comment

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* fix lint

* fix lint

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>

3e696922