GitLab

[Dist] Add support for Parquet-formatted edges files, remove some assumptions on edge file number. (#5051)

* [Dist] Add support for Parquet-formatted edges files, remove some assumptions on edge file number.

* [Dist] Add parquet edges option to unit tests.
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

* Add SpatialEncoder and SpatialEncoder3d

* Optimize the code execution efficiency

* Fixed certain problems according to Dongyu's suggestions.

* Fix an error about probability of division by zero in PathEcoder; Change certain designs in SpatialEncoder

* Fix a typo

* polish the docstring

* fix doc

* lint
Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-146.ap-northeast-1.compute.internal>
Co-authored-by: rudongyu <ru_dongyu@outlook.com>

* update

* lint

* lint

* comments
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* new print func

* lintrunner

* pass gcn

* example tests

* pass gcnii, hgnn, sgc, sign

* black

* test file

* add torchmetrics

* migrate appnp, c_and_s, twirls

* fix style

* minor fix

* fix hypergraphatt and twirls

* pass gat

* fix torchmetric

* lintrunner
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* [Sparse] Support BSpMM and BSDDMM

* Update SpMM and SDDMM error messages

* Use TORCH_CHECK

* Update error string

* add coalesce() and has_duplicate()

* Add duplicate check in spspmm

* black

* fix

* clang-format

* lintrunner

* minor fix

* fix

Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* Update

* update

* Update

* Update

* Update

* format

* Update

* lint

* Update

* lint

* CI

* CI

* Update

* address comments
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* rename

* next time i should use lintrunner

* [Sparse] Add SpSpMM

* Update matmul interface

* address comments

* fix test utils to generate only coalesced matrices

* fix linter

* fix ut

* fix

* rm print
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* initial commit

* lint

* rename

* address comments

* add doctests

* address comments

* remove additional test

* add description

* lint

* lint

* lint

* Update test_reduction.py
Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com>

* [Sparse] Support SpMM and SDDMM

* Add SDDMMAutoGrad

* Rename SpMMImpl and SDDMMImpl and move them to dgl_sparse/src/matmul.h

* Update comments

* Update

* Minor fix

Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>

[Sparse] Add Two Sparse Matrices with Different Sparsity, DiagMatrix.dense, Add SparseMatrix and DiagMatrix (#5044)

* Update

* undo wrong edit

* lint

* fix import

* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* replace LooseVersion with packaging.version

* import packaging from setuptools

* import packaging.version in dgl.utils

* format

* revert changes to apps/kg

* revert changes to _dataloading

* [Dist] enable to chunk node/edge data into arbitrary number of chunks

* [Dist] enable to split node/edge data into arbitrary parts

* refine code

* Format boolean to uint8 forcely to avoid dist.scatter failure

* convert boolean to int8 before scatter and revert it after scatter

* refine code

* fix test

* refine code

* move test utilities into utils.py

* update comment

* fix empty data

* update

* update

* fix empty data issue

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* release unnecessary mem

* remove unnecessary shuffle data

* separate array_split into standalone utility

* add example
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

* Update

* Update

* lint

* CI

* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* [Dist] generate partition meta for ParMETIS

* Revert "[Sparse] Add SpMM and SDDMM. (#4999)"

This reverts commit 15365d78.

* lint

* [Sparse] Add SpMM and SDDMM

* Update

* Add CSR and CSC SpMM tests

* Update

* Update

* Update

* Update

* update

* lint

* CI

* CI

* CI
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* fix empty tensor is treated as pinned

* avoid calling cudaHostGetDevicePointer on nullptr

* update empty array

* add a comment

* add lint workflow

* add line number

* address comments

* update package versions

* remove autopep8 config

* Add GraphTransformerLayer (dense)

* beautify the python code with black

* refine according to mufei's comments

* fix AttributeError in unit test

* rename module as GraphormerLayer

* fix name issue
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* update agg function with new bindings

* handle optional import in __init__

* raise error in RelGraphConvAgg when pylibcugraphops not imported

* Update tests/cugraph/cugraph-ops/test_cugraph_relgraphconv.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update tests/cugraph/cugraph-ops/test_cugraph_relgraphconv.py
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* use keyword args for readability

* add missing docstring to pass CI

* catch ImportError rather than ModuleNotFoundError
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update

* update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* Add PathEncoder to transformer.py

* add blank line at the and

* rename variabl sp to shortest_path

* Fixed corresponding problems

* Fixed certain bugs when running on CUDA

* changed clamp min from 0 to 1
Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-146.ap-northeast-1.compute.internal>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update

* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-36-188.ap-northeast-1.compute.internal>

* fix unit test

* oops

* [Sparse] Support sparse format conversion

* Update

* Update

Signed-off-by: Serge Panev <spanev@nvidia.com>
Signed-off-by: Serge Panev <spanev@nvidia.com>

* [Dist] instantiate NodeDataView in lazy mode

* fix test failure

* init node/edge data store at the very beginning

* fix test failures

* refine comment

* add more tests