GitLab

* Update

* lint

* Fix

* Update

* Update

* Update

* update

* Fix

* Update

* Fix

* Fix

* Fix

* CI

* Update

* Update

* Update

* update test
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>

* fix test stream

* init cusparse handle

* [Sparse] Add reduction operators

* Fix linting problem

* Fix linting problem

* Fix linting problem

* Fix docstring

* Fix test cases

* Fix linting problem

* Skip reduction unittests

* initial commit

* address comments

* lint check
Co-authored-by: Israt Nisa <nisisrat@amazon.com>

* [Sparse] Add Transposition

* Fix docstring

* Fix linting problem

* Minor fix

* Minor fix
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal>

* fix

* Update hetero.py

* why did i remove this
Co-authored-by: Xin Yao <xiny@nvidia.com>

* add set_stream

* add .record_stream for NDArray and HeteroGraph

* refactor dgl stream Python APIs

* test record_stream

* add unit test for record stream

* use pytorch's stream

* fix lint

* fix cpu build

* address comments

* address comments

* add record stream tests for dgl.graph

* record frames and update dataloder

* add docstring

* update frame

* add backend check for record_stream

* remove CUDAThreadEntry::stream

* record stream for newly created formats

* fix bug

* fix cpp test

* fix None c_void_p to c_handle

* init

* Add api doc for sparse library

* support op btwn matrices with differnt sparsity

* Fixed docstring

* addresses comments

* lint check

* change keyword format to fmt
Co-authored-by: Israt Nisa <nisisrat@amazon.com>

[Feature] Added exclude_self and output_batch to knn graph construction (Issues #4323 #4316) (#4389)

* * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph
* Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information

* * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing

* * Added test to ensure dgl.remove_self_loop function correctly updates batch information

* * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph

* * Formatting

* * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph

* * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass
* Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed

* * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments

* * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test

* * Added doc comments for new parameters

* * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph
* Added test cases for more than k coincident points

* * Updated doc comments for output_batch parameters for clarity

* * Linter formatting fixes

* * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu

* * Rewording in doc comments

* * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* rgcn-ogbn-mag

* Add link in README.md

* correct code-format,add the reset_parameters function to the HeteroEmbedding module

* add the annotation in hetero.py

* add a unit test

* modify format

* Update
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-50-143.us-west-2.compute.internal>

* Fix doc and default settings for RelGraphConv

* Add unit test

* Split msg in two lines to pass CI-lint

* test ddp dataloader

* add pure_gpu for edgedataloader

* resolve ddp issue

* enable use for weighted neighbor sampler and biased random walk

* add unit tests

* fix for mxnet/tf

* fix typo

* Fix test_csrmm for tensor core

* unset allow tf32 flag

* update test unified tensor

* skip fp16 for CPU

Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>
Co-authored-by: Xin Yao <xiny@nvidia.com>

* Add uva by default to embedding

* More updates

* Update optimizer

* Add new uva functions

* Expose new pinned memory function

* Add unit tests

* Update formatting

* Fix unit test

* Handle auto UVA case when training is on CPU

* Allow per-embedding decisions for whether to use UVA

* Address spares_optim.py comments

* Remove unused templates

* Update unit test

* Use dgl allocate memory for pinning

* allow automatically unpin

* workaround for d2h copy with a different dtype

* fix linting

* update error message

* update copyright
Co-authored-by: Xin Yao <xiny@nvidia.com>
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* Explicitly unpin tensoradapter allocated arrays

* Undo unrelated change

* Add unit test

* update unit test

* add pinned_by_dgl flag to NDArray::Container

* use dgl.ndarray for holding the pinning status

* update multi-gpu uva inference

* reinterpret cast NDArray::Container* to DLTensor* in MoveAsDLTensor

* update unpin column and examples

* add unit test for unpin column
Co-authored-by: Dominique LaSalle <dlasalle@nvidia.com>
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>

* fix

* fix

* Update utils.py

* enhance EGATConv| nfeats as tuples

* egatconv modified for bipartite graphs

* modified docstrings

* added/modified unittests for EGATConv

* Update egatconv.py

* rectified lint errors
Co-authored-by: rijulizer <riju.mukherjee@gmail.com>
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

This reverts commit fdd1fe19.

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* add label propagation module

* fix prev bug in example

* add dgn

* fix linting and doc issues

* update label propagation & dgn

* update label propagation & dgn

* update example

* fix unit test

* fix agg heritage issue

* fix agg issue

* fix lint

* fix idx

* fix lp gpu issue

* Update

* Update
Co-authored-by: mufeili <mufeili1996@gmail.com>

Prevent users from attempting to pin PyTorch non-contiguous tensors or views only encompassing part of tensor. (#3992)

* Disable pinning non-contiguous memory

* Prevent views from being converted for write

* Fix linting

* Add unit tests

* Improve error message for users

* Switch to pytest function

* exclude mxnet and tensorflow from inplace pinning

* Add skip

* Restrict to pytorch backend

* Use backend to retrieve device

* Fix capitalization in decorator
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>

* Explicitly unpin tensoradapter allocated arrays

* Undo unrelated change

* Add unit test

* update unit test

* Update

* Update

* Update

* Update
Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com>

* enable UVA sampling with CPU indices

* add docs

* add more docs

* lint

* fix

* fix

* better error message

* use mp.Barrier instead of queues

* revert

* revert

* oops

* revert dgl.multiprocessing.spawn

* Update pytorch.py

* add EGNN & PNA

* fix egnn issues

* fix pna issues

* update pna conv

* add doc strings

* update pnaconv

* fix unused args issue

* fix moment aggregation issue
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* radius graph

* remove trailing whitespaces from docs

* disable invalid name for transform func

* disable radius graph module invalid name

* move pylint disable before init

* fix missing nodes from point set

* update docs indexing

* add compute mode as optional param

* radius graph test

* remove trailing whitespaces

* fix precision when comparing tensors
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Fix
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>

* fix

* unit test

* fix uva sampling with features

* fix

* add is_listlike function to distinguish strings from sequence

* fix
Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com>

* Update

* Fix

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* Update

* try fix

* try fix

* try fix

* try fix

* Revert "try fix"

This reverts commit a3fa0b1e9c0ab892cc3a22acf3770903db8b14a7.

* try fix shared memory

* try fix shared memory

* try fix image version

* fix

* WIP: TypedLinear and new RelGraphConv

* wip

* further simplify RGCN

* a bunch of tweak for performance; add basic cpu support

* update on segmm

* wip: segment.cu

* new backward kernel works

* fix a bunch of bugs in kernel; leave idx_a for future

* add nn test for typed_linear

* rgcn nn test

* bugfix in corner case; update RGCN README

* doc

* fix cpp lint

* fix lint

* fix ut

* wip: hgtconv; presorted flag for rgcn

* hgt code and ut; WIP: some fix on reorder graph

* better typed linear init

* fix ut

* fix lint; add docstring

* fixes

* fix

* more fixes

* update

* oops

* lint?

* temporarily revert - will fix in another PR

* more fixes

* skipping mxnet test

* address comments

* fix DDP

* fix edge dataloader exclusion problems

* stupid bug

* fix

* use_uvm option

* fix

* fixes

* fixes

* fixes

* fixes

* add evaluation for cluster gcn and ddp

* stupid bug again

* fixes

* move sanity checks to only support DGLGraphs

* pytorch lightning compatibility fixes

* remove

* poke

* more fixes

* fix

* fix

* disable test

* docstrings

* why is it getting a memory leak?

* fix

* update

* updates and temporarily disable forkingpickler

* update

* fix?

* fix?

* oops

* oops

* fix

* lint

* huh

* uh

* update

* fix

* made it memory efficient

* refine exclude interface

* fix tutorial

* fix tutorial

* fix graph duplication in CPU dataloader workers

* lint

* lint

* Revert "lint"

This reverts commit 805484dd553695111b5fb37f2125214a6b7276e9.

* Revert "lint"

This reverts commit 0bce411b2b415c2ab770343949404498436dc8b2.

* Revert "fix graph duplication in CPU dataloader workers"

This reverts commit 9e3a8cf34c175d3093c773f6bb023b155f2bd27f.
Co-authored-by: xiny <xiny@nvidia.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>

* modify hetero

* modify rst document

* update hetero

* update hetero

* update hetero

* update hetero

* Update

* Update

* Update

* Update

* 20220216

* Update

* Update

* Fix
Co-authored-by: Mufei Li <mufeili1996@gmail.com>
Co-authored-by: ShelkerX <925089962@qq.com>

* implement pin_memory/unpin_memory/is_pinned for dgl.graph

* update python docstring

* update c++ docstring

* add test

* fix the broken UnifiedTensor

* XPU_SWITCH for kDLCPUPinned

* a rough version ready for testing

* eliminate extra context parameter for pin/unpin

* update train_sampling

* fix linting

* fix typo

* multi-gpu uva sampling case

* disable new format materialization for pinned graphs

* update python doc for pin_memory_

* fix unit test

* UVA sampling for link prediction

* dispatch most csr ops

* update graphsage example to combine uva sampling and UnifiedTensor

* update graphsage example to combine uva sampling and UnifiedTensor

* update graphsage example to combine uva sampling and UnifiedTensor

* update doc

* update examples

* change unitgraph and heterograph's PinMemory to in-place

* update examples for multi-gpu uva sampling

* update doc

* fix linting

* fix cpu build

* fix is_pinned for DistGraph

* fix is_pinned for DistGraph

* update graphsage unsupervised example

* update doc for gpu sampling

* update some check for sampling device switching

* fix linting

* adapt for new dataloader

* fix linting

* fix

* fix some name issue

* adjust device check

* add unit test for uva sampling & fix some zero_copy bug

* fix linting

* update num_threads in graphsage examples
Co-authored-by: Quan (Andy) Gan <coin2028@hotmail.com>
Co-authored-by: Jinjing Zhou <VoVAllen@users.noreply.github.com>