[Deprecation] Examples (#4751)

* Update from master (#4584) * [Example][Refactor] Refactor graphsage multigpu and full-graph example (#4430) * Add refactors for multi-gpu and full-graph example * Fix format * Update * Update * Update * [Cleanup] Remove async_transferer (#4505) * Remove async_transferer * remove test * Remove AsyncTransferer Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Xin Yao <yaox12@outlook.com> * [Cleanup] Remove duplicate entries of CUB submodule (issue# 4395) (#4499) * remove third_part/cub * remove from third_party Co-authored-by: Israt Nisa <nisisrat@amazon.com> Co-authored-by: Xin Yao <xiny@nvidia.com> * [Bug] Enable turn on/off libxsmm at runtime (#4455) * enable turn on/off libxsmm at runtime by adding a global config and related API Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal> * [Feature] Unify the cuda stream used in core library (#4480) * Use an internal cuda stream for CopyDataFromTo * small fix white space * Fix to compile * Make stream optional in copydata for compile * fix lint issue * Update cub functions to use internal stream * Lint check * Update CopyTo/CopyFrom/CopyFromTo to use internal stream * Address comments * Fix backward CUDA stream * Avoid overloading CopyFromTo() * Minor comment update * Overload copydatafromto in cuda device api Co-authored-by: xiny <xiny@nvidia.com> * [Feature] Added exclude_self and output_batch to knn graph construction (Issues #4323 #4316) (#4389) * * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph * Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information * * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing * * Added test to ensure dgl.remove_self_loop function correctly updates batch information * * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph * * Formatting * * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph * * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass * Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed * * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments * * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test * * Added doc comments for new parameters * * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph * Added test cases for more than k coincident points * * Updated doc comments for output_batch parameters for clarity * * Linter formatting fixes * * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu * * Rewording in doc comments * * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * [CI] only known devs are authorized to trigger CI (#4518) * [CI] only known devs are authorized to trigger CI * fix if author is null * add comments * [Readability] Auto fix setup.py and update-version.py (#4446) * Auto fix update-version * Auto fix setup.py * Auto fix update-version * Auto fix setup.py * [Doc] Change random.py to random_partition.py in guide on distributed partition pipeline (#4438) * Update distributed-preprocessing.rst * Update Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> * fix unpinning when tensoradaptor is not available (#4450) * [Doc] fix print issue in tutorial (#4459) * [Example][Refactor] Refactor RGCN example (#4327) * Refactor full graph entity classification * Refactor rgcn with sampling * README update * Update * Results update * Respect default setting of self_loop=false in entity.py * Update * Update README * Update for multi-gpu * Update * [doc] fix invalid link in user guide (#4468) * [Example] directional_GSN for ogbg-molpcba (#4405) * version-1 * version-2 * version-3 * update examples/README * Update .gitignore * update performance in README, delete scripts * 1st approving review * 2nd approving review Co-authored-by: Mufei Li <mufeili1996@gmail.com> * Clarify the message name, which is 'm'. (#4462) Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> * [Refactor] Auto fix view.py. (#4461) Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * [Example] SEAL for OGBL (#4291) * [Example] SEAL for OGBL * update index * update * fix readme typo * add seal sampler * modify set ops * prefetch * efficiency test * update * optimize * fix ScatterAdd dtype issue * update sampler style * update Co-authored-by: Quan Gan <coin2028@hotmail.com> * [CI] use https instead of http (#4488) * [BugFix] fix crash due to incorrect dtype in dgl.to_block() (#4487) * [BugFix] fix crash due to incorrect dtype in dgl.to_block() * fix test failure in TF * [Feature] Make TensorAdapter Stream Aware (#4472) * Allocate tensors in DGL's current stream * make tensoradaptor stream-aware * replace TAemtpy with cpu allocator * fix typo * try fix cpu allocation * clean header * redirect AllocDataSpace as well * resolve comments * [Build][Doc] Specify the sphinx version (#4465) Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * reformat * reformat * Auto fix update-version * Auto fix setup.py * reformat * reformat Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> Co-authored-by: Mufei Li <mufeili1996@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Chang Liu <chang.liu@utexas.edu> Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: rudongyu <ru_dongyu@outlook.com> Co-authored-by: Quan Gan <coin2028@hotmail.com> * Move mock version of dgl_sparse library to DGL main repo (#4524) * init * Add api doc for sparse library * support op btwn matrices with differnt sparsity * Fixed docstring * addresses comments * lint check * change keyword format to fmt Co-authored-by: Israt Nisa <nisisrat@amazon.com> * [DistPart] expose timeout config for process group (#4532) * [DistPart] expose timeout config for process group * refine code * Update tools/distpartitioning/data_proc_pipeline.py Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * [Feature] Import PyTorch's CUDA stream management (#4503) * add set_stream * add .record_stream for NDArray and HeteroGraph * refactor dgl stream Python APIs * test record_stream * add unit test for record stream * use pytorch's stream * fix lint * fix cpu build * address comments * address comments * add record stream tests for dgl.graph * record frames and update dataloder * add docstring * update frame * add backend check for record_stream * remove CUDAThreadEntry::stream * record stream for newly created formats * fix bug * fix cpp test * fix None c_void_p to c_handle * [examples]educe memory consumption (#4558) * [examples]educe memory consumption * reffine help message * refine * [Feature][REVIEW] Enable DGL cugaph nightly CI (#4525) * Added cugraph nightly scripts * Removed nvcr.io//nvidia/pytorch:22.04-py3 reference Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> * Revert "[Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)" (#4563) This reverts commit ec171c64 . * [Misc] Add flake8 lint workflow. (#4566) * Add pyproject.toml for autopep8. * Add pyproject.toml for autopep8. * Add flake8 annotation in workflow. * remove * add * clean up Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Misc] Try use official pylint workflow. (#4568) * polish update_version * update pylint workflow. * add * revert. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [CI] refine stage logic (#4565) * [CI] refine stage logic * refine * refine * remove (#4570) Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Add Pylint workflow for flake8. (#4571) * remove * Add pylint. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Misc] Update the python version in Pylint workflow for flake8. (#4572) * remove * Add pylint. * Change the python version for pylint. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Update pylint. (#4574) Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Misc] Use another workflow. (#4575) * Update pylint. * Use another workflow. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Update pylint. (#4576) Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Update pylint.yml * Update pylint.yml * Delete pylint.yml * [Misc]Add pyproject.toml for autopep8 & black. (#4543) * Add pyproject.toml for autopep8. * Add pyproject.toml for autopep8. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Feature] Bump DLPack to v0.7 and decouple DLPack from the core library (#4454) * rename `DLContext` to `DGLContext` * rename `kDLGPU` to `kDLCUDA` * replace DLTensor with DGLArray * fix linting * Unify DGLType and DLDataType to DGLDataType * Fix FFI * rename DLDeviceType to DGLDeviceType * decouple dlpack from the core library * fix bug * fix lint * fix merge * fix build * address comments * rename dl_converter to dlpack_convert * remove redundant comments Co-authored-by: Chang Liu <chang.liu@utexas.edu> Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Xin Yao <yaox12@outlook.com> Co-authored-by: Israt Nisa <neesha295@gmail.com> Co-authored-by: Israt Nisa <nisisrat@amazon.com> Co-authored-by: peizhou001 <110809584+peizhou001@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal> Co-authored-by: ndickson-nvidia <99772994+ndickson-nvidia@users.noreply.github.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com> Co-authored-by: rudongyu <ru_dongyu@outlook.com> Co-authored-by: Quan Gan <coin2028@hotmail.com> Co-authored-by: Vibhu Jawa <vibhujawa@gmail.com> * [Deprecation] Dataset Attributes (#4546) * Update * CI * CI * Update Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> * [Example] Bug Fix (#4665) * Update * CI * CI * Update * Update Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> * Update * Update (#4724) Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Chang Liu <chang.liu@utexas.edu> Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Xin Yao <yaox12@outlook.com> Co-authored-by: Israt Nisa <neesha295@gmail.com> Co-authored-by: Israt Nisa <nisisrat@amazon.com> Co-authored-by: peizhou001 <110809584+peizhou001@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal> Co-authored-by: ndickson-nvidia <99772994+ndickson-nvidia@users.noreply.github.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com> Co-authored-by: rudongyu <ru_dongyu@outlook.com> Co-authored-by: Quan Gan <coin2028@hotmail.com> Co-authored-by: Vibhu Jawa <vibhujawa@gmail.com>

[Deprecation] Examples (#4751)
* Update from master (#4584) * [Example][Refactor] Refactor graphsage multigpu and full-graph example (#4430) * Add refactors for multi-gpu and full-graph example * Fix format * Update * Update * Update * [Cleanup] Remove async_transferer (#4505) * Remove async_transferer * remove test * Remove AsyncTransferer Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Xin Yao <yaox12@outlook.com> * [Cleanup] Remove duplicate entries of CUB submodule (issue# 4395) (#4499) * remove third_part/cub * remove from third_party Co-authored-by: Israt Nisa <nisisrat@amazon.com> Co-authored-by: Xin Yao <xiny@nvidia.com> * [Bug] Enable turn on/off libxsmm at runtime (#4455) * enable turn on/off libxsmm at runtime by adding a global config and related API Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal> * [Feature] Unify the cuda stream used in core library (#4480) * Use an internal cuda stream for CopyDataFromTo * small fix white space * Fix to compile * Make stream optional in copydata for compile * fix lint issue * Update cub functions to use internal stream * Lint check * Update CopyTo/CopyFrom/CopyFromTo to use internal stream * Address comments * Fix backward CUDA stream * Avoid overloading CopyFromTo() * Minor comment update * Overload copydatafromto in cuda device api Co-authored-by: xiny <xiny@nvidia.com> * [Feature] Added exclude_self and output_batch to knn graph construction (Issues #4323 #4316) (#4389) * * Added "exclude_self" and "output_batch" options to knn_graph and segmented_knn_graph * Updated out-of-date comments on remove_edges and remove_self_loop, since they now preserve batch information * * Changed defaults on new knn_graph and segmented_knn_graph function parameters, for compatibility; pytorch/test_geometry.py was failing * * Added test to ensure dgl.remove_self_loop function correctly updates batch information * * Added new knn_graph and segmented_knn_graph parameters to dgl.nn.KNNGraph and dgl.nn.SegmentedKNNGraph * * Formatting * * Oops, I missed the one in segmented_knn_graph when I fixed the similar thing in knn_graph * * Fixed edge case handling when invalid k specified, since it still needs to be handled consistently for tests to pass * Fixed context of batch info, since it must match the context of the input position data for remove_self_loop to succeed * * Fixed batch info resulting from knn_graph when output_batch is true, for case of 3D input tensor, representing multiple segments * * Added testing of new exclude_self and output_batch parameters on knn_graph and segmented_knn_graph, and their wrappers, KNNGraph and SegmentedKNNGraph, into the test_knn_cuda test * * Added doc comments for new parameters * * Added correct handling for uncommon case of k or more coincident points when excluding self edges in knn_graph and segmented_knn_graph * Added test cases for more than k coincident points * * Updated doc comments for output_batch parameters for clarity * * Linter formatting fixes * * Extracted out common function for test_knn_cpu and test_knn_cuda, to add the new test cases to test_knn_cpu * * Rewording in doc comments * * Removed output_batch parameter from knn_graph and segmented_knn_graph, in favour of always setting the batch information, except in knn_graph if x is a 2D tensor Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * [CI] only known devs are authorized to trigger CI (#4518) * [CI] only known devs are authorized to trigger CI * fix if author is null * add comments * [Readability] Auto fix setup.py and update-version.py (#4446) * Auto fix update-version * Auto fix setup.py * Auto fix update-version * Auto fix setup.py * [Doc] Change random.py to random_partition.py in guide on distributed partition pipeline (#4438) * Update distributed-preprocessing.rst * Update Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> * fix unpinning when tensoradaptor is not available (#4450) * [Doc] fix print issue in tutorial (#4459) * [Example][Refactor] Refactor RGCN example (#4327) * Refactor full graph entity classification * Refactor rgcn with sampling * README update * Update * Results update * Respect default setting of self_loop=false in entity.py * Update * Update README * Update for multi-gpu * Update * [doc] fix invalid link in user guide (#4468) * [Example] directional_GSN for ogbg-molpcba (#4405) * version-1 * version-2 * version-3 * update examples/README * Update .gitignore * update performance in README, delete scripts * 1st approving review * 2nd approving review Co-authored-by: Mufei Li <mufeili1996@gmail.com> * Clarify the message name, which is 'm'. (#4462) Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> * [Refactor] Auto fix view.py. (#4461) Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * [Example] SEAL for OGBL (#4291) * [Example] SEAL for OGBL * update index * update * fix readme typo * add seal sampler * modify set ops * prefetch * efficiency test * update * optimize * fix ScatterAdd dtype issue * update sampler style * update Co-authored-by: Quan Gan <coin2028@hotmail.com> * [CI] use https instead of http (#4488) * [BugFix] fix crash due to incorrect dtype in dgl.to_block() (#4487) * [BugFix] fix crash due to incorrect dtype in dgl.to_block() * fix test failure in TF * [Feature] Make TensorAdapter Stream Aware (#4472) * Allocate tensors in DGL's current stream * make tensoradaptor stream-aware * replace TAemtpy with cpu allocator * fix typo * try fix cpu allocation * clean header * redirect AllocDataSpace as well * resolve comments * [Build][Doc] Specify the sphinx version (#4465) Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * reformat * reformat * Auto fix update-version * Auto fix setup.py * reformat * reformat Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> Co-authored-by: Mufei Li <mufeili1996@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Chang Liu <chang.liu@utexas.edu> Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: rudongyu <ru_dongyu@outlook.com> Co-authored-by: Quan Gan <coin2028@hotmail.com> * Move mock version of dgl_sparse library to DGL main repo (#4524) * init * Add api doc for sparse library * support op btwn matrices with differnt sparsity * Fixed docstring * addresses comments * lint check * change keyword format to fmt Co-authored-by: Israt Nisa <nisisrat@amazon.com> * [DistPart] expose timeout config for process group (#4532) * [DistPart] expose timeout config for process group * refine code * Update tools/distpartitioning/data_proc_pipeline.py Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> * [Feature] Import PyTorch's CUDA stream management (#4503) * add set_stream * add .record_stream for NDArray and HeteroGraph * refactor dgl stream Python APIs * test record_stream * add unit test for record stream * use pytorch's stream * fix lint * fix cpu build * address comments * address comments * add record stream tests for dgl.graph * record frames and update dataloder * add docstring * update frame * add backend check for record_stream * remove CUDAThreadEntry::stream * record stream for newly created formats * fix bug * fix cpp test * fix None c_void_p to c_handle * [examples]educe memory consumption (#4558) * [examples]educe memory consumption * reffine help message * refine * [Feature][REVIEW] Enable DGL cugaph nightly CI (#4525) * Added cugraph nightly scripts * Removed nvcr.io//nvidia/pytorch:22.04-py3 reference Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> * Revert "[Feature][REVIEW] Enable DGL cugaph nightly CI (#4525)" (#4563) This reverts commit ec171c64 . * [Misc] Add flake8 lint workflow. (#4566) * Add pyproject.toml for autopep8. * Add pyproject.toml for autopep8. * Add flake8 annotation in workflow. * remove * add * clean up Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Misc] Try use official pylint workflow. (#4568) * polish update_version * update pylint workflow. * add * revert. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [CI] refine stage logic (#4565) * [CI] refine stage logic * refine * refine * remove (#4570) Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Add Pylint workflow for flake8. (#4571) * remove * Add pylint. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Misc] Update the python version in Pylint workflow for flake8. (#4572) * remove * Add pylint. * Change the python version for pylint. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Update pylint. (#4574) Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Misc] Use another workflow. (#4575) * Update pylint. * Use another workflow. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Update pylint. (#4576) Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * Update pylint.yml * Update pylint.yml * Delete pylint.yml * [Misc]Add pyproject.toml for autopep8 & black. (#4543) * Add pyproject.toml for autopep8. * Add pyproject.toml for autopep8. Co-authored-by: Steve <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> * [Feature] Bump DLPack to v0.7 and decouple DLPack from the core library (#4454) * rename `DLContext` to `DGLContext` * rename `kDLGPU` to `kDLCUDA` * replace DLTensor with DGLArray * fix linting * Unify DGLType and DLDataType to DGLDataType * Fix FFI * rename DLDeviceType to DGLDeviceType * decouple dlpack from the core library * fix bug * fix lint * fix merge * fix build * address comments * rename dl_converter to dlpack_convert * remove redundant comments Co-authored-by: Chang Liu <chang.liu@utexas.edu> Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Xin Yao <yaox12@outlook.com> Co-authored-by: Israt Nisa <neesha295@gmail.com> Co-authored-by: Israt Nisa <nisisrat@amazon.com> Co-authored-by: peizhou001 <110809584+peizhou001@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal> Co-authored-by: ndickson-nvidia <99772994+ndickson-nvidia@users.noreply.github.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com> Co-authored-by: rudongyu <ru_dongyu@outlook.com> Co-authored-by: Quan Gan <coin2028@hotmail.com> Co-authored-by: Vibhu Jawa <vibhujawa@gmail.com> * [Deprecation] Dataset Attributes (#4546) * Update * CI * CI * Update Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> * [Example] Bug Fix (#4665) * Update * CI * CI * Update * Update Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> * Update * Update (#4724) Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Chang Liu <chang.liu@utexas.edu> Co-authored-by: nv-dlasalle <63612878+nv-dlasalle@users.noreply.github.com> Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Xin Yao <yaox12@outlook.com> Co-authored-by: Israt Nisa <neesha295@gmail.com> Co-authored-by: Israt Nisa <nisisrat@amazon.com> Co-authored-by: peizhou001 <110809584+peizhou001@users.noreply.github.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-19-194.ap-northeast-1.compute.internal> Co-authored-by: ndickson-nvidia <99772994+ndickson-nvidia@users.noreply.github.com> Co-authored-by: Minjie Wang <wmjlyjemaine@gmail.com> Co-authored-by: Rhett Ying <85214957+Rhett-Ying@users.noreply.github.com> Co-authored-by: Hongzhi (Steve), Chen <chenhongzhi.nkcs@gmail.com> Co-authored-by: Ubuntu <ubuntu@ip-172-31-34-29.ap-northeast-1.compute.internal> Co-authored-by: Ubuntu <ubuntu@ip-172-31-9-26.ap-northeast-1.compute.internal> Co-authored-by: Zhiteng Li <55398076+ZHITENGLI@users.noreply.github.com> Co-authored-by: rudongyu <ru_dongyu@outlook.com> Co-authored-by: Quan Gan <coin2028@hotmail.com> Co-authored-by: Vibhu Jawa <vibhujawa@gmail.com>
ec17136e · Mufei Li · GitHub · 1990e797 · 1990e797 · 1990e797
Unverified Commit ec17136e authored Oct 26, 2022 by Mufei Li Committed by GitHub Oct 26, 2022
8 changed files
--- a/examples/pytorch/_deprecated/adaptive_sampling/adaptive_sampling.py
+++ b/examples/pytorch/_deprecated/adaptive_sampling/adaptive_sampling.py
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-import dgl
-import dgl.function as fn
-import numpy as np
-import scipy.sparse as ssp
-from dgl.data import citation_graph as citegrh
-import networkx as nx
-##load data
-##cora dataset have 2708 nodes, 1208 of them is used as train set 1000 of them is used as test set
-data = citegrh.load_cora()
-adj = nx.adjacency_matrix(data.graph)
-# reorder
-n_nodes = 2708
-ids_shuffle = np.arange(n_nodes)
-np.random.shuffle(ids_shuffle)
-adj = adj[ids_shuffle, :][:, ids_shuffle]
-data.features = data.features[ids_shuffle]
-data.labels = data.labels[ids_shuffle]
-##train-test split
-train_nodes = np.arange(1208)
-test_nodes = np.arange(1708, 2708)
-train_adj = adj[train_nodes, :][:, train_nodes]
-test_adj = adj[test_nodes, :][:, test_nodes]
-trainG = dgl.DGLGraphStale(train_adj)
-allG = dgl.DGLGraphStale(adj)
-h = torch.tensor(data.features[train_nodes], dtype=torch.float32)
-test_h = torch.tensor(data.features[test_nodes], dtype=torch.float32)
-all_h = torch.tensor(data.features, dtype=torch.float32)
-train_nodes = torch.tensor(train_nodes)
-test_nodes = torch.tensor(test_nodes)
-y_train = torch.tensor(data.labels[train_nodes])
-y_test = torch.tensor(data.labels[test_nodes])
-input_size = h.shape[1]
-output_size = data.num_labels
-##configuration
-config = {
-    'n_epoch': 300,
-    'lamb': 0.5,
-    'lr': 1e-3,
-    'weight_decay': 5e-4,
-    'hidden_size': 16,
-    ##sample size for each layer during training
-    'batch_size': 256,
-    ##sample size for each layer during test
-    'test_batch_size': 64,
-    'test_layer_sizes': [64 * 8, 64 * 8],
-}
-class NodeSampler(object):
-    """Minibatch sampler that samples batches of nodes uniformly from the given graph and list of seeds.
-    """
-    def __init__(self, graph, seeds, batch_size):
-        self.seeds = seeds
-        self.batch_size = batch_size
-        self.graph = graph
-    def __len__(self):
-        return len(self.seeds) // self.batch_size
-    def __iter__(self):
-        """Returns
-        (1) The seed node IDs, for NodeFlow generation,
-        (2) Indices of the seeds in the original seed array, as auxiliary data.
-        """
-        batches = torch.randperm(len(self.seeds)).split(self.batch_size)
-        for i in range(len(self)):
-            if len(batches[i]) < self.batch_size:
-                break
-            else:
-                yield self.seeds[batches[i]], batches[i]
-def create_nodeflow(layer_mappings, block_mappings, block_aux_data,
-                    data_dict, num_nodes_dict, seed_map):
-    hg = dgl.heterograph(data_dict, num_nodes_dict=num_nodes_dict)
-    hg.layer_mappings = layer_mappings
-    hg.block_mappings = block_mappings
-    hg.block_aux_data = block_aux_data
-    hg.seed_map = seed_map
-    return hg
-def normalize_adj(adj):
-    """Symmetrically normalize adjacency matrix."""
-    adj = ssp.eye(adj.shape[0]) + adj
-    adj = ssp.coo_matrix(adj)
-    rowsum = np.array(adj.sum(1))
-    d_inv_sqrt = np.power(rowsum, -0.5).flatten()
-    d_inv_sqrt[np.isinf(d_inv_sqrt)] = 0.
-    d_mat_inv_sqrt = ssp.diags(d_inv_sqrt)
-    return adj.dot(d_mat_inv_sqrt).transpose().dot(d_mat_inv_sqrt)
-class AdaptGenerator(object):
-    """
-    Nodeflow generator used for adaptive sampling
-    """
-    def __init__(self, graph, num_blocks, node_feature=None, sampler=None, num_workers=0, coalesce=False,
-                 sampler_weights=None, layer_nodes=None):
-        self.node_feature = node_feature
-        adj = graph.adjacency_matrix_scipy(transpose=False)
-        adj.data = np.ones(adj.nnz)
-        self.norm_adj = normalize_adj(adj).tocsr()
-        self.layer_nodes = layer_nodes
-        if sampler_weights is not None:
-            self.sample_weights = sampler_weights
-        else:
-            self.sample_weights = nn.Parameter(torch.randn((input_size, 2), dtype=torch.float32))
-            nn.init.xavier_uniform_(self.sample_weights)
-        self.graph = graph
-        self.num_blocks = num_blocks
-        self.coalesce = coalesce
-        self.sampler = sampler
-    def __iter__(self):
-        for sampled in self.sampler:
-            seeds = sampled[0]
-            auxiliary = sampled[1:]
-            hg = self(seeds, *auxiliary)
-            yield (hg, auxiliary[0])
-    def __call__(self, seeds, *auxiliary):
-        """
-           The __call__ function must take in an array of seeds, and any auxiliary data, and
-           return a NodeFlow grown from the seeds and conditioned on the auxiliary data.
-        """
-        curr_frontier = seeds  # Current frontier to grow neighbors from
-        layer_mappings = []  # Mapping from layer node ID to parent node ID
-        block_mappings = []  # Mapping from block edge ID to parent edge ID, or -1 if nonexistent
-        block_aux_data = []
-        data_dict = dict()
-        num_nodes_dict = dict()
-        if self.coalesce:
-            curr_frontier = torch.LongTensor(np.unique(seeds.numpy()))
-            invmap = {x: i for i, x in enumerate(curr_frontier.numpy())}
-            seed_map = [invmap[x] for x in seeds.numpy()]
-        else:
-            seed_map = list(range(len(seeds)))
-        layer_mappings.append(curr_frontier.numpy())
-        for i in reversed(range(self.num_blocks)):
-            neighbor_nodes, neighbor_edges, num_neighbors, aux_result = self.stepback(curr_frontier, i, *auxiliary)
-            prev_frontier_srcs = neighbor_nodes
-            # The un-coalesced mapping from block edge ID to parent edge ID
-            prev_frontier_edges = neighbor_edges.numpy()
-            nodes_idx_map = dict({*zip(neighbor_nodes.numpy(), range(len(aux_result)))})
-            # Coalesce nodes
-            if self.coalesce:
-                prev_frontier = np.unique(prev_frontier_srcs.numpy())
-                prev_frontier_invmap = {x: j for j, x in enumerate(prev_frontier)}
-                block_srcs = np.array([prev_frontier_invmap[s] for s in prev_frontier_srcs.numpy()])
-            else:
-                prev_frontier = prev_frontier_srcs.numpy()
-                block_srcs = np.arange(len(prev_frontier_edges))
-            aux_result = aux_result[[nodes_idx_map[i] for i in prev_frontier]]
-            block_dsts = np.arange(len(curr_frontier)).repeat(num_neighbors)
-            data_dict.update({
-                ('layer%d' % i, 'block%d' % i, 'layer%d' % (i + 1)): (block_srcs, block_dsts)
-            })
-            num_nodes_dict.update({
-                'layer%d' % i: len(prev_frontier),
-                'layer%d' % (i + 1): len(curr_frontier)
-            })
-            layer_mappings.insert(0, prev_frontier)
-            block_mappings.insert(0, prev_frontier_edges)
-            block_aux_data.insert(0, aux_result)
-            curr_frontier = torch.LongTensor(prev_frontier)
-        return create_nodeflow(
-            layer_mappings=layer_mappings,
-            block_mappings=block_mappings,
-            block_aux_data=block_aux_data,
-            data_dict=data_dict,
-            num_nodes_dict=num_nodes_dict,
-            seed_map=seed_map)
-    def stepback(self, curr_frontier, layer_index, *auxiliary):
-        """Function that takes in the node set in the current layer, and returns the
-        neighbors of each node.
-        Parameters
-        ----------
-        curr_frontier : Tensor
-        auxiliary : any auxiliary data yielded by the sampler
-        Returns
-        -------
-        neighbor_nodes, incoming_edges, num_neighbors, auxiliary: Tensor, Tensor, Tensor, any
-            num_neighbors[i] contains the number of neighbors generated for curr_frontier[i]
-            neighbor_nodes[sum(num_neighbors[0:i]):sum(num_neighbors[0:i+1])] contains the actual
-            neighbors as node IDs in the original graph for curr_frontier[i].
-            incoming_edges[sum(num_neighbors[0:i]):sum(num_neighbors[0:i+1])] contains the actual
-            incoming edges as edge IDs in the original graph for curr_frontier[i], or -1 if the
-            edge does not exist, or if we don't care about the edge, in the original graph.
-            auxiliary could be of any type containing block-specific additional data.
-        """
-        # Relies on that the same dst node of in_edges are contiguous, and the dst nodes
-        # are ordered the same as curr_frontier.
-        sample_weights = self.sample_weights
-        layer_size = self.layer_nodes[layer_index]
-        src, des, eid = self.graph.in_edges(curr_frontier, form='all')
-        neighbor_nodes = torch.unique(torch.cat((curr_frontier, src), dim=0), sorted=False)
-        sparse_adj = self.norm_adj[curr_frontier, :][:, neighbor_nodes]
-        square_adj = sparse_adj.multiply(sparse_adj).sum(0)
-        tensor_adj = torch.FloatTensor(square_adj.A[0])
-        ##compute sampling probability for next layer which is decided by :
-        # 1. attention part derived from node hidden feature
-        # 2. adjacency part derived from graph structure
-        hu = torch.matmul(self.node_feature[neighbor_nodes], sample_weights[:, 0])
-        hv = torch.sum(torch.matmul(self.node_feature[curr_frontier], sample_weights[:, 1]))
-        adj_part = torch.sqrt(tensor_adj)
-        attention_part = F.relu(hv + hu) + 1
-        gu = F.relu(hu) + 1
-        probas = adj_part * attention_part * gu
-        probas = probas / torch.sum(probas)
-        ##build graph between candidates and curr_frontier
-        candidates = neighbor_nodes[probas.multinomial(num_samples=layer_size, replacement=True)]
-        ivmap = {x: i for i, x in enumerate(neighbor_nodes.numpy())}
-        # use matrix operation in pytorch to avoid for-loop
-        curr_padding = curr_frontier.repeat_interleave(len(candidates))
-        cand_padding = candidates.repeat(len(curr_frontier))
-        ##the edges between candidates and curr_frontier composed of
-        # 1. edges in orginal graph
-        # 2. edges between same node(self-loop)
-        has_loops = curr_padding == cand_padding
-        has_edges = self.graph.has_edges_between(cand_padding, curr_padding)
-        loops_or_edges = (has_edges.bool() + has_loops).int()
-        # get neighbor_nodes and corresponding sampling probability
-        num_neighbors = loops_or_edges.reshape((len(curr_frontier), -1)).sum(1)
-        sample_neighbor = cand_padding[loops_or_edges.bool()]
-        q_prob = probas[[ivmap[i] for i in sample_neighbor.numpy()]]
-        # get the edge mapping ,-1 if the edge dostn't exist
-        eids = torch.zeros(torch.sum(num_neighbors), dtype=torch.int64) - 1
-        has_edge_ids = torch.where(has_edges)[0]
-        all_ids = torch.where(loops_or_edges)[0]
-        edges_ids_map = torch.where(has_edge_ids[:, None] == all_ids[None, :])[1]
-        u, v, e = self.graph.edge_ids(cand_padding, curr_padding, return_uv=True)
-        eids[edges_ids_map] = e
-        return sample_neighbor, eids, num_neighbors, q_prob
-class AdaptSAGEConv(nn.Module):
-    def __init__(self,
-                 in_feats,
-                 out_feats,
-                 aggregator_type='mean',
-                 feat_drop=0.,
-                 bias=True,
-                 norm=None,
-                 activation=None,
-                 G=None):
-        super(AdaptSAGEConv, self).__init__()
-        self._in_feats = in_feats
-        self._out_feats = out_feats
-        self.norm = norm
-        self.feat_drop = nn.Dropout(feat_drop)
-        self.activation = activation
-        # self.fc_self = nn.Linear(in_feats, out_feats, bias=bias).double()
-        self.fc_neigh = nn.Linear(in_feats, out_feats, bias=bias)
-        self.reset_parameters()
-        self.G = G
-    def reset_parameters(self):
-        gain = nn.init.calculate_gain('relu')
-        # nn.init.xavier_uniform_(self.fc_self.weight, gain=gain)
-        nn.init.xavier_uniform_(self.fc_neigh.weight, gain=gain)
-    def forward(self, graph, hidden_feat, node_feat, layer_id, sample_weights, norm_adj=None, var_loss=None,
-                is_test=False):
-        """
-        graph: Bipartite.  Has two edge types.  The first one represents the connection to
-        the desired nodes from neighbors.  The second one represents the computation
-        dependence of the desired nodes themselves.
-        :type graph: dgl.DGLHeteroGraph
-        """
-        # local_var is not implemented for heterograph
-        # graph = graph.local_var()
-        neighbor_etype_name = 'block%d' % layer_id
-        src_name = 'layer%d' % layer_id
-        dst_name = 'layer%d' % (layer_id + 1)
-        graph.nodes[src_name].data['hidden_feat'] = hidden_feat
-        graph.nodes[src_name].data['node_feat'] = node_feat[graph.layer_mappings[layer_id]]
-        ##normalized degree for node_i is defined as  1/sqrt(d_i+1)
-        ##use the training graph during training and whole graph during testing
-        if not is_test:
-            graph.nodes[src_name].data['norm_deg'] = 1 / torch.sqrt(
-                trainG.in_degrees(graph.layer_mappings[layer_id]).float() + 1)
-            graph.nodes[dst_name].data['norm_deg'] = 1 / torch.sqrt(
-                trainG.in_degrees(graph.layer_mappings[layer_id + 1]).float() + 1)
-        else:
-            graph.nodes[src_name].data['norm_deg'] = 1 / torch.sqrt(
-                allG.in_degrees(graph.layer_mappings[layer_id]).float() + 1)
-            graph.nodes[dst_name].data['norm_deg'] = 1 / torch.sqrt(
-                allG.in_degrees(graph.layer_mappings[layer_id + 1]).float() + 1)
-        graph.nodes[dst_name].data['node_feat'] = node_feat[graph.layer_mappings[layer_id + 1]]
-        graph.nodes[src_name].data['q_probs'] = graph.block_aux_data[layer_id]
-        def send_func(edges):
-            hu = torch.matmul(edges.src['node_feat'], sample_weights[:, 0])
-            hv = torch.matmul(edges.dst['node_feat'], sample_weights[:, 1])
-            ##attention coeffient is adjusted by normalized degree and sampling probability
-            attentions = edges.src['norm_deg'] * edges.dst['norm_deg'] * (F.relu(hu + hv) + 0.1) / edges.src[
-                'q_probs'] / len(hu)
-            hidden = edges.src['hidden_feat'] * torch.reshape(attentions, [-1, 1])
-            return {"hidden": hidden}
-        recv_func = dgl.function.sum('hidden', 'neigh')
-        # def receive_fuc(nodes):
-        # aggregate from neighbors
-        graph[neighbor_etype_name].update_all(message_func=send_func, reduce_func=recv_func)
-        h_neigh = graph.nodes[dst_name].data['neigh']
-        rst = self.fc_neigh(h_neigh)
-        # activation
-        if self.activation is not None:
-            rst = self.activation(rst)
-        # normalization
-        if self.norm is not None:
-            rst = self.norm(rst)
-        # compute the variance loss according to the formula in orginal paper
-        if var_loss and not is_test:
-            pre_sup = self.fc_neigh(hidden_feat)  # u*h
-            ##normalized adjacency matrix for nodeflow layer
-            support = norm_adj[graph.layer_mappings[layer_id + 1], :][:, graph.layer_mappings[layer_id]]  ##v*u
-            hu = torch.matmul(node_feat[graph.layer_mappings[layer_id]], sample_weights[:, 0])
-            hv = torch.matmul(node_feat[graph.layer_mappings[layer_id + 1]], sample_weights[:, 1])
-            attentions = (F.relu(torch.reshape(hu, [1, -1]) + torch.reshape(hv, [-1, 1])) + 1) / graph.block_aux_data[
-                layer_id] / len(hu)
-            adjust_support = torch.tensor(support.A, dtype=torch.float32) * attentions
-            support_mean = adjust_support.sum(0)
-            mu_v = torch.mean(rst, dim=0)  # h
-            diff = torch.reshape(support_mean, [-1, 1]) * pre_sup - torch.reshape(mu_v, [1, -1])
-            loss = torch.sum(diff * diff) / len(hu) / len(hv)
-            return rst, loss
-        return rst
-class AdaptGraphSAGENet(nn.Module):
-    def __init__(self, sample_weights, node_feature, hidden_size):
-        super().__init__()
-        self.layers = nn.ModuleList([
-            AdaptSAGEConv(input_size, hidden_size, 'mean', bias=False, activation=F.relu),
-            AdaptSAGEConv(hidden_size, output_size, 'mean', bias=False, activation=F.relu),
-        ])
-        self.sample_weights = sample_weights
-        self.node_feature = node_feature
-        self.norm_adj = normalize_adj(trainG.adjacency_matrix_scipy(transpose=False))
-    def forward(self, nf, h, is_test=False):
-        for i, layer in enumerate(self.layers):
-            if i == len(self.layers) - 1 and not is_test:
-                h, loss = layer(nf, h, self.node_feature, i, self.sample_weights, norm_adj=self.norm_adj, var_loss=True,
-                                is_test=is_test)
-            else:
-                h = layer(nf, h, self.node_feature, i, self.sample_weights, is_test=is_test)
-        if is_test:
-            return h
-        return h, loss
-def main(args):
-    config.update(args)
-    config['layer_sizes'] = [int(config['node_per_layer']) for _ in range(2)]
-    # Create a sampler for training nodes and testing nodes
-    train_sampler = NodeSampler(graph=trainG, seeds=train_nodes, batch_size=config['batch_size'])
-    test_sampler = NodeSampler(graph=allG, seeds=test_nodes, batch_size=config['test_batch_size'])
-    ##Generator for training
-    train_generator = AdaptGenerator(graph=trainG, node_feature=all_h, layer_nodes=config['layer_sizes'],
-                                     sampler=train_sampler,
-                                     num_blocks=len(config['layer_sizes']), coalesce=True)
-    # Generator for testing
-    test_sample_generator = AdaptGenerator(graph=allG, node_feature=all_h, sampler=test_sampler,
-                                           num_blocks=len(config['test_layer_sizes']),
-                                           sampler_weights=train_generator.sample_weights,
-                                           layer_nodes=config['test_layer_sizes'],
-                                           coalesce=True)
-    model = AdaptGraphSAGENet(train_generator.sample_weights, all_h, config['hidden_size'])
-    params = list(model.parameters())
-    params.append(train_generator.sample_weights)
-    opt = torch.optim.Adam(params=params, lr=config['lr'])
-    # model.train()
-    lamb, weight_decay = config['lamb'], config['weight_decay']
-    for epoch in range(config['n_epoch']):
-        train_accs = []
-        for nf, sample_indices in train_generator:
-            seed_map = nf.seed_map
-            train_y_hat, varloss = model(nf, h[nf.layer_mappings[0]])
-            train_y_hat = train_y_hat[seed_map]
-            y_train_batch = y_train[sample_indices]
-            y_pred = torch.argmax(train_y_hat, dim=1)
-            train_acc = torch.sum(torch.eq(y_pred, y_train_batch)).item() / config['batch_size']
-            train_accs.append(train_acc)
-            loss = F.cross_entropy(train_y_hat.squeeze(), y_train_batch)
-            l2_loss = torch.norm(params[0])
-            total_loss = varloss * lamb + loss + l2_loss * weight_decay
-            opt.zero_grad()
-            total_loss.backward()
-            opt.step()
-            # print(train_sampler.sample_weight)
-        test_accs = []
-        for test_nf, test_sample_indices in test_sample_generator:
-            seed_map = test_nf.seed_map
-            # print(test_sample_indices)
-            test_y_hat = model(test_nf, all_h[test_nf.layer_mappings[0]], is_test=True)
-            # print("test",test_y_hat)
-            test_y_hat = test_y_hat[seed_map]
-            y_test_batch = y_test[test_sample_indices]
-            y_pred = torch.argmax(test_y_hat, dim=1)
-            test_acc = torch.sum(torch.eq(y_pred, y_test_batch)).item() / len(y_pred)
-            test_accs.append(test_acc)
-        print("eqoch{} train accuracy {}, regloss {}, loss {} ,test accuracy {}".format(epoch, np.mean(train_acc),
-                                                                                        varloss.item() * lamb,
-                                                                                        total_loss.item(),
-                                                                                        np.mean(test_accs)))
-if __name__ == "__main__":
-    import argparse
-    parser = argparse.ArgumentParser('Adaptive Sampling for CCN')
-    parser.add_argument('-b', '--batch_size', type=int, default=256,
-                        help='batch size')
-    parser.add_argument('-l', '--node_per_layer', type=float, default=256,
-                        help='sampling size for each layer')
-    args = parser.parse_args().__dict__
-    main(args)
--- a/examples/pytorch/_deprecated/sampling/README.md
+++ b/examples/pytorch/_deprecated/sampling/README.md
-# Stochastic Training for Graph Convolutional Networks
-DEPRECATED!!
-* Paper: [Control Variate](https://arxiv.org/abs/1710.10568)
-* Paper: [Skip Connection](https://arxiv.org/abs/1809.05343)
-* Author's code: [https://github.com/thu-ml/stochastic_gcn](https://github.com/thu-ml/stochastic_gcn)
-Dependencies
------------
- PyTorch 0.4.1+
- requests
-``bash
-pip install torch requests
-``
-### Neighbor Sampling & Skip Connection
-#### cora 
-Test accuracy ~83% with --num-neighbors 2, ~84% by training on the full graph
-```
-DGLBACKEND=pytorch python3 gcn_ns_sc.py --dataset cora --self-loop --num-neighbors 2 --batch-size 1000000 --test-batch-size 1000000
-```
-#### citeseer 
-Test accuracy ~69% with --num-neighbors 2, ~70% by training on the full graph
-```
-DGLBACKEND=pytorch python3 gcn_ns_sc.py --dataset citeseer --self-loop --num-neighbors 2 --batch-size 1000000 --test-batch-size 1000000
-```
-#### pubmed 
-Test accuracy ~76% with --num-neighbors 3, ~77% by training on the full graph
-```
-DGLBACKEND=pytorch python3 gcn_ns_sc.py --dataset pubmed --self-loop --num-neighbors 3 --batch-size 1000000 --test-batch-size 1000000
-```
-### Control Variate & Skip Connection
-#### cora 
-Test accuracy ~84% with --num-neighbors 1, ~84% by training on the full graph
-```
-DGLBACKEND=pytorch python3 gcn_cv_sc.py --dataset cora --self-loop --num-neighbors 1 --batch-size 1000000 --test-batch-size 1000000
-```
-#### citeseer 
-Test accuracy ~69% with --num-neighbors 1, ~70% by training on the full graph
-```
-DGLBACKEND=pytorch python3 gcn_cv_sc.py --dataset citeseer --self-loop --num-neighbors 1 --batch-size 1000000 --test-batch-size 1000000
-```
-#### pubmed 
-Test accuracy ~77% with --num-neighbors 1, ~77% by training on the full graph
-```
-DGLBACKEND=pytorch python3 gcn_cv_sc.py --dataset pubmed --self-loop --num-neighbors 1 --batch-size 1000000 --test-batch-size 1000000
-```
--- a/examples/pytorch/_deprecated/sampling/dis_sampling/README.md
+++ b/examples/pytorch/_deprecated/sampling/dis_sampling/README.md
-# Stochastic Training for Graph Convolutional Networks Using Distributed Sampler
-* Paper: [Control Variate](https://arxiv.org/abs/1710.10568)
-* Paper: [Skip Connection](https://arxiv.org/abs/1809.05343)
-* Author's code: [https://github.com/thu-ml/stochastic_gcn](https://github.com/thu-ml/stochastic_gcn)
-Dependencies
------------
- PyTorch 0.4.1+
- requests
-``bash
-pip install torch requests
-``
-### Usage Guide
-Assume that the user has already launched two instances (`instance_0` & `instance_1`) on AWS EC2, and also these two instances have the correct authority to access each other by TCP/IP protocol. Now we can treat `instance_0` as `Trainer` and `instance_1` as `Sampler`. Then, the user can start the trainer process and sampler process on these two instances separately. We have already provided a set of scripts to start the trainer and sampler process and users just need to change the `--ip` to their own IP address.
-Once we start the trainer process, users will see the following logging output:
-```
-[04:48:20] .../socket_communicator.cc:68: Bind to 127.0.0.1:2049
-[04:48:20] .../socket_communicator.cc:74: Listen on 127.0.0.1:2049, wait sender connect ...
-```
-After that user can start the sampler process. For the sampler instance_0, users can change the `--num-sampler` option to set the number of the sampler. The `sampler.py` script will start `--num-sampler` processes concurrently to maximalize the system utilization. Users can also launch many samplers in parallel across a set of machines. For example, if we have `10` sampler instance and for each instance, we set the `--num-sampler` to `2`, we need to set the `--num-sampler` of the trainer instance to `20`.
-### Neighbor Sampling & Skip Connection
-#### cora
-Test accuracy ~83% with --num-neighbors 2, ~84% by training on the full graph
-Trainer side:
-```
-DGLBACKEND=pytorch python3 gcn_ns_sc_train.py --dataset cora --self-loop --num-neighbors 2 --batch-size 1000000 --test-batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-Sampler side:
-```
-OMP_NUM_THREADS=1 DGLBACKEND=pytorch python3 sampler.py --model gcn_ns --dataset cora --self-loop --num-neighbors 2 --batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-#### citeseer 
-Test accuracy ~69% with --num-neighbors 2, ~70% by training on the full graph
-Trainer side:
-```
-DGLBACKEND=pytorch python3 gcn_ns_sc_train.py --dataset citeseer --self-loop --num-neighbors 2 --batch-size 1000000 --test-batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-Sampler side:
-```
-OMP_NUM_THREADS=1 DGLBACKEND=pytorch python3 sampler.py --model gcn_ns --dataset citeseer --self-loop --num-neighbors 2 --batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-#### pubmed 
-Test accuracy ~76% with --num-neighbors 3, ~77% by training on the full graph
-Trainer side:
-```
-DGLBACKEND=pytorch python3 gcn_ns_sc_train.py --dataset pubmed --self-loop --num-neighbors 3 --batch-size 1000000 --test-batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-Sampler side:
-```
-OMP_NUM_THREADS=1 DGLBACKEND=pytorch python3 sampler.py --model gcn_ns --dataset pubmed --self-loop --num-neighbors 3 --batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-### Control Variate & Skip Connection
-#### cora
-Test accuracy ~84% with --num-neighbors 1, ~84% by training on the full graph
-Trainer side:
-```
-DGLBACKEND=pytorch python3 gcn_cv_sc_train.py --dataset cora --self-loop --num-neighbors 1 --batch-size 1000000 --test-batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-Sampler side:
-```
-OMP_NUM_THREADS=1 DGLBACKEND=pytorch python3 sampler.py --model gcn_cv --dataset cora --self-loop --num-neighbors 1 --batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-#### citeseer
-Test accuracy ~69% with --num-neighbors 1, ~70% by training on the full graph
-Trainer side:
-```
-DGLBACKEND=pytorch python3 gcn_cv_sc_train.py --dataset citeseer --self-loop --num-neighbors 1 --batch-size 1000000 --test-batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-Sampler side:
-```
-OMP_NUM_THREADS=1 DGLBACKEND=pytorch python3 sampler.py --model gcn_cv --dataset citeseer --self-loop --num-neighbors 1 --batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-#### pubmed
-Test accuracy ~77% with --num-neighbors 1, ~77% by training on the full graph
-Trainer side:
-```
-DGLBACKEND=pytorch python3 gcn_cv_sc_train.py --dataset pubmed --self-loop --num-neighbors 1 --batch-size 1000000 --test-batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
-Sampler side:
-```
-OMP_NUM_THREADS=1 DGLBACKEND=pytorch python3 sampler.py --model gcn_cv --dataset pubmed --self-loop --num-neighbors 1 --batch-size 1000000 --ip 127.0.0.1:50051 --num-sampler 1
-```
--- a/examples/pytorch/_deprecated/sampling/dis_sampling/gcn_cv_sc_train.py
+++ b/examples/pytorch/_deprecated/sampling/dis_sampling/gcn_cv_sc_train.py
-import os, sys
-import argparse, time, math
-import numpy as np
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-import dgl
-import dgl.function as fn
-from dgl import DGLGraph
-from dgl.data import register_data_args, load_data
-parentdir=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
-sys.path.insert(0, parentdir)
-from gcn_cv_sc import NodeUpdate, GCNSampling, GCNInfer
-def main(args):
-    # load and preprocess dataset
-    data = load_data(args)
-    if args.self_loop and not args.dataset.startswith('reddit'):
-        data.graph.add_edges_from([(i,i) for i in range(len(data.graph))])
-    train_nid = np.nonzero(data.train_mask)[0].astype(np.int64)
-    test_nid = np.nonzero(data.test_mask)[0].astype(np.int64)
-    features = torch.FloatTensor(data.features)
-    labels = torch.LongTensor(data.labels)
-    if hasattr(torch, 'BoolTensor'):
-        train_mask = torch.BoolTensor(data.train_mask)
-        val_mask = torch.BoolTensor(data.val_mask)
-        test_mask = torch.BoolTensor(data.test_mask)
-    else:
-        train_mask = torch.ByteTensor(data.train_mask)
-        val_mask = torch.ByteTensor(data.val_mask)
-        test_mask = torch.ByteTensor(data.test_mask)
-    in_feats = features.shape[1]
-    n_classes = data.num_labels
-    n_edges = data.graph.number_of_edges()
-    n_train_samples = train_mask.int().sum().item()
-    n_val_samples = val_mask.int().sum().item()
-    n_test_samples = test_mask.int().sum().item()
-    print("""----Data statistics------'
-      #Edges %d
-      #Classes %d
-      #Train samples %d
-      #Val samples %d
-      #Test samples %d""" %
-          (n_edges, n_classes,
-              n_train_samples,
-              n_val_samples,
-              n_test_samples))
-    # create GCN model
-    g = DGLGraph(data.graph, readonly=True)
-    norm = 1. / g.in_degrees().float().unsqueeze(1)
-    if args.gpu < 0:
-        cuda = False
-    else:
-        cuda = True
-        torch.cuda.set_device(args.gpu)
-        features = features.cuda()
-        labels = labels.cuda()
-        train_mask = train_mask.cuda()
-        val_mask = val_mask.cuda()
-        test_mask = test_mask.cuda()
-        norm = norm.cuda()
-    g.ndata['features'] = features
-    num_neighbors = args.num_neighbors
-    n_layers = args.n_layers
-    g.ndata['norm'] = norm
-    g.update_all(fn.copy_src(src='features', out='m'),
-                 fn.sum(msg='m', out='preprocess'),
-                 lambda node : {'preprocess': node.data['preprocess'] * node.data['norm']})
-    for i in range(n_layers):
-        g.ndata['h_{}'.format(i)] = torch.zeros(features.shape[0], args.n_hidden).to(device=features.device)
-    g.ndata['h_{}'.format(n_layers-1)] = torch.zeros(features.shape[0], 2*args.n_hidden).to(device=features.device)
-    model = GCNSampling(in_feats,
-                        args.n_hidden,
-                        n_classes,
-                        n_layers,
-                        F.relu,
-                        args.dropout)
-    loss_fcn = nn.CrossEntropyLoss()
-    infer_model = GCNInfer(in_feats,
-                           args.n_hidden,
-                           n_classes,
-                           n_layers,
-                           F.relu)
-    if cuda:
-        model.cuda()
-        infer_model.cuda()
-    # use optimizer
-    optimizer = torch.optim.Adam(model.parameters(),
-                                 lr=args.lr,
-                                 weight_decay=args.weight_decay)
-    # Create sampler receiver
-    sampler = dgl.contrib.sampling.SamplerReceiver(graph=g, addr=args.ip, num_sender=args.num_sampler)
-    for epoch in range(args.n_epochs):
-        for nf in sampler:
-            for i in range(n_layers):
-                agg_history_str = 'agg_h_{}'.format(i)
-                g.pull(nf.layer_parent_nid(i+1).long(), fn.copy_src(src='h_{}'.format(i), out='m'),
-                       fn.sum(msg='m', out=agg_history_str),
-                       lambda node : {agg_history_str: node.data[agg_history_str] * node.data['norm']})
-            node_embed_names = [['preprocess', 'h_0']]
-            for i in range(1, n_layers):
-                node_embed_names.append(['h_{}'.format(i), 'agg_h_{}'.format(i-1)])
-            node_embed_names.append(['agg_h_{}'.format(n_layers-1)])
-            nf.copy_from_parent(node_embed_names=node_embed_names)
-            model.train()
-            # forward
-            pred = model(nf)
-            batch_nids = nf.layer_parent_nid(-1).to(device=pred.device).long()
-            batch_labels = labels[batch_nids]
-            loss = loss_fcn(pred, batch_labels)
-            optimizer.zero_grad()
-            loss.backward()
-            optimizer.step()
-            node_embed_names = [['h_{}'.format(i)] for i in range(n_layers)]
-            node_embed_names.append([])
-            nf.copy_to_parent(node_embed_names=node_embed_names)
-        for infer_param, param in zip(infer_model.parameters(), model.parameters()):
-            infer_param.data.copy_(param.data)
-        num_acc = 0.
-        for nf in dgl.contrib.sampling.NeighborSampler(g, args.test_batch_size,
-                                                       g.number_of_nodes(),
-                                                       neighbor_type='in',
-                                                       num_workers=32,
-                                                       num_hops=n_layers,
-                                                       seed_nodes=test_nid):
-            node_embed_names = [['preprocess']]
-            for i in range(n_layers):
-                node_embed_names.append(['norm'])
-            nf.copy_from_parent(node_embed_names=node_embed_names)
-            infer_model.eval()
-            with torch.no_grad():
-                pred = infer_model(nf)
-                batch_nids = nf.layer_parent_nid(-1).to(device=pred.device).long()
-                batch_labels = labels[batch_nids]
-                num_acc += (pred.argmax(dim=1) == batch_labels).sum().cpu().item()
-        print("Test Accuracy {:.4f}". format(num_acc/n_test_samples))
-if __name__ == '__main__':
-    parser = argparse.ArgumentParser(description='GCN')
-    register_data_args(parser)
-    parser.add_argument("--dropout", type=float, default=0.5,
-            help="dropout probability")
-    parser.add_argument("--gpu", type=int, default=-1,
-            help="gpu")
-    parser.add_argument("--lr", type=float, default=3e-2,
-            help="learning rate")
-    parser.add_argument("--n-epochs", type=int, default=200,
-            help="number of training epochs")
-    parser.add_argument("--batch-size", type=int, default=1000,
-            help="train batch size")
-    parser.add_argument("--test-batch-size", type=int, default=1000,
-            help="test batch size")
-    parser.add_argument("--num-neighbors", type=int, default=2,
-            help="number of neighbors to be sampled")
-    parser.add_argument("--n-hidden", type=int, default=16,
-            help="number of hidden gcn units")
-    parser.add_argument("--n-layers", type=int, default=1,
-            help="number of hidden gcn layers")
-    parser.add_argument("--self-loop", action='store_true',
-            help="graph self-loop (default=False)")
-    parser.add_argument("--weight-decay", type=float, default=5e-4,
-            help="Weight for L2 loss")
-    parser.add_argument("--ip", type=str, default='127.0.0.1:50051',
-            help="IP address")
-    parser.add_argument("--num-sampler", type=int, default=1,
-            help="number of sampler")
-    args = parser.parse_args()
-    print(args)
-    main(args)
--- a/examples/pytorch/_deprecated/sampling/dis_sampling/gcn_ns_sc_train.py
+++ b/examples/pytorch/_deprecated/sampling/dis_sampling/gcn_ns_sc_train.py
-import os, sys
-import argparse, time, math
-import numpy as np
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-from functools import partial
-import dgl
-import dgl.function as fn
-from dgl import DGLGraph
-from dgl.data import register_data_args, load_data
-parentdir=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
-sys.path.insert(0, parentdir)
-from gcn_ns_sc import NodeUpdate, GCNSampling, GCNInfer
-def main(args):
-    # load and preprocess dataset
-    data = load_data(args)
-    if args.self_loop and not args.dataset.startswith('reddit'):
-        data.graph.add_edges_from([(i,i) for i in range(len(data.graph))])
-    train_nid = np.nonzero(data.train_mask)[0].astype(np.int64)
-    test_nid = np.nonzero(data.test_mask)[0].astype(np.int64)
-    features = torch.FloatTensor(data.features)
-    labels = torch.LongTensor(data.labels)
-    if hasattr(torch, 'BoolTensor'):
-        train_mask = torch.BoolTensor(data.train_mask)
-        val_mask = torch.BoolTensor(data.val_mask)
-        test_mask = torch.BoolTensor(data.test_mask)
-    else:
-        train_mask = torch.ByteTensor(data.train_mask)
-        val_mask = torch.ByteTensor(data.val_mask)
-        test_mask = torch.ByteTensor(data.test_mask)
-    in_feats = features.shape[1]
-    n_classes = data.num_labels
-    n_edges = data.graph.number_of_edges()
-    n_train_samples = train_mask.int().sum().item()
-    n_val_samples = val_mask.int().sum().item()
-    n_test_samples = test_mask.int().sum().item()
-    print("""----Data statistics------'
-      #Edges %d
-      #Classes %d
-      #Train samples %d
-      #Val samples %d
-      #Test samples %d""" %
-          (n_edges, n_classes,
-              n_train_samples,
-              n_val_samples,
-              n_test_samples))
-    # create GCN model
-    g = DGLGraph(data.graph, readonly=True)
-    norm = 1. / g.in_degrees().float().unsqueeze(1)
-    if args.gpu < 0:
-        cuda = False
-    else:
-        cuda = True
-        torch.cuda.set_device(args.gpu)
-        features = features.cuda()
-        labels = labels.cuda()
-        train_mask = train_mask.cuda()
-        val_mask = val_mask.cuda()
-        test_mask = test_mask.cuda()
-        norm = norm.cuda()
-    g.ndata['features'] = features
-    num_neighbors = args.num_neighbors
-    g.ndata['norm'] = norm
-    model = GCNSampling(in_feats,
-                        args.n_hidden,
-                        n_classes,
-                        args.n_layers,
-                        F.relu,
-                        args.dropout)
-    if cuda:
-        model.cuda()
-    loss_fcn = nn.CrossEntropyLoss()
-    infer_model = GCNInfer(in_feats,
-                           args.n_hidden,
-                           n_classes,
-                           args.n_layers,
-                           F.relu)
-    if cuda:
-        infer_model.cuda()
-    # use optimizer
-    optimizer = torch.optim.Adam(model.parameters(),
-                                 lr=args.lr,
-                                 weight_decay=args.weight_decay)
-    # Create sampler receiver
-    sampler = dgl.contrib.sampling.SamplerReceiver(graph=g, addr=args.ip, num_sender=args.num_sampler)
-    # initialize graph
-    dur = []
-    for epoch in range(args.n_epochs):
-        for nf in sampler:
-            nf.copy_from_parent()
-            model.train()
-            # forward
-            pred = model(nf)
-            batch_nids = nf.layer_parent_nid(-1).to(device=pred.device, dtype=torch.long)
-            batch_labels = labels[batch_nids]
-            loss = loss_fcn(pred, batch_labels)
-            optimizer.zero_grad()
-            loss.backward()
-            optimizer.step()
-        for infer_param, param in zip(infer_model.parameters(), model.parameters()):
-            infer_param.data.copy_(param.data)
-        num_acc = 0.
-        for nf in dgl.contrib.sampling.NeighborSampler(g, args.test_batch_size,
-                                                       g.number_of_nodes(),
-                                                       neighbor_type='in',
-                                                       num_workers=32,
-                                                       num_hops=args.n_layers+1,
-                                                       seed_nodes=test_nid):
-            nf.copy_from_parent()
-            infer_model.eval()
-            with torch.no_grad():
-                pred = infer_model(nf)
-                batch_nids = nf.layer_parent_nid(-1).to(device=pred.device, dtype=torch.long)
-                batch_labels = labels[batch_nids]
-                num_acc += (pred.argmax(dim=1) == batch_labels).sum().cpu().item()
-        print("Test Accuracy {:.4f}". format(num_acc/n_test_samples))
-if __name__ == '__main__':
-    parser = argparse.ArgumentParser(description='GCN')
-    register_data_args(parser)
-    parser.add_argument("--dropout", type=float, default=0.5,
-            help="dropout probability")
-    parser.add_argument("--gpu", type=int, default=-1,
-            help="gpu")
-    parser.add_argument("--lr", type=float, default=3e-2,
-            help="learning rate")
-    parser.add_argument("--n-epochs", type=int, default=200,
-            help="number of training epochs")
-    parser.add_argument("--batch-size", type=int, default=1000,
-            help="batch size")
-    parser.add_argument("--test-batch-size", type=int, default=1000,
-            help="test batch size")
-    parser.add_argument("--num-neighbors", type=int, default=3,
-            help="number of neighbors to be sampled")
-    parser.add_argument("--n-hidden", type=int, default=16,
-            help="number of hidden gcn units")
-    parser.add_argument("--n-layers", type=int, default=1,
-            help="number of hidden gcn layers")
-    parser.add_argument("--self-loop", action='store_true',
-            help="graph self-loop (default=False)")
-    parser.add_argument("--weight-decay", type=float, default=5e-4,
-            help="Weight for L2 loss")
-    parser.add_argument("--ip", type=str, default='127.0.0.1:50051',
-            help="IP address")
-    parser.add_argument("--num-sampler", type=int, default=1,
-            help="number of sampler")
-    args = parser.parse_args()
-    print(args)
-    main(args)
--- a/examples/pytorch/_deprecated/sampling/dis_sampling/sampler.py
+++ b/examples/pytorch/_deprecated/sampling/dis_sampling/sampler.py
-import argparse, time, math
-import numpy as np
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-from functools import partial
-import dgl
-import dgl.function as fn
-from dgl import DGLGraph
-from dgl.data import register_data_args, load_data
-from dgl.contrib.sampling import SamplerPool
-class MySamplerPool(SamplerPool):
-    def worker(self, args):
-        number_hops = 1
-        if args.model == "gcn_ns":
-            number_hops = args.n_layers + 1
-        elif args.model == "gcn_cv":
-            number_hops = args.n_layers
-        else:
-            print("unknown model. Please choose from gcn_ns and gcn_cv")
-        # Start sender
-        namebook = { 0:args.ip }
-        sender = dgl.contrib.sampling.SamplerSender(namebook)
-        # load and preprocess dataset
-        data = load_data(args)
-        if args.self_loop and not args.dataset.startswith('reddit'):
-            data.graph.add_edges_from([(i,i) for i in range(len(data.graph))])
-        train_nid = np.nonzero(data.train_mask)[0].astype(np.int64)
-        test_nid = np.nonzero(data.test_mask)[0].astype(np.int64)
-        # create GCN model
-        g = DGLGraph(data.graph, readonly=True)
-        while True:
-            idx = 0
-            for nf in dgl.contrib.sampling.NeighborSampler(g, args.batch_size,
-                                                           args.num_neighbors,
-                                                           neighbor_type='in',
-                                                           shuffle=True,
-                                                           num_workers=32,
-                                                           num_hops=number_hops,
-                                                           seed_nodes=train_nid):
-                print("send train nodeflow: %d" % (idx))
-                sender.send(nf, 0)
-                idx += 1
-            sender.signal(0)
-def main(args):
-    pool = MySamplerPool()
-    pool.start(args.num_sampler, args)
-if __name__ == '__main__':
-    parser = argparse.ArgumentParser(description='GCN')
-    register_data_args(parser)
-    parser.add_argument("--model", type=str,
-                        help="select a model. Valid models: gcn_ns, gcn_cv")
-    parser.add_argument("--dropout", type=float, default=0.5,
-            help="dropout probability")
-    parser.add_argument("--gpu", type=int, default=-1,
-            help="gpu")
-    parser.add_argument("--lr", type=float, default=3e-2,
-            help="learning rate")
-    parser.add_argument("--n-epochs", type=int, default=200,
-            help="number of training epochs")
-    parser.add_argument("--batch-size", type=int, default=1000,
-            help="batch size")
-    parser.add_argument("--test-batch-size", type=int, default=1000,
-            help="test batch size")
-    parser.add_argument("--num-neighbors", type=int, default=3,
-            help="number of neighbors to be sampled")
-    parser.add_argument("--n-hidden", type=int, default=16,
-            help="number of hidden gcn units")
-    parser.add_argument("--n-layers", type=int, default=1,
-            help="number of hidden gcn layers")
-    parser.add_argument("--self-loop", action='store_true',
-            help="graph self-loop (default=False)")
-    parser.add_argument("--weight-decay", type=float, default=5e-4,
-            help="Weight for L2 loss")
-    parser.add_argument("--ip", type=str, default='127.0.0.1:50051',
-            help="IP address")
-    parser.add_argument("--num-sampler", type=int, default=1,
-            help="number of sampler")
-    args = parser.parse_args()
-    print(args)
-    main(args)
--- a/examples/pytorch/_deprecated/sampling/gcn_cv_sc.py
+++ b/examples/pytorch/_deprecated/sampling/gcn_cv_sc.py
-import argparse, time, math
-import numpy as np
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-import dgl
-import dgl.function as fn
-from dgl import DGLGraphStale
-from dgl.data import register_data_args, load_data
-class NodeUpdate(nn.Module):
-    def __init__(self, layer_id, in_feats, out_feats, dropout, activation=None, test=False, concat=False):
-        super(NodeUpdate, self).__init__()
-        self.layer_id = layer_id
-        self.linear = nn.Linear(in_feats, out_feats)
-        self.dropout = None
-        if dropout != 0:
-            self.dropout = nn.Dropout(p=dropout)
-        self.activation = activation
-        self.concat = concat
-        self.test = test
-    def forward(self, node):
-        h = node.data['h']
-        if self.test:
-            norm = node.data['norm']
-            h = h * norm
-        else:
-            agg_history_str = 'agg_h_{}'.format(self.layer_id-1)
-            agg_history = node.data[agg_history_str]
-            # control variate
-            h = h + agg_history
-            if self.dropout:
-                h = self.dropout(h)
-        h = self.linear(h)
-        if self.concat:
-            h = torch.cat((h, self.activation(h)), dim=1)
-        elif self.activation:
-            h = self.activation(h)
-        return {'activation': h}
-class GCNSampling(nn.Module):
-    def __init__(self,
-                 in_feats,
-                 n_hidden,
-                 n_classes,
-                 n_layers,
-                 activation,
-                 dropout):
-        super(GCNSampling, self).__init__()
-        self.n_layers = n_layers
-        self.dropout = None
-        if dropout != 0:
-            self.dropout = nn.Dropout(p=dropout)
-        self.activation = activation
-        # input layer
-        self.linear = nn.Linear(in_feats, n_hidden)
-        self.layers = nn.ModuleList()
-        # hidden layers
-        for i in range(1, n_layers):
-            skip_start = (i == n_layers-1)
-            self.layers.append(NodeUpdate(i, n_hidden, n_hidden, dropout, activation, concat=skip_start))
-        # output layer
-        self.layers.append(NodeUpdate(n_layers, 2*n_hidden, n_classes, dropout))
-    def forward(self, nf):
-        h = nf.layers[0].data['preprocess']
-        if self.dropout:
-            h = self.dropout(h)
-        h = self.linear(h)
-        skip_start = (0 == self.n_layers-1)
-        if skip_start:
-            h = torch.cat((h, self.activation(h)), dim=1)
-        else:
-            h = self.activation(h)
-        for i, layer in enumerate(self.layers):
-            new_history = h.clone().detach()
-            history_str = 'h_{}'.format(i)
-            history = nf.layers[i].data[history_str]
-            h = h - history
-            nf.layers[i].data['h'] = h
-            nf.block_compute(i,
-                             fn.copy_src(src='h', out='m'),
-                             lambda node : {'h': node.mailbox['m'].mean(dim=1)},
-                             layer)
-            h = nf.layers[i+1].data.pop('activation')
-            # update history
-            if i < nf.num_layers-1:
-                nf.layers[i].data[history_str] = new_history
-        return h
-class GCNInfer(nn.Module):
-    def __init__(self,
-                 in_feats,
-                 n_hidden,
-                 n_classes,
-                 n_layers,
-                 activation):
-        super(GCNInfer, self).__init__()
-        self.n_layers = n_layers
-        self.activation = activation
-        # input layer
-        self.linear = nn.Linear(in_feats, n_hidden)
-        self.layers = nn.ModuleList()
-        # hidden layers
-        for i in range(1, n_layers):
-            skip_start = (i == n_layers-1)
-            self.layers.append(NodeUpdate(i, n_hidden, n_hidden, 0, activation, True, concat=skip_start))
-        # output layer
-        self.layers.append(NodeUpdate(n_layers, 2*n_hidden, n_classes, 0, None, True))
-    def forward(self, nf):
-        h = nf.layers[0].data['preprocess']
-        h = self.linear(h)
-        skip_start = (0 == self.n_layers-1)
-        if skip_start:
-            h = torch.cat((h, self.activation(h)), dim=1)
-        else:
-            h = self.activation(h)
-        for i, layer in enumerate(self.layers):
-            nf.layers[i].data['h'] = h
-            nf.block_compute(i,
-                             fn.copy_src(src='h', out='m'),
-                             fn.sum(msg='m', out='h'),
-                             layer)
-            h = nf.layers[i+1].data.pop('activation')
-        return h
-def main(args):
-    # load and preprocess dataset
-    data = load_data(args)
-    if args.self_loop and not args.dataset.startswith('reddit'):
-        data.graph.add_edges_from([(i,i) for i in range(len(data.graph))])
-    train_nid = np.nonzero(data.train_mask)[0].astype(np.int64)
-    test_nid = np.nonzero(data.test_mask)[0].astype(np.int64)
-    features = torch.FloatTensor(data.features)
-    labels = torch.LongTensor(data.labels)
-    if hasattr(torch, 'BoolTensor'):
-        train_mask = torch.BoolTensor(data.train_mask)
-        val_mask = torch.BoolTensor(data.val_mask)
-        test_mask = torch.BoolTensor(data.test_mask)
-    else:
-        train_mask = torch.ByteTensor(data.train_mask)
-        val_mask = torch.ByteTensor(data.val_mask)
-        test_mask = torch.ByteTensor(data.test_mask)
-    in_feats = features.shape[1]
-    n_classes = data.num_labels
-    n_edges = data.graph.number_of_edges()
-    n_train_samples = train_mask.int().sum().item()
-    n_val_samples = val_mask.int().sum().item()
-    n_test_samples = test_mask.int().sum().item()
-    print("""----Data statistics------'
-      #Edges %d
-      #Classes %d
-      #Train samples %d
-      #Val samples %d
-      #Test samples %d""" %
-          (n_edges, n_classes,
-              n_train_samples,
-              n_val_samples,
-              n_test_samples))
-    # create GCN model
-    g = DGLGraphStale(data.graph, readonly=True)
-    norm = 1. / g.in_degrees().float().unsqueeze(1)
-    if args.gpu < 0:
-        cuda = False
-    else:
-        cuda = True
-        torch.cuda.set_device(args.gpu)
-        features = features.cuda()
-        labels = labels.cuda()
-        train_mask = train_mask.cuda()
-        val_mask = val_mask.cuda()
-        test_mask = test_mask.cuda()
-        norm = norm.cuda()
-    g.ndata['features'] = features
-    num_neighbors = args.num_neighbors
-    n_layers = args.n_layers
-    g.ndata['norm'] = norm
-    g.update_all(fn.copy_src(src='features', out='m'),
-                 fn.sum(msg='m', out='preprocess'),
-                 lambda node : {'preprocess': node.data['preprocess'] * node.data['norm']})
-    for i in range(n_layers):
-        g.ndata['h_{}'.format(i)] = torch.zeros(features.shape[0], args.n_hidden).to(device=features.device)
-    g.ndata['h_{}'.format(n_layers-1)] = torch.zeros(features.shape[0], 2*args.n_hidden).to(device=features.device)
-    model = GCNSampling(in_feats,
-                        args.n_hidden,
-                        n_classes,
-                        n_layers,
-                        F.relu,
-                        args.dropout)
-    loss_fcn = nn.CrossEntropyLoss()
-    infer_model = GCNInfer(in_feats,
-                           args.n_hidden,
-                           n_classes,
-                           n_layers,
-                           F.relu)
-    if cuda:
-        model.cuda()
-        infer_model.cuda()
-    # use optimizer
-    optimizer = torch.optim.Adam(model.parameters(),
-                                 lr=args.lr,
-                                 weight_decay=args.weight_decay)
-    for epoch in range(args.n_epochs):
-        for nf in dgl.contrib.sampling.NeighborSampler(g, args.batch_size,
-                                                       num_neighbors,
-                                                       neighbor_type='in',
-                                                       shuffle=True,
-                                                       num_workers=32,
-                                                       num_hops=n_layers,
-                                                       seed_nodes=train_nid):
-            for i in range(n_layers):
-                agg_history_str = 'agg_h_{}'.format(i)
-                g.pull(nf.layer_parent_nid(i+1).long(), fn.copy_src(src='h_{}'.format(i), out='m'),
-                       fn.sum(msg='m', out=agg_history_str),
-                       lambda node : {agg_history_str: node.data[agg_history_str] * node.data['norm']})
-            node_embed_names = [['preprocess', 'h_0']]
-            for i in range(1, n_layers):
-                node_embed_names.append(['h_{}'.format(i), 'agg_h_{}'.format(i-1)])
-            node_embed_names.append(['agg_h_{}'.format(n_layers-1)])
-            nf.copy_from_parent(node_embed_names=node_embed_names)
-            model.train()
-            # forward
-            pred = model(nf)
-            batch_nids = nf.layer_parent_nid(-1).to(device=pred.device).long()
-            batch_labels = labels[batch_nids]
-            loss = loss_fcn(pred, batch_labels)
-            optimizer.zero_grad()
-            loss.backward()
-            optimizer.step()
-            node_embed_names = [['h_{}'.format(i)] for i in range(n_layers)]
-            node_embed_names.append([])
-            nf.copy_to_parent(node_embed_names=node_embed_names)
-        for infer_param, param in zip(infer_model.parameters(), model.parameters()):
-            infer_param.data.copy_(param.data)
-        num_acc = 0.
-        for nf in dgl.contrib.sampling.NeighborSampler(g, args.test_batch_size,
-                                                       g.number_of_nodes(),
-                                                       neighbor_type='in',
-                                                       num_workers=32,
-                                                       num_hops=n_layers,
-                                                       seed_nodes=test_nid):
-            node_embed_names = [['preprocess']]
-            for i in range(n_layers):
-                node_embed_names.append(['norm'])
-            nf.copy_from_parent(node_embed_names=node_embed_names)
-            infer_model.eval()
-            with torch.no_grad():
-                pred = infer_model(nf)
-                batch_nids = nf.layer_parent_nid(-1).to(device=pred.device).long()
-                batch_labels = labels[batch_nids]
-                num_acc += (pred.argmax(dim=1) == batch_labels).sum().cpu().item()
-        print("Test Accuracy {:.4f}". format(num_acc/n_test_samples))
-if __name__ == '__main__':
-    parser = argparse.ArgumentParser(description='GCN')
-    register_data_args(parser)
-    parser.add_argument("--dropout", type=float, default=0.5,
-            help="dropout probability")
-    parser.add_argument("--gpu", type=int, default=-1,
-            help="gpu")
-    parser.add_argument("--lr", type=float, default=3e-2,
-            help="learning rate")
-    parser.add_argument("--n-epochs", type=int, default=200,
-            help="number of training epochs")
-    parser.add_argument("--batch-size", type=int, default=1000,
-            help="train batch size")
-    parser.add_argument("--test-batch-size", type=int, default=1000,
-            help="test batch size")
-    parser.add_argument("--num-neighbors", type=int, default=2,
-            help="number of neighbors to be sampled")
-    parser.add_argument("--n-hidden", type=int, default=16,
-            help="number of hidden gcn units")
-    parser.add_argument("--n-layers", type=int, default=1,
-            help="number of hidden gcn layers")
-    parser.add_argument("--self-loop", action='store_true',
-            help="graph self-loop (default=False)")
-    parser.add_argument("--weight-decay", type=float, default=5e-4,
-            help="Weight for L2 loss")
-    args = parser.parse_args()
-    print(args)
-    main(args)
--- a/examples/pytorch/_deprecated/sampling/gcn_ns_sc.py
+++ b/examples/pytorch/_deprecated/sampling/gcn_ns_sc.py
-import argparse, time, math
-import numpy as np
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-from functools import partial
-import dgl
-import dgl.function as fn
-from dgl import DGLGraphStale
-from dgl.data import register_data_args, load_data
-class NodeUpdate(nn.Module):
-    def __init__(self, in_feats, out_feats, activation=None, test=False, concat=False):
-        super(NodeUpdate, self).__init__()
-        self.linear = nn.Linear(in_feats, out_feats)
-        self.activation = activation
-        self.concat = concat
-        self.test = test
-    def forward(self, node):
-        h = node.data['h']
-        if self.test:
-            h = h * node.data['norm']
-        h = self.linear(h)
-        # skip connection
-        if self.concat:
-            h = torch.cat((h, self.activation(h)), dim=1)
-        elif self.activation:
-            h = self.activation(h)
-        return {'activation': h}
-class GCNSampling(nn.Module):
-    def __init__(self,
-                 in_feats,
-                 n_hidden,
-                 n_classes,
-                 n_layers,
-                 activation,
-                 dropout):
-        super(GCNSampling, self).__init__()
-        self.n_layers = n_layers
-        if dropout != 0:
-            self.dropout = nn.Dropout(p=dropout)
-        else:
-            self.dropout = None
-        self.layers = nn.ModuleList()
-        # input layer
-        skip_start = (0 == n_layers-1)
-        self.layers.append(NodeUpdate(in_feats, n_hidden, activation, concat=skip_start))
-        # hidden layers
-        for i in range(1, n_layers):
-            skip_start = (i == n_layers-1)
-            self.layers.append(NodeUpdate(n_hidden, n_hidden, activation, concat=skip_start))
-        # output layer
-        self.layers.append(NodeUpdate(2*n_hidden, n_classes))
-    def forward(self, nf):
-        nf.layers[0].data['activation'] = nf.layers[0].data['features']
-        for i, layer in enumerate(self.layers):
-            h = nf.layers[i].data.pop('activation')
-            if self.dropout:
-                h = self.dropout(h)
-            nf.layers[i].data['h'] = h
-            nf.block_compute(i,
-                             fn.copy_src(src='h', out='m'),
-                             lambda node : {'h': node.mailbox['m'].mean(dim=1)},
-                             layer)
-        h = nf.layers[-1].data.pop('activation')
-        return h
-class GCNInfer(nn.Module):
-    def __init__(self,
-                 in_feats,
-                 n_hidden,
-                 n_classes,
-                 n_layers,
-                 activation):
-        super(GCNInfer, self).__init__()
-        self.n_layers = n_layers
-        self.layers = nn.ModuleList()
-        # input layer
-        skip_start = (0 == n_layers-1)
-        self.layers.append(NodeUpdate(in_feats, n_hidden, activation, test=True, concat=skip_start))
-        # hidden layers
-        for i in range(1, n_layers):
-            skip_start = (i == n_layers-1)
-            self.layers.append(NodeUpdate(n_hidden, n_hidden, activation, test=True, concat=skip_start))
-        # output layer
-        self.layers.append(NodeUpdate(2*n_hidden, n_classes, test=True))
-    def forward(self, nf):
-        nf.layers[0].data['activation'] = nf.layers[0].data['features']
-        for i, layer in enumerate(self.layers):
-            h = nf.layers[i].data.pop('activation')
-            nf.layers[i].data['h'] = h
-            nf.block_compute(i,
-                             fn.copy_src(src='h', out='m'),
-                             fn.sum(msg='m', out='h'),
-                             layer)
-        h = nf.layers[-1].data.pop('activation')
-        return h
-def main(args):
-    # load and preprocess dataset
-    data = load_data(args)
-    if args.self_loop and not args.dataset.startswith('reddit'):
-        data.graph.add_edges_from([(i,i) for i in range(len(data.graph))])
-    train_nid = np.nonzero(data.train_mask)[0].astype(np.int64)
-    test_nid = np.nonzero(data.test_mask)[0].astype(np.int64)
-    features = torch.FloatTensor(data.features)
-    labels = torch.LongTensor(data.labels)
-    if hasattr(torch, 'BoolTensor'):
-        train_mask = torch.BoolTensor(data.train_mask)
-        val_mask = torch.BoolTensor(data.val_mask)
-        test_mask = torch.BoolTensor(data.test_mask)
-    else:
-        train_mask = torch.ByteTensor(data.train_mask)
-        val_mask = torch.ByteTensor(data.val_mask)
-        test_mask = torch.ByteTensor(data.test_mask)
-    in_feats = features.shape[1]
-    n_classes = data.num_labels
-    n_edges = data.graph.number_of_edges()
-    n_train_samples = train_mask.int().sum().item()
-    n_val_samples = val_mask.int().sum().item()
-    n_test_samples = test_mask.int().sum().item()
-    print("""----Data statistics------'
-      #Edges %d
-      #Classes %d
-      #Train samples %d
-      #Val samples %d
-      #Test samples %d""" %
-          (n_edges, n_classes,
-              n_train_samples,
-              n_val_samples,
-              n_test_samples))
-    # create GCN model
-    g = DGLGraphStale(data.graph, readonly=True)
-    norm = 1. / g.in_degrees().float().unsqueeze(1)
-    if args.gpu < 0:
-        cuda = False
-    else:
-        cuda = True
-        torch.cuda.set_device(args.gpu)
-        features = features.cuda()
-        labels = labels.cuda()
-        train_mask = train_mask.cuda()
-        val_mask = val_mask.cuda()
-        test_mask = test_mask.cuda()
-        norm = norm.cuda()
-    g.ndata['features'] = features
-    num_neighbors = args.num_neighbors
-    g.ndata['norm'] = norm
-    model = GCNSampling(in_feats,
-                        args.n_hidden,
-                        n_classes,
-                        args.n_layers,
-                        F.relu,
-                        args.dropout)
-    if cuda:
-        model.cuda()
-    loss_fcn = nn.CrossEntropyLoss()
-    infer_model = GCNInfer(in_feats,
-                           args.n_hidden,
-                           n_classes,
-                           args.n_layers,
-                           F.relu)
-    if cuda:
-        infer_model.cuda()
-    # use optimizer
-    optimizer = torch.optim.Adam(model.parameters(),
-                                 lr=args.lr,
-                                 weight_decay=args.weight_decay)
-    # initialize graph
-    dur = []
-    for epoch in range(args.n_epochs):
-        for nf in dgl.contrib.sampling.NeighborSampler(g, args.batch_size,
-                                                       args.num_neighbors,
-                                                       neighbor_type='in',
-                                                       shuffle=True,
-                                                       num_workers=32,
-                                                       num_hops=args.n_layers+1,
-                                                       seed_nodes=train_nid):
-            nf.copy_from_parent()
-            model.train()
-            # forward
-            pred = model(nf)
-            batch_nids = nf.layer_parent_nid(-1).to(device=pred.device, dtype=torch.long)
-            batch_labels = labels[batch_nids]
-            loss = loss_fcn(pred, batch_labels)
-            optimizer.zero_grad()
-            loss.backward()
-            optimizer.step()
-        for infer_param, param in zip(infer_model.parameters(), model.parameters()):
-            infer_param.data.copy_(param.data)
-        num_acc = 0.
-        for nf in dgl.contrib.sampling.NeighborSampler(g, args.test_batch_size,
-                                                       g.number_of_nodes(),
-                                                       neighbor_type='in',
-                                                       num_workers=32,
-                                                       num_hops=args.n_layers+1,
-                                                       seed_nodes=test_nid):
-            nf.copy_from_parent()
-            infer_model.eval()
-            with torch.no_grad():
-                pred = infer_model(nf)
-                batch_nids = nf.layer_parent_nid(-1).to(device=pred.device, dtype=torch.long)
-                batch_labels = labels[batch_nids]
-                num_acc += (pred.argmax(dim=1) == batch_labels).sum().cpu().item()
-        print("Test Accuracy {:.4f}". format(num_acc/n_test_samples))
-if __name__ == '__main__':
-    parser = argparse.ArgumentParser(description='GCN neighbor sampling')
-    register_data_args(parser)
-    parser.add_argument("--dropout", type=float, default=0.5,
-            help="dropout probability")
-    parser.add_argument("--gpu", type=int, default=-1,
-            help="gpu")
-    parser.add_argument("--lr", type=float, default=3e-2,
-            help="learning rate")
-    parser.add_argument("--n-epochs", type=int, default=200,
-            help="number of training epochs")
-    parser.add_argument("--batch-size", type=int, default=1000,
-            help="batch size")
-    parser.add_argument("--test-batch-size", type=int, default=1000,
-            help="test batch size")
-    parser.add_argument("--num-neighbors", type=int, default=3,
-            help="number of neighbors to be sampled")
-    parser.add_argument("--n-hidden", type=int, default=16,
-            help="number of hidden gcn units")
-    parser.add_argument("--n-layers", type=int, default=1,
-            help="number of hidden gcn layers")
-    parser.add_argument("--self-loop", action='store_true',
-            help="graph self-loop (default=False)")
-    parser.add_argument("--weight-decay", type=float, default=5e-4,
-            help="Weight for L2 loss")
-    args = parser.parse_args()
-    print(args)
-    main(args)