GitLab

* The start of experiments of Jiahang Li on GraphSAINT.

* a nightly build

* a nightly build

Check the basic pipeline of codes. Next to check the details of samplers , GCN layer (forward propagation) and loss (backward propagation)

* a night build

* Implement GraphSAINT with torch.dataloader

There're still some bugs with sampling in training procedure

* Test validity

Succeed in testing validity on ppi_node experiments without testing other setup.
1. Online sampling on ppi_node experiments performs perfectly.
2. Sampling speed is a bit slow because the operations on [dgl.subgraphs], next step is to improve this part by putting the conversion into parallelism
3. Figuring out why offline+online sampling method performs bad, which does not make sense
4. Doing experiments on other setup

* Implement saint with torch.dataloader

Use torch.dataloader to speed up saint sampling with experiments. Except experiments on too large dataset Amazon, we've done some experiments on other four datasets including ppi, flickr, reddit and yelp. Preliminary experimental results show consumed time and metrics reach not bad level. Next step is to employ more accurate profiler which is the line_profiler to test consumed period, and adjust num_workers to speed up sampling procedures on same certain datasets faster.

* a nightly build

* Update .gitignore

* reorganize codes

Reorganize some codes and comments.

* a nightly build

* Update .gitignore

* fix bugs

Fix bugs about why fully offline sampling and author's version don't work

* reorganize files and codes

Reorganize files and codes then do some experiments to test the performance of offline sampling and online sampling

* do some experiments and update README

* a nightly build

* a nightly build

* Update README.md

* delete unnecessary files

* Update README.md

* a nightly update

1. handle directory named 'graphsaintdata'
2. control graph shift between gpu and cpu related to large dataset ('amazon')
3. remove parameter 'train'
4. refine annotations of the sampler
5. update README.md including updating dataset info, dependencies info, etc

* a nightly update

explain config differences in TEST part
remove a sampling time variant
make 'online' an argument
change 'norm' to 'sampler'
explain parameters in README.md

* Update README.md

* a nightly build

* make online an argument
* refine README.md
* refine codes of `collate_fn` in sampler.py, in training phase only return one subgraph, no need to check if the number of subgraphs larger than 1

* Update sampler.py

check the problem on flickr is about overfitting.

* a nightly update

Fix the overfitting problem of `flickr` dataset. We need to restrict the number of subgraphs (also the number of iterations) used in each epoch of training phase. Or it might overfit when validating at the end of each epoch. The method to limit the number is a formula specified by the author.

* Set up a new flag `full` specifying if the number of subgraphs used in training phase equals to that of pre-sampled subgraphs

* Modify codes and annotations related the new flag

* Add a new parameter called `node_budget` in the base class `SAINTSampler` to compute the specific formula

* set `gpu` as a command line argument

* Update README.md

* Finish the experiments on Flickr, which is done after adding new flag `full`

* a nightly update

* use half of edges in the original graph to do sampling
* test dgl.random.choice with or without replacement with half of edges
~ next is to test what if put the calculating probability part out of __getitem__ can speed up sampling and try to implement sampling method of author

* employ cython to implement edge sampling for per edge

* employ cython to implement edge sampling for per edge
* doing experiments to test consumed time and performance
** the consumed time decreased to approximately 480s, the performance decrease about 5 points.
* deprecate cython implementation

* Revert "employ cython to implement edge sampling for per edge"

* This reverts commit 4ba4f092
* Deprecate cython implementation
* Reserve half-edges mechanism

* a nightly update

* delete unnecessary annotations
Co-authored-by: Mufei Li <mufeili1996@gmail.com>

* PPIDataset

* Revert "PPIDataset"

This reverts commit 264bd0c960cfa698a7bb946dad132bf52c2d0c8a.

* update data rst

* update data doc and docstring

* API doc rst for dataset

* docstring

* update api doc

* add url format

* update docstring

* update citation graph

* update knowledge graph

* update gc datasets

* fix index

* Rst fix (#3)

* Fix syntax

* syntax

* update docstring

* update doc (#4)

* final update

* fix rdflib

* fix rdf
Co-authored-by: HuXiangkun <huxk_hit@qq.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-51-214.ec2.internal>
Co-authored-by: xiang song(charlie.song) <classicxsong@gmail.com>

* Merge

* [Graph][CUDA] Graph on GPU and many refactoring (#1791)

* change edge_ids behavior and C++ impl

* fix unittests; remove utils.Index in edge_id

* pass mx and th tests

* pass tf test

* add aten::Scatter_

* Add nonzero; impl CSRGetDataAndIndices/CSRSliceMatrix

* CSRGetData and CSRGetDataAndIndices passed tests

* CSRSliceMatrix basic tests

* fix bug in empty slice

* CUDA CSRHasDuplicate

* has_node; has_edge_between

* predecessors, successors

* deprecate send/recv; fix send_and_recv

* deprecate send/recv; fix send_and_recv

* in_edges; out_edges; all_edges; apply_edges

* in deg/out deg

* subgraph/edge_subgraph

* adj

* in_subgraph/out_subgraph

* sample neighbors

* set/get_n/e_repr

* wip: working on refactoring all idtypes

* pass ndata/edata tests on gpu

* fix

* stash

* workaround nonzero issue

* stash

* nx conversion

* test_hetero_basics except update rou...

* fix

* fix

* fix

* fix

* fix

* 111

* fix

* fix

* fix

* test

* ff

* fix

* ff

* fix

* f

* merge

* fix

* add working scripts

* add frcnn training script

* remove redundent files

* refactor validation computation, will optimize sgdet and training

* validation finally finished

* f-rcnn training

* test reldn

* rm file

* update reldn training

* data preprocess to h5

* temp

* use coco json

* fix conflict

* new obj dataset for detection

* update training

* before cleanup

* remove abundant files

* add arg parse to train

* cleanup code file

* update

* fix

* add readme

* add ipynb as demo

* add demo pic

* update readme

* add demo script

* improve paths

* improve readme

* add docstrings

* fix args description

* update readme

* add models from s3

* update README
Co-authored-by: Minjie Wang <minjie.wang@nyu.edu>

* add gin model

* convert dataset.py to data_ont_the_fly way and put it into dgl.data module

* convert dataset.py to data_ont_the_fly way and put it into dgl.data module
python code checked

* modified document and reference TUDataset; checked python part and bypass cpp part due to error

* change tensor to numpy in dataset and transform in collate@Dataloader

* Change minor format issue

Change minor format issue

* moved logging; adjusted tqdm etc

* fixing builtin src*edge shape mismatch

* bundled function refactor (?)

* fixing names

* readout prototype

* oops

* more fixes

* removing readout prototype

* sum_on() with SPMV, fixing batching with 0 edges

* readouts with segmented sum

* typo (??????)

* fixes NLTK dependency (#125)

* misc fixes including #126 (pushing again)

* sanity check for mxnet

* fixes NLTK dependency (#125) and #126

* reverting to sum_nodes/edges

* model code for generative graphs

* batched version for dynamic graph generation using padding

* renaming function train back to forward

* remove old util function for padding DGMG

* override networkx clear to reset state, add dgl.nn

* Dynamic graph without batching

* use relative import path

* load dataset, pad batch

* bug fix

* experimental batch and unbatch

* dgmg batched version

* minor tweak

* move preprocessing padding into data loading

* batch graph test code

* minor

* batched graph class and test cases

* make dgl.nn.gcn a simple layer plus minor fix

* update dgmg model

* test forward using attribute field

* use frame append, minor changes

* moving networkx operations out of forward

* revert some changes

* remove structural immutability check

* fix edge list order problem in cached graph.

* minor fix

* fix bug in edge iter

* SPMV works

* gcn spmv on CPU

* change gcn style

* fix cached graph performance; fixed gcn dataset bug

* reorg dir

* non-batch spmv; partial update problem with shape change

* fix reorder problem; finish gcn-batch impl

* pop API

* GPU context

Initial example file for ginipath