README.md

## How to load ogb data
To load ogb dataset, you need to run the following command, which will output a network file, ogbl-collab-net.txt:
```
python3 load_dataset.py --name ogbl-collab
```
Or you can run the code directly with:
```
python3 deepwalk --ogbl_name xxx --load_from_ogbl
```
However, ogb.linkproppred might not be compatible with mixed training with multi-gpu. If you want to do mixed training, please use no more than 1 gpu by the command above.

## Evaluation
For evaluatation we follow the code mlp.py provided by ogb [here](https://github.com/snap-stanford/ogb/blob/master/examples/linkproppred/collab/mlp.py).

## Used config
ogbl-collab
```
python3 deepwalk.py --ogbl_name ogbl-collab --load_from_ogbl --save_in_pt --output_emb_file embedding.pt --num_walks 50 --window_size 20 --walk_length 40 --lr 0.1 --negative 1 --neg_weight 1 --lap_norm 0.005 --mix --adam --gpus 0 --num_threads 4 --print_interval 2000 --print_loss --batch_size 32
cd ./ogb/blob/master/examples/linkproppred/collab/
cp embedding_pt_file_path ./
python3 mlp.py --device 0 --runs 10 --use_node_embedding
```

ogbl-ddi
```
python3 deepwalk.py --ogbl_name ogbl-ddi --load_from_ogbl --save_in_pt --output_emb_file ddi-embedding.pt --num_walks 50 --window_size 2 --walk_length 80 --lr 0.1 --negative 1 --neg_weight 1 --lap_norm 0.05 --only_gpu --adam --gpus 0 --num_threads 4 --print_interval 2000 --print_loss --batch_size 16 --use_context_weight
cd ./ogb/blob/master/examples/linkproppred/ddi/
cp embedding_pt_file_path ./
python3 mlp.py --device 0 --runs 10 --epochs 100
```

ogbl-ppa
```
python3 deepwalk.py --ogbl_name ogbl-ppa --load_from_ogbl --save_in_pt --output_emb_file ppa-embedding.pt --negative 1 --neg_weight 1 --batch_size 64 --print_interval 2000 --print_loss --window_size 2 --num_walks 30 --walk_length 80 --lr 0.1 --lap_norm 0.02 --adam --mix --gpus 0 --use_context_weight --num_threads 4
cp embedding_pt_file_path ./
python3 mlp.py --device 2 --runs 10
```

ogbl-citation
```
python3 deepwalk.py --ogbl_name ogbl-citation --load_from_ogbl --save_in_pt --output_emb_file embedding.pt --window_size 2 --num_walks 10 --negative 1 --neg_weight 1 --walk_length 80 --batch_size 128 --print_loss --print_interval 1000 --mix --adam --gpus 0 --use_context_weight --num_threads 4 --lap_norm 0.05 --lr 0.1
cp embedding_pt_file_path ./
python3 mlp.py --device 2 --runs 10 --use_node_embedding
```

## Result
ogbl-collab
<br>#params: 61258346(model) + 131841(mlp) = 61390187
<br>Hits@10
<br>&emsp;Highest Train: 74.83 ± 4.79
<br>&emsp;Highest Valid: 40.03 ± 2.98
<br>&emsp;&emsp;Final Train: 74.51 ± 4.92
<br>&emsp;&emsp;Final Test: 31.13 ± 2.47
<br>Hits@50
<br>&emsp;Highest Train: 98.83 ± 0.15
<br>&emsp;Highest Valid: 60.61 ± 0.32
<br>&emsp;&emsp;Final Train: 98.74 ± 0.17
<br>&emsp;&emsp;Final Test: 50.37 ± 0.34
<br>Hits@100
<br>&emsp;Highest Train: 99.86 ± 0.04
<br>&emsp;Highest Valid: 66.64 ± 0.32
<br>&emsp;&emsp;Final Train: 99.84 ± 0.06
<br>&emsp;&emsp;Final Test: 56.88 ± 0.37

<br>obgl-ddi
<br>#params: 1444840(model) + 99073(mlp) = 1543913
<br>&emsp;Hits@10
<br>&emsp;Highest Train: 36.09 ± 2.47
<br>&emsp;Highest Valid: 32.83 ± 2.30
<br>&emsp;&emsp;Final Train: 36.06 ± 2.45
<br>&emsp;&emsp;Final Test: 11.76 ± 3.91
<br>&emsp;Hits@20
<br>&emsp;Highest Train: 45.59 ± 2.45
<br>&emsp;Highest Valid: 42.00 ± 2.36
<br>&emsp;&emsp;Final Train: 45.56 ± 2.50
<br>&emsp;&emsp;Final Test: 22.46 ± 2.90
<br>&emsp;Hits@30
<br>&emsp;Highest Train: 51.58 ± 2.41
<br>&emsp;Highest Valid: 47.82 ± 2.19
<br>&emsp;&emsp;Final Train: 51.58 ± 2.42
<br>&emsp;&emsp;Final Test: 30.17 ± 3.39


<br>ogbl-ppa
<br>#params: 150024820(model) + 113921(mlp) = 150138741
<br>Hits@10
<br>&emsp;Highest Train: 3.58 ± 0.90
<br>&emsp;Highest Valid: 2.88 ± 0.76
<br>&emsp;&emsp;Final Train: 3.58 ± 0.90
<br>&emsp;&emsp;Final Test: 1.45 ± 0.65
<br>&emsp;Hits@50
<br>&emsp;Highest Train: 18.21 ± 2.29
<br>&emsp;Highest Valid: 15.75 ± 2.10
<br>&emsp;&emsp;Final Train: 18.21 ± 2.29
<br>&emsp;&emsp;Final Test: 11.70 ± 0.97
<br>&emsp;Hits@100
<br>&emsp;Highest Train: 31.16 ± 2.23
<br>&emsp;Highest Valid: 27.52 ± 2.07
<br>&emsp;&emsp;Final Train: 31.16 ± 2.23
<br>&emsp;&emsp;Final Test: 23.02 ± 1.63

<br>ogbl-citation
<br>#params: 757811178(model) + 131841(mlp) = 757943019
<br>MRR
<br>&emsp;Highest Train: 0.8797 ± 0.0007
<br>&emsp;Highest Valid: 0.8139 ± 0.0005
<br>&emsp;&emsp;Final Train: 0.8792 ± 0.0008
<br>&emsp;&emsp;Final Test: 0.8148 ± 0.0004