Commits · 628bf0dd17975af779e86dd437a8dca04594aa9b · wuxk1 / Megatron-LM

"pretrain_bert_ict.py" did not exist on "a1d04b7939c3cf3cd19f93a0043e81750d6fba38"

14 Jul, 2020 1 commit
- Use the new allgather implementation · 628bf0dd
  Neel Kant authored Jul 13, 2020
  
  628bf0dd
10 Jul, 2020 1 commit
- added allgather and allreduce · 98feae4e
  mohammad authored Jul 09, 2020
  
  98feae4e
07 Jul, 2020 2 commits
- Made topk accuracy reporting optional · de6640be
  Neel Kant authored Jul 07, 2020
  
  de6640be
- Addressed Jared's comments · 8d7f508a
  Neel Kant authored Jul 06, 2020
  
  8d7f508a
02 Jul, 2020 1 commit
- One sentence options · 03feecbc
  Neel Kant authored Jul 02, 2020
  
  03feecbc
29 Jun, 2020 1 commit
- Create tensors on cuda rather than copying · 76928caa
  Neel Kant authored Jun 29, 2020
  
  76928caa
26 Jun, 2020 2 commits
- Cosmetic changes · 2a3b445d
  Neel Kant authored Jun 26, 2020
  
  2a3b445d
- Merge branch 'master' into ict-merge · ac967fa0
  Neel Kant authored Jun 26, 2020
  
  ac967fa0
24 Jun, 2020 4 commits
- Update README · 7b3baaaa
  Neel Kant authored Jun 24, 2020
  
  7b3baaaa
- Update README · 47996737
  Neel Kant authored Jun 24, 2020
  
  47996737
- Fix accidental comment · 1f514406
  Neel Kant authored Jun 23, 2020
  
  1f514406
- More ict_merge changes and interactive testing · 3354081f
  Neel Kant authored Jun 23, 2020
  
  3354081f
22 Jun, 2020 1 commit
- Address most of comments from code review w/ Mohammad · 6495927e
  Neel Kant authored Jun 22, 2020
  
  6495927e
11 Jun, 2020 1 commit
- Merge branch 'mmap_data_samples' into 'master' · 46a536cc
  Mohammad Shoeybi authored Jun 10, 2020
```
mmap sample map

See merge request ADLR/megatron-lm!88
```
  46a536cc
10 Jun, 2020 3 commits
- gpt2 mmap sample map · dd76c914
  Raul Puri authored Jun 10, 2020
  
  dd76c914
- mmap sample map · 2b47c5e7
  Raul Puri authored Jun 10, 2020
  
  2b47c5e7
- Ran and cleaned up · fcc500d6
  Neel Kant authored Jun 10, 2020
  
  fcc500d6
05 Jun, 2020 8 commits
- Merge branch 'cross_entropy_mem_opt' into 'master' · cfe35fd3
  Jared Casper authored Jun 05, 2020
```
Option to perform language model cross entropy calculations in fp16

See merge request ADLR/megatron-lm!86
```
  cfe35fd3
- addressed jareds comments · 197c132e
  mohammad authored Jun 05, 2020
  
  197c132e
- added fp16 lm cross entropy to bert · 78022005
  mohammad authored Jun 05, 2020
  
  78022005
- added fp16 cross entropy loss option for gpt2 · 22e3c7e6
  mohammad authored Jun 05, 2020
  
  22e3c7e6
- added fp16 cross entropy loss option for gpt2 · acfe848e
  mohammad authored Jun 05, 2020
  
  acfe848e
- Some changes noticed late · c044f59a
  Neel Kant authored Jun 05, 2020
  
  c044f59a
- Prune changes to only be related to ICT · 32bb4edc
  Neel Kant authored Jun 05, 2020
  
  32bb4edc
- testing · 2ede8235
  mohammad authored Jun 05, 2020
  
  2ede8235
04 Jun, 2020 2 commits
- Merge branch 'master' into realm-mlm · 674814a5
  Neel Kant authored Jun 04, 2020
  
  674814a5
- Temporarily modify ict_eval_bm25 to build full wiki index · 8b355d93
  Neel Kant authored Jun 04, 2020
  
  8b355d93
28 May, 2020 1 commit
- Add bm25 evaluation code · 8409e1c1
  Neel Kant authored May 28, 2020
  
  8409e1c1
27 May, 2020 3 commits
- Correct indexer seq length · dfb907fe
  Neel Kant authored May 27, 2020
  
  dfb907fe
- Merge branch 'realm-mlm' of https://gitlab-master.nvidia.com/ADLR/megatron-lm into realm-mlm · ab5c4f92
  Neel Kant authored May 27, 2020
  
  ab5c4f92
- Correct CrossEntropyLoss · 0e8f4331
  Neel Kant authored May 27, 2020
  
  0e8f4331
26 May, 2020 1 commit
- Corrected realm example building, misc improvements for async concurrency · 2fd4ea6c
  Neel Kant authored May 25, 2020
  
  2fd4ea6c
24 May, 2020 3 commits
- Fix token alignment, add mpu checkpointing, misc training code · 8e22824e
  Neel Kant authored May 24, 2020
  
  8e22824e
- Use Faiss GPU index and report retrieval utility · 8573ab35
  Neel Kant authored May 24, 2020
  
  8573ab35
- Restructure Indexer classes · e59496bf
  Neel Kant authored May 24, 2020
  
  e59496bf
21 May, 2020 2 commits
- Fix autoresume by removing code from indexer · e0a1caba
  Neel Kant authored May 21, 2020
  
  e0a1caba
- Change sync variable to gloo backend · 05ea0cca
  Neel Kant authored May 21, 2020
  
  05ea0cca
20 May, 2020 1 commit
- Async works for total 8 GPU, indexer debug mode · a670b6c9
  Neel Kant authored May 19, 2020
  
  a670b6c9
19 May, 2020 2 commits
- Full cycle of communication complete. Also added BasicIndexBuilder · 5684f904
  Neel Kant authored May 19, 2020
  
  5684f904
- Reorganize indexer. Things run up to saving model checkpoint and repeating · d4b00be0
  Neel Kant authored May 19, 2020
  
  d4b00be0