- 12 May, 2022 1 commit
-
-
John Kamalu authored
tools/merge_datasets.py - tool to merge multiple dataset files into a single dataset - testing conducted and included in the megatron-testing repo https://gitlab-master.nvidia.com/ADLR/megatron-testing tools/preprocess_data.py - magic numbers changed to required command line arguments megatron/data/indexed_dataset.py - when merging, fix to properly update document index - testing conducted and included in the megatron-testing repo (see above) - fix follows this history https://github.com/bigscience-workshop/Megatron-DeepSpeed/pull/66
-
- 10 Dec, 2021 1 commit
-
-
zihanl authored
-
- 22 Nov, 2021 2 commits
- 21 Nov, 2021 1 commit
-
-
zihanl authored
-
- 19 Oct, 2021 1 commit
-
-
rprenger authored
-
- 15 Oct, 2021 1 commit
-
-
mshoeybi authored
-
- 13 Oct, 2021 1 commit
-
-
mshoeybi authored
-
- 11 Oct, 2021 1 commit
-
-
mshoeybi authored
-
- 08 Oct, 2021 1 commit
-
-
Jared Casper authored
-
- 20 Sep, 2021 1 commit
-
-
Mohammad Shoeybi authored
-
- 14 Sep, 2021 1 commit
-
-
rprenger authored
-
- 08 Sep, 2021 1 commit
-
-
rprenger authored
-
- 29 Aug, 2021 1 commit
-
-
zihanl authored
-
- 27 Aug, 2021 1 commit
-
-
Ryan Prenger authored
-
- 26 Aug, 2021 1 commit
-
-
rprenger authored
-
- 19 Jul, 2021 1 commit
-
-
rprenger authored
-
- 12 Jul, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 09 Jul, 2021 1 commit
-
-
rprenger authored
Refactoring code so server code is more independent of sampling and adding a CLI. CLI still has URL of server hard-coded
-
- 30 Jun, 2021 2 commits
- 23 Jun, 2021 1 commit
-
-
rprenger authored
-
- 19 May, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 14 May, 2021 1 commit
-
-
Jared Casper authored
-
- 21 Apr, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 20 Apr, 2021 2 commits
-
-
Mostofa Patwary authored
-
Mostofa Patwary authored
-
- 02 Apr, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 01 Apr, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 30 Mar, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 26 Mar, 2021 1 commit
-
-
mpatwary authored
-
- 24 Mar, 2021 2 commits
-
-
Jared Casper authored
-
Jared Casper authored
-
- 19 Mar, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 08 Mar, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 05 Mar, 2021 3 commits
-
-
Mostofa Patwary authored
-
Mostofa Patwary authored
-
Mostofa Patwary authored
-
- 02 Mar, 2021 1 commit
-
-
Mostofa Patwary authored
-
- 26 Feb, 2021 1 commit
-
-
Mostofa Patwary authored
-