Commits · 8a13fabe565e915747116840a4cb33ac56e527ad · OpenDAS / Megatron-LM

12 Dec, 2024 3 commits
- Merge branch 'main' into 'main' · 8a13fabe
  wxj authored Dec 12, 2024
```
添加优化项, 添加qwen和llama3

See merge request !5
```
  8a13fabe
- Update Llama_pretraining.sh · 425a2473
  wxj authored Dec 12, 2024
  
  425a2473
- 合并优化项 · 4923c381
  wxj authored Dec 12, 2024
  
  4923c381
09 Dec, 2024 7 commits
- Merge branch 'main' into 'main' · f5ca0d94
  wxj authored Dec 09, 2024
```
Main

See merge request !4
```
  f5ca0d94
- 更新llama预训练脚本 · bc09f8ef
  wxj authored Dec 09, 2024
  
  bc09f8ef
- 更新readme和预训练脚本 · 5bd8b291
  wxj authored Dec 09, 2024
  
  5bd8b291
- Merge branch 'main' into 'main' · d520d24f
  silencealiang authored Dec 09, 2024
```
megatron升级v0.10

See merge request !3
```
  d520d24f
- add · 481609bb
  xingjinliang authored Dec 09, 2024
  
  481609bb
- Update README.md · c0a53a34
  silencealiang authored Dec 09, 2024
  
  c0a53a34
- 升级megatron_v0.10[B · 3bec6514
  xingjinliang authored Dec 09, 2024
  
  3bec6514
29 Apr, 2024 2 commits
- Merge branch 'megatron-lm_dtk24.04' into 'main' · 3aca1415
  liangjing authored Apr 29, 2024
```
Megatron lm dtk24.04

See merge request !1
```
  3aca1415
- update · 1005e9d3
  liangjj authored Apr 29, 2024
  
  1005e9d3
27 Mar, 2024 1 commit
- v1 · 051f58f1
  liangjing authored Mar 27, 2024
  
  051f58f1
26 Apr, 2023 2 commits
- Merge branch 'main' of https://github.com/NVIDIA/Megatron-LM · 0024a5c6
  zhuwenwen authored Apr 26, 2023
  
  0024a5c6
- update main · b004456b
  zhuwenwen authored Apr 26, 2023
  
  b004456b
19 Apr, 2023 8 commits
- Merge branch 'enable_autocast' into 'main' · 3db2063b
  Jared Casper authored Apr 19, 2023
```
Add ability to enable autocast through schedules

See merge request ADLR/megatron-lm!569
```
  3db2063b
- address review comments · 98bb40d9
  Abhinav Khattar authored Apr 19, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  98bb40d9
- keep previous autocast check · 630811b2
  Abhinav Khattar authored Apr 19, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  630811b2
- add enable autocast · 513d7d8e
  Abhinav Khattar authored Apr 19, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  513d7d8e
- Merge branch 'jbarker/apex_layernorm' into 'main' · 2699f93e
  Jared Casper authored Apr 19, 2023
```
replace custom layer_norm_cuda with Apex layer_norm_cuda

See merge request ADLR/megatron-lm!566
```
  2699f93e
- replace custom layer_norm_cuda with Apex layer_norm_cuda · d650e6a2
  Jon Barker authored Apr 19, 2023
  
  d650e6a2
- Merge branch 'checkpoint_util_fix' into 'main' · 378db1cd
  Jared Casper authored Apr 19, 2023
```
Some quick fixes to checkpoint_util.

See merge request ADLR/megatron-lm!568
```
  378db1cd
- Some quick fixes to checkpoint_util. · be1a575e
  Jared Casper authored Apr 19, 2023
  
  be1a575e
14 Apr, 2023 1 commit
- Merge branch 'test-update' into 'main' · 8dbd0757
  Jared Casper authored Apr 13, 2023
```
Update tests with new gold numbers

See merge request ADLR/megatron-lm!565
```
  8dbd0757
13 Apr, 2023 1 commit
- Recent commit changed the bias dropout add jitted function which slightly... · 08e62070
  Jared Casper authored Apr 13, 2023
```
Recent commit changed the bias dropout add jitted function which slightly changes numbers, manually tested to be accurate.
```
  08e62070
09 Apr, 2023 1 commit
- Merge branch 'add_helper_fns' into 'main' · ee084835
  Jared Casper authored Apr 08, 2023
```
Add parallel_state helper functions

See merge request ADLR/megatron-lm!564
```
  ee084835
07 Apr, 2023 4 commits
- add needed helper functions · 3ce6a1c2
  Abhinav Khattar authored Apr 07, 2023
```
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
```
  3ce6a1c2
- Merge branch 'nextlm-fixes' into 'main' · 9c5e287d
  Jared Casper authored Apr 06, 2023
```
Some quick fixes.

See merge request ADLR/megatron-lm!563
```
  9c5e287d
- Some quick fixes. · 45084d46
  Jared Casper authored Apr 06, 2023
  
  45084d46
- Merge branch 'flash_attention_inference' into 'main' · cdd2afdf
  Jared Casper authored Apr 06, 2023
```
Flash Attention inference fix

See merge request ADLR/megatron-lm!562
```
  cdd2afdf
06 Apr, 2023 10 commits
- dropout variable · a7cce320
  Jimmy Zhang authored Apr 06, 2023
  
  a7cce320
- refactor flash attention · 254e8815
  Jimmy Zhang authored Apr 06, 2023
  
  254e8815
- Flash Attention inference fix · f1a50a3c
  Jimmy Zhang authored Mar 03, 2023
  
  f1a50a3c
- Merge branch 'nextlm-merge-sentencepiece' into 'main' · 717c5274
  Jared Casper authored Apr 06, 2023
```
Add support for GPTSentencePieceTokenizer and related fixes.

See merge request ADLR/megatron-lm!561
```
  717c5274
- Merge branch 'untie_embeddings' into 'main' · 018391a6
  Jared Casper authored Apr 06, 2023
```
Untie Embeddings

See merge request ADLR/megatron-lm!558
```
  018391a6
- Missed some changes from next-best-lm branch. · 3207c19a
  Jared Casper authored Apr 06, 2023
  
  3207c19a
- Add support for GPTSentencePieceTokenizer and related fixes. · 46ffb75c
  Jared Casper authored Apr 06, 2023
  
  46ffb75c
- Merge branch 'next-best-lm/merge-layernorm-1p-main' into 'main' · 4e891fe9
  Jared Casper authored Apr 05, 2023
```
layernorm1p added

See merge request ADLR/megatron-lm!557
```
  4e891fe9
- Merge branch 'next-best-lm/merge-rope-main' into 'main' · 7bd25e26
  Jared Casper authored Apr 05, 2023
```
merging rope to main

See merge request ADLR/megatron-lm!556
```
  7bd25e26
- Add assert in BertModel that untie_embeddings_and_output_weights is False · 1e2a0405
  Jared Casper authored Apr 05, 2023
  
  1e2a0405