Merge branch 'main' into 'main'

megatron升级v0.10 See merge request !3

Merge branch 'main' into 'main'
megatron升级v0.10 See merge request !3
d520d24f · silencealiang · 3aca1415 · 481609bb · d520d24f · d520d24f
Commit d520d24f authored Dec 09, 2024 by silencealiang
20 changed files
--- a/megatron/training/theoretical_memory_usage.py
+++ b/megatron/training/theoretical_memory_usage.py
--- a/megatron/tokenizer/__init__.py
+++ b/megatron/tokenizer/__init__.py
--- a/megatron/tokenizer/bert_tokenization.py
+++ b/megatron/tokenizer/bert_tokenization.py
--- a/megatron/tokenizer/gpt2_tokenization.py
+++ b/megatron/tokenizer/gpt2_tokenization.py
--- a/megatron/training/tokenizer/multimodal_tokenizer.py
+++ b/megatron/training/tokenizer/multimodal_tokenizer.py
--- a/megatron/tokenizer/tokenizer.py
+++ b/megatron/tokenizer/tokenizer.py
--- a/megatron/training.py
+++ b/megatron/training.py
--- a/megatron/utils.py
+++ b/megatron/utils.py
--- a/megatron/training/yaml_arguments.py
+++ b/megatron/training/yaml_arguments.py
--- a/mypy.ini
+++ b/mypy.ini
--- a/pretrain_bert.py
+++ b/pretrain_bert.py
--- a/pretrain_gpt.py
+++ b/pretrain_gpt.py
--- a/pretrain_gpt_core.py
+++ b/pretrain_gpt_core.py
--- a/pretrain_ict.py
+++ b/pretrain_ict.py
--- a/pretrain_mamba.py
+++ b/pretrain_mamba.py
--- a/pretrain_retro.py
+++ b/pretrain_retro.py
--- a/pretrain_t5.py
+++ b/pretrain_t5.py
--- a/pretrain_vision_classify.py
+++ b/pretrain_vision_classify.py
--- a/pretrain_vision_dino.py
+++ b/pretrain_vision_dino.py
--- a/pretrain_vision_inpaint.py
+++ b/pretrain_vision_inpaint.py