Merge branch 'main' into 'main'

megatron升级v0.10 See merge request !3

Merge branch 'main' into 'main'
megatron升级v0.10 See merge request !3
d520d24f · silencealiang · 3aca1415 · 481609bb · d520d24f · d520d24f
Commit d520d24f authored Dec 09, 2024 by silencealiang
20 changed files
--- a/megatron/mpu/tests/test_initialize.py
+++ b/megatron/mpu/tests/test_initialize.py
--- a/megatron/mpu/tests/test_layers.py
+++ b/megatron/mpu/tests/test_layers.py
--- a/megatron/mpu/tests/test_random.py
+++ b/megatron/mpu/tests/test_random.py
--- a/megatron/memory.py
+++ b/megatron/memory.py
--- a/megatron/microbatches.py
+++ b/megatron/microbatches.py
--- a/megatron/model/distributed.py
+++ b/megatron/model/distributed.py
--- a/megatron/optimizer/__init__.py
+++ b/megatron/optimizer/__init__.py
--- a/megatron/optimizer/distrib_optimizer.py
+++ b/megatron/optimizer/distrib_optimizer.py
--- a/megatron/text_generation/forward_step.py
+++ b/megatron/text_generation/forward_step.py
--- a/megatron/__init__.py
+++ b/megatron/__init__.py
--- a/megatron/training/activations.py
+++ b/megatron/training/activations.py
--- a/megatron/arguments.py
+++ b/megatron/arguments.py
--- a/megatron/training/async_utils.py
+++ b/megatron/training/async_utils.py
--- a/megatron/checkpointing.py
+++ b/megatron/checkpointing.py
--- a/megatron/dist_signal_handler.py
+++ b/megatron/dist_signal_handler.py
--- a/megatron/training/ft_integration.py
+++ b/megatron/training/ft_integration.py
--- a/megatron/global_vars.py
+++ b/megatron/global_vars.py
--- a/megatron/initialize.py
+++ b/megatron/initialize.py
--- a/megatron/training/log_handler.py
+++ b/megatron/training/log_handler.py
--- a/megatron/training/one_logger_utils.py
+++ b/megatron/training/one_logger_utils.py