Merge pull request #5278 from ver217/sync/npu

[sync] sync npu branch with main

Merge pull request #5278 from ver217/sync/npu
[sync] sync npu branch with main
d66e6988 · Frank Lee · GitHub · 9102d655 · 14846934 · d66e6988
Unverified Commit d66e6988 authored Jan 18, 2024 by Frank Lee Committed by GitHub Jan 18, 2024
20 changed files
--- a/colossalai/shardformer/policies/falcon.py
+++ b/colossalai/shardformer/policies/falcon.py
--- a/colossalai/shardformer/policies/gptj.py
+++ b/colossalai/shardformer/policies/gptj.py
--- a/colossalai/shardformer/policies/llama.py
+++ b/colossalai/shardformer/policies/llama.py
--- a/colossalai/shardformer/policies/mistral.py
+++ b/colossalai/shardformer/policies/mistral.py
--- a/colossalai/shardformer/policies/opt.py
+++ b/colossalai/shardformer/policies/opt.py
--- a/colossalai/shardformer/policies/whisper.py
+++ b/colossalai/shardformer/policies/whisper.py
--- a/colossalai/shardformer/shard/shard_config.py
+++ b/colossalai/shardformer/shard/shard_config.py
--- a/colossalai/shardformer/shard/sharder.py
+++ b/colossalai/shardformer/shard/sharder.py
--- a/colossalai/tensor/colo_parameter.py
+++ b/colossalai/tensor/colo_parameter.py
--- a/colossalai/tensor/d_tensor/comm_spec.py
+++ b/colossalai/tensor/d_tensor/comm_spec.py
--- a/colossalai/utils/memory.py
+++ b/colossalai/utils/memory.py
--- a/colossalai/zero/__init__.py
+++ b/colossalai/zero/__init__.py
--- a/colossalai/zero/gemini/__init__.py
+++ b/colossalai/zero/gemini/__init__.py
--- a/docs/README-zh-Hans.md
+++ b/docs/README-zh-Hans.md
--- a/docs/source/en/features/lazy_init.md
+++ b/docs/source/en/features/lazy_init.md
@@ -73,4 +73,4 @@ And some models are not supported at all which will raise an error. We tested mo
 | Blip2Model                    | transformers |
 | Blip2ForConditionalGeneration | transformers |
-<!-- doc-test-command: torchrun --standalone --nproc_per_node=2 lazy_iniy.py  -->
+<!-- doc-test-command: torchrun --standalone --nproc_per_node=2 lazy_init.py  -->
--- a/docs/source/en/features/pipeline_parallel.md
+++ b/docs/source/en/features/pipeline_parallel.md
--- a/docs/source/en/features/shardformer.md
+++ b/docs/source/en/features/shardformer.md
--- a/docs/source/en/get_started/installation.md
+++ b/docs/source/en/get_started/installation.md
--- a/docs/source/zh-Hans/basics/booster_api.md
+++ b/docs/source/zh-Hans/basics/booster_api.md
--- a/docs/source/zh-Hans/features/lazy_init.md
+++ b/docs/source/zh-Hans/features/lazy_init.md