Skip to content

GitLab

"...test_cli/git@developer.sourcefind.cn:wangsen/mineru.git" did not exist on "59c023f177e75cf2abf6d9208e37ab459a7be446"
Unverified Commit 066fd047 authored by Stas Bekman's avatar Stas Bekman Committed by GitHub
Browse files

correct TP implementation resources (#13248) 

fix a few implementation links
parent 4d10474f
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 2 deletions
docs/source/parallelism.md
View file @ 066fd047
...@@ -220,9 +220,12 @@ Special considerations: TP requires very fast network, and therefore it's not ad ...@@ -220,9 +220,12 @@ Special considerations: TP requires very fast network, and therefore it's not ad
This section is based on the original much more [detailed TP overview](https://github.com/huggingface/transformers/issues/10321#issuecomment-783543530). This section is based on the original much more [detailed TP overview](https://github.com/huggingface/transformers/issues/10321#issuecomment-783543530).
by [@anton-l](https://github.com/anton-l). by [@anton-l](https://github.com/anton-l).
Implementations: Alternative names:
- DeepSpeed calls it [tensor slicing](https://www.deepspeed.ai/features/#model-parallelism) - DeepSpeed calls it [tensor slicing](https://www.deepspeed.ai/features/#model-parallelism)
- [Megatron-LM](https://github.com/NVIDIA/Megatron-LM) has an internal implementation.
Implementations:
- [Megatron-LM](https://github.com/NVIDIA/Megatron-LM) has an internal implementation, as it's very model-specific
- [parallelformers](https://github.com/tunib-ai/parallelformers) (only inference at the moment)
🤗 Transformers status: 🤗 Transformers status:
- core: not yet implemented in the core - core: not yet implemented in the core
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment