更新transformer代码

c0f05c10 · hepj · c056df78 · c0f05c10 · c0f05c10 · c0f05c10
Commit c0f05c10 authored Nov 29, 2022 by hepj
15 changed files
--- a/PyTorch/NLP/new-Transformer/scripts/read_binarized.py
+++ b/PyTorch/NLP/new-Transformer/scripts/read_binarized.py
--- a/PyTorch/NLP/new-Transformer/scripts/rm_pt.py
+++ b/PyTorch/NLP/new-Transformer/scripts/rm_pt.py
--- a/PyTorch/NLP/new-Transformer/scripts/sacrebleu.sh
+++ b/PyTorch/NLP/new-Transformer/scripts/sacrebleu.sh
--- a/PyTorch/NLP/new-Transformer/scripts/shard_docs.py
+++ b/PyTorch/NLP/new-Transformer/scripts/shard_docs.py
--- a/PyTorch/NLP/new-Transformer/scripts/split_train_valid_docs.py
+++ b/PyTorch/NLP/new-Transformer/scripts/split_train_valid_docs.py
--- a/PyTorch/NLP/new-Transformer/scripts/spm_decode.py
+++ b/PyTorch/NLP/new-Transformer/scripts/spm_decode.py
--- a/PyTorch/NLP/new-Transformer/scripts/spm_encode.py
+++ b/PyTorch/NLP/new-Transformer/scripts/spm_encode.py
--- a/PyTorch/NLP/new-Transformer/scripts/spm_train.py
+++ b/PyTorch/NLP/new-Transformer/scripts/spm_train.py
--- a/PyTorch/NLP/new-Transformer/scripts/test_fsdp.sh
+++ b/PyTorch/NLP/new-Transformer/scripts/test_fsdp.sh
--- a/PyTorch/NLP/new-Transformer/setup.cfg
+++ b/PyTorch/NLP/new-Transformer/setup.cfg
--- a/PyTorch/NLP/new-Transformer/setup.py
+++ b/PyTorch/NLP/new-Transformer/setup.py
--- a/PyTorch/NLP/new-Transformer/single.sh
+++ b/PyTorch/NLP/new-Transformer/single.sh
--- a/PyTorch/NLP/new-Transformer/single_fp16.sh
+++ b/PyTorch/NLP/new-Transformer/single_fp16.sh
--- a/PyTorch/NLP/new-Transformer/single_process.sh
+++ b/PyTorch/NLP/new-Transformer/single_process.sh
--- a/PyTorch/NLP/new-Transformer/train.py
+++ b/PyTorch/NLP/new-Transformer/train.py