Unverified Commit 4c3d98dd authored by Stas Bekman's avatar Stas Bekman Committed by GitHub
Browse files

[s2s finetune_trainer] add instructions for distributed training (#8884)

parent aa60b230
...@@ -213,6 +213,11 @@ To see all the possible command line options, run: ...@@ -213,6 +213,11 @@ To see all the possible command line options, run:
python finetune_trainer.py --help python finetune_trainer.py --help
``` ```
For multi-gpu training use `torch.distributed.launch`, e.g. with 2 gpus:
```bash
python -m torch.distributed.launch --nproc_per_node=2 finetune_trainer.py ...
```
**At the moment, `Seq2SeqTrainer` does not support *with teacher* distillation.** **At the moment, `Seq2SeqTrainer` does not support *with teacher* distillation.**
All `Seq2SeqTrainer`-based fine-tuning scripts are included in the `builtin_trainer` directory. All `Seq2SeqTrainer`-based fine-tuning scripts are included in the `builtin_trainer` directory.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment