Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
chenpangpang
transformers
Commits
7a68d401
Unverified
Commit
7a68d401
authored
Jul 27, 2020
by
Sam Shleifer
Committed by
GitHub
Jul 27, 2020
Browse files
[s2s] Don't mention packed data in README (#6079)
parent
b7345d22
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
examples/seq2seq/README.md
examples/seq2seq/README.md
+3
-3
No files found.
examples/seq2seq/README.md
View file @
7a68d401
...
...
@@ -89,20 +89,20 @@ Then you can finetune mbart_cc25 on english-romanian with the following command.
Best performing command:
```
bash
# optionally
export
ENRO_DIR
=
'wmt_en_ro
_packed_train_200
'
# Download instructions above
export
ENRO_DIR
=
'wmt_en_ro'
# Download instructions above
# export WANDB_PROJECT="MT" # optional
export
MAX_LEN
=
200
export
BS
=
4
export
GAS
=
8
# gradient accumulation steps
./train_mbart_cc25_enro.sh
--output_dir
enro_finetune_baseline
--label_smoothing
0.1
--fp16_opt_level
=
O1
--logger_name
wandb
--sortish_sampler
```
This should take <
2
h/epoch on a 16GB v100 and achieve val_avg_ BLEU score above 25. (you can see in wandb or metrics.json).
This should take <
6
h/epoch on a 16GB v100 and achieve val_avg_ BLEU score above 25. (you can see
metrics
in wandb or metrics.json).
To get results in line with fairseq, you need to do some postprocessing.
MultiGPU command
(using 8 GPUS as an example)
```
bash
export
ENRO_DIR
=
'wmt_en_ro
_packed_train_200
'
# Download instructions above
export
ENRO_DIR
=
'wmt_en_ro'
# Download instructions above
# export WANDB_PROJECT="MT" # optional
export
MAX_LEN
=
200
export
BS
=
4
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment