Skip to content

GitLab

Switch branch/tag
  1. 08 Oct, 2019 3 commits
    • Jerry Ma's avatar
      Add printing of PyTorch memory summary on OOM (#885) · 63b6b3f4
      Jerry Ma authored 
      Summary:
PyTorch now has more comprehensive memory instrumentation, added in https://github.com/pytorch/pytorch/pull/27361 . This PR makes fairseq print a summary table of the memory state when an OOM occurs.
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/885

Differential Revision: D17820445

Pulled By: jma127

fbshipit-source-id: 1887417c7648d703f78e1cff9f2a5b89901f49d0
      63b6b3f4
    • Jungo Kasai's avatar
      ensemble levts · 34e79c58
      Jungo Kasai authored 
      Summary:
Add ensemble wrappers to the levenshtein NAT.
Levenshtein
Final softmax ensemble over the pipeline of three steps: deletion, placeholder insertion, and word selection.
1. Deletion
2. Placeholder Insertion
3. Word Selection

Each step involves scoring, averaging the scores over the ensemble, and then make hard decisions with argmax. Then next step follows. We cannot do the three steps in parallel by design.

Reviewed By: kahne

Differential Revision: D17723202

fbshipit-source-id: 05f7a4fcd922a972cc4796ca397e8220f0b4d53e
      34e79c58
    • Changhan Wang's avatar
      fix max lengths in Levenshtein Tramsformer · c2165224
      Changhan Wang authored 
      Summary: Fix the max length calculation in Levenshtein Transformer

Reviewed By: jhcross

Differential Revision: D17672946

fbshipit-source-id: e5efbe7e56cf879d3e822864e4398f99f45b04d4
      c2165224
  3. 07 Oct, 2019 1 commit
    • Nayan Singhal's avatar
      Setting Global sync to 50 in BMUF · 6f58e15e
      Nayan Singhal authored 
      Summary:
In all our final settings, we are using global_sync = 50 and we get comparable results with DDP and caffe2.

Setting the default global-sync-iter = 50
and users can just define --use-bmuf to enable it for training.

Reviewed By: skritika

Differential Revision: D17765094

fbshipit-source-id: 369591eeff266d757f89e1fc8dda01711146fdbc
      6f58e15e
  5. 05 Oct, 2019 1 commit
  7. 04 Oct, 2019 2 commits
  9. 01 Oct, 2019 1 commit
  11. 30 Sep, 2019 2 commits
  13. 29 Sep, 2019 2 commits
  15. 28 Sep, 2019 1 commit
  17. 27 Sep, 2019 5 commits
  19. 26 Sep, 2019 1 commit
  21. 24 Sep, 2019 1 commit
  23. 23 Sep, 2019 3 commits
  25. 20 Sep, 2019 3 commits
    • Myle Ott's avatar
      Remove extraneous call to RNG in multi-GPU code path · 10f9349e
      Myle Ott authored 
      Summary: Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/865

Differential Revision: D17510276

Pulled By: myleott

fbshipit-source-id: 24119402ad5fe95a1312fadb77bafe49a9197c6b
      10f9349e
    • Myle Ott's avatar
      Update README.race.md · e869c80d
      Myle Ott authored 
      Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/1155

Differential Revision: D17509762

Pulled By: myleott

fbshipit-source-id: 4de535289c1f35abff0d8142d8580f3ede039f47
      e869c80d
    • Naman Goyal's avatar
      added multilingual masked LM training (#849) · 32335404
      Naman Goyal authored 
      Summary:
The multilingual-RoBERTa training is working with aconneau XLM data.

Two pieces remaining:

1) `XLM` limits batch to be from same language, I am not 100% sure about the reason for that, but should be easy to implement, basically we can add `batch_by_size_and_language` instead of default `batch_by_size` function. If it's not critical, I would want to leave it out as it keeps the code very clean and simple.

2) `sample_ratio` in `ConcatDataset` works with `int` by tiling the datasets based on ratio. Currently I am handling it by sounding off the ratio to `first decimal` and then multiplying by `10`. We can see if some such simple heuristics are good enough, there are other options (we can talk about them offline).
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/849

Differential Revision: D17162460

fbshipit-source-id: d967f3d872f7a1f0aa4ea418bd362b68af9e432f
      32335404
  27. 19 Sep, 2019 2 commits
    • Jerry Ma's avatar
      Add dataset class for weighted sampling with replacement. (#861) · a8a85c26
      Jerry Ma authored 
      Summary:
As discussed with Naman earlier today. Weighted sampling with
replacement can be done on a per-epoch basis using `set_epoch()`
functionality, which generates the samples as a function of random seed
and epoch.

Additionally, `FairseqTask` needs to set the starting epoch for the
dataset at the very beginning of iterator construction.

Not yet implemented is the per-epoch iterator construction, which
is necessary to actually regenerate the batches for each epoch.
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/861

Differential Revision: D17460687

Pulled By: jma127

fbshipit-source-id: 1c2a54f04ac96b3561c100a6fd66a9fccbe3c658
      a8a85c26
    • Myle Ott's avatar
      Add cython language_level hints · 0eaaf355
      Myle Ott authored 
      Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/1147

Differential Revision: D17468447

Pulled By: myleott

fbshipit-source-id: 0dbac04b92c8df74ad991d5e92cd02036d662369
      0eaaf355
  29. 18 Sep, 2019 3 commits
  31. 17 Sep, 2019 2 commits
  33. 16 Sep, 2019 1 commit
    • Naman Goyal's avatar
      added fast stats sync option (#858) · e1ba32aa
      Naman Goyal authored 
      Summary:
Added `--fast-stat-sync` option.
This avoids pickle and achieves `~7%` more `wps` on 16 nodes.
It is less flexible as it just aggregates only basic stats and it ignores the aggregate function defined by criterion.

Let me know what you think myleott
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/858

Differential Revision: D17398770

fbshipit-source-id: 36261a1d970e67deeda8211af8f009ef9b4f9c14
      e1ba32aa
  35. 12 Sep, 2019 1 commit
  37. 05 Sep, 2019 1 commit
    • Roman Rädle's avatar
      Return predicted token for RoBERTa filling mask · 3e3fe722
      Roman Rädle authored 
      Summary:
Added the `predicted_token` to each `topk` filled output item

Updated RoBERTa filling mask example in README.md

Reviewed By: myleott

Differential Revision: D17188810

fbshipit-source-id: 5fdc57ff2c13239dabf13a8dad43ae9a55e8931c
      3e3fe722
  39. 04 Sep, 2019 1 commit
    • Peng-Jen Chen's avatar
      Fix multilingual translation bug for to-many case · 1566cfb9
      Peng-Jen Chen authored 
      Summary:
The logic for adding decoder side language token was wrongly implemented.
The way we inject the language token is by replacing the eos symbol with language token symbol. However, the parameter for source / target eos symbol was not set correctly.

Reviewed By: tangyuq

Differential Revision: D17129108

fbshipit-source-id: 6fae385b787370656fd7ca7ab74e6bb91fe5463b
      1566cfb9
  41. 03 Sep, 2019 2 commits
  43. 01 Sep, 2019 1 commit