Skip to content

GitLab

Switch branch/tag
  1. 26 Jun, 2019 3 commits
  3. 25 Jun, 2019 1 commit
    • freewym's avatar
      avoid "divided by zero error" in logging_outputs when --use-bmuf is e… (#812) · b3864b28
      freewym authored 
      Summary:
… enabled.

When doing multi-gpu training with --use-bmuf turned on and --global-sync-iter > 1, each replica may not sync with other replicas at each iteration. So logging_outputs only has stats of their own.  On the other hand, logging_outputs may be empty at the end of an epoch after "a dummy iteration" because the number of replicas does not divide the number of batches of the training data. If this happens, sample_size and ntokens would be 0 for some replica  and cause "divided by 0" error. This fix sets *loss to 0 if sample_size/ntokens is 0.
Pull Request resolved: https://github.com/pytorch/fairseq/pull/812

Reviewed By: myleott, yqwangustc

Differential Revision: D15908614

Pulled By: nayansinghal

fbshipit-source-id: c92e8e095f012bdb4ef753a3c627fd215afa215d
      b3864b28
  5. 24 Jun, 2019 1 commit
  7. 23 Jun, 2019 3 commits
  9. 21 Jun, 2019 2 commits
  11. 20 Jun, 2019 6 commits
    • Matt Le's avatar
      Use bert init for xlm_base · 6be5f07c
      Matt Le authored 
      Summary:
Use bert init for xlm_base.  This seems to be much closer to what is done in the [XLM](https://github.com/facebookresearch/XLM/blob/master/src/model/transformer.py#L44) repo.

At update 10 with BERT init (f121471600), loss starts at 14.234

At update 10 without BERT init (f121471612), loss starts at 154.423

Reviewed By: liezl200, pipibjc

Differential Revision: D15874836

fbshipit-source-id: f81bf83a078992d7476ba7fdf263b731a9f5b66d
      6be5f07c
    • Myle Ott's avatar
      v0.7.1: fix PyPI setup and tests · 881381cf
      Myle Ott authored 
      Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/818

Differential Revision: D15916265

Pulled By: myleott

fbshipit-source-id: c66c0bd988d3472c4150226952f34ee8d4c3db86
      881381cf
    • davidecaroselli's avatar
      Enhanced MMapIndexedDataset: less memory, higher speed (#816) · 9462a819
      davidecaroselli authored 
      Summary:
I have made an upgrade to my previous implementation of MMapIndexedDataset, now:
- It uses up to **4 times less memory and disk space**
- Words per second is slightly improved thanks to less memory access
Pull Request resolved: https://github.com/pytorch/fairseq/pull/816

Differential Revision: D15899848

Pulled By: myleott

fbshipit-source-id: 9ddeb4809729ef69cc6b0867b33ee71184d845e6
      9462a819
    • Peng-Jen Chen's avatar
      Better explain the inference argument format of multilingual translation · 9c3bb5c6
      Peng-Jen Chen authored 
      Summary:
In https://github.com/pytorch/fairseq/issues/656, people are often confused about how to set multilingual translation parameters at inference time.

This diff add more checks to ensure the arguments (`--lang-pairs`, `--encoder-langtok`, `--decoder-langtok`) load from checkpoint are consistent with arguments specified in generate/interactive command line.
We also add a section in example page to explain how to set the arguments

Reviewed By: myleott

Differential Revision: D15682169

fbshipit-source-id: 64e6db94cd72ea7ce2d0aa1067c9c2dcd3b8a2ac
      9c3bb5c6
    • alexeib's avatar
      wav2vec model (#654) · 392fce8a
      alexeib authored 
      Summary:
Merging wav2vec to master. Includes renames (Cpc -> wav2vec) and some light example files.
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/654

Differential Revision: D15913409

Pulled By: alexeib

fbshipit-source-id: f723e6f211706cd9431c7d76dc12c4e80c9cfc80
      392fce8a
    • Myle Ott's avatar
      v0.7.0 (#817) · bd710e75
      Myle Ott authored 
      Summary:
Notable (possibly breaking) changes:
- d45db804: Remove checkpoint utility functions from utils.py into checkpoint_utils.py
- f2563c21: Move LM definitions into separate files
- dffb1674: Updates to model API:
  - `FairseqModel` -> `FairseqEncoderDecoderModel`
  - add `FairseqDecoder.extract_features` and `FairseqDecoder.output_layer`
  - `encoder_out_dict` -> `encoder_out`
  - rm unused `remove_head` functions
- 34726d56: Move `distributed_init` into `DistributedFairseqModel`
- cf17068a: Simplify distributed launch by automatically launching multiprocessing on each node for all visible GPUs (allows launching just one job per node instead of one per GPU)
- d45db804: Change default LR scheduler from `reduce_lr_on_plateau` to `fixed`
- 96ac28d3: Rename `--sampling-temperature` -> `--temperature`
- fc1a19a3: Deprecate dummy batches
- a1c997bd: Add memory mapped datasets
- 0add50c2: Allow cycling over multiple datasets, where each one becomes an "epoch"

Plus many additional features and bugfixes
Pull Request resolved: https://github.com/pytorch/fairseq/pull/817

Differential Revision: D15913844

Pulled By: myleott

fbshipit-source-id: d5b5d678efdd9dd3e4d7ca848ddcf1ec2b21bf6b
      bd710e75
  13. 19 Jun, 2019 4 commits
  15. 15 Jun, 2019 1 commit
  17. 13 Jun, 2019 1 commit
  19. 12 Jun, 2019 3 commits
    • Nayan Singhal's avatar
      Add Model Averaging · 6982c404
      Nayan Singhal authored 
      Summary:
Implemented model averaging for fairseq.
Removed the ddp wrapper if global optimizer is provided.
Syncing all the models based on the iteration provide in the input

TODO:
1) Fix throughput and wps meter. Need to check other meters too.
2) Replace Model average code with BMUF algorithm implementation.

Reviewed By: myleott

Differential Revision: D15711044

fbshipit-source-id: 58a4af74db2a61d06762597b95836cbeb1ed82cc
      6982c404
    • Myle Ott's avatar
      Add more torch.hub deps · 78c2fcf0
      Myle Ott authored 
      Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/801

Differential Revision: D15781975

Pulled By: myleott

fbshipit-source-id: b86276cd3a40138c09494637c43ce52a56c4aced
      78c2fcf0
    • Myle Ott's avatar
      Add missing dependencies to hubconf · 37df862e
      Myle Ott authored 
      Summary: Pull Request resolved: https://github.com/pytorch/fairseq/pull/799

Differential Revision: D15773932

Pulled By: myleott

fbshipit-source-id: 650c0621bedb3b7ecebc0654d8e10d7692c50994
      37df862e
  21. 11 Jun, 2019 7 commits
  23. 10 Jun, 2019 2 commits
    • Myle Ott's avatar
      More generator features for demo (#791) · 4868c182
      Myle Ott authored 
      Summary:
- make it possible to load file_utils.py without the dependencies
- add some more demo features
Pull Request resolved: https://github.com/pytorch/fairseq/pull/791

Differential Revision: D15739950

Pulled By: myleott

fbshipit-source-id: 38df5209973a6fe2e3651575b97134e096aaf5bf
      4868c182
    • freewym's avatar
      fix log printing in progress bar (#778) · a58c1127
      freewym authored 
      Summary:
In the current progress bar, the counter for log_interval will always start from 0, which is not correct if  reloading from a checkpoint in the middle of an epoch. This fix obtains the offset from the iterator to set the counter correctly.
Pull Request resolved: https://github.com/pytorch/fairseq/pull/778

Differential Revision: D15739953

Pulled By: myleott

fbshipit-source-id: a1d13403ec5783b22e01d7cb63874fd8dea7f8b0
      a58c1127
  25. 07 Jun, 2019 1 commit
  27. 06 Jun, 2019 1 commit
  29. 04 Jun, 2019 4 commits