GitLab

Summary:
Possibly breaking changes:
- Set global numpy seed (4a7cd582)
- Split `in_proj_weight` into separate k, v, q projections in MultiheadAttention (fdf4c3e9)
- TransformerEncoder returns namedtuples instead of dict (27568a7e)

New features:
- Add `--fast-stat-sync` option (e1ba32aa)
- Add `--empty-cache-freq` option (315c463d)
- Support criterions with parameters (ba5f829f)

New papers:
- Simple and Effective Noisy Channel Modeling for Neural Machine Translation (49177c99)
- Levenshtein Transformer (86857a58, ...)
- Cross+Self-Attention for Transformer Models (4ac2c5f2)
- Jointly Learning to Align and Translate with Transformer Models (1c667929)
- Reducing Transformer Depth on Demand with Structured Dropout (dabbef46)
- Unsupervised Cross-lingual Representation Learning at Scale (XLM-RoBERTa) (e23e5eaa)
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (a92bcdad)
- CamemBERT: a French BERT (b31849aa)

Speed improvements:
- Add CUDA kernels for LightConv and DynamicConv (f840564d)
- Cythonization of various dataloading components (4fc39538, ...)
- Don't project mask tokens for MLM training (718677eb)
Pull Request resolved: https://github.com/pytorch/fairseq/pull/1452

Differential Revision: D18798409

Pulled By: myleott

fbshipit-source-id: 860a0d5aaf7377c8c9bd63cdb3b33d464f0e1727

Summary: Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/835

Differential Revision: D16904038

Pulled By: myleott

fbshipit-source-id: 2c9d0b913f8d688297ac80fcabd905bd1397f66a

Summary:
Implementation of noisy channel model reranking for release with paper
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/667

Reviewed By: michaelauli

Differential Revision: D15901665

Pulled By: nng555

fbshipit-source-id: 2de2c518be8e5828ffad72db3e741b0940623373

Summary:
CUDA code for light/dynamicconv kernels, including pytorch modules. Modules can be built by running setup.py in each respective folder, and can then be imported and used like any other module.
Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/547

Reviewed By: myleott, shubho

Differential Revision: D15703660

Pulled By: nng555

fbshipit-source-id: e9c913753be3a1cd571965f7200df6678b644520