GitLab

Changes:
- 7d19e36: Add `--sampling` flag to generate.py to sample instead of doing beam search
- c777340: Add `scripts/average_checkpoints.py` to average multiple checkpoints into a combined model
- 3ea882c: Add `--max-update` option to train.py to stop training after a given number of updates
- small bugfixes for distributed training, LSTM, inverse square root LR scheduler

Update training commands

specify a single GPU setup for IWSLT14

Update training commands in data/README to match the latest version of this project according to #132.

Continue from 3c072958: add omitted "\".

Update training commands in data/README to match the latest version of this project according to #132.

- Motivation: in the previous data/README, the commands are obsolete and will cause the error "unrecognized arguments: --label-smoothing 0.1 --force-anneal 50". 
- What's changed: add arguments "--criterion label_smoothed_cross_entropy" and "--lr-scheduler fixed" to the training commands of all 3 datasets.
- Result: the new commands run without error on all 3 datasets.

Change "awailable" to "available".

Change "awailable" to "available".

Oss merge internal

* Allow more flexible pre-processing and generation

* Addressing CR comments

* small fix

Remove custom ConvTBC code

Oss merge internal changes

* Making our code compatible with the latest pytorch

* revert

* torch.nn.utils.clip_grad_norm now returns tensor

* Add prefix

* Fixes

* Keep original scores with prefix

* Improve prefix code

* Replace 'repeat' with 'expand'

This PR includes breaking API changes to modularize fairseq-py and adds support for distributed training across multiple nodes.

Changes:
- c7033ef: add support for distributed training! See updated README for usage.
- e016299: modularize fairseq-py, adding support for register_model, register_criterion, register_optimizer, etc.
- 154e440: update LSTM implementation to use PackedSequence objects in the encoder, better following best practices and improving perf
- 90c2973 and 1da6265: improve unit test coverage

Prepare scripts for WMT14 (#88)

BLEU ratio should be predlen/reflen not reflen/predlen

To be compatible with multi-bleu.
This seems to only affect the result_string.

See https://arxiv.org/abs/1711.05101