GitLab

default samples_per_token in eval_lm

Also show a nicer error message.

When downloading files in examples directory (e.g. when running
`prepare-iwslt14.sh`), git sees them as untracked files but they should
not be committed.
Add a .gitignore script that ignore everything in the examples
subdirectories except for .sh and .md files.

Two tiny changes to train/eval_lm. For train fix an off by one, while for eval_lm make it work when the task is translation'

Fix preprocessed test set download links

Changelog:
- 97b58b46: add Transformer model from Vaswani et al. (2017)
- b2374e52: faster Transformer inference with improved caching
- 2d27ae08: simulate large mini-batch training with delayed updates (`--update-freq`)
- 7ee1d284: add FP16 training support (`--fp16`)
- 2a84f46b: faster inference by removing completed sentences from the batch
- 663fd806: batched interactive generation
- 4c2ef2de: add language modeling / gated convolutional model from Dauphin et al. (2017)
- b59815bc: add Hierarchical Neural Story Generation model from Fan et al. (2018)
- ff68a9ef: add FairseqTask to modularize task definitions (e.g., translation, language modeling)

Add transformer models and replace list with table

A Task defines the data format, stores shared state (e.g., dictionaries) and provides helpers for building the model/criterion and calculating the loss.

Changes:
- Add TranslationTask and LanguageModelingTask. New tasks can be registered with @register_task decorator.
- Add EpochBatchIterator to encapsulate batching and saving/restoring dataloader position
- Remove LEFT_PAD_* constants and make them configurable per task

torch.arange default return type is changed in the latest pytorch version https://github.com/pytorch/pytorch/pull/7016

Co-authored-by: pmichel31415 <pmichel@fb.com>