• Myle Ott's avatar
    Add FairseqTask · ff68a9ef
    Myle Ott authored
    A Task defines the data format, stores shared state (e.g., dictionaries) and provides helpers for building the model/criterion and calculating the loss.
    
    Changes:
    - Add TranslationTask and LanguageModelingTask. New tasks can be registered with @register_task decorator.
    - Add EpochBatchIterator to encapsulate batching and saving/restoring dataloader position
    - Remove LEFT_PAD_* constants and make them configurable per task
    ff68a9ef
utils.py 4.99 KB