• Alex Xiao's avatar
    Refactor data sharding to be specified via caller of task rather than task itself · 99fbd317
    Alex Xiao authored
    Summary: Modifying number of shards internally to disable data sharding for batch iteration is dangerous because the caller of these tasks is not limited to fairspeq/train. So therefore we should put the onus of data sharding properly on the caller rather than the task itself.
    
    Reviewed By: myleott
    
    Differential Revision: D18456424
    
    fbshipit-source-id: d46be16c441c50082f9a768d0b259e6c28a4b67b
    99fbd317
trainer.py 24.3 KB