1. 19 Sep, 2019 1 commit
    • Jerry Ma's avatar
      Add dataset class for weighted sampling with replacement. (#861) · a8a85c26
      Jerry Ma authored
      Summary:
      As discussed with Naman earlier today. Weighted sampling with
      replacement can be done on a per-epoch basis using `set_epoch()`
      functionality, which generates the samples as a function of random seed
      and epoch.
      
      Additionally, `FairseqTask` needs to set the starting epoch for the
      dataset at the very beginning of iterator construction.
      
      Not yet implemented is the per-epoch iterator construction, which
      is necessary to actually regenerate the batches for each epoch.
      Pull Request resolved: https://github.com/fairinternal/fairseq-py/pull/861
      
      Differential Revision: D17460687
      
      Pulled By: jma127
      
      fbshipit-source-id: 1c2a54f04ac96b3561c100a6fd66a9fccbe3c658
      a8a85c26