1. 20 Nov, 2020 1 commit
    • Quentin Lhoest's avatar
      Fix rag finetuning + add finetuning test (#8585) · 8062fa63
      Quentin Lhoest authored
      * replace init_ddp_connection for index init
      
      * style
      
      * add finetune test
      
      * add test data
      
      * move generate tensors to device
      
      * add test on EM metric
      
      * style
      
      * allow multi process test
      
      * keep gloo process group for retrieval
      
      * add multi-gpu test
      
      * use custom accelerator
      
      * clean test finetune
      
      * minor
      
      * style
      
      * style
      
      * typo
      
      * use python call instead of imported main fumction
      
      * return_dict fix in modeling_rag
      
      * use float32 in retrieval
      
      * store as float32 as well in the custom knowledge dataset example
      
      * style
      
      * rename to finetune_rag
      
      * style
      
      * update readme
      
      * rename utils and callbacks to utils_rag and callbacks_rag
      
      * fix test
      
      * patrick's comments
      
      * generate dummy data in the finetue test script
      
      * remove dummy data files
      
      * style
      8062fa63