Skip to content

GitLab

"...research_projects/seq2seq-distillation/utils copy.py" did not exist on "de9e2979647338bc9617dae68c5e9dccc413fb9f"
Switch branch/tag
  1. 18 Feb, 2021 2 commits
  3. 11 Feb, 2021 2 commits
  5. 09 Feb, 2021 1 commit
  7. 31 Jan, 2021 1 commit
  9. 29 Jan, 2021 2 commits
  11. 28 Jan, 2021 2 commits
  13. 27 Jan, 2021 1 commit
  15. 26 Jan, 2021 1 commit
    • Sylvain Gugger's avatar
      Smdistributed trainer (#9798) · 0d0efd3a
      Sylvain Gugger authored 
      * Add a debug print

* Adapt Trainer to use smdistributed if available

* Forgotten parenthesis

* Real check for sagemaker

* Donforget to define device...

* Woopsie, local)rank is defined differently

* Update since local_rank has the proper value

* Remove debug statement

* More robust check for smdistributed

* Quality

* Deal with key not present error
      0d0efd3a
  17. 22 Jan, 2021 1 commit
  19. 20 Jan, 2021 2 commits
  21. 14 Jan, 2021 2 commits
  23. 13 Jan, 2021 2 commits
    • Sylvain Gugger's avatar
      Fix data parallelism in Trainer (#9566) · 04dc65e5
      Sylvain Gugger authored 
      

* Fix data parallelism in Trainer

* Update src/transformers/training_args.py
Co-authored-by: default avatarLysandre Debut <lysandre@huggingface.co>
Co-authored-by: default avatarLysandre Debut <lysandre@huggingface.co>
      04dc65e5
    • Stas Bekman's avatar
      [trainer] deepspeed integration (#9211) · 2df34f4a
      Stas Bekman authored 
      

* deepspeed integration

* style

* add test

* ds wants to do its own backward

* fp16 assert

* Update src/transformers/training_args.py
Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style

* for clarity extract what args are being passed to deepspeed

* introduce the concept of self.wrapped_model

* s/self.wrapped_model/self.model_wrapped/

* complete transition to self.wrapped_model / self.model

* fix

* doc

* give ds its own init

* add custom overrides, handle bs correctly

* fix test

* clean up model_init logic, fix small bug

* complete fix

* collapse --deepspeed_config into --deepspeed

* style

* start adding doc notes

* style

* implement hf2ds optimizer and scheduler configuration remapping

* oops

* call get_num_training_steps absolutely when needed

* workaround broken auto-formatter

* deepspeed_config arg is no longer needed - fixed in deepspeed master

* use hf's fp16 args in config

* clean

* start on the docs

* rebase cleanup

* finish up --fp16

* clarify the supported stages

* big refactor thanks to discovering deepspeed.init_distributed

* cleanup

* revert fp16 part

* add checkpoint-support

* more init ds into integrations

* extend docs

* cleanup

* unfix docs

* clean up old code

* imports

* move docs

* fix logic

* make it clear which file it's referring to

* document nodes/gpus

* style

* wrong format

* style

* deepspeed handles gradient clipping

* easier to read

* major doc rewrite

* Apply suggestions from code review
Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>

* docs

* switch to AdamW optimizer

* style

* Apply suggestions from code review
Co-authored-by: default avatarLysandre Debut <lysandre@huggingface.co>

* clarify doc
Co-authored-by: default avatarSylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: default avatarLysandre Debut <lysandre@huggingface.co>
      2df34f4a
  25. 11 Jan, 2021 1 commit
    • Stas Bekman's avatar
      [trainer] remove `--model_parallel` (#9451) · 33b74228
      Stas Bekman authored 
      

* fix bad merge - dropped code

* remove --model_parallel

* Deal with TrainingArguments

* Use a private attr and fix batch sizes

* fix _n_gpu

* add is_parallel helper wrapper

* fix attribute

* introduce a new attribute is_model_parallel

* docs

* docs

* Put back init False and rearrange doc

* Ignore non-init args in HFArgumentParser
Co-authored-by: default avatarSylvain Gugger <sylvain.gugger@gmail.com>
      33b74228
  27. 05 Jan, 2021 2 commits
  29. 22 Dec, 2020 1 commit
  31. 18 Dec, 2020 1 commit
  33. 16 Dec, 2020 1 commit
  35. 15 Dec, 2020 3 commits
  37. 14 Dec, 2020 1 commit
  39. 07 Dec, 2020 1 commit
  41. 01 Dec, 2020 2 commits
  43. 23 Nov, 2020 2 commits
  45. 20 Nov, 2020 1 commit
  47. 17 Nov, 2020 1 commit
  49. 03 Nov, 2020 1 commit
  51. 30 Oct, 2020 1 commit
  53. 29 Oct, 2020 1 commit
  55. 27 Oct, 2020 1 commit