Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "38ccb8b1a321fa70926236a22cfd7911a993b53e"
Switch branch/tag
  1. 12 Jul, 2022 1 commit
  3. 07 Jul, 2022 1 commit
  5. 04 Jul, 2022 1 commit
  7. 29 Jun, 2022 1 commit
  9. 22 Jun, 2022 1 commit
  11. 14 Jun, 2022 1 commit
  13. 01 Jun, 2022 1 commit
  15. 27 May, 2022 1 commit
  17. 20 May, 2022 1 commit
  19. 14 May, 2022 1 commit
  21. 13 May, 2022 1 commit
  23. 11 May, 2022 1 commit
  25. 06 May, 2022 1 commit
  27. 03 May, 2022 1 commit
  29. 22 Apr, 2022 1 commit
  31. 21 Apr, 2022 1 commit
  33. 20 Apr, 2022 1 commit
  35. 14 Apr, 2022 1 commit
  37. 08 Apr, 2022 1 commit
  39. 07 Apr, 2022 1 commit
  41. 30 Mar, 2022 1 commit
  43. 22 Mar, 2022 1 commit
  45. 18 Mar, 2022 1 commit
  47. 14 Mar, 2022 1 commit
  49. 15 Feb, 2022 1 commit
  51. 04 Feb, 2022 1 commit
  53. 19 Jan, 2022 1 commit
  55. 18 Jan, 2022 1 commit
  57. 17 Jan, 2022 1 commit
  59. 04 Jan, 2022 1 commit
  61. 27 Dec, 2021 1 commit
  63. 21 Dec, 2021 1 commit
    • Xin Zhang's avatar
      add example of self-supervised SimCLR training - V2 (#50) · 648f8063
      Xin Zhang authored 
      * add example of self-supervised SimCLR training

* simclr v2, replace nvidia dali dataloader

* updated

* sync to latest code writing style

* sync to latest code writing style and modify README

* detail README & standardize dataset path
      648f8063
  65. 13 Dec, 2021 1 commit
  67. 09 Dec, 2021 1 commit
    • Frank Lee's avatar
      Develop/experiments (#59) · da01c234
      Frank Lee authored 
      

* Add gradient accumulation, fix lr scheduler

* fix FP16 optimizer and adapted torch amp with tensor parallel (#18)

* fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes

* fixed trainer

* Revert "fixed trainer"

This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097.

* improved consistency between trainer, engine and schedule (#23)
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>

* Split conv2d, class token, positional embedding in 2d, Fix random number in ddp
Fix convergence in cifar10, Imagenet1000

* Integrate 1d tensor parallel in Colossal-AI (#39)

* fixed 1D and 2D convergence (#38)

* optimized 2D operations

* fixed 1D ViT convergence problem

* Feature/ddp (#49)

* remove redundancy func in setup (#19) (#20)

* use env to control the language of doc (#24) (#25)

* Support TP-compatible Torch AMP and Update trainer API (#27)

* Add gradient accumulation, fix lr scheduler

* fix FP16 optimizer and adapted torch amp with tensor parallel (#18)

* fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes

* fixed trainer

* Revert "fixed trainer"

This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097.

* improved consistency between trainer, engine and schedule (#23)
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatarver217 <lhx0217@gmail.com>

* add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29)

* add explanation for ViT example (#35) (#36)

* support torch ddp

* fix loss accumulation

* add log for ddp

* change seed

* modify timing hook
Co-authored-by: default avatarFrank Lee <somerlee.9@gmail.com>
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatarbinmakeswell <binmakeswell@gmail.com>

* Feature/pipeline (#40)

* remove redundancy func in setup (#19) (#20)

* use env to control the language of doc (#24) (#25)

* Support TP-compatible Torch AMP and Update trainer API (#27)

* Add gradient accumulation, fix lr scheduler

* fix FP16 optimizer and adapted torch amp with tensor parallel (#18)

* fixed bugs in compatibility between torch amp and tensor parallel and performed some minor fixes

* fixed trainer

* Revert "fixed trainer"

This reverts commit 2e0b0b76990e8d4e337add483d878c0f61cf5097.

* improved consistency between trainer, engine and schedule (#23)
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatarver217 <lhx0217@gmail.com>

* add an example of ViT-B/16 and remove w_norm clipping in LAMB (#29)

* add explanation for ViT example (#35) (#36)

* optimize communication of pipeline parallel

* fix grad clip for pipeline
Co-authored-by: default avatarFrank Lee <somerlee.9@gmail.com>
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatarbinmakeswell <binmakeswell@gmail.com>

* optimized 3d layer to fix slow computation ; tested imagenet performance with 3d; reworked lr_scheduler config definition; fixed launch args; fixed some printing issues; simplified apis of 3d layers (#51)

* Update 2.5d layer code to get a similar accuracy on imagenet-1k dataset

* update api for better usability (#58)

update api for better usability
Co-authored-by: default avatar1SAA <c2h214748@gmail.com>
Co-authored-by: default avatarver217 <lhx0217@gmail.com>
Co-authored-by: default avatarpuck_WCR <46049915+WANG-CR@users.noreply.github.com>
Co-authored-by: default avatarbinmakeswell <binmakeswell@gmail.com>
Co-authored-by: default avatarアマデウス <kurisusnowdeng@users.noreply.github.com>
Co-authored-by: default avatarBoxiangW <45734921+BoxiangW@users.noreply.github.com>
      da01c234
  69. 02 Dec, 2021 3 commits
  71. 29 Nov, 2021 1 commit
  73. 18 Nov, 2021 2 commits