Skip to content

GitLab

Switch branch/tag
  1. 16 Sep, 2023 2 commits
  3. 13 Sep, 2023 2 commits
  5. 11 Sep, 2023 2 commits
  7. 09 Sep, 2023 1 commit
  9. 07 Sep, 2023 1 commit
  11. 06 Sep, 2023 1 commit
  13. 05 Sep, 2023 2 commits
  15. 04 Sep, 2023 4 commits
  17. 03 Sep, 2023 2 commits
  19. 30 Aug, 2023 1 commit
  21. 29 Aug, 2023 1 commit
  23. 28 Aug, 2023 1 commit
  25. 27 Aug, 2023 1 commit
  27. 26 Aug, 2023 3 commits
  29. 25 Aug, 2023 2 commits
  31. 22 Aug, 2023 1 commit
  33. 21 Aug, 2023 1 commit
  35. 19 Aug, 2023 2 commits
  37. 18 Aug, 2023 2 commits
    • Xuechen Li's avatar
      support when num_heads is not divisible by world_size; resolves #459 (#461) · bb4cded1
      Xuechen Li authored 
      * uneql rank.

* trim.

* enable passing in number of heads for each rank.

* simplify.

* simplify.

* cleanup.

* fix col parallel.

* fix bug with row parallel.

* fit out proj.

* refac.

* fix sharding logic.

* refac sharding.

* refac.

* support multiple of.

* make fn reuseable.

* fix bug in dimensions.

* scaffold.

* test uneven heads.

* fix test by adding barrier.

* refac.

* reuse code.

* clean up.
      bb4cded1
    • Tri Dao's avatar
      [ViT] Minor fix so it runs · a81900d4
      Tri Dao authored
      a81900d4
  39. 16 Aug, 2023 1 commit
  41. 15 Aug, 2023 1 commit
    • Xuechen Li's avatar
      enable loading hf llama checkpoints for training (#446) · 0f7853c6
      Xuechen Li authored 
      * prelim.

* add hf convertion fn.

* mlp.

* change name.

* fix bug.

* inverse permute.

* change comment.

* revert style changes.

* fix.

* add doc.

* revert.

* enable load safe.

* fix safe load.

* fix import.

* fix typing-related lints.

* fix ckpt loading logic.

* make single gpu work.

* test with parallel.

* ckpt format.

* enable pretrained state dict.

* remove unused imports.

* remove unused.

* mark idea related.
      0f7853c6
  43. 13 Aug, 2023 1 commit
  45. 01 Aug, 2023 2 commits
  47. 29 Jul, 2023 1 commit
  49. 26 Jul, 2023 2 commits