Skip to content

GitLab

Switch branch/tag
  1. 18 Aug, 2023 1 commit
    • Xuechen Li's avatar
      support when num_heads is not divisible by world_size; resolves #459 (#461) · bb4cded1
      Xuechen Li authored 
      * uneql rank.

* trim.

* enable passing in number of heads for each rank.

* simplify.

* simplify.

* cleanup.

* fix col parallel.

* fix bug with row parallel.

* fit out proj.

* refac.

* fix sharding logic.

* refac sharding.

* refac.

* support multiple of.

* make fn reuseable.

* fix bug in dimensions.

* scaffold.

* test uneven heads.

* fix test by adding barrier.

* refac.

* reuse code.

* clean up.
      bb4cded1
  3. 17 Aug, 2023 1 commit
  5. 10 Aug, 2023 1 commit
  7. 26 Jul, 2023 2 commits
  9. 23 Jul, 2023 2 commits
  11. 22 Jul, 2023 1 commit
  13. 21 Jul, 2023 1 commit
  15. 17 Jul, 2023 1 commit
  17. 02 Jul, 2023 1 commit
  19. 02 Jun, 2023 1 commit
  21. 30 May, 2023 1 commit
  23. 06 May, 2023 1 commit
  25. 21 Apr, 2023 2 commits
  27. 19 Apr, 2023 1 commit
  29. 18 Apr, 2023 2 commits
  31. 14 Apr, 2023 1 commit
  33. 13 Apr, 2023 1 commit
  35. 31 Mar, 2023 1 commit
  37. 29 Mar, 2023 1 commit
  39. 22 Mar, 2023 1 commit
  41. 18 Jan, 2023 1 commit
  43. 16 Jan, 2023 3 commits
  45. 15 Jan, 2023 1 commit
  47. 08 Jan, 2023 1 commit
  49. 07 Jan, 2023 3 commits
  51. 04 Jan, 2023 1 commit
  53. 02 Jan, 2023 1 commit
  55. 01 Jan, 2023 1 commit
  57. 28 Dec, 2022 2 commits
  59. 25 Dec, 2022 2 commits