Skip to content

GitLab

Switch branch/tag
  1. 18 Aug, 2023 1 commit
    • Xuechen Li's avatar
      support when num_heads is not divisible by world_size; resolves #459 (#461) · bb4cded1
      Xuechen Li authored 
      * uneql rank.

* trim.

* enable passing in number of heads for each rank.

* simplify.

* simplify.

* cleanup.

* fix col parallel.

* fix bug with row parallel.

* fit out proj.

* refac.

* fix sharding logic.

* refac sharding.

* refac.

* support multiple of.

* make fn reuseable.

* fix bug in dimensions.

* scaffold.

* test uneven heads.

* fix test by adding barrier.

* refac.

* reuse code.

* clean up.
      bb4cded1
  3. 17 Aug, 2023 1 commit
  5. 29 Jul, 2023 1 commit
  7. 26 Jul, 2023 1 commit
  9. 23 Jul, 2023 4 commits
  11. 30 May, 2023 1 commit
  13. 06 May, 2023 1 commit
  15. 05 May, 2023 1 commit
  17. 21 Apr, 2023 2 commits
  19. 19 Apr, 2023 1 commit
  21. 18 Apr, 2023 1 commit
  23. 13 Apr, 2023 1 commit
  25. 31 Mar, 2023 1 commit
  27. 29 Mar, 2023 1 commit
  29. 22 Mar, 2023 1 commit
  31. 23 Jan, 2023 1 commit
  33. 19 Jan, 2023 1 commit
  35. 18 Jan, 2023 1 commit
  37. 16 Jan, 2023 1 commit
  39. 15 Jan, 2023 1 commit
  41. 07 Jan, 2023 2 commits
  43. 01 Jan, 2023 2 commits
  45. 28 Dec, 2022 1 commit
  47. 27 Dec, 2022 2 commits
  49. 23 Dec, 2022 1 commit
  51. 21 Dec, 2022 1 commit
  53. 19 Dec, 2022 1 commit
  55. 23 Nov, 2022 1 commit
  57. 14 Nov, 2022 1 commit