Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "0ddfaa0b37399ebe634760b584f25f6fed70a0b7"
Switch branch/tag
  1. 07 Feb, 2024 3 commits
    • Hongxin Liu's avatar
      [moe] fix mixtral checkpoint io (#5314) · b60be18d
      Hongxin Liu authored
      b60be18d
    • Hongxin Liu's avatar
      [moe] support mixtral (#5309) · da39d21b
      Hongxin Liu authored 
      * [moe] add mixtral block for single expert

* [moe] mixtral block fwd support uneven ep

* [moe] mixtral block bwd support uneven ep

* [moe] add mixtral moe layer

* [moe] simplify replace

* [meo] support save sharded mixtral

* [meo] support load sharded mixtral

* [meo] support save sharded optim

* [meo] integrate moe manager into plug

* [meo] fix optimizer load

* [meo] fix mixtral layer
      da39d21b
    • Xuanlei Zhao's avatar
      [moe] init mixtral impl · 7d8e0338
      Xuanlei Zhao authored
      7d8e0338