- 25 May, 2024 1 commit
-
-
zhuwenwen authored
-
- 22 May, 2024 2 commits
- 21 May, 2024 1 commit
-
-
zhuwenwen authored
-
- 18 May, 2024 1 commit
-
-
zhuwenwen authored
-
- 16 May, 2024 5 commits
- 15 May, 2024 2 commits
- 12 May, 2024 5 commits
- 09 May, 2024 1 commit
-
-
zhuwenwen authored
-
- 07 May, 2024 1 commit
-
-
zhuwenwen authored
-
- 06 May, 2024 1 commit
-
-
zhuwenwen authored
-
- 05 May, 2024 2 commits
- 04 May, 2024 5 commits
-
-
DearPlanet authored
-
Simon Mo authored
-
Michael Goin authored
[Kernel] Support MoE Fp8 Checkpoints for Mixtral (Static Weights with Dynamic/Static Activations) (#4527) Follow on to #4332 to enable FP8 checkpoint loading for Mixtral and supersedes #4436. This PR enables the following checkpoint loading features for Mixtral: Supports loading fp8 checkpoints for Mixtral, such as this "nm-testing/Mixtral-8x7B-Instruct-v0.1-FP8" test model Supports static or dynamic activation quantization with static weight quantization (all per tensor) Supports different scales for each expert weight Supports Fp8 in QKV layer Notes: The Expert Gate/Router always runs at half / full precision for now. If there are different weight scales between QKV layer (for separate QKV weights), they are re-quantized using layer.weight_scale.max() so we can have a single gemm for performance.
-
SangBin Cho authored
-
Cody Yu authored
-
- 03 May, 2024 10 commits
-
-
youkaichao authored
-
Cade Daniel authored
-
Lily Liu authored
Co-authored-by:LiuXiaoxuanPKU <llilyliupku@gmail.com>
-
Sebastian Schoennenbeck authored
-
Michael Goin authored
-
SangBin Cho authored
-
youkaichao authored
-
DefTruth authored
-
Yang, Bo authored
-
youkaichao authored
-
- 02 May, 2024 3 commits
-
-
SangBin Cho authored
-
Alexei-V-Ivanov-AMD authored
Co-authored-by:simon-mo <simon.mo@hey.com>
-
Michał Moskal authored
Co-authored-by:SangBin Cho <rkooo567@gmail.com>
-
