- 12 Jun, 2024 12 commits
-
-
Michael Goin authored
-
Cyrus Leung authored
-
youkaichao authored
-
Travis Johnson authored
Signed-off-by:
Travis Johnson <tsjohnso@us.ibm.com> Co-authored-by:
Sanger Steel <sangersteel@gmail.com> Co-authored-by:
Roger Wang <ywang@roblox.com>
-
Arthur Kim authored
-
Cody Yu authored
Inspired by #5146, this PR improves FP8 quantize kernel by vectorizing data transfer to better utilize memory bandwidth. Microbenchmark shows that this improved kernel can achieve 1.0x-1.5x speedup (especially when hidden size is large). In details, we applied 3 optimizations: - Use inverted scale so that most divisions are changed to multiplications. - Unroll the loop by 4 times to improve ILP. - Use vectorized 4 to transfer data between HBM and SRAM.
-
Kevin H. Luu authored
Signed-off-by:kevin <kevin@anyscale.com>
-
Li, Jiang authored
-
Woosuk Kwon authored
-
SangBin Cho authored
-
Simon Mo authored
Revert "[CI/Build] Add `is_quant_method_supported` to control quantization test configurations" (#5463)
-
Michael Goin authored
-
- 11 Jun, 2024 18 commits
-
-
youkaichao authored
-
Nick Hill authored
Co-authored-by:Antoni Baum <antoni.baum@protonmail.com>
-
youkaichao authored
-
sasha0552 authored
-
Ali Panahi authored
-
Junichi Sato authored
-
Cade Daniel authored
-
Kuntai Du authored
Co-authored-by:simon-mo <simon.mo@hey.com>
-
Cade Daniel authored
-
SangBin Cho authored
Co-authored-by:
DarkLight1337 <tlleungac@connect.ust.hk> Co-authored-by:
-
Woosuk Kwon authored
-
Roger Wang authored
-
youkaichao authored
-
Cyrus Leung authored
-
maor-ps authored
Co-authored-by: -
Nick Hill authored
-
Kevin H. Luu authored
Signed-off-by: -
Michael Goin authored
-
- 10 Jun, 2024 10 commits
-
-
Simon Mo authored
-
Woosuk Kwon authored
-
Roger Wang authored
-
Kevin H. Luu authored
Signed-off-by: -
Kevin H. Luu authored
Signed-off-by: -
Cyrus Leung authored
-
Itay Etelis authored
-
Cyrus Leung authored
Co-authored-by: -
Cyrus Leung authored
-
youkaichao authored
-
