- 14 Jun, 2024 1 commit
-
-
Simon Mo authored
-
- 13 Jun, 2024 11 commits
-
-
Antoni Baum authored
-
Tyler Michael Smith authored
-
Antoni Baum authored
-
Cody Yu authored
Co-authored-by:Philipp Moritz <pcmoritz@gmail.com>
-
Tyler Michael Smith authored
Co-authored-by:
Michael Goin <michael@neuralmagic.com> Co-authored-by:
youkaichao <youkaichao@gmail.com> Co-authored-by:
zifeitong <zifei.tong@parasail.io> Co-authored-by:
Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
-
Cyrus Leung authored
Co-authored-by:Roger Wang <ywang@roblox.com>
-
Cyrus Leung authored
-
Li, Jiang authored
Co-authored-by:Jianan Gu <jianan.gu@intel.com>
-
wenyujin333 authored
Tune Qwen2-57B-A14B configs based on #4921 Throughput Performance command: python benchmarks/benchmark_throughput.py --model=Qwen/Qwen2-57B-A14B-Instruct --input-len 1000 --output-len 50 -tp 2 A100 GPU benchmark no config w/ PR tp=2 10.53 requests/s, 11058.17 tokens/s 12.47 requests/s, 13088.57 tokens/s tp=4 17.77 requests/s, 18662.95 tokens/s 20.20 requests/s, 21212.32 tokens/s
-
Dipika Sikka authored
Co-authored-by:Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
-
youkaichao authored
[Core][Distributed] add coordinator to reduce code duplication in tp and pp (#5293)
-
- 12 Jun, 2024 8 commits
-
-
Isotr0py authored
-
Michael Goin authored
-
Michael Goin authored
-
youkaichao authored
-
Travis Johnson authored
Signed-off-by:
Travis Johnson <tsjohnso@us.ibm.com> Co-authored-by:
Sanger Steel <sangersteel@gmail.com> Co-authored-by:
Roger Wang <ywang@roblox.com>
-
Arthur Kim authored
-
Li, Jiang authored
-
Woosuk Kwon authored
-
- 11 Jun, 2024 9 commits
-
-
Nick Hill authored
Co-authored-by:Antoni Baum <antoni.baum@protonmail.com>
-
youkaichao authored
-
sasha0552 authored
-
Ali Panahi authored
-
Junichi Sato authored
-
Woosuk Kwon authored
-
Cyrus Leung authored
-
maor-ps authored
Co-authored-by:DarkLight1337 <tlleungac@connect.ust.hk>
-
Nick Hill authored
-
- 10 Jun, 2024 8 commits
-
-
Simon Mo authored
-
Cyrus Leung authored
-
Itay Etelis authored
-
Cyrus Leung authored
Co-authored-by:Roger Wang <ywang@roblox.com>
-
Cyrus Leung authored
-
youkaichao authored
-
Roger Wang authored
-
Dipika Sikka authored
Co-authored-by:Michael Goin <michael@neuralmagic.com>
-
- 09 Jun, 2024 3 commits
-
-
Bla_ckB authored
-
bnellnm authored
-
youkaichao authored
[Core][CUDA Graph] add output buffer for cudagraph to reduce memory footprint (#5074)
-