- 07 Jun, 2024 8 commits
-
-
limingshu authored
-
Antoni Baum authored
-
Tyler Michael Smith authored
Switching from torch._scaled_mm to vLLM's cutlass fp8 kernels when supported as we are seeing 5-15% improvement in e2e performance on neuralmagic/Meta-Llama-3-8B-Instruct-FP8 see https://docs.google.com/spreadsheets/d/1GiAnmzyGHgZ6zL_LDSTm35Bdrt4A8AaFEurDlISYYA4/ for some quick e2e benchmarks and #5144 for comparisons across different GEMM sizes.
-
youkaichao authored
-
Itay Etelis authored
-
Jie Fu (傅杰) authored
-
Antoni Baum authored
-
Antoni Baum authored
-
- 06 Jun, 2024 4 commits
-
-
Matthew Goldey authored
-
Philipp Moritz authored
-
liuyhwangyh authored
Co-authored-by:mulin.lyh <mulin.lyh@taobao.com>
-
Cyrus Leung authored
-
- 05 Jun, 2024 13 commits
-
-
Breno Faria authored
Co-authored-by:
Simon Mo <simon.mo@hey.com> Co-authored-by:
Breno Faria <breno.faria@intrafind.com>
-
Woosuk Kwon authored
-
Alex Wu authored
-
Michael Goin authored
-
Nick Hill authored
-
Nick Hill authored
-
Philipp Moritz authored
-
DriverSong authored
Co-authored-by:qiujiawei9 <qiujiawei9@jd.com>
-
Cody Yu authored
-
tomeras91 authored
[Frontend] OpenAI API server: Add `add_special_tokens` to ChatCompletionRequest (default False) (#5278)
-
Woosuk Kwon authored
-
Woosuk Kwon authored
-
zifeitong authored
-
- 04 Jun, 2024 3 commits
-
-
zifeitong authored
-
Woosuk Kwon authored
-
Toshiki Kataoka authored
-
- 03 Jun, 2024 5 commits
-
-
Breno Faria authored
-
Kaiyang Chen authored
-
Tyler Michael Smith authored
-
Antoni Baum authored
-
Cyrus Leung authored
-
- 02 Jun, 2024 2 commits
-
-
Divakar Verma authored
This PR enables the fused topk_softmax kernel used in moe layer for HIP
-
Avinash Raj authored
-
- 01 Jun, 2024 5 commits
-
-
Robert Shaw authored
Co-authored-by:mgoin <michael@neuralmagic.com>
-
Zhuohan Li authored
-
chenqianfzh authored
-
Nadav Shmayovits authored
-
Ye Cao authored
Signed-off-by:Ye Cao <caoye.cao@alibaba-inc.com>
-
