- 27 Dec, 2024 2 commits
-
-
Robert Shaw authored
-
Simon Mo authored
Signed-off-by:
mgoin <michael@neuralmagic.com> Co-authored-by:
robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
-
- 26 Dec, 2024 1 commit
-
-
Michael Goin authored
Signed-off-by:
simon-mo <simon.mo@hey.com> Signed-off-by:
simon-mo <xmo@berkeley.edu> Co-authored-by:
HandH1998 <1335248067@qq.com>
-
- 06 Nov, 2024 1 commit
-
-
Michael Goin authored
Signed-off-by:
-
- 28 Oct, 2024 1 commit
-
-
wangshuai09 authored
Signed-off-by:wangshuai09 <391746016@qq.com>
-
- 18 Sep, 2024 1 commit
-
-
Cyrus Leung authored
-
- 30 Aug, 2024 1 commit
-
-
Wenxiang authored
Co-authored-by:
Your Name <you@example.com> Co-authored-by:
Zeqi Lin <zelin@microsoft.com> Co-authored-by:
Zeqi Lin <Zeqi.Lin@microsoft.com>
-
- 27 Aug, 2024 1 commit
-
-
Dipika Sikka authored
Co-authored-by:ElizaWszola <eliza@neuralmagic.com>
-
- 22 Aug, 2024 2 commits
-
-
Dipika Sikka authored
-
Michael Goin authored
-
- 21 Aug, 2024 1 commit
-
-
Dipika Sikka authored
Co-authored-by:
-
- 16 Aug, 2024 2 commits
-
-
Mor Zusman authored
-
Charlie Fu authored
-
- 13 Aug, 2024 1 commit
-
-
Dipika Sikka authored
-
- 07 Aug, 2024 1 commit
-
-
Michael Goin authored
-
- 29 Jul, 2024 1 commit
-
-
Thomas Parnell authored
Signed-off-by:Thomas Parnell <tpa@zurich.ibm.com>
-
- 25 Jul, 2024 1 commit
-
-
Robert Shaw authored
Co-authored-by:
-
- 23 Jul, 2024 2 commits
-
-
Michael Goin authored
-
Michael Goin authored
-
- 20 Jul, 2024 1 commit
-
-
Robert Shaw authored
-
- 19 Jul, 2024 1 commit
-
-
Robert Shaw authored
-
- 16 Jul, 2024 1 commit
-
-
Michael Goin authored
-
- 14 Jul, 2024 1 commit
-
-
Robert Shaw authored
-
- 11 Jul, 2024 1 commit
-
-
Robert Shaw authored
Co-authored-by:
-
- 07 Jul, 2024 1 commit
-
-
Robert Shaw authored
Co-authored-by:Robert Shaw <rshaw@neuralmagic>
-
- 03 Jul, 2024 2 commits
-
-
Michael Goin authored
-
youkaichao authored
-
- 02 Jul, 2024 1 commit
-
-
Robert Shaw authored
Co-authored-by:
-
- 01 Jul, 2024 1 commit
-
-
youkaichao authored
-
- 30 Jun, 2024 1 commit
-
-
Robert Shaw authored
Co-authored-by:
-
- 28 Jun, 2024 1 commit
-
-
Robert Shaw authored
Co-authored-by:
-
- 20 Jun, 2024 1 commit
-
-
Tyler Michael Smith authored
-
- 14 Jun, 2024 1 commit
-
-
Tyler Michael Smith authored
-
- 13 Jun, 2024 2 commits
-
-
Tyler Michael Smith authored
-
Tyler Michael Smith authored
Co-authored-by:
youkaichao <youkaichao@gmail.com> Co-authored-by:
zifeitong <zifei.tong@parasail.io> Co-authored-by:
Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
-
- 08 Jun, 2024 2 commits
-
-
Michael Goin authored
-
Cheng Li authored
Bug description: With torch 2.4.0.dev20240603+cu121, cutlass_fp8_supported outputs False, and the (capability, version) before the comparison is (90, 11111111112) This PR fixes the support check for FP8 CUTLASS ( cutlass_fp8_supported) which was introduced in https://github.com/vllm-project/vllm/pull/5183.
-
- 07 Jun, 2024 1 commit
-
-
Tyler Michael Smith authored
Switching from torch._scaled_mm to vLLM's cutlass fp8 kernels when supported as we are seeing 5-15% improvement in e2e performance on neuralmagic/Meta-Llama-3-8B-Instruct-FP8 see https://docs.google.com/spreadsheets/d/1GiAnmzyGHgZ6zL_LDSTm35Bdrt4A8AaFEurDlISYYA4/ for some quick e2e benchmarks and #5144 for comparisons across different GEMM sizes.
-
- 05 Jun, 2024 1 commit
-
-
Cody Yu authored
-
- 22 May, 2024 1 commit
-
-
Cody Yu authored
The 2nd PR for #4532. This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).
-
