"tests/entrypoints/llm/test_gpu_utilization.py" did not exist on "efbce85f4d375d7851a491a0126a224e25d9f91d"
- 22 Jul, 2024 1 commit
-
-
zhuwenwen authored
-
- 20 Jul, 2024 5 commits
- 17 Jul, 2024 1 commit
-
-
zhuwenwen authored
-
- 09 Jul, 2024 1 commit
-
-
huangwb authored
-
- 08 Jul, 2024 1 commit
-
-
zhuwenwen authored
-
- 06 Jul, 2024 1 commit
-
-
zhuwenwen authored
-
- 01 Jul, 2024 1 commit
-
-
zhuwenwen authored
-
- 28 Jun, 2024 1 commit
-
-
zhuwenwen authored
-
- 11 Jun, 2024 1 commit
-
-
Nick Hill authored
-
- 10 Jun, 2024 3 commits
-
-
Cyrus Leung authored
-
Cyrus Leung authored
Co-authored-by:Roger Wang <ywang@roblox.com>
-
Dipika Sikka authored
Co-authored-by:Michael Goin <michael@neuralmagic.com>
-
- 09 Jun, 2024 1 commit
-
-
bnellnm authored
-
- 08 Jun, 2024 2 commits
-
-
Michael Goin authored
-
Cheng Li authored
Bug description: With torch 2.4.0.dev20240603+cu121, cutlass_fp8_supported outputs False, and the (capability, version) before the comparison is (90, 11111111112) This PR fixes the support check for FP8 CUTLASS ( cutlass_fp8_supported) which was introduced in https://github.com/vllm-project/vllm/pull/5183.
-
- 07 Jun, 2024 5 commits
-
-
Calvinn Ng authored
Co-authored-by:team <calvinn.ng@ahrefs.com>
-
Dipika Sikka authored
Co-authored-by:
Varun Sundar Rabindranath <varunsundar08@gmail.com> Co-authored-by:
Varun Sundar Rabindranath <varun@neuralmagic.com>
-
Tyler Michael Smith authored
Switching from torch._scaled_mm to vLLM's cutlass fp8 kernels when supported as we are seeing 5-15% improvement in e2e performance on neuralmagic/Meta-Llama-3-8B-Instruct-FP8 see https://docs.google.com/spreadsheets/d/1GiAnmzyGHgZ6zL_LDSTm35Bdrt4A8AaFEurDlISYYA4/ for some quick e2e benchmarks and #5144 for comparisons across different GEMM sizes.
-
Antoni Baum authored
-
Antoni Baum authored
-
- 06 Jun, 2024 1 commit
-
-
Philipp Moritz authored
-
- 05 Jun, 2024 5 commits
-
-
Breno Faria authored
Co-authored-by:
Simon Mo <simon.mo@hey.com> Co-authored-by:
Breno Faria <breno.faria@intrafind.com>
-
Woosuk Kwon authored
-
Philipp Moritz authored
-
Cody Yu authored
-
Woosuk Kwon authored
-
- 04 Jun, 2024 2 commits
-
-
Woosuk Kwon authored
-
Toshiki Kataoka authored
-
- 03 Jun, 2024 3 commits
-
-
Breno Faria authored
-
Tyler Michael Smith authored
-
Cyrus Leung authored
-
- 02 Jun, 2024 1 commit
-
-
Divakar Verma authored
This PR enables the fused topk_softmax kernel used in moe layer for HIP
-
- 01 Jun, 2024 3 commits
-
-
chenqianfzh authored
-
Ye Cao authored
Signed-off-by:Ye Cao <caoye.cao@alibaba-inc.com>
-
Tyler Michael Smith authored
-
- 31 May, 2024 1 commit
-
-
Cody Yu authored
-
