Commits · 1d0c9d6b2d74e6884c6c63f4856ac0a2cddd0508 · OpenDAS / vllm_cscc

05 May, 2025 1 commit
- [Kernel] some optimizations for dense marlin and moe marlin (#16850) · 1d0c9d6b
  Jinzhen Lin authored May 06, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  1d0c9d6b
03 May, 2025 2 commits
- [Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558) · e3d0a1d1
  rasmith authored May 02, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  e3d0a1d1
- fix typo in logging (#17605) · 9b103a1d
  Eric Hartford authored May 02, 2025
  
  9b103a1d
02 May, 2025 3 commits
- Support W8A8 INT8 MoE for compressed-tensors (#16745) · 868c546d
  Michael Goin authored May 02, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  868c546d
- Check if bitblas is installed during support check (#17572) · b4003d11
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b4003d11
- [Bugfix] Disable gptq_bitblas for <SM80 to fix GPTQ on V100/T4 (#17541) · 24aebae1
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  24aebae1
01 May, 2025 1 commit
- [Bugfix] Temporarily disable gptq_bitblas on ROCm (#17411) · 1144a8ef
  NaLan ZeYu authored May 01, 2025
```
Signed-off-by: Yan Cangang <nalanzeyu@gmail.com>
```
  1144a8ef
30 Apr, 2025 2 commits
- [Fix] Support passing args to logger (#17425) · da4e7687
  Aaron Pham authored Apr 30, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  da4e7687
- Improve configs - `ModelConfig` (#17130) · 13698db6
  Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  13698db6
29 Apr, 2025 1 commit
- Fix: Spelling of inference (#17387) · 0ed27ef6
  a2q1p authored Apr 30, 2025
  
  0ed27ef6
28 Apr, 2025 3 commits
- [Bugfix] Fix moe weight losing all extra attrs after `process_weights_after_loading`. (#16854) · ed246203
  Charlie Fu authored Apr 28, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  ed246203
- Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) · c7941cca
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c7941cca
- Make name of `compressed-tensors` quant method consistent across vLLM (#17255) · b6dd32aa
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b6dd32aa
26 Apr, 2025 2 commits
- [ROCm][Misc] Follow-ups for Skinny Gemms on ROCm. (#17011) · 54271bb7
  Charlie Fu authored Apr 26, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  54271bb7
- [AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215) · 68af5f6c
  rasmith authored Apr 25, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  68af5f6c
25 Apr, 2025 2 commits

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

[FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) · eef36472

vllmellm authored Apr 25, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

eef36472

22 Apr, 2025 4 commits

[Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
8d32dc60
[Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
188b7f9b

[Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7

Varun Sundar Rabindranath authored Apr 21, 2025


Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>

7b8a2ab7

[ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) · 5b794cae

kliuae authored Apr 22, 2025


Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>

5b794cae

15 Apr, 2025 2 commits
- [Misc] Update `compressed-tensors` WNA16 to support zero-points (#14211) · 54a66e5f
  Dipika Sikka authored Apr 15, 2025
  
  54a66e5f
- [Kernel] moe wna16 marlin kernel (#14447) · d06ba4ed
  Jinzhen Lin authored Apr 15, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  d06ba4ed
13 Apr, 2025 1 commit
- Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537) · d085a440
  Michael Goin authored Apr 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d085a440
11 Apr, 2025 2 commits
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee
- [Bugfix] Fix bugs of running Quark quantized models (#16236) · 9e90c9f7
  chaow-amd authored Apr 11, 2025
```
Signed-off-by: chaow <chaow@amd.com>
```
  9e90c9f7
10 Apr, 2025 1 commit
- [Kernel] Use moe_wna16 kernel for compressed tensors wna16 moe models (#16038) · c70cf0fe
  Michael Goin authored Apr 10, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  c70cf0fe
09 Apr, 2025 1 commit
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
08 Apr, 2025 2 commits
- [Bugfix] Handle `process_weights_after_loading` for `QKVCrossParallelLinear` (#15328) · 40b4284f
  Isotr0py authored Apr 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  40b4284f
- [ROCm][Bugfix][FP8] Make fp8 quant respect fused modules mapping (#16031) · 21802c4b
  Michael Goin authored Apr 07, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  21802c4b
07 Apr, 2025 2 commits

Torchao (#14231) · 652907b3
Driss Guessous authored Apr 07, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
652907b3

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

55dcce91

06 Apr, 2025 1 commit
- [Bugfix] fix use_atomic_add support of marlin kernel when using v1 engine (#15946) · 2fa66ef7
  Jinzhen Lin authored Apr 06, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  2fa66ef7
02 Apr, 2025 1 commit
- [Kernel] Add more dtype support for GGUF dequantization (#15879) · 90969fb3
  LukasBluebaum authored Apr 02, 2025
```
Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>
```
  90969fb3
01 Apr, 2025 1 commit
- Add option to use DeepGemm contiguous grouped gemm kernel for fused MoE operations. (#13932) · e59ca942
  bnellnm authored Apr 01, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  e59ca942
29 Mar, 2025 2 commits
- [FEAT] [ROCm] Add AITER int8 scaled gemm kernel (#15433) · 4965ec42
  TJian authored Mar 29, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  4965ec42
- [TPU][V1][Bugfix] Fix w8a8 recompiilation with GSM8K (#15714) · da461f3c
  Nicolò Lucchesi authored Mar 29, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  da461f3c
28 Mar, 2025 2 commits
- [Quantization][FP8] Adding support for fp8 gemm layer input in fp8 (#14578) · 4d0ec372
  Gregory Shtrasberg authored Mar 27, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  4d0ec372
- [Quantization][V1] BitsAndBytes support V1 (#15611) · 726efc6a
  Jee Jee Li authored Mar 28, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  726efc6a
27 Mar, 2025 1 commit

[Quantization] Fp8 Channelwise Dynamic Per Token GroupedGEMM (#15587) · 43ed4143

Robert Shaw authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>

43ed4143