Commits · a9944aabfa0eb0f133cf869b3ed5defb44ed7d33 · OpenDAS / vllm_cscc · GitLab

15 May, 2025 4 commits
- fix: typos (#18151) · a9944aab
  omahs authored May 15, 2025
```
Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>
```
  a9944aab
- [Bugfix] Fix FusedMoEPrepareAndFinalize for cuda-disalike backends (#18178) · 70f8b967
  Mengqing Cao authored May 15, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  70f8b967
- [Model] Allow the use of sliding window in Qwen2 (#17772) · dd2a9459
  inkcherry authored May 15, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  dd2a9459
- Update deprecated type hinting in `models` (#18132) · 26d04193
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  26d04193
14 May, 2025 14 commits
- Add support for loading torchao models with `AOPerModuleConfig` (#17826) · 79747367
  Jerry Zhang authored May 14, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  79747367
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b
  Ekagra Ranjan authored May 14, 2025
```
[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  418d2f8b
- [Model] GritLM supports other attention backends (#18109) · d62a076e
  Cyrus Leung authored May 14, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d62a076e
- [FEAT] [ROCm]: Add AITER CK 2 Stages MoE support (#17110) · 612c2edb
  TJian authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  612c2edb
- [Bugfix] Fix QKVCrossParallelLinear::sync_weight_attrs for PyTorch compile (#17844) · 38fe728d
  Andrzej Kotłowski authored May 14, 2025
```
Signed-off-by: Andrzej Kotłowski <akotlowski@habana.ai>
```
  38fe728d
- [Model] Add packed_modules_mapping for Qwen3-MOE (#18118) · 63dc3426
  Jee Jee Li authored May 14, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  63dc3426
- [New Model]: support GTE NewModel (#17986) · 63ad6222
  wang.yuqi authored May 14, 2025
  
  63ad6222
- [BugFix][AMD] Compatible patch for AITER lib after 04/20 (#17912) · 4f8b3732
  qli88 authored May 14, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  4f8b3732
- [FEAT] [ROCm] [V1]: Add AITER biased group topk for DeepSeekV3 (#17955) · 2d912fb6
  vllmellm authored May 14, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  2d912fb6
- [Bugfix] Fix FP8 Marlin MoE and enable for compressed-tensors models (#18026) · 9a2a6357
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  9a2a6357
- [core][distributed] add ep group and all2all interface (#18077) · 6266c57b
  youkaichao authored May 14, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6266c57b
- [Hardware/NVIDIA/Modelopt] Fix modelopt forward method for v1 torch.compile (#18101) · 65f0f74b
  Pavani Majety authored May 13, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  65f0f74b
- [FEAT] [ROCm]: Add AITER Block-Scaled GEMM Feature (#14968) · 40de1ef4
  vllmellm authored May 14, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  40de1ef4
13 May, 2025 6 commits
- Update deprecated type hinting in `model_executor/layers` (#18056) · 6223dd81
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6223dd81
- [Bugfix] Fix marlin moe fallback logic for llama4 (#18042) · ea6ae8cb
  Michael Goin authored May 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ea6ae8cb
- [Bugfix] Fixes for new marlin moe usage (#18017) · 1df491c5
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  1df491c5
- [Misc] Slight spelling modification (#18039) · c06af9a9
  Jee Jee Li authored May 13, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c06af9a9
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
- Use NVFP4 Marlin for CompressedTensorsW4A16Fp4 (#18000) · 307939f2
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  307939f2
12 May, 2025 4 commits
- Update some more deprecated type hinting (#17998) · 9d7ea9db
  Harry Mellor authored May 13, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  9d7ea9db
- [Model] Support MiMo-7B inference with MTP (#17433) · acee8f48
  bwshen-mi authored May 13, 2025
```
Signed-off-by: wp-alpha <wangpeng66@xiaomi.com>
Co-authored-by: wangpeng66 <wangpeng66@xiaomi.com>
```
  acee8f48
- Fix FBGEMM integration (#18002) · f065de4e
  Michael Goin authored May 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f065de4e
- [Model] Broadcast Ovis2 implementation to fit Ovis1.6 (#17861) · 021c16c7
  Isotr0py authored May 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  021c16c7
11 May, 2025 4 commits
- [BugFix] [ROCm]: Bugfix and handle addition case of input for `rocm_aiter_rms_norm` (#17857) · a810b5b0
  TJian authored May 11, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  a810b5b0
- [New Model]: nomic-embed-text-v2-moe (#17785) · e4b87133
  wang.yuqi authored May 11, 2025
  
  e4b87133
- [Misc] Add compressed-tensors NVFP4A16 emulation support (#17914) · cd3edfc9
  Dipika Sikka authored May 11, 2025
```
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
```
  cd3edfc9
- [Kernel] fp4 marlin kernel (#17687) · d74e5f37
  Jinzhen Lin authored May 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d74e5f37
09 May, 2025 7 commits
- [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) · 0c0fdae8
  Pavani Majety authored May 09, 2025
  
  0c0fdae8
- Update CT WNA16MarlinMoE integration (#16666) · 22481fbf
  Michael Goin authored May 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  22481fbf
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394
- [FEAT][ROCm]: Support AITER MLA on V1 Engine (#17523) · 3c9396a6
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>
```
  3c9396a6
08 May, 2025 1 commit
- Add cutlass support for blackwell fp8 blockwise gemm (#14383) · 376786fa
  Shu Wang authored May 08, 2025
```
Signed-off-by: Shu Wang <shuw@nvidia.com>
```
  376786fa