Commits · cabaf4eff3c7df30d785769d5a0a1fa1a1c48a8a · OpenDAS / vllm_cscc · GitLab

31 Jan, 2025 1 commit

[Attention] MLA decode optimizations (#12528) · cabaf4ef

Lucas Wilkinson authored Jan 31, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

cabaf4ef

29 Jan, 2025 1 commit
- [V1] Improve Error Message for Unsupported Config (#12535) · 5f671cb4
  Robert Shaw authored Jan 28, 2025
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  5f671cb4
28 Jan, 2025 1 commit
- Update `pre-commit` hooks (#12475) · 823ab796
  Harry Mellor authored Jan 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  823ab796
22 Jan, 2025 2 commits
- [Bugfix] Fix HPU multiprocessing executor (#12167) · 96f6a759
  Konrad Zawora authored Jan 22, 2025
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  96f6a759
- [ci/lint] Add back default arg for pre-commit (#12279) · 64ea24d0
  Kevin H. Luu authored Jan 21, 2025
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  64ea24d0
21 Jan, 2025 1 commit
- [Platform] improve platforms getattr (#12264) · c6461280
  Mengqing Cao authored Jan 21, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  c6461280
20 Jan, 2025 1 commit
- fix: update platform detection for M-series arm based MacBook processors (#12227) · af69a6ad
  Işık authored Jan 20, 2025
```
Signed-off-by: isikhi <huseyin.isik000@gmail.com>
```
  af69a6ad
18 Jan, 2025 1 commit
- [misc] fix cross-node TP (#12166) · 2b835032
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2b835032
15 Jan, 2025 3 commits
- [Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6
  kewang-xlnx authored Jan 16, 2025
```
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  de0526f6
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
- [Platform] move current_memory_usage() into platform (#11369) · 9ddac563
  Shanshan Shen authored Jan 15, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
  9ddac563
14 Jan, 2025 1 commit
- [Bugfix][Kernel] Give unique name to BlockSparseFlashAttention (#12040) · a2d2acb4
  Chen Zhang authored Jan 14, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a2d2acb4
13 Jan, 2025 3 commits
- [Platform] Move get_punica_wrapper() function to Platform (#11516) · a7d59688
  Shanshan Shen authored Jan 13, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
  a7d59688
- [platform] add device_control env var (#12009) · 458e63a2
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  458e63a2
- [platform] add ray_device_key (#11948) · 89ce62a3
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  89ce62a3
10 Jan, 2025 3 commits
- [platform] support custom torch.compile backend key (#11318) · 20410b2f
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  20410b2f
- [platform] support pytorch custom op pluggable (#11328) · ef725fea
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  ef725fea
- [BUGFIX] Fix `UnspecifiedPlatform` package name (#11916) · 61af6332
  Kunshang Ji authored Jan 10, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  61af6332
09 Jan, 2025 1 commit

[platform] Allow platform specify attention backend (#11609) · 405eb8e3

wangxiyuan authored Jan 09, 2025


Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

405eb8e3

08 Jan, 2025 1 commit
- [TPU][Quantization] TPU `W8A8` (#11785) · 56fe4c29
  Robert Shaw authored Jan 08, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  56fe4c29
06 Jan, 2025 1 commit
- [Doc][2/N] Reorganize Models and Usage sections (#11755) · ee77fdb5
  Cyrus Leung authored Jan 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ee77fdb5
30 Dec, 2024 1 commit
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
27 Dec, 2024 1 commit
- [Platform] Move model arch check to platform (#11503) · 6c6f7fe8
  Mengqing Cao authored Dec 27, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  6c6f7fe8
23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
17 Dec, 2024 1 commit
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
13 Dec, 2024 1 commit

[Refactor]A simple device-related refactor (#11163) · d1fa714c

Chenguang Li authored Dec 13, 2024


Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>

d1fa714c

10 Dec, 2024 2 commits
- [Bugfix] cuda error running llama 3.2 (#11047) · 82c73fd5
  Gene Der Su authored Dec 09, 2024
  
  82c73fd5
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
09 Dec, 2024 2 commits
- [ROCm][bugfix] scpecilative decoding worker class (#11035) · b63ba848
  Gregory Shtrasberg authored Dec 09, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b63ba848
- [Platform] Move `async output` check to platform (#10768) · aea2fc38
  wangxiyuan authored Dec 10, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  aea2fc38
05 Dec, 2024 1 commit
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
03 Dec, 2024 1 commit
- [Bugfix] Only require XGrammar on x86 (#10865) · 7090c27b
  Michael Goin authored Dec 03, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  7090c27b
29 Nov, 2024 1 commit
- [platform] Add verify_quantization in platform. (#10757) · 661175bc
  wangxiyuan authored Nov 29, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  661175bc
27 Nov, 2024 1 commit
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 1 commit
- [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) · f5792c7c
  Conroy Cheers authored Nov 27, 2024
```
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
```
  f5792c7c
23 Nov, 2024 2 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- [Platforms] Refactor openvino code (#10573) · 86a44fb8
  JiHuazhong authored Nov 23, 2024
```
Signed-off-by: statelesshz <hzji210@gmail.com>
```
  86a44fb8
22 Nov, 2024 2 commits
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
21 Nov, 2024 1 commit
- [misc] improve error message (#10553) · cf656f5a
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cf656f5a