Commits · afd0da2186c1d58fb48e138df0a2f548612b5d7d · OpenDAS / vllm_cscc

31 Jan, 2025 1 commit

[Attention] MLA decode optimizations (#12528) · cabaf4ef

Lucas Wilkinson authored Jan 31, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Alexander Matveev <59768536+alexm-neuralmagic@users.noreply.github.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>

cabaf4ef

15 Jan, 2025 2 commits

[Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6

kewang-xlnx authored Jan 16, 2025


Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

de0526f6

[Platform] move current_memory_usage() into platform (#11369) · 9ddac563
Shanshan Shen authored Jan 15, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
9ddac563

13 Jan, 2025 3 commits
- [Platform] Move get_punica_wrapper() function to Platform (#11516) · a7d59688
  Shanshan Shen authored Jan 13, 2025
```
Signed-off-by: Shanshan Shen <467638484@qq.com>
```
  a7d59688
- [platform] add device_control env var (#12009) · 458e63a2
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  458e63a2
- [platform] add ray_device_key (#11948) · 89ce62a3
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  89ce62a3
09 Jan, 2025 1 commit

[platform] Allow platform specify attention backend (#11609) · 405eb8e3

wangxiyuan authored Jan 09, 2025


Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

405eb8e3

03 Jan, 2025 1 commit
- update qwen2 and mixtral layout · 57adffa2
  zhuwenwen authored Jan 03, 2025
  
  57adffa2
27 Dec, 2024 1 commit
- [Platform] Move model arch check to platform (#11503) · 6c6f7fe8
  Mengqing Cao authored Dec 27, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  6c6f7fe8
17 Dec, 2024 1 commit
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
09 Dec, 2024 2 commits
- [ROCm][bugfix] scpecilative decoding worker class (#11035) · b63ba848
  Gregory Shtrasberg authored Dec 09, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b63ba848
- [Platform] Move `async output` check to platform (#10768) · aea2fc38
  wangxiyuan authored Dec 10, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  aea2fc38
03 Dec, 2024 1 commit
- add large_gpu_test · 0dc55ec0
  zhuwenwen authored Dec 03, 2024
  
  0dc55ec0
29 Nov, 2024 1 commit
- [platform] Add verify_quantization in platform. (#10757) · 661175bc
  wangxiyuan authored Nov 29, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  661175bc
22 Nov, 2024 2 commits
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
21 Nov, 2024 2 commits
- [Platforms] Add `device_type` in `Platform` (#10508) · 9d827170
  Mengqing Cao authored Nov 21, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  9d827170
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
19 Nov, 2024 1 commit
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
29 Sep, 2024 1 commit
- [CI/Build] Add test decorator for minimum GPU memory (#8925) · 26a68d5d
  Cyrus Leung authored Sep 29, 2024
  
  26a68d5d
25 Sep, 2024 1 commit
- remove reason of unsupported · a087cda8
  zhuwenwen authored Sep 25, 2024
  
  a087cda8
18 Sep, 2024 1 commit
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
28 Aug, 2024 1 commit
- [hardware][rocm] allow rocm to override default env var (#7926) · bc6e42a9
  youkaichao authored Aug 27, 2024
  
  bc6e42a9
17 Aug, 2024 1 commit
- [misc] use nvml to get consistent device name (#7582) · eed020f6
  youkaichao authored Aug 16, 2024
  
  eed020f6
03 Jul, 2024 1 commit
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee