Commits · 458e63a2c6b18e7febfa30cecb59461f96fbe324 · OpenDAS / vllm_cscc

13 Jan, 2025 2 commits
- [platform] add device_control env var (#12009) · 458e63a2
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  458e63a2
- [platform] add ray_device_key (#11948) · 89ce62a3
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  89ce62a3
10 Jan, 2025 3 commits
- [platform] support custom torch.compile backend key (#11318) · 20410b2f
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  20410b2f
- [platform] support pytorch custom op pluggable (#11328) · ef725fea
  wangxiyuan authored Jan 10, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  ef725fea
- [BUGFIX] Fix `UnspecifiedPlatform` package name (#11916) · 61af6332
  Kunshang Ji authored Jan 10, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
```
  61af6332
09 Jan, 2025 1 commit

[platform] Allow platform specify attention backend (#11609) · 405eb8e3

wangxiyuan authored Jan 09, 2025


Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

405eb8e3

08 Jan, 2025 1 commit
- [TPU][Quantization] TPU `W8A8` (#11785) · 56fe4c29
  Robert Shaw authored Jan 08, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  56fe4c29
06 Jan, 2025 1 commit
- [Doc][2/N] Reorganize Models and Usage sections (#11755) · ee77fdb5
  Cyrus Leung authored Jan 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ee77fdb5
30 Dec, 2024 1 commit
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
27 Dec, 2024 1 commit
- [Platform] Move model arch check to platform (#11503) · 6c6f7fe8
  Mengqing Cao authored Dec 27, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  6c6f7fe8
23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
17 Dec, 2024 1 commit
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
13 Dec, 2024 1 commit

[Refactor]A simple device-related refactor (#11163) · d1fa714c

Chenguang Li authored Dec 13, 2024


Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>

d1fa714c

10 Dec, 2024 2 commits
- [Bugfix] cuda error running llama 3.2 (#11047) · 82c73fd5
  Gene Der Su authored Dec 09, 2024
  
  82c73fd5
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
09 Dec, 2024 2 commits
- [ROCm][bugfix] scpecilative decoding worker class (#11035) · b63ba848
  Gregory Shtrasberg authored Dec 09, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b63ba848
- [Platform] Move `async output` check to platform (#10768) · aea2fc38
  wangxiyuan authored Dec 10, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  aea2fc38
05 Dec, 2024 1 commit
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
03 Dec, 2024 1 commit
- [Bugfix] Only require XGrammar on x86 (#10865) · 7090c27b
  Michael Goin authored Dec 03, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  7090c27b
29 Nov, 2024 1 commit
- [platform] Add verify_quantization in platform. (#10757) · 661175bc
  wangxiyuan authored Nov 29, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  661175bc
27 Nov, 2024 1 commit
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
26 Nov, 2024 1 commit
- [Hardware][NVIDIA] Add non-NVML CUDA mode for Jetson (#9735) · f5792c7c
  Conroy Cheers authored Nov 27, 2024
```
Signed-off-by: Conroy Cheers <conroy@corncheese.org>
```
  f5792c7c
23 Nov, 2024 2 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- [Platforms] Refactor openvino code (#10573) · 86a44fb8
  JiHuazhong authored Nov 23, 2024
```
Signed-off-by: statelesshz <hzji210@gmail.com>
```
  86a44fb8
22 Nov, 2024 2 commits
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
21 Nov, 2024 4 commits
- [misc] improve error message (#10553) · cf656f5a
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cf656f5a
- [platforms] improve error message for unspecified platforms (#10520) · aaddce5d
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  aaddce5d
- [Platforms] Add `device_type` in `Platform` (#10508) · 9d827170
  Mengqing Cao authored Nov 21, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  9d827170
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
20 Nov, 2024 3 commits
- [platforms] restore xpu check for parallel config (#10479) · 772a6673
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  772a6673
- [Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355) · 63f1fde2
  Li, Jiang authored Nov 20, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  63f1fde2
- [Platforms] Refactor xpu code (#10468) · d5b28447
  Mengqing Cao authored Nov 20, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  d5b28447
19 Nov, 2024 2 commits
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 1 commit
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
17 Nov, 2024 2 commits
- [platforms] refactor cpu code (#10402) · 8d74b5ae
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8d74b5ae
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
06 Nov, 2024 1 commit

[Hardware][Intel-Gaudi] Add Intel Gaudi (HPU) inference backend (#6143) · a02a50e6

Konrad Zawora authored Nov 06, 2024


Signed-off-by: yuwenzho <yuwen.zhou@intel.com>
Signed-off-by: Chendi.Xue <chendi.xue@intel.com>
Signed-off-by: Bob Zhu <bob.zhu@intel.com>
Signed-off-by: zehao-intel <zehao.huang@intel.com>
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Sanju C Sudhakaran <scsudhakaran@habana.ai>
Co-authored-by: Michal Adamczyk <madamczyk@habana.ai>
Co-authored-by: Marceli Fylcek <mfylcek@habana.ai>
Co-authored-by: Himangshu Lahkar <49579433+hlahkar@users.noreply.github.com>
Co-authored-by: Vivek Goel <vgoel@habana.ai>
Co-authored-by: yuwenzho <yuwen.zhou@intel.com>
Co-authored-by: Dominika Olszewska <dolszewska@habana.ai>
Co-authored-by: barak goldberg <149692267+bgoldberg-habana@users.noreply.github.com>
Co-authored-by: Michal Szutenberg <37601244+szutenberg@users.noreply.github.com>
Co-authored-by: Jan Kaniecki <jkaniecki@habana.ai>
Co-authored-by: Agata Dobrzynie...

a02a50e6

04 Nov, 2024 1 commit
- [Bugfix][OpenVINO] Fix circular reference #9939 (#9974) · ccb5376a
  Mengqing Cao authored Nov 04, 2024
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
  ccb5376a