Commits · 9d72daf4ced05a5fec1ad8ea2914a39296f402da · OpenDAS / vllm_cscc · GitLab

24 Mar, 2025 20 commits
- [V1][Perf] Simpler request output queues (#15156) · 9d72daf4
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  9d72daf4
- [Doc] Update docs on handling OOM (#15357) · 6dd55af6
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  6dd55af6
- [DOC] Add Kubernetes deployment guide with CPUs (#14865) · 3eb08ed9
  Yuan Tang authored Mar 24, 2025
  
  3eb08ed9
- [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral (#12303) · 5eeadc26
  liuzhenwei authored Mar 25, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  5eeadc26
- [V1] Aggregate chunked prompt logprobs in model runner (#14875) · 3aee6573
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3aee6573
- [MISC] Refine no available block debug msg (#15076) · 9cc64514
  Yi Liu authored Mar 25, 2025
```
Signed-off-by: Yi Liu <yiliu4@habana.ai>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Yi Liu <yiliu4@habana.ai>
```
  9cc64514
- [V1][Minor] fix comments (#15392) · 0893567d
  Chen1022 authored Mar 24, 2025
```
Signed-off-by: chenjincong <chenjincong@baidu.com>
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: chenjincong <chenjincong@baidu.com>
```
  0893567d
- [Core] Don't force uppercase for VLLM_LOGGING_LEVEL (#15306) · 8abe69b4
  Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  8abe69b4
- [Core] Integrate `fastsafetensors` loader for loading model weights (#10647) · 761702fd
  Manish Sethi authored Mar 24, 2025
```
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
```
  761702fd
- [distributed] fix dp group (#15355) · 9606d572
  youkaichao authored Mar 24, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  9606d572
- [Bugfix] Fix chat template loading (#15143) · cbcdf2c6
  Cyrus Leung authored Mar 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  cbcdf2c6
- Fix zmq IPv6 URL format error (#15341) · 038de04d
  Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  038de04d
- [Kernel] allow non-contiguous input for marlin kernel (#14658) · 6b3cc75b
  Jinzhen Lin authored Mar 24, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  6b3cc75b
- Revert "[CI/Build] Use uv python for docker rather than ppa:deadsnakess/ppa (#13569)" (#15377) · 7ffcccfa
  Simon Mo authored Mar 24, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  7ffcccfa
- [Misc] Update guided decoding logs to debug (#15310) · cc8accfd
  sfbemerk authored Mar 24, 2025
```
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
```
  cc8accfd
- [Bugfix][V1] Avoid importing PreTrainedModel (#15366) · 948ab03e
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Mar 24, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  948ab03e
- [Misc] Remove ignore_reinit_error for ray.init() (#15373) · 5797fb97
  Rui Qiao authored Mar 24, 2025
  
  5797fb97
- [Misc] Upgrade BNB version (#15183) · 3892e58a
  Jee Jee Li authored Mar 24, 2025
  
  3892e58a
- Fix non-contiguous input passed to Marlin kernel (#15319) · d20e2611
  Qubitium-ModelCloud authored Mar 24, 2025
  
  d20e2611
- [Fix] [torch.compile] Improve UUID system for custom passes (#15249) · f622dbcf
  Luka Govedič authored Mar 23, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  f622dbcf
23 Mar, 2025 12 commits
- [V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f
  Lucas Wilkinson authored Mar 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dccf535f
- [Misc][Doc] Add note regarding loading `generation_config` by default (#15281) · 9c5c81b0
  Roger Wang authored Mar 23, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  9c5c81b0
- [Frontend] Support tool calling and reasoning parser (#14511) · d6cd59f1
  Robin authored Mar 24, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
  d6cd59f1
- [V1][Spec Decode] Use better defaults for N-gram (#15358) · bc8ed3c4
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  bc8ed3c4
- [V1][Spec Decode] Respect prompt_lookup_max (#15348) · b9bd76ca
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b9bd76ca
- [Bugfix] consider related env vars for torch.compiled cache hash (#14953) · 6ebaf9ac
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <31974251+DefTruth@users.noreply.github.com>
```
  6ebaf9ac
- [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) · f90d34b4
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  f90d34b4
- [ci/build] fix broken tests in LLM.collective_rpc (#15350) · f68cce8e
  youkaichao authored Mar 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  f68cce8e
- [ci/build] update torch nightly version for GH200 (#15135) · 09b6a955
  youkaichao authored Mar 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  09b6a955
- [V1][Usage] Refactor speculative decoding configuration and tests (#14434) · 50c9636d
  shangmingc authored Mar 23, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50c9636d
- Fix v1 supported oracle for worker-cls and worker-extension-cls (#15324) · 0661cfef
  hijkzzz authored Mar 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  0661cfef
- [doc] Add back previous news (#15331) · a827aa81
  Chen Zhang authored Mar 23, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  a827aa81
22 Mar, 2025 8 commits
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
- [BugFix][Typing] Fix Imprecise Type Annotations (#15208) · dd861b99
  Wang Ran (汪然) authored Mar 23, 2025
```
Signed-off-by: Wang Ran (汪然) <wrran@outlook.com>
```
  dd861b99
- [V1] Add `disable-any-whitespace` option support for xgrammar (#15316) · eb63ea1e
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  eb63ea1e
- [Model] Support Tele-FLM Model (#15023) · 2f4bd358
  Naitong Yu authored Mar 22, 2025
```
Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>
```
  2f4bd358
- [Bugfix] LoRA V0 - Fix case where `max_num_seqs` is between cudagraph capture sizes (#15308) · 8a8b30ea
  Varun Sundar Rabindranath authored Mar 22, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8a8b30ea
- [Bugfix] Fix torch.compile raise FileNotFoundError (#15278) · 2fa0e139
  Jee Jee Li authored Mar 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  2fa0e139
- [Doc] add load_format items in docs (#14804) · 1c2bec0f
  wwl2755 authored Mar 22, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  1c2bec0f
- [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) · ec870fba
  TJian authored Mar 22, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  ec870fba