Commits · 651cf0fec19ed1da44ad266066f86b74e5246006 · OpenDAS / vllm_cscc · GitLab

04 Apr, 2025 11 commits
- [V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906) · 651cf0fe
  Nick Hill authored Apr 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  651cf0fe
- [Bugfix] Fix default behavior/fallback for pp in v1 (#16057) · 4708f13a
  Michael Goin authored Apr 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4708f13a
- [ROCm][Bugfix] Bring back fallback to eager mode removed in #14917, but for ROCm only (#15413) · a6d042df
  Gregory Shtrasberg authored Apr 04, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a6d042df
- [ROCm][Bugfix] Use platform specific FP8 dtype (#15717) · 40a36ccf
  Gregory Shtrasberg authored Apr 04, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  40a36ccf
- [Distributed] [ROCM] Fix custom allreduce enable checks (#16010) · ef608c37
  Ilya Markov authored Apr 04, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  ef608c37
- [CPU] Change default block_size for CPU backend (#16002) · 2386803f
  Li, Jiang authored Apr 05, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  2386803f
- [Hardware][Gaudi][BugFix] fix arguments of hpu fused moe (#15945) · 0812d8dd
  liuzhenwei authored Apr 05, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  0812d8dd
- [Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (#15939) · bf7e3c51
  Jonghyun Choe authored Apr 05, 2025
```
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
```
  bf7e3c51
- [V1][Spec Decode] Avoid logging useless nan metrics (#16023) · a35a8a83
  Mark McLoughlin authored Apr 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  a35a8a83
- [TPU][V1] Remove ragged attention kernel parameter hard coding (#16041) · fadc59c0
  Chengji Yao authored Apr 04, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  fadc59c0
- [Misc] improve gguf check (#15974) · 86cbd2ee
  Reid authored Apr 04, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  86cbd2ee
03 Apr, 2025 14 commits
- [TPU] Support sliding window and logit soft capping in the paged attention kernel for TPU. (#15732) · b6be6f8d
  iefgnoix authored Apr 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b6be6f8d
- [Minor] Fused experts refactor (#15914) · 15ba07ef
  bnellnm authored Apr 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  15ba07ef
- [Neuron][kernel] Fuse kv cache into a single tensor (#15911) · d2b58ca2
  Liangfu Chen authored Apr 03, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  d2b58ca2
- [SupportsQuant] Chameleon, Chatglm, Commandr (#15952) · 82e7e19a
  Kyle Sayers authored Apr 03, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  82e7e19a
- [SupportsQuant] Bert, Blip, Blip2, Bloom (#15573) · 421c4629
  Kyle Sayers authored Apr 03, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  421c4629
- [Bugfix][V1] Fix bug from putting llm_engine.model_executor in a background process (#15367) · 463bbb18
  wwl2755 authored Apr 03, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  463bbb18
- [misc] improve error message for "Failed to infer device type" (#15994) · 5e125e74
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  5e125e74
- [ROCM][V0] PA kennel selection when no sliding window provided (#15982) · 57a810db
  Aleksandr Malyshev authored Apr 02, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
  57a810db
- [bugfix] add seed in torchrun_example.py (#15980) · 8b664706
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8b664706
- fix: better error message for get_config close #13889 (#15943) · 37bfee92
  yihong authored Apr 03, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  37bfee92
- [ROCM][KERNEL] Paged attention for V1 (#15720) · e73ff24e
  Aleksandr Malyshev authored Apr 02, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
```
  e73ff24e
- [V1][TPU] Do not compile sampling more than needed (#15883) · bd7599d3
  Nicolò Lucchesi authored Apr 03, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  bd7599d3
- [TPU] optimize the all-reduce performance (#15903) · 01b61136
  Chengji Yao authored Apr 02, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  01b61136
- [V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) · 1b84eff0
  Hyesoo Yang authored Apr 02, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
```
  1b84eff0
02 Apr, 2025 15 commits
- [V1] Support Mistral3 in V1 (#15950) · f021b979
  Michael Goin authored Apr 02, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f021b979
- [misc] instruct pytorch to use nvml-based cuda check (#15951) · 1cab43c2
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1cab43c2
- [Misc] V1 LoRA support CPU offload (#15843) · 58e234a7
  Jee Jee Li authored Apr 02, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  58e234a7
- [Model] use AutoWeightsLoader in model load_weights (#15770) · e86c414d
  rongfu.leng authored Apr 02, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  e86c414d
- [CPU][Bugfix] Using custom allreduce for CPU backend (#15934) · 550b2801
  Li, Jiang authored Apr 02, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  550b2801
- [Frontend] Implement Tool Calling with `tool_choice='required'` (#13483) · cefb9e5a
  Matthias Matt authored Apr 02, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Matt, Matthias <matthias.matt@tuwien.ac.at>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  cefb9e5a
- [Metrics] Hide deprecated metrics (#15458) · 98d7367b
  Mark McLoughlin authored Apr 02, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  98d7367b
- [Bugfix] Fix the issue where the model name is empty string, causing no... · 594a8b90
  Chauncey authored Apr 02, 2025
```
[Bugfix] Fix the issue where the model name is empty string, causing no response with the model name. (#15938)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  594a8b90
- [V1] Fix json_object support with xgrammar (#15488) · 14e53ed1
  Russell Bryant authored Apr 02, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  14e53ed1
- [core] Add tags parameter to wake_up() (#15500) · ddb94c26
  Eric Tang authored Apr 02, 2025
```
Signed-off-by: Eric <erictang000@gmail.com>
```
  ddb94c26
- [Kernel] Add more dtype support for GGUF dequantization (#15879) · 90969fb3
  LukasBluebaum authored Apr 02, 2025
```
Signed-off-by: lukas.bluebaum <lukas.bluebaum@aleph-alpha.com>
```
  90969fb3
- [Bugfix] Fix cache block size calculation for CPU MLA (#15848) · 2edc87b1
  Thien Tran authored Apr 02, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  2edc87b1
- [Misc] Replace print with logger (#15923) · cdb57015
  Chauncey authored Apr 02, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  cdb57015
- [V1][Bugfix] Fix typo in MoE TPU checking (#15927) · 0e00d40e
  Roger Wang authored Apr 01, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  0e00d40e
- [V1][Minor] Enhance SpecDecoding Metrics Log in V1 (#15902) · 274d8e88
  Woosuk Kwon authored Apr 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  274d8e88