Commits · 95862f7b4de2fbe99529a4832e2d8424bc9ee247 · OpenDAS / vllm_cscc · GitLab

04 Apr, 2025 9 commits
- [Benchmark][Doc] Update throughput benchmark and README (#15998) · 95862f7b
  Ziji Shi (Steven) authored Apr 04, 2025
```
Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  95862f7b
- [Bugfix][kernels] Fix half2float conversion in gguf kernels (#15995) · 230b131b
  Isotr0py authored Apr 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  230b131b
- [Hardware][Gaudi][BugFix] fix arguments of hpu fused moe (#15945) · 0812d8dd
  liuzhenwei authored Apr 05, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  0812d8dd
- [Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (#15939) · bf7e3c51
  Jonghyun Choe authored Apr 05, 2025
```
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
```
  bf7e3c51
- [V1][Spec Decode] Avoid logging useless nan metrics (#16023) · a35a8a83
  Mark McLoughlin authored Apr 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  a35a8a83
- doc: add info for macos clang errors (#16049) · 4ef0bb1f
  yihong authored Apr 04, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  4ef0bb1f
- [TPU][V1] Remove ragged attention kernel parameter hard coding (#16041) · fadc59c0
  Chengji Yao authored Apr 04, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  fadc59c0
- [Misc] improve gguf check (#15974) · 86cbd2ee
  Reid authored Apr 04, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  86cbd2ee
- [ROCm] Tweak the benchmark script to run on ROCm (#14252) · 092475f7
  Huy Do authored Apr 03, 2025
  
  092475f7
03 Apr, 2025 21 commits
- [Bugfix] Fix function names in test_block_fp8.py (#16033) · dcc56d62
  bnellnm authored Apr 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  dcc56d62
- [TPU] Switch Test to Non-Sliding Window (#15981) · f15e70d9
  Robert Shaw authored Apr 03, 2025
```
Signed-off-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
```
  f15e70d9
- [TPU] Support sliding window and logit soft capping in the paged attention kernel for TPU. (#15732) · b6be6f8d
  iefgnoix authored Apr 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b6be6f8d
- Re-enable the AMD Testing for the passing tests. (#15586) · 03a70eac
  Alexei-V-Ivanov-AMD authored Apr 03, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
```
  03a70eac
- [Misc][Performance] Advance tpu.txt to the most recent nightly torch … (#16024) · 45b1ff7a
  yarongmu-google authored Apr 03, 2025
  
  45b1ff7a
- [Minor] Fused experts refactor (#15914) · 15ba07ef
  bnellnm authored Apr 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  15ba07ef
- [Neuron][kernel] Fuse kv cache into a single tensor (#15911) · d2b58ca2
  Liangfu Chen authored Apr 03, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  d2b58ca2
- [SupportsQuant] Chameleon, Chatglm, Commandr (#15952) · 82e7e19a
  Kyle Sayers authored Apr 03, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  82e7e19a
- [SupportsQuant] Bert, Blip, Blip2, Bloom (#15573) · 421c4629
  Kyle Sayers authored Apr 03, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  421c4629
- fix: tiny fix make format.sh excutable (#16015) · 84884cd9
  yihong authored Apr 03, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  84884cd9
- [doc] update contribution link (#15922) · a43aa183
  Reid authored Apr 03, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  a43aa183
- [Bugfix][V1] Fix bug from putting llm_engine.model_executor in a background process (#15367) · 463bbb18
  wwl2755 authored Apr 03, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  463bbb18
- [misc] improve error message for "Failed to infer device type" (#15994) · 5e125e74
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  5e125e74
- [Benchmark] Add AIMO Dataset to Benchmark (#15955) · 06f21ce7
  Ziji Shi (Steven) authored Apr 02, 2025
```
Signed-off-by: Ziji Shi <shi.ziji.sm@gmail.com>
Signed-off-by: StevenShi-23 <shi.ziji.sm@gmail.com>
```
  06f21ce7
- [ROCM][V0] PA kennel selection when no sliding window provided (#15982) · 57a810db
  Aleksandr Malyshev authored Apr 02, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
  57a810db
- [bugfix] add seed in torchrun_example.py (#15980) · 8b664706
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8b664706
- fix: better error message for get_config close #13889 (#15943) · 37bfee92
  yihong authored Apr 03, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  37bfee92
- [ROCM][KERNEL] Paged attention for V1 (#15720) · e73ff24e
  Aleksandr Malyshev authored Apr 02, 2025
```
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>
```
  e73ff24e
- [V1][TPU] Do not compile sampling more than needed (#15883) · bd7599d3
  Nicolò Lucchesi authored Apr 03, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  bd7599d3
- [TPU] optimize the all-reduce performance (#15903) · 01b61136
  Chengji Yao authored Apr 02, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  01b61136
- [V1][TPU] TPU-optimized top-p implementation (avoids scattering). (#15736) · 1b84eff0
  Hyesoo Yang authored Apr 02, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: root <root@t1v-n-822696b7-w-0.us-central2-b.c.tpu-prod-env-large-adhoc.internal>
```
  1b84eff0
02 Apr, 2025 10 commits
- Fix `huggingface-cli[hf-xet]` -> `huggingface-cli[hf_xet]` (#15969) · 55acf86b
  Harry Mellor authored Apr 03, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  55acf86b
- [V1] Support Mistral3 in V1 (#15950) · f021b979
  Michael Goin authored Apr 02, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f021b979
- [misc] instruct pytorch to use nvml-based cuda check (#15951) · 1cab43c2
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1cab43c2
- Restricted cmake to be less than version 4 as 4.x breaks the build of… (#15859) · 8bd651b3
  Nishidha authored Apr 02, 2025
```
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
```
  8bd651b3
- [Misc] V1 LoRA support CPU offload (#15843) · 58e234a7
  Jee Jee Li authored Apr 02, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  58e234a7
- [Model] use AutoWeightsLoader in model load_weights (#15770) · e86c414d
  rongfu.leng authored Apr 02, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  e86c414d
- [CPU][Bugfix] Using custom allreduce for CPU backend (#15934) · 550b2801
  Li, Jiang authored Apr 02, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  550b2801
- [Frontend] Implement Tool Calling with `tool_choice='required'` (#13483) · cefb9e5a
  Matthias Matt authored Apr 02, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Signed-off-by: Matt, Matthias <matthias.matt@tuwien.ac.at>
Co-authored-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  cefb9e5a
- [Metrics] Hide deprecated metrics (#15458) · 98d7367b
  Mark McLoughlin authored Apr 02, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  98d7367b
- [Bugfix] Fix the issue where the model name is empty string, causing no... · 594a8b90
  Chauncey authored Apr 02, 2025
```
[Bugfix] Fix the issue where the model name is empty string, causing no response with the model name. (#15938)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  594a8b90