Commits · f2c3f66d59f9e38aa94985b54f370219222e7bd1 · OpenDAS / vllm_cscc

31 May, 2025 2 commits
- [FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825) · 0f5e0d56
  vllmellm authored May 31, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  0f5e0d56
- [Frontend] Add rerank support to run_batch endpoint (#16278) · dff80b0e
  Pooya Davoodi authored May 31, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  dff80b0e
30 May, 2025 6 commits
- Tool parser regex timeout handling (#18960) · 1dab4d57
  Will Eaton authored May 30, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  1dab4d57
- [VLM] Add PP support and fix GPTQ inference for Ovis models (#18958) · 5a864163
  Isotr0py authored May 31, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5a864163
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
- [Feature] minicpm eagle support (#18943) · e1fadf11
  Shawn Huang authored May 30, 2025
```
Signed-off-by: huangyuxiang03 <huangyx0321@gmail.com>
Co-authored-by: huangyuxiang03 <huangyx0321@gmail.com>
```
  e1fadf11
- [Bugfix][TPU] Fix tpu model runner testcase failure (#18810) · fba02e3b
  Carol Zheng authored May 30, 2025
```
Signed-off-by: Carol Zheng <cazheng@google.com>
```
  fba02e3b
- [Misc] Update type annotation for rotary embedding `base` (#18914) · 1aa2f81b
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1aa2f81b
29 May, 2025 7 commits
- [TPU] remove transpose ops in moe kernel (#18923) · a1cc9f33
  Chengji Yao authored May 29, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a1cc9f33
- [BugFix] Make DP work with connector-delayed new requests (#18559) · d1d61f33
  Nick Hill authored May 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Will Eaton <weaton@redhat.com>
```
  d1d61f33
- [V1] Allocate kv_cache with stride order for V1 (#18775) · 32ce3cf7
  Nicolò Lucchesi authored May 29, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  32ce3cf7
- [Deprecation] Disallow pos-args other than `model` when initializing `LLM` (#18802) · c2903403
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c2903403
- [Bugfix] Fix the failing gte embedding test (#18720) · c9479b29
  Isotr0py authored May 29, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c9479b29
- [Neuron] Add multi-LoRA support for Neuron. (#18284) · 972eddf7
  Satyajith Chilappagari authored May 29, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
```
  972eddf7
- Add ability to use CUDAGraphs with use_inductor=False (#17345) · 26b4fa45
  Richard Zou authored May 28, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  26b4fa45
28 May, 2025 9 commits

[Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py... · 269d9017

Hongxia Yang authored May 28, 2025


[Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix (#18100)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

269d9017

[Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend (#15655) · 643622ba

Akshat Tripathi authored May 28, 2025


Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Signed-off-by: xihajun <junfan@krai.ai>
Signed-off-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Signed-off-by: Jorge de Freitas <jorge@krai.ai>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: xihajun <junfan@krai.ai>
Co-authored-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Co-authored-by: Jorge de Freitas <jorge@krai.ai>

643622ba

[V1][Metrics] Remove metrics that were deprecated in 0.8 (#18837) · 0e98964e
Mark McLoughlin authored May 28, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
0e98964e
[Core] Add Lora Support to Beam Search (#18346) · 321331b8
Alex Brooks authored May 28, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
321331b8

[Frontend] add run batch to CLI (#18804) · 435fa954

Reid authored May 28, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

435fa954

Enable Pydantic mypy checks and convert configs to Pydantic dataclasses (#17599) · 4c2b38ce
Harry Mellor authored May 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
4c2b38ce
[CI] improve embed testing (#18747) · de65fc8e
wang.yuqi authored May 28, 2025

de65fc8e
[Bugfix][FailingTest]Fix test_model_load_with_params.py (#18758) · b78f844a
Rabi Mishra authored May 28, 2025
```
Signed-off-by: rabi <ramishra@redhat.com>
```
b78f844a
[Bugfix] Fix nomic max_model_len (#18755) · 3e9ce609
wang.yuqi authored May 28, 2025

3e9ce609

27 May, 2025 4 commits
- [Neuron] Support quantization on neuron (#18283) · e0cbad4e
  Satyajith Chilappagari authored May 27, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
```
  e0cbad4e
- [Bugfix] Mistral tool calling when content is list (#18729) · 58738772
  Michael Goin authored May 27, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  58738772
- [V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010) · 06a03380
  Mark McLoughlin authored May 27, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  06a03380
- [V1][Quantization] Add CUDA graph compatible v1 GGUF support (#18646) · 1f1b1bc0
  Isotr0py authored May 27, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1f1b1bc0
26 May, 2025 5 commits
- [Doc] Move examples and further reorganize user guide (#18666) · 82e2339b
  Cyrus Leung authored May 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  82e2339b
- [CI] fix dump_input for str type (#18697) · 5a2c76cb
  Ning Xie authored May 26, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  5a2c76cb
- [CI/Build] Replace `math.isclose` with `pytest.approx` (#18703) · 38b13dfe
  Cyrus Leung authored May 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  38b13dfe
- [CI] add missing argument (#18694) · 4ea62c0e
  Ning Xie authored May 26, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  4ea62c0e
- [CI/Build][Doc] Update `gte-Qwen2-1.5B-instruct` usage (#18683) · fba06427
  Cyrus Leung authored May 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  fba06427
25 May, 2025 4 commits
- [Bugfix] Fix profiling dummy data for Pixtral (#18677) · 57fd13a7
  Cyrus Leung authored May 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  57fd13a7
- Speed up the `kernels/quantization/` tests (#18669) · 63934543
  Michael Goin authored May 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  63934543
- [VLM] Initialize video input support for InternVL models (#18499) · 75f81750
  Isotr0py authored May 25, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  75f81750
- [Misc][ModelScope] Change to use runtime VLLM_USE_MODELSCOPE (#18655) · 6ab681bc
  Mengqing Cao authored May 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  6ab681bc
24 May, 2025 3 commits
- [V1][Spec Decode] Support multi-layer eagle draft model (#18030) · c1e4a405
  qizixi authored May 24, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c1e4a405
- [Model] Add support for Qwen2.5-Omni-7B-AWQ (Qwen2_5OmniForConditionalGeneration) (#18647) · a8593205
  Yuanhao WU authored May 24, 2025
  
  a8593205
- [V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424) · d55e446d
  qizixi authored May 23, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  d55e446d