Commits · c48334d405d6e0e106b4b6d922df8ff8ce58b64d · OpenDAS / vllm_cscc · GitLab

26 Apr, 2025 18 commits
- [Hardware][Intel-Gaudi] Update hpu-extension and update bucketing system for HPU device (#17186) · c48334d4
  Agata Dobrzyniewicz authored Apr 26, 2025
```
Signed-off-by: Agata Dobrzyniewicz <adobrzyniewicz@habana.ai>
```
  c48334d4
- [Bugfix] Fix standard models tests (#17217) · 909fdaf1
  Cyrus Leung authored Apr 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  909fdaf1
- [Bugfix] Fix missing int type for `-n` in multi-image example (#17223) · 8c1c926d
  Isotr0py authored Apr 26, 2025
  
  8c1c926d
- [Core] Remove prompt string from engine core data structures (#17214) · df6f3ce8
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  df6f3ce8
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [BugFix] Avoid race conditions in zero-copy tensor transmission (#17203) · b07bf83c
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b07bf83c
- [V1][Metrics] Allow V1 AsyncLLM to use custom logger (#14661) · 53e8cf53
  Zijing Liu authored Apr 25, 2025
```
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  53e8cf53
- [ROCm][Misc] Follow-ups for Skinny Gemms on ROCm. (#17011) · 54271bb7
  Charlie Fu authored Apr 26, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  54271bb7
- Allocate kv_cache with stride order (#16605) · 9e96f56e
  Shu Wang authored Apr 26, 2025
```
Signed-off-by: shuw <shuw@nvidia.com>
```
  9e96f56e
- [Minor][Models] Fix Return Types of Llama & Eagle (#17220) · b2789112
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b2789112
- [Doc] Minor fix for the vLLM TPU setup page (#17206) · 7bd0c774
  yarongmu-google authored Apr 25, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  7bd0c774
- [Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213) · 1cf0719e
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1cf0719e
- [doc] add Anything LLM integration (#17216) · 537d5ee0
  Reid authored Apr 26, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  537d5ee0
- [MISC][AMD] Add unused annotation to rocm kernel file (#17097) · c8e5be35
  Lu Fang authored Apr 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  c8e5be35
- [Bugfix] [pytorch] Patch AOTAutogradCache._get_shape_env (#17142) · a6e72e1e
  James Wu authored Apr 25, 2025
```
Signed-off-by: James Wu <jjwu@meta.com>
```
  a6e72e1e
- [v1] [P/D] Adding LMCache KV connector for v1 (#16625) · 5e83a727
  Yihua Cheng authored Apr 25, 2025
  
  5e83a727
- [AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary (#17215) · 68af5f6c
  rasmith authored Apr 25, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  68af5f6c
- [Bugfix] gemma[2,3] interleaved attention when sliding window is disabled (#17180) · 8de2901f
  Chen Zhang authored Apr 26, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  8de2901f
25 Apr, 2025 22 commits
- [Misc] Refine ray_serve_deepseek example (#17204) · c53e0730
  Rui Qiao authored Apr 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c53e0730
- [V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6
  Benjamin Chislett authored Apr 25, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>
```
  a0e619e6
- [BugFix][Frontend] Fix `LLM.chat()` tokenization (#16081) · 70116459
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  70116459
- Fix Python packaging edge cases (#17159) · 65e262b9
  Christian Heimes authored Apr 26, 2025
```
Signed-off-by: Christian Heimes <christian@python.org>
```
  65e262b9
- [Bugfix] Fix hybrid model tests (#17182) · 43faa046
  Cyrus Leung authored Apr 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  43faa046
- [V1] Move usage stats to worker and start logging TPU hardware (#16211) · 48cb2109
  Daniel Li authored Apr 25, 2025
  
  48cb2109
- [Security] Use safe serialization and fix zmq setup for mooncake pipe (#17192) · a5450f11
  Russell Bryant authored Apr 25, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  a5450f11
- [Misc] Inline Molmo requirements (#17190) · 9d98ab5e
  Cyrus Leung authored Apr 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9d98ab5e
- [doc] update wrong hf model links (#17184) · df5c8795
  Reid authored Apr 26, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  df5c8795
- Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
  Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  423e9f1c
- Bump Transformers to 4.51.3 (#17116) · 0bd7f8fc
  Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0bd7f8fc
- [Bugfix] Fix Mistral ChatCompletionRequest Body Exception (#16769) · d5615af9
  Jasmond L authored Apr 25, 2025
```
Signed-off-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d5615af9
- [Bugfix] Fix mistral model tests (#17181) · 19dcc02a
  Cyrus Leung authored Apr 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  19dcc02a
- [Doc] Move todo out of beam search docstring (#17183) · 7feae92c
  Alex Brooks authored Apr 25, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  7feae92c
- [Doc] Add two links to disagg_prefill.md (#17168) · f851b842
  Michael Yao authored Apr 25, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  f851b842
- Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 (#17158) · fc966e9c
  Lu Fang authored Apr 25, 2025
  
  fc966e9c
- [Doc] Add headings to improve gptqmodel.md (#17164) · ef19e67d
  Michael Yao authored Apr 25, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  ef19e67d
- [Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3
  rasmith authored Apr 25, 2025
```
[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>
```
  a41351f3
- [Bugfix] remove fallback in guided_json (int range, patterns) (#16725) · 6aae216b
  Sangyeon Cho authored Apr 25, 2025
```
Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>
```
  6aae216b
- [Perf]Optimize rotary_emb implementation to use Triton operator for improved... · b22980a1
  yexin(叶鑫) authored Apr 25, 2025
```
[Perf]Optimize rotary_emb implementation to use Triton operator for improved inference performance (#16457)
Signed-off-by: cynthieye <yexin93@qq.com>
Co-authored-by: MagnetoWang <magnetowang@outlook.com>
```
  b22980a1
- [Misc] Benchmark Serving Script Support Appending Results (#17028) · 881f7358
  Lucas Wilkinson authored Apr 25, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  881f7358
- [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton (#15099) · 2f540455
  Mengqing Cao authored Apr 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  2f540455