Commits · 8ddd1cf26ae484481126f6f9af198879b2da8a41 · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "a9e532afe2a1ae65c917ae977bf9090806e14721"

23 May, 2025 24 commits
- [v1] Redo "Support multiple KV cache groups in GPU model runner (#17945)" (#18593) · 6550114c
  Chen Zhang authored May 24, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  6550114c
- Refactor pplx init logic to make it modular (prepare for deepep) (#18200) · 6a7988c5
  youkaichao authored May 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6a7988c5
- Include private attributes in API documentation (#18614) · d4c29197
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d4c29197
- [Bugfix] Fix transformers model impl ignored for mixtral quant (#18602) · 6220f3c6
  Tristan Leclercq authored May 23, 2025
```
Signed-off-by: Tristan Leclercq <tristanleclercq@gmail.com>
```
  6220f3c6
- Replace `{func}` with mkdocs style links (#18610) · 2edb533a
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2edb533a
- [Hardware][CPU] Update intel_extension_for_pytorch 2.7.0 and move to... · 7ab056c2
  Kay Yan authored May 23, 2025
```
[Hardware][CPU] Update intel_extension_for_pytorch 2.7.0 and move to `requirements/cpu.txt`  (#18542)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
```
  7ab056c2
- [V0][Bugfix] Fix parallel sampling performance regression when guided decoding is enabled (#17731) · e493e485
  Madeesh Kannan authored May 23, 2025
```
Signed-off-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  e493e485
- [Bugfix][Model] Fix baichuan model loader for tp (#18597) · 4ce64e2d
  Mengqing Cao authored May 23, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  4ce64e2d
- Revert "[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034)" (#18600) · fbb13a2c
  Cyrus Leung authored May 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  fbb13a2c
- Migrate docs from Sphinx to MkDocs (#18145) · a1fe24d9
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a1fe24d9
- [Feature][V1]: suupports cached_tokens in response usage (#18149) · b046cf79
  Chauncey authored May 23, 2025
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
```
  b046cf79
- [Feature]Add async tensor parallelism using compilation pass (#17882) · 71ea614d
  cascade authored May 23, 2025
```
Signed-off-by: cascade812 <cascade812@outlook.com>
```
  71ea614d
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (#18034) · 4c611348
  RonaldBXu authored May 23, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  4c611348
- [Hardware] correct method signatures for HPU,ROCm,XPU (#18551) · 60cad94b
  Ning Xie authored May 23, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  60cad94b
- [Misc] Replace `cuda` hard code with `current_platform` (#16983) · 9c1baa5b
  Shanshan Shen authored May 23, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  9c1baa5b
- [Neuron] Remove bypass on EAGLEConfig and add a test (#18514) · ed5d4082
  aws-elaineyz authored May 22, 2025
```
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
```
  ed5d4082
- [Spec Decode] Make EAGLE3 draft token ID mapping optional (#18488) · 583507d1
  Benjamin Chislett authored May 22, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  583507d1
- [Bugfix] Set `KVTransferConfig.engine_id` in post_init (#18576) · e44d8ce8
  lkchen authored May 22, 2025
```
Signed-off-by: Linkun Chen <github@lkchen.net>
```
  e44d8ce8
- [BugFix] Increase TP execute_model timeout (#18558) · 93ecb813
  Nick Hill authored May 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  93ecb813
- [Misc] refactor: simplify input validation and num_requests handling in _convert_v1_inputs (#18482) · fae453f8
  CYJiang authored May 23, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  fae453f8
- Enable hybrid attention models for Transformers backend (#18494) · 4b0da7b6
  Harry Mellor authored May 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b0da7b6
- [V1][Spec Decoding] Use model_loader.get_model() to load models (#18273) · c6b636f9
  Mark McLoughlin authored May 23, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  c6b636f9
- Re-submit: Fix: Proper RGBA -> RGB conversion for PIL images. (#18569) · 04eb88dc
  Chenheli Hua authored May 22, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
  04eb88dc
- [Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization... · c32e249a
  Sanger Steel authored May 22, 2025
```
[Frontend] [Core] Add Tensorizer support for V1, LoRA adapter serialization and deserialization (#17926)
Signed-off-by: Sanger Steel <sangersteel@gmail.com>
```
  c32e249a
22 May, 2025 16 commits
- [Frontend][Bug Fix] Update llama4 pythonic jinja template and llama4_pythonic parser (#17917) · c91fe7b1
  Kai Wu authored May 22, 2025
```
Signed-off-by: Kai Wu <kaiwu@meta.com>
```
  c91fe7b1
- [V1][Spec Decode][Bugfix] Load quantize weights for EAGLE (#18290) · a04720bc
  Ekagra Ranjan authored May 22, 2025
  
  a04720bc
- [Build/CI] Fix CUDA 11.8 build (#17679) · 6e588da0
  Tyler Michael Smith authored May 22, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  6e588da0
- [Compile][Platform] Make PiecewiseBackend pluggable and extendable (#18076) · f8d2cc5f
  Mengqing Cao authored May 23, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  f8d2cc5f
- [Platform] Move platform check to right place (#18470) · 721fb9b1
  wangxiyuan authored May 23, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  721fb9b1
- [Misc] Call `ndarray.tobytes()` directly instead of `ndarray.data.tobytes()` (#18347) · 54631f82
  Lukas Geiger authored May 22, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  54631f82
- [BugFix][CPU] Fix x86 SHM distributed module initialization (#18536) · 93f71673
  Li, Jiang authored May 22, 2025
```
Signed-off-by: jiang.li <jiang1.li@intel.com>
```
  93f71673
- [Bugfix] Use random hidden states in dummy sampler run (#18543) · 4e04eceb
  Bowen Wang authored May 22, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
```
  4e04eceb
- [Bugfix] Add kwargs to RequestOutput __init__ to be forward compatible (#18513) · a35a4947
  lkchen authored May 22, 2025
```
Signed-off-by: Linkun <github@lkchen.net>
```
  a35a4947
- [Bugfix] Fix MRoPE Errors in the Qwen-VL Model When Processing Pure Text (#18526) · f6037d19
  燃 authored May 22, 2025
```
Co-authored-by: 松灵 <wpf272043@alibaba-inc.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f6037d19
- Order sequence ids + config update to support specifying custom quantization layers (#18279) · fa72f9a8
  aws-elaineyz authored May 22, 2025
```
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Tailin Pan <tailinpa@amazon.com>
Co-authored-by: Rishabh Rajesh <rishyraj@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Maxwell Goldberg <mgld@amazon.com>
Co-authored-by: Aakash Shetty <sheaak@amazon.com>
```
  fa72f9a8
- Update default neuron config for speculation (#18274) · ebed81fb
  aws-elaineyz authored May 22, 2025
```
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Aakash Shetty <sheaak@amazon.com>
```
  ebed81fb
- [Bugfix][Model] Make Olmo2Model weight loading return loaded weights (#18504) · 51797775
  Shane A authored May 21, 2025
```
Signed-off-by: Shane A <shanea@allenai.org>
```
  51797775
- [BugFix][DP] Send DP wave completion only from `dp_rank==0` (#18502) · cf5984b2
  Nick Hill authored May 21, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: kourosh hakhamaneshi <kourosh@anyscale.com>
```
  cf5984b2
- [Bugfix] Inconsistent token calculation compared to HF in llava family (#18479) · d022115c
  youngrok cha authored May 22, 2025
```
Signed-off-by: jaycha <jaycha@ncsoft.com>
```
  d022115c
- Intialize io_thread_pool attribute in the beginning. (#18331) · acb54ca8
  Rabi Mishra authored May 22, 2025
```
Signed-off-by: rabi <ramishra@redhat.com>
```
  acb54ca8