Commits · 8820821b59121192205077d0cdc56490e215b9fb · OpenDAS / vllm_cscc

26 May, 2025 2 commits

[Misc] Fixed the abnormally high TTFT issue in the PD disaggregation example (#18644) · 8820821b

AlexZhao authored May 26, 2025


Signed-off-by: zhaohaidao <zhaohaidao2008@hotmail.com>
Signed-off-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>
Co-authored-by: zhaohaiyuan <zhaohaiyuan@xiaohongshu.com>

8820821b

[CI/Build][Doc] Update `gte-Qwen2-1.5B-instruct` usage (#18683) · fba06427

Cyrus Leung authored May 26, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>

fba06427

25 May, 2025 16 commits
- [Core][Multimodal] Convert PIL Image to array without data copy when hashing (#18682) · 6071e989
  Lukas Geiger authored May 25, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  6071e989
- [Bugfix] Fix profiling dummy data for Pixtral (#18677) · 57fd13a7
  Cyrus Leung authored May 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  57fd13a7
- [Misc] small improve (#18680) · 3a886bd5
  Reid authored May 25, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  3a886bd5
- [CI/build] fix no regex (#18676) · 35be8fad
  Reid authored May 25, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  35be8fad
- [Bugfix] Fix cpu usage and cache hit stats reporting on cpu environment (#18674) · f2faac74
  Yuqi Zhang authored May 25, 2025
```
Signed-off-by: zzzyq <zhangyuqi94@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  f2faac74
- [doc] improve readability (#18675) · 279f8545
  Reid authored May 25, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  279f8545
- [doc] fix broken links (#18671) · 624b77a2
  Reid authored May 25, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  624b77a2
- [Misc] Reduce logs on startup (#18649) · 503f8487
  Cyrus Leung authored May 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  503f8487
- [BUGFIX] catch subclass first for try...except (#18672) · 44073a7a
  Ning Xie authored May 25, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  44073a7a
- Speed up the `kernels/quantization/` tests (#18669) · 63934543
  Michael Goin authored May 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  63934543
- [VLM] Initialize video input support for InternVL models (#18499) · 75f81750
  Isotr0py authored May 25, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  75f81750
- [Misc][ModelScope] Change to use runtime VLLM_USE_MODELSCOPE (#18655) · 6ab681bc
  Mengqing Cao authored May 25, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  6ab681bc
- [Misc]Replace `cuda` hard code with `current_platform` in Ray (#14668) · cebc22f3
  Chenguang Li authored May 25, 2025
```
Signed-off-by: noemotiovon <757486878@qq.com>
```
  cebc22f3
- [MISC] correct signature for LoaderFunction (#18670) · 6c6dcd86
  Ning Xie authored May 25, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  6c6dcd86
- [V1] Fix _pickle.PicklingError: Can't pickle <class... · 7891fdf0
  Seiji Eicher authored May 24, 2025
```
[V1] Fix _pickle.PicklingError: Can't pickle <class 'transformers_modules.deepseek-ai.DeepSeek-V2-Lite... (#18640)
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  7891fdf0
- [BugFix][Spec Decode] Improve Prefix Caching Logic in Speculative Decoding (#18668) · 6825d9a9
  Woosuk Kwon authored May 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6825d9a9
24 May, 2025 18 commits
- [CI/Build] fix permission denied issue (#18645) · b554ab73
  Reid authored May 25, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  b554ab73
- fix(regression): clone from reference items (#18662) · 9ea7f1ab
  Aaron Pham authored May 24, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  9ea7f1ab
- [CI] enforce import regex instead of re (#18665) · 2807271c
  Aaron Pham authored May 24, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  2807271c
- [BugFix] Fix import error for fused_moe (#18642) · b9018a3f
  wangxiyuan authored May 24, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  b9018a3f
- [MISC] typo fix and clean import (#18664) · 4ceafb62
  Ning Xie authored May 24, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  4ceafb62
- [CI/Build] `chmod +x` to `cleanup_pr_body.sh` (#18650) · 2e670578
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2e670578
- [Doc] Reorganize user guide (#18661) · 1cb194a0
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1cb194a0
- [Model] use AutoWeightsLoader for gpt2 (#18625) · 2cd4d58d
  ztang2370 authored May 24, 2025
```
Signed-off-by: zt2370 <ztang2370@gmail.com>
```
  2cd4d58d
- [Doc] Add community links (#18657) · 6d166a8d
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6d166a8d
- [Doc] Fix indentation problems in V0 Paged Attention docs (#18659) · ef1dd687
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ef1dd687
- [MISC][pre-commit] Add pre-commit check for triton import (#17716) · e77dc4ba
  Mengqing Cao authored May 24, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  e77dc4ba
- [Doc] Update README links, mark external links (#18635) · 07458a51
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  07458a51
- [V1][Spec Decode] Support multi-layer eagle draft model (#18030) · c1e4a405
  qizixi authored May 24, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  c1e4a405
- [Model] Add support for Qwen2.5-Omni-7B-AWQ (Qwen2_5OmniForConditionalGeneration) (#18647) · a8593205
  Yuanhao WU authored May 24, 2025
  
  a8593205
- [Frontend] improve vllm serve --help display (#18643) · 441dc63a
  Reid authored May 24, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  441dc63a
- [V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424) · d55e446d
  qizixi authored May 23, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  d55e446d
- FIX MOE issue in AutoRound format (#18586) · ec82c3e3
  Wenhua Cheng authored May 24, 2025
```
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>
```
  ec82c3e3
- config.py: Clarify that only local GGUF checkpoints are supported. (#18623) · 45ab403a
  Mathieu Borderé authored May 24, 2025
```
Signed-off-by: Mathieu Bordere <mathieu@letmetweakit.com>
```
  45ab403a
23 May, 2025 4 commits
- [Bugfix][Nixl] Fix Preemption Bug (#18631) · 2b10ba74
  Robert Shaw authored May 23, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  2b10ba74
- [Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454) · 4fc1bf81
  Feng XiaoLong authored May 24, 2025
```
Signed-off-by: Crucifixion-Fxl <xmufxl@gmail.com>
Co-authored-by: Crucifixion-Fxl <xmufxl@gmail.com>
```
  4fc1bf81
- [ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control... · f2036734
  Pavani Majety authored May 23, 2025
```
[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control blockscale tensor allocation (#18160)
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  f2036734
- [Doc] Update references to doc files (#18637) · 7d921649
  Cyrus Leung authored May 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7d921649