Commits · 6bd1dd9d2625ac0ac81e8b5c740aa855399990b9 · OpenDAS / vllm_cscc · GitLab

06 Mar, 2025 17 commits
- [Kernel] [V1] Improved performance for V1 Triton (ROCm) backend (#14152) · 6bd1dd9d
  Thomas Parnell authored Mar 06, 2025
  
  6bd1dd9d
- [Doc] Correct beam_search using in generative_models.md (#14363) · 4f27044a
  Irina Yuryeva authored Mar 06, 2025
  
  4f27044a
- [Doc] Update reasoning with stream example to use OpenAI library (#14077) · 0ddc991f
  Yanyi Liu authored Mar 06, 2025
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
```
  0ddc991f
- [Frontend][Docs] Transcription API streaming (#13301) · fa82b938
  Nicolò Lucchesi authored Mar 06, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  fa82b938
- [Core] Optimizing cross-attention `QKVParallelLinear` computation (#12325) · 69ff99fd
  Nicolò Lucchesi authored Mar 06, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
```
  69ff99fd
- [V1][VLM][Pixtral-HF] Support Pixtral-HF on V1 (#14275) · 5d802522
  lkchen authored Mar 06, 2025
```
Signed-off-by: Linkun Chen <github@lkchen.net>
```
  5d802522
- [Model] Update Paligemma multimodal processing with PromptUpdate (#14015) · 17699280
  kYLe authored Mar 06, 2025
```
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  17699280
- [Hardware] Update the flash attn tag to support Blackwell (#14244) · ed6ea065
  Pavani Majety authored Mar 05, 2025
  
  ed6ea065
- [Bugfix][CI] ALiBi test case in xformers multi_query_kv_attention (#11301) · 5ee10e99
  Nicolò Lucchesi authored Mar 06, 2025
  
  5ee10e99
- [V1] LoRA - Enable more V1 tests (#14315) · 3dbd2d81
  Varun Sundar Rabindranath authored Mar 05, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  3dbd2d81
- [Bugfix][Structured Output] Support outlines engine with reasoning outputs for DeepSeek R1 (#14114) · f5f7f00c
  Ce Gao authored Mar 06, 2025
  
  f5f7f00c
- [misc] Mention `ray list nodes` command to troubleshoot ray issues (#14318) · abcc61e0
  Rui Qiao authored Mar 05, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  abcc61e0
- [BugFix] MLA + V1, illegal memory access and accuracy issues (#14253) · f6bb18fd
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f6bb18fd
- [Build] Add UV_HTTP_TIMEOUT to avoid timeout during installation (#13850) · 71eaf896
  Yuan Tang authored Mar 05, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  71eaf896
- Add benchmark for DeepGEMM and vLLM Block FP8 Dense GEMM (#13917) · ca100c90
  Michael Goin authored Mar 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ca100c90
- [CI/Build] Use spawn multiprocessing mode for V1 test pipeline (#14243) · ffad9439
  Russell Bryant authored Mar 05, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ffad9439
- [BugFix] Fix prefix caching V0 MLA (#14255) · 4dacaa4a
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Ying Zhong <zhongyingmatrix@gmail.com>
```
  4dacaa4a
05 Mar, 2025 23 commits
- [Bugfix] Remove num_tokens_across_dp (#14302) · a7ea35aa
  Tyler Michael Smith authored Mar 05, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  a7ea35aa
- [Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due... · 1e3e76b6
  pyc96 authored Mar 05, 2025
```
[Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due to shape mismatch (#14237)
Signed-off-by: pyc96 <pychen96@gmail.com>
```
  1e3e76b6
- [V1][Easy] Add empty allowed_token_ids in the v1 sampler test (#14308) · 53ea6ad8
  Lu Fang authored Mar 05, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  53ea6ad8
- [misc] Add FlashMLA as a new option of VLLM_ATTENTION_BACKEND env (#14267) · 1b7624bf
  Serena authored Mar 06, 2025
  
  1b7624bf
- [V1][BugFix] Fix for mixed top_k batch (#14301) · ac60dc7f
  Nick Hill authored Mar 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Ye Cao <caoye.cao@alibaba-inc.com>
```
  ac60dc7f
- Deprecate `best_of` Sampling Parameter in anticipation for vLLM V1 (#13997) · a4f1ee35
  Vincent authored Mar 05, 2025
```
Signed-off-by: vincent-4 <vincentzhongy+githubvincent4@gmail.com>
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a4f1ee35
- [V1][Minor] Remove obsolete FIXME comment (#14304) · a32c8669
  Nick Hill authored Mar 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  a32c8669
- [Docs] Add Meta Slides (#14297) · ca2ca8de
  Simon Mo authored Mar 05, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  ca2ca8de
- [Bugfix] Fix broken vision language example (#14292) · f71b00a1
  Isotr0py authored Mar 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f71b00a1
- prefix_caching.md: Fixed typo (#14293) · 8f808cf8
  DaividFrank authored Mar 05, 2025
```
Signed-off-by: Daivid Savernin-Frenk <daivid.frank@TurboNext.ai>
```
  8f808cf8
- [Misc] Add Qwen2MoeForCausalLM moe tuning support (#14276) · 7bab4bb0
  Jee Jee Li authored Mar 05, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7bab4bb0
- [LoRA] Remove linear hack outside transformers backend (#14177) · e17e4488
  Isotr0py authored Mar 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e17e4488
- [V1][Frontend] Add Testing For V1 Runtime Parameters (#14159) · 257e200a
  Robert Shaw authored Mar 05, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  257e200a
- Small update for external_launcher backend docs (#14288) · 47d4a7e0
  Zhe Zhang authored Mar 05, 2025
  
  47d4a7e0
- [Doc] [3/N] Refer code examples for common cases in dev multimodal processor (#14278) · 7f89a594
  Cyrus Leung authored Mar 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7f89a594
- [Doc] Update nginx guide: remove privileged from vllm container run and add target GPU ID (#14217) · 961644e6
  Iacopo Poli authored Mar 05, 2025
```
Signed-off-by: Iacopo Poli <iacopo@lighton.ai>
```
  961644e6
- [Bugfix][V1] Fix allowed_token_ids for v1 Sampler (#14169) · 8d6cd32b
  Lu Fang authored Mar 05, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  8d6cd32b
- [Misc][V1] Avoid using `envs.VLLM_USE_V1` in mm processing (#14256) · ec79b67c
  Roger Wang authored Mar 04, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  ec79b67c
- [Frontend] Allow return_tokens_as_token_ids to be passed as a request param (#14066) · 32985bed
  Benjamin Chislett authored Mar 05, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  32985bed
- Temporarily disable test_awq_gemm_opcheck (#14251) · dae9ec46
  Michael Goin authored Mar 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  dae9ec46
- [platforms] improve rocm debugging info (#14257) · 6eaf9302
  youkaichao authored Mar 05, 2025
  
  6eaf9302
- [V1] EP/TP MoE + DP Attention (#13931) · 72c62eae
  Tyler Michael Smith authored Mar 05, 2025
  
  72c62eae
- [Model] New model support for Phi-4-multimodal-instruct (#14119) · 0a995d54
  Congcong Chen authored Mar 04, 2025
  
  0a995d54