Commits · 04421dff8a05a81bbd5ed19545175066163a4082 · OpenDAS / vllm_cscc · GitLab

10 Mar, 2025 17 commits
- [V1] Prevent xgrammar from breaking TPU support (#14575) · 04421dff
  Russell Bryant authored Mar 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  04421dff
- Fix typo in benchmark_serving_structured_output.py (#14566) · 432d6dad
  Russell Bryant authored Mar 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  432d6dad
- [V1] LoRA - Add triton kernels for V1 (#13096) · 5ff0d325
  Varun Sundar Rabindranath authored Mar 10, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  5ff0d325
- [Minor] Update the tqdm bar for parallel sampling (#14571) · 0967110e
  Woosuk Kwon authored Mar 10, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  0967110e
- [Perf] Improve MLA on V1 (#14540) · fb0acb6c
  Simon Mo authored Mar 10, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  fb0acb6c
- [Bugfix][v1] fixed llava-hf/llava-1.5-7b-hf is broken on V1 (#14554) · 92b0ce2a
  Chauncey authored Mar 11, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  92b0ce2a
- [Docs] Make installation URLs nicer (#14556) · bc2d4473
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bc2d4473
- Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3b352a2f
- [V1][Bugfix] Fix handing of `second_per_grid_ts` for Qwen2-VL & Qwen2.5-VL (#14548) · dea985ae
  Roger Wang authored Mar 10, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  dea985ae
- Correct capitalisation: `Github` -> `GitHub` (#14561) · 39be3035
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  39be3035
- [Doc] Update PaliGemma note to a warning (#14565) · 001a9c7b
  Cyrus Leung authored Mar 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  001a9c7b
- [Kernel] Add more dtype support for GGUF kernels (#14043) · 89cdaa83
  Szymon Ożóg authored Mar 10, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
```
  89cdaa83
- [Frontend] support image embeds (#13955) · b0746fae
  Chauncey authored Mar 10, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  b0746fae
- [Docs] Mention `model_impl` arg when explaining Transformers fallback (#14552) · 60a98b2d
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  60a98b2d
- [Misc] Add log information for handle_process_request. (#14130) · 460f553a
  Chauncey authored Mar 10, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  460f553a
- [Feature] Consolidate performance benchmark datasets (#14036) · 1253b157
  Jennifer Zhao authored Mar 10, 2025
```
Signed-off-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Jennifer Zhao <7443418+JenZhao@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  1253b157
- [Bugfix] Wrong requirements path - rocm (#14527) · dc74613f
  Martin Hoyer authored Mar 10, 2025
```
Signed-off-by: Martin Hoyer <mhoyer@redhat.com>
```
  dc74613f
09 Mar, 2025 8 commits
- [Misc] Ensure out-of-tree quantization method recognize by cli args (#14328) · a21076ed
  Yanyi Liu authored Mar 09, 2025
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
```
  a21076ed
- [Hardware][TPU] Fix the recompiling issue in logits processor after warmup (#14510) · 212007b1
  Chengji Yao authored Mar 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  212007b1
- [Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB quantization work (#14498) · fb16eea4
  Isotr0py authored Mar 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  fb16eea4
- [Bugfix] Fix tqdm progress bar when SamplingParams.n > 1 (#12428) · 73ae0b44
  Yuchen Yan authored Mar 09, 2025
```
Signed-off-by: Yuchen Yan <740987012@qq.com>
```
  73ae0b44
- [Feat] Support chunked prefill for LMCache connector (#14505) · 6d7f0377
  Jiayi Yao authored Mar 08, 2025
```
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
```
  6d7f0377
- [V1][TPU] Remove unnecessary padding for running on TPU. (#14467) · 10f75527
  iefgnoix authored Mar 08, 2025
  
  10f75527
- [Attention] Default to FlashMLA backend for MLA (#14451) · b0d54194
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  b0d54194
- Revert "[V1][Core] Fix memory issue with logits & sampling" (#14504) · 5f0b53c6
  Robert Shaw authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  5f0b53c6
08 Mar, 2025 15 commits
- [V1] Support bad_words in sampler (#13376) · eb8b5eb1
  22quinn authored Mar 08, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  eb8b5eb1
- [Misc] Upgrade to Python 3.9 typing for additional directories (#14492) · 95132900
  Cyrus Leung authored Mar 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  95132900
- Update CODEOWNERS for structured output (#14496) · 0d5e73d3
  Russell Bryant authored Mar 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  0d5e73d3
- [Bugfix] Fix profiling OOM and decouple encoder multimodal profiling (#14361) · 609ef61f
  Isotr0py authored Mar 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  609ef61f
- [Bugfix] DeepSeek Accuracy (#14476) · db84f5eb
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  db84f5eb
- Move requirements into their own directory (#12547) · 206e2577
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  206e2577
- [Misc] Don't run ruff at all on 3rd party libs (#14493) · e02883c4
  Cyrus Leung authored Mar 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e02883c4
- [benchmarks] Add option to use unique jsonschema for each request (#14457) · 9085aabd
  Russell Bryant authored Mar 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9085aabd
- [V1][Core] Fix memory issue with logits & sampling (#13776) · 8d5aa466
  Roger Wang authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  8d5aa466
- [Misc] add `use_tqdm_on_load` to reduce logs (#14407) · 0b7f06b4
  Aaron Pham authored Mar 08, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  0b7f06b4
- [VLM] Add TP support for Phi-4-MM (#14453) · 03fe18ae
  Isotr0py authored Mar 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  03fe18ae
- [V1] TPU - Add tensor parallel support via Ray (#13618) · cb8bdfad
  Alexander Matveev authored Mar 08, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  cb8bdfad
- [CI/Build] Use a fixed seed to avoid flaky tests (#14480) · 33f227e1
  Cyrus Leung authored Mar 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  33f227e1
- Add RLHF document (#14482) · cfd0ae82
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cfd0ae82
- [Build/BugFix] Fix hopper 12.8 build (#14354) · 7caff01a
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  7caff01a