Commits · 951329003203ba34f52b82ed12bcf91b349c2fa2 · OpenDAS / vllm_cscc · GitLab

08 Mar, 2025 26 commits
- [Misc] Upgrade to Python 3.9 typing for additional directories (#14492) · 95132900
  Cyrus Leung authored Mar 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  95132900
- Update CODEOWNERS for structured output (#14496) · 0d5e73d3
  Russell Bryant authored Mar 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  0d5e73d3
- [Bugfix] Fix profiling OOM and decouple encoder multimodal profiling (#14361) · 609ef61f
  Isotr0py authored Mar 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  609ef61f
- [Bugfix] DeepSeek Accuracy (#14476) · db84f5eb
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  db84f5eb
- Move requirements into their own directory (#12547) · 206e2577
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  206e2577
- [Misc] Don't run ruff at all on 3rd party libs (#14493) · e02883c4
  Cyrus Leung authored Mar 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e02883c4
- [benchmarks] Add option to use unique jsonschema for each request (#14457) · 9085aabd
  Russell Bryant authored Mar 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9085aabd
- [V1][Core] Fix memory issue with logits & sampling (#13776) · 8d5aa466
  Roger Wang authored Mar 08, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  8d5aa466
- [Misc] add `use_tqdm_on_load` to reduce logs (#14407) · 0b7f06b4
  Aaron Pham authored Mar 08, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  0b7f06b4
- [VLM] Add TP support for Phi-4-MM (#14453) · 03fe18ae
  Isotr0py authored Mar 08, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  03fe18ae
- [V1] TPU - Add tensor parallel support via Ray (#13618) · cb8bdfad
  Alexander Matveev authored Mar 08, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  cb8bdfad
- [CI/Build] Use a fixed seed to avoid flaky tests (#14480) · 33f227e1
  Cyrus Leung authored Mar 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  33f227e1
- Add RLHF document (#14482) · cfd0ae82
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cfd0ae82
- [Build/BugFix] Fix hopper 12.8 build (#14354) · 7caff01a
  Lucas Wilkinson authored Mar 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  7caff01a
- Add training doc signposting to TRL (#14439) · be0b399d
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  be0b399d
- [Bugfix] Make the deviceprofiler include LoRA memory. (#14469) · b8b0ccbd
  Jee Jee Li authored Mar 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b8b0ccbd
- [Doc] Added QwQ-32B to the supported models list in the reasoning out… (#14479) · c908a07f
  Robin authored Mar 08, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
  c908a07f
- [Doc]add doc for Qwen models tool calling (#14478) · 7b6fd6e4
  Robin authored Mar 08, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
  7b6fd6e4
- Default to `generation_config` from model (#12622) · 47512b32
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  47512b32
- [CI/Build] refactor: set timezone of container to UTC (#12888) · 3b9c6c69
  Roger Meier authored Mar 08, 2025
```
Signed-off-by: Roger Meier <r.meier@siemens.com>
```
  3b9c6c69
- [core] add `extra_args` to `SamplingParams` (#13300) · 4aae6676
  Aviv Keshet authored Mar 07, 2025
```
Signed-off-by: Aviv Keshet <akeshet@scaledcognition.com>
```
  4aae6676
- [MISC][V1] Register process killing handler only in the main thread (#14380) · 9f3bc0f5
  Cody Yu authored Mar 07, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  9f3bc0f5
- [Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and... · 980385f8
  Mathis Felardos authored Mar 08, 2025
```
[Bugfix][Disaggregated] Add a check in send_kv_caches_and_hidden_states and fix the reshape of the KVCache (#14369)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
```
  980385f8
- Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) · ca7a2d5f
  Tyler Michael Smith authored Mar 08, 2025
  
  ca7a2d5f
- [Bugfix][V1] Handle MLA in kv_cache_interface (#14462) · 33368140
  Tyler Michael Smith authored Mar 08, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  33368140
- [V1] Prompt logprobs + APC compatibility; prompt logprobs reqs cannot fill APC (#13949) · ef640440
  afeldman-nm authored Mar 07, 2025
  
  ef640440
07 Mar, 2025 14 commits
- [Bugfix] Fix torch_xla which can't handle None seed introduced in #14274 (#14459) · 66e16a03
  yarongmu-google authored Mar 07, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  66e16a03
- [V1][Metrics] Fix traceback with preemptions+LoRA (#14220) · e1f0835a
  Mark McLoughlin authored Mar 07, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  e1f0835a
- [V1] Eagerly remove finished requests from the batch (#14388) · 8ed5421a
  Nick Hill authored Mar 07, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  8ed5421a
- [v1] torch.compile integration explanation (#14437) · c6359e8c
  youkaichao authored Mar 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c6359e8c
- [Misc] Add Phi4-MM example (#14343) · 952a0749
  Jee Jee Li authored Mar 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  952a0749
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
- [Benchmarks] Make detokenization optional in benchmark scripts (#11697) · 58abe354
  Jeremy Arnold authored Mar 07, 2025
```
Signed-off-by: Jeremy Arnold <Jeremy.Arnold@amd.com>
```
  58abe354
- [Doc] Update prefix_caching.md to match the example image (#14420) · f7ebad23
  York-RDWang authored Mar 07, 2025
  
  f7ebad23
- [V1][Core] Support for Structured Outputs (#12388) · 80e9afb5
  Aaron Pham authored Mar 07, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  80e9afb5
- Use the optimized block sizes after tuning the kernel. (#14329) · 1e3598ed
  iefgnoix authored Mar 07, 2025
  
  1e3598ed
- Fix missing `kv_caches` and `attn_metadata` in `OpenVINOCausalLM` (#14271) · f7a6bd0f
  Harry Mellor authored Mar 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f7a6bd0f
- [BUGFIX] Skip tokenization support for throughput benchmark (#12712) · 0ca3b8e0
  Aleksandr Malyshev authored Mar 07, 2025
```
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
  0ca3b8e0
- [Misc] Set default value of seed to None (#14274) · cc102814
  மனோஜ்குமார் பழனிச்சாமி authored Mar 07, 2025
```
Signed-off-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
```
  cc102814
- [Bugfix] Clean up multi-modal processors (#14417) · 05fb6718
  Cyrus Leung authored Mar 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  05fb6718