Commits · 1bff42c4b74658122b6e54f04bb122ee09b53330 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "36bf8150cc3a048d69d9d2196128462014b9599d"

09 Apr, 2025 27 commits
- [Misc] refactor Structured Outputs example (#16322) · 1bff42c4
  Reid authored Apr 10, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  1bff42c4
- [Hardware] add platform-specific request validation api (#16291) · cb391d85
  Joe Runde authored Apr 09, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cb391d85
- [Build/CI] Add tracing deps to vllm container image (#15224) · fee5b8d3
  Russell Bryant authored Apr 09, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  fee5b8d3
- Fix `benchmark_throughput.py --backend=hf` (#16352) · b2ce859b
  Michael Goin authored Apr 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b2ce859b
- [CI]Fix hpu docker and numpy version for CI (#16355) · 566f10a9
  Chendi.Xue authored Apr 09, 2025
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  566f10a9
- [Bugfix] catch AssertionError in MistralTokenizer as ValueError (#16344) · c3b51891
  Guillaume Calmettes authored Apr 09, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  c3b51891
- [Bugfix] Fix profiling.py (#16202) · a25866ac
  zh Wang authored Apr 10, 2025
```
Signed-off-by: zh Wang <rekind133@outlook.com>
```
  a25866ac
- Revert "Update label-tpu mergify and remove removal bot" (#16350) · 098900d7
  Michael Goin authored Apr 09, 2025
  
  098900d7
- [Bugfix][Frontend] respect provided default guided decoding backend (#15476) · 98d01d3c
  Guillaume Calmettes authored Apr 09, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  98d01d3c
- [Model] Add `SupportsMultiModal.get_language_model` interface (#16007) · d55244df
  Nicolò Lucchesi authored Apr 09, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d55244df
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
- update neuron config (#16289) · 24834f48
  ajayvohra2005 authored Apr 09, 2025
```
Signed-off-by: Ajay Vohra <ajayvohr@amazon.com>
```
  24834f48
- [BugFix] llama4 qknorm should be not shared across head (#16311) · ec7da6fc
  Lucia Fang authored Apr 09, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ec7da6fc
- [BugFix] logger is not callable (#16312) · 819d548e
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  819d548e
- Update label-tpu mergify and remove removal bot (#16298) · 477d2a8a
  Michael Goin authored Apr 09, 2025
  
  477d2a8a
- [Bugfix] Avoid transferring cached multi-modal items from P0 to P1 (#16273) · e484e028
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e484e028
- [Misc] Fix test_sharded_state_loader.py(#16004) (#16005) · 24f6b9a7
  Accelerator1996 authored Apr 09, 2025
```
Signed-off-by: lvfei.lv <lvfei.lv@alibaba-inc.com>
```
  24f6b9a7
- [BugFix] Fix fusion test and add them to CI (#16287) · 9cdde472
  Luka Govedič authored Apr 09, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  9cdde472
- [TPU] Update PyTorch/XLA (#16288) · b1eb4ca1
  Chengji Yao authored Apr 08, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  b1eb4ca1
- [CI][Bugfix] Fix bad tolerance for test_batch_base64_embedding (#16221) · 87b4ac56
  Michael Goin authored Apr 08, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  87b4ac56
- [Core] Upgrade to xgrammar 0.1.18, add cache size limit (#16283) · cb84e45a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cb84e45a
- [Feature] Estimate max-model-len use available KV cache memory (#16168) · 4716377f
  rongfu.leng authored Apr 09, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4716377f
- [Bugfix] fix gettid method is not define (#16084) · 4e9cf8c1
  rongfu.leng authored Apr 09, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  4e9cf8c1
- [Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and... · 2976dc27
  TJian authored Apr 09, 2025
```
[Bug] [ROCm] Fix Llama 4 Enablement Bug on ROCm: V0 ROCmFlashAttentionImpl and Triton Fused MoE bugs (#16198)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: kliuae <kuanfu.liu@embeddedllm.com>
```
  2976dc27
- [Model] Add smolvlm support (#16017) · 102bf967
  Chauncey authored Apr 09, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  102bf967
- Add support to modelopt quantization of Mixtral model (#15961) · 1f4b09b5
  yueshen2016 authored Apr 08, 2025
```
Signed-off-by: Yue <yueshen@nvidia.com>
```
  1f4b09b5
- [CI/Build] Fix CI LoRA failure (#16270) · 86c3369e
  Jee Jee Li authored Apr 09, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  86c3369e
08 Apr, 2025 13 commits
- [V1] Update structured output offline inference example (#15721) · 2755c34a
  Russell Bryant authored Apr 08, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  2755c34a
- [Bugfix] fix deepseek fp16 scale bug (#14809) · db104221
  Jinzhen Lin authored Apr 09, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  db104221
- [BugFix] Fix Llama4 - Index Error When Single Request Near Max Context (#16209) · e1a2c699
  Lucas Wilkinson authored Apr 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  e1a2c699
- Add warning that content below line in template will be removed (#16276) · 0115ccd5
  Harry Mellor authored Apr 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0115ccd5
- [Bugfix] Handle `process_weights_after_loading` for `QKVCrossParallelLinear` (#15328) · 40b4284f
  Isotr0py authored Apr 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  40b4284f
- [Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156) · 4ebc0b96
  Cyrus Leung authored Apr 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4ebc0b96
- [Misc] Avoid stripping meaningful whitespace from `nvidia-smi topo -m` output... · dc96fd54
  Kero Liang authored Apr 09, 2025
```
[Misc] Avoid stripping meaningful whitespace from `nvidia-smi topo -m` output in collect_env.py (#16272)
Signed-off-by: imkero <kerorek@outlook.com>
```
  dc96fd54
- [New Model]: jinaai/jina-embeddings-v3 (#16120) · 1f5d13ab
  wang.yuqi authored Apr 08, 2025
  
  1f5d13ab
- Update to transformers==4.51.1 (#16257) · 90cb44eb
  Harry Mellor authored Apr 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  90cb44eb
- [Bugfix] Remove triton do_bench fast_flush arg (#16256) · e11880de
  Kebe authored Apr 08, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  e11880de
- [BugFix][ROCm] Fix GGUF MoE Dispatch Block_Dim for ROCm (#16247) · 9351f91b
  TY-AMD authored Apr 08, 2025
```
Signed-off-by: Tianyuan Wu <Tianyuan.Wu@amd.com>
```
  9351f91b
- [Model] use AutoWeightsLoader for phimoe,qwen2_moe,qwen3_moe (#16203) · 5a1e1c83
  rongfu.leng authored Apr 08, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  5a1e1c83
- [Misc] Add warning for multimodal data in LLM.beam_search (#16241) · 69ecaa7c
  Alex Brooks authored Apr 08, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  69ecaa7c