Commits · b8b904795d3033c21bcb8f5c36e135e75dc1baf2 · OpenDAS / vllm_cscc

"fern/pages/kubernetes/quickstart.md" did not exist on "4a71802842e86950ce15b17c03011e6044eebe6e"

31 May, 2025 5 commits
- fix security issue of logging llm output (#18980) · b8b90479
  Lucia Fang authored May 31, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
```
  b8b90479
- [Bugfix]: Fix the incompatibility issue with Structured Outputs when Thinking is disabled (#18879) · ba5111f2
  Chauncey authored May 31, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ba5111f2
- [Misc] Fix estimated max model len msg (#18966) · 1e123529
  Yong Hoon Shin authored May 31, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  1e123529
- [Frontend] Add rerank support to run_batch endpoint (#16278) · dff80b0e
  Pooya Davoodi authored May 31, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  dff80b0e
- create util function for batched arange (#18937) · 7782464a
  Yu Guo authored May 30, 2025
  
  7782464a
30 May, 2025 18 commits
- Tool parser regex timeout handling (#18960) · 1dab4d57
  Will Eaton authored May 30, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  1dab4d57
- [Misc] add group_size is -1 in awq quantization (#18910) · 7f21e805
  rongfu.leng authored May 31, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  7f21e805
- [VLM] Add PP support and fix GPTQ inference for Ovis models (#18958) · 5a864163
  Isotr0py authored May 31, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5a864163
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
- Improve "failed to get the hash of the compiled graph" error (#18956) · 84ec470f
  Richard Zou authored May 30, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  84ec470f
- [Feature] minicpm eagle support (#18943) · e1fadf11
  Shawn Huang authored May 30, 2025
```
Signed-off-by: huangyuxiang03 <huangyx0321@gmail.com>
Co-authored-by: huangyuxiang03 <huangyx0321@gmail.com>
```
  e1fadf11
- [Bugfix][TPU] Fix tpu model runner testcase failure (#18810) · fba02e3b
  Carol Zheng authored May 30, 2025
```
Signed-off-by: Carol Zheng <cazheng@google.com>
```
  fba02e3b
- [Misc]Fix typo (#18947) · 4577fc9a
  Always-Naive authored May 30, 2025
  
  4577fc9a
- [Bugfix][Failing Test] Fix test_vllm_port.py (#18618) · 5f1d0c81
  Rabi Mishra authored May 30, 2025
```
Signed-off-by: rabi <ramishra@redhat.com>
```
  5f1d0c81
- [Model] Use in-place adds in SigLIP (#18922) · c3bb9f23
  Lukas Geiger authored May 30, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  c3bb9f23
- [Deprecation] Remove mean pooling default for `Qwen2EmbeddingModel` (#18913) · 4f4a6b84
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4f4a6b84
- [Bugfix] Remove NVFP4 scales assertions to fix load_format=dummy (#18861) · 4d0a1541
  Michael Goin authored May 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4d0a1541
- [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. (#18938) · 77b6e74f
  vllmellm authored May 30, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  77b6e74f
- [Model] Use AutoWeightsLoader for mamba2 (#18918) · 3987e2ae
  iLeGend authored May 30, 2025
```
Signed-off-by: iLeGend <824040212@qq.com>
```
  3987e2ae
- [Bugfix] Consistent ascii handling in tool parsers (#18883) · 77164dad
  Chauncey authored May 30, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  77164dad
- improve the robustness of parsing vlms config in AutoRound (#18894) · 3de3eadf
  Wenhua Cheng authored May 30, 2025
```
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com>
```
  3de3eadf
- [Misc] Update type annotation for rotary embedding `base` (#18914) · 1aa2f81b
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1aa2f81b
- [Bugfix] Fix PP default fallback behavior for V1 (#18915) · d54af615
  Michael Goin authored May 29, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d54af615
29 May, 2025 17 commits
- [TPU] remove transpose ops in moe kernel (#18923) · a1cc9f33
  Chengji Yao authored May 29, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a1cc9f33
- Use standalone_compile by default in torch >= 2.8.0 (#18846) · a521ef06
  Richard Zou authored May 29, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  a521ef06
- [P/D] NixlConnector DP fixes (#18903) · 64eaf5fe
  Will Eaton authored May 29, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  64eaf5fe
- [BugFix] Make DP work with connector-delayed new requests (#18559) · d1d61f33
  Nick Hill authored May 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Will Eaton <weaton@redhat.com>
```
  d1d61f33
- [V1] Allocate kv_cache with stride order for V1 (#18775) · 32ce3cf7
  Nicolò Lucchesi authored May 29, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  32ce3cf7
- [Misc] Remove duplicate init for self.vllm_config (#18896) · d58f9c7f
  CYJiang authored May 30, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  d58f9c7f
- [Deprecation] Disallow pos-args other than `model` when initializing `LLM` (#18802) · c2903403
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c2903403
- [ROCm][V0][Attention] Revert to the previous FA triton kernel (#18226) · 1b7cfd5a
  Gregory Shtrasberg authored May 29, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  1b7cfd5a
- [Attention][V1] Toggle for v1 attention backend (#18275) · da4b69d0
  Gregory Shtrasberg authored May 29, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  da4b69d0
- Fix an error in dummy weight loading for quantization models (#18855) · f8977c23
  Chenyaaang authored May 29, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  f8977c23
- [Bugfix] Ensure tensors are contiguous during serialisation (#18860) · 0b1447f8
  Lukas Geiger authored May 29, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  0b1447f8
- [Misc] Replace TODO in serving transcription (#18895) · 24d0ef89
  Nicolò Lucchesi authored May 29, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  24d0ef89
- [LoRA] Add LoRA support for InternVL (#18842) · 34d6c447
  Jee Jee Li authored May 29, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  34d6c447
- [Neuron] Add multi-LoRA support for Neuron. (#18284) · 972eddf7
  Satyajith Chilappagari authored May 29, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
```
  972eddf7
- Skip device and quant Pydantic validation to make plugin device work (#18843) · 3c49dbdd
  Yikun Jiang authored May 29, 2025
```
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
```
  3c49dbdd
- [Doc][Neuron] Update documentation for Neuron (#18868) · 1661a9c2
  aws-elaineyz authored May 28, 2025
```
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
```
  1661a9c2
- [Bugfix][TPU] fix moe custom kernel import (#18853) · 8e882ffd
  Chengji Yao authored May 28, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  8e882ffd