Commits · d6fd3a33b863003929fc3eef5dd9828219e04ab0 · OpenDAS / vllm_cscc

01 Jun, 2025 9 commits

[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp... · d6fd3a33

zhrrr authored Jun 02, 2025


[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context (#18935)
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

d6fd3a33

[doc] wrong output (#19000) · 432ec992

Reid authored Jun 01, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

432ec992

[BugFix] Fix incorrect metrics shutdown error log message (#18992) · 2b102d51
Nick Hill authored May 31, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
2b102d51
[BugFix] fix data parallel construct ipv6 url addres (#18991) · aa54a7bf
rongfu.leng authored Jun 01, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
aa54a7bf
Let max_num_batched_tokens use human_readable_int for large numbers (#18968) · 2ad6194a
Michael Goin authored May 31, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
2ad6194a

[doc] small fix - mkdocs (#18996) · c594cbf5

Reid authored Jun 01, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

c594cbf5

[LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with... · a35ca765

Isotr0py authored Jun 01, 2025


[LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with arbitrary components (#18987)
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>

a35ca765

[Core] Rework dtype resolution (#18751) · 6aa8f9a4
Cyrus Leung authored Jun 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6aa8f9a4
[Bugfix] Fix EAGLE3 broken logits (#18909) · 1bc86a3d
Benjamin Chislett authored May 31, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
1bc86a3d

31 May, 2025 16 commits
- [Misc][Benchmark] Add support for CustomDataset (#18511) · bbfa0c61
  Ekagra Ranjan authored May 31, 2025
  
  bbfa0c61
- [Misc] add return token strs for tokenize (#18941) · 20079c6e
  Reid authored Jun 01, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  20079c6e
- [BugFix] Fix multi-node offline data-parallel (#18981) · 9a1b9b99
  Nick Hill authored May 31, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com>
```
  9a1b9b99
- [P/D] NixlConnector use cache device index for memory registration (#18969) · 8bf507d7
  ptarasiewiczNV authored May 31, 2025
```
Signed-off-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
```
  8bf507d7
- [ROCm][Kernel] Add gfx950 support for skinny gemms (#18010) · 306d6040
  Charlie Fu authored May 31, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  306d6040
- [Bugfix] Fix for issue 17396 (#18773) · f2c3f66d
  Fred Reiss authored May 31, 2025
```
Signed-off-by: Fred Reiss <frreiss@us.ibm.com>
```
  f2c3f66d
- [FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 (#18825) · 0f5e0d56
  vllmellm authored May 31, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  0f5e0d56
- [BugFix] Pydantic part 2 (#18911) · c55d8046
  Luka Govedič authored May 31, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  c55d8046
- [doc] fix the list rendering issue - security.md (#18982) · 749f5bdd
  Reid authored May 31, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  749f5bdd
- [Neuron] Add Multi-Modal model support for Neuron (#18921) · 2a50ef57
  Satyajith Chilappagari authored May 31, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
Co-authored-by: Rohith Nallamaddi <nalrohit@amazon.com>
Co-authored-by: FeliciaLuo <luof@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
```
  2a50ef57
- fix security issue of logging llm output (#18980) · b8b90479
  Lucia Fang authored May 31, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Lucia (Lu) Fang <fanglu@meta.com>
```
  b8b90479
- [Bugfix]: Fix the incompatibility issue with Structured Outputs when Thinking is disabled (#18879) · ba5111f2
  Chauncey authored May 31, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  ba5111f2
- [Misc] Fix estimated max model len msg (#18966) · 1e123529
  Yong Hoon Shin authored May 31, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  1e123529
- [Frontend] Add rerank support to run_batch endpoint (#16278) · dff80b0e
  Pooya Davoodi authored May 31, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  dff80b0e
- create util function for batched arange (#18937) · 7782464a
  Yu Guo authored May 30, 2025
  
  7782464a
- [Docs] Correct multiprocessing design doc (#18964) · 0f71e240
  Lukas Geiger authored May 31, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  0f71e240
30 May, 2025 15 commits
- Tool parser regex timeout handling (#18960) · 1dab4d57
  Will Eaton authored May 30, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  1dab4d57
- [Misc] add group_size is -1 in awq quantization (#18910) · 7f21e805
  rongfu.leng authored May 31, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  7f21e805
- [VLM] Add PP support and fix GPTQ inference for Ovis models (#18958) · 5a864163
  Isotr0py authored May 31, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5a864163
- Benchmark script for fp8 vs bf16 gemm (#17126) · f49239cb
  Michael Goin authored May 30, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f49239cb
- [Perf] API-server scaleout with many-to-many server-engine comms (#17546) · 2dbe8c07
  Nick Hill authored May 30, 2025
  
  2dbe8c07
- Improve "failed to get the hash of the compiled graph" error (#18956) · 84ec470f
  Richard Zou authored May 30, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  84ec470f
- [Docs] Update SECURITY.md with link to our security guide (#18961) · b29ca5c4
  Russell Bryant authored May 30, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b29ca5c4
- [doc] show the count for fork and watch (#18950) · ec6833c5
  Reid authored May 30, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  ec6833c5
- [Feature] minicpm eagle support (#18943) · e1fadf11
  Shawn Huang authored May 30, 2025
```
Signed-off-by: huangyuxiang03 <huangyx0321@gmail.com>
Co-authored-by: huangyuxiang03 <huangyx0321@gmail.com>
```
  e1fadf11
- [CI/Build] remove regex from build dependencies (#18945) · 43ff405b
  Daniele authored May 30, 2025
```
Signed-off-by: Daniele Trifirò <dtrifiro@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  43ff405b
- [Bugfix][TPU] Fix tpu model runner testcase failure (#18810) · fba02e3b
  Carol Zheng authored May 30, 2025
```
Signed-off-by: Carol Zheng <cazheng@google.com>
```
  fba02e3b
- [Misc]Fix typo (#18947) · 4577fc9a
  Always-Naive authored May 30, 2025
  
  4577fc9a
- [Bugfix][Failing Test] Fix test_vllm_port.py (#18618) · 5f1d0c81
  Rabi Mishra authored May 30, 2025
```
Signed-off-by: rabi <ramishra@redhat.com>
```
  5f1d0c81
- [Model] Use in-place adds in SigLIP (#18922) · c3bb9f23
  Lukas Geiger authored May 30, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  c3bb9f23
- [doc] add mkdocs doc (#18930) · 8f8900ce
  Reid authored May 30, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  8f8900ce