Commits · 308fa287a84a16ec5d81f662ca219ff5dc8929f5 · OpenDAS / vllm_cscc

"lib/runtime/examples/system_metrics/Cargo.toml" did not exist on "d38325c21f932395db5428fe463f8856a4e3afc1"

23 Aug, 2025 5 commits
- Add glm4.5v tp2,4 fp8 config on H100_80GB (#23443) · 308fa287
  Chenxi Yang authored Aug 22, 2025
```
Co-authored-by: Chenxi Yang <cxyang@meta.com>
```
  308fa287
- Quantization: support FP4 quantized models on AMD CDNA2/CDNA3 GPUs (#22527) · fa78de9d
  Daifeng Li authored Aug 23, 2025
```
Signed-off-by: feng <fengli1702@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  fa78de9d
- [Model] Support DP for ViT on MiniCPM-V-4 (#23327) · 23c939fd
  WeiQing Chen authored Aug 23, 2025
```
Signed-off-by: ycyaw66 <497410282@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
```
  23c939fd
- [BugFix] Fix `MinPLogitsProcessor.update_states()` (#23401) · add1adfe
  Nick Hill authored Aug 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  add1adfe
- [BugFix] Fix batch updates for pooling models (#23398) · c80c53a3
  Nick Hill authored Aug 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  c80c53a3
22 Aug, 2025 25 commits
- [NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6
  elvischenv authored Aug 23, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  24d0c9e6
- [BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being... · cc7ae5e7
  rasmith authored Aug 22, 2025
```
[BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being called when it should when using quantized FP8 model (#22281)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  cc7ae5e7
- [PERF] PyTorch Symmetric Memory All-Reduce (#20759) · 0313cf85
  Ilya Markov authored Aug 22, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  0313cf85
- add an env var for path to pre-downloaded flashinfer cubin files (#22675) · da65bec3
  Shiyan Deng authored Aug 22, 2025
  
  da65bec3
- [Quantization] Allow GGUF quantization to skip unquantized layer (#23188) · 4645024d
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  4645024d
- [Bugfix] Fix broken Florence-2 model (#23426) · cd7a3df2
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  cd7a3df2
- [Model] Add Ovis2.5 PP support (#23405) · 32d2b406
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  32d2b406
- [Doc]: fix various typos in multiple files (#23179) · 22cf679a
  Didier Durand authored Aug 22, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
  22cf679a
- [BugFix] Fix the issue where image embeddings were incorrectly split.… (#23366) · 424fb7a5
  bppps authored Aug 23, 2025
```
Signed-off-by: bppps <bpppsaka@gmail.com>
Co-authored-by: zouyu.zzx <zouyu.zzx@alibaba-inc.com>
Co-authored-by: bppps <bpppsaka@gmail.com>
```
  424fb7a5
- [Speculators][Speculative Decoding] Fix Qwen 2 Eagle3 Support (#23337) · 88491c1b
  PapaGoose authored Aug 22, 2025
  
  88491c1b
- [Misc] local import code clean (#23420) · 325aa3de
  Ning Xie authored Aug 22, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  325aa3de
- [misc] Remove outdate comment about runai_model_streamer (#23421) · 695e7adc
  杨朱 · Kiki authored Aug 22, 2025
```
Signed-off-by: carlory <baofa.fan@daocloud.io>
```
  695e7adc
- [Attention] Allow V1 flash_attn to support cross-attention (#23297) · 281710ef
  Russell Bryant authored Aug 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  281710ef
- [Misc] Move M-RoPE init logic to _init_mrope_positions (#23422) · 808d2e9a
  Woosuk Kwon authored Aug 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  808d2e9a
- [Bugfix] Fix pooling models on CPU backend (#23392) · 88016c37
  Li, Jiang authored Aug 22, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  88016c37
- Migrate MiniCPMOAudioInputs to TensorSchema (#21847) · 99872085
  Benji Beck authored Aug 22, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  99872085
- [gpt-oss] add input/output usage in responses api when harmony context is leveraged (#22667) · 0ba1b54a
  Guillaume Calmettes authored Aug 22, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  0ba1b54a
- [P/D][Nixl] Make kv cache register compatible with hybrid memory allocator (#23079) · 53415653
  Flora Feng authored Aug 21, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  53415653
- [Attention] Refactor AttentionMetadata Preparation for Encoder-only Models (#23154) · 17373dcd
  Chen Zhang authored Aug 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  17373dcd
- [New Model] Add Seed-Oss model (#23241) · 59640693
  Bin Jia authored Aug 22, 2025
```
Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  59640693
- [Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) · 394591e3
  Wentao Ye authored Aug 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  394591e3
- Migrate MllamaImagePixelInputs to TensorSchema (#22020) · 0b9cc56f
  Benji Beck authored Aug 21, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  0b9cc56f
- [Deprecation] Remove `prompt_token_ids` arg fallback in `LLM.generate` and `LLM.embed` (#18800) · 8896eb72
  Cyrus Leung authored Aug 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8896eb72
- [Kernel] Add FP8 support with FlashMLA backend (#22668) · 19fe1a05
  Matthew Bonanni authored Aug 21, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  19fe1a05
- [Core] Support custom executor qualname (#23314) · 480bdf5a
  22quinn authored Aug 21, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  480bdf5a
21 Aug, 2025 10 commits
- [Feature][Responses API] Support logprobs(non-stream) (#23319) · 5368f768
  Kebe authored Aug 22, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
  5368f768
- [Perf] Small optimizations for silu_mul_fp8_quant_deep_gemm (#23265) · 3bbe11cc
  Michael Goin authored Aug 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  3bbe11cc
- [Structured Outputs] Refactor bitmask construction into get_grammar_bitmask (#23361) · 800349c2
  Woosuk Kwon authored Aug 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  800349c2
- [Core] Always use tensor cores for Flashinfer Decode Wrapper (#23214) · 1d353b63
  Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  1d353b63
- [Misc] Convert VLLM_TORCH_PROFILER_DIR path to absolute (#23191) · 34962746
  Ning Xie authored Aug 22, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  34962746
- [BugFix][gpt-oss] Fix Chat Completion with Multiple Output Message (#23318) · 8a193031
  Chen Zhang authored Aug 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  8a193031
- [Misc] Misc code cleanup/simplification (#23304) · 603fbbbc
  Nick Hill authored Aug 21, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  603fbbbc
- [Bugfix] Fix port conflict by obtaining a list of open ports upfront (#21894) · 10f535c0
  Ming Yang authored Aug 21, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  10f535c0
- [Bug] Fix R1 Accuracy 0 Bug (#23294) · 48bfb0c9
  Wentao Ye authored Aug 21, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  48bfb0c9
- Fix nvfp4 swizzling (#23140) · 0278f1ac
  Yi Liu authored Aug 22, 2025
```
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  0278f1ac