Commits · 582bbe6bd708d01d74d6d02d6ef59b4c3c34a7b1 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "bdd8981dab8d8c6ae88a3f605d04ec5243088e5a"

20 Aug, 2025 31 commits
- [Fix] correct tool_id for kimi-k2 when use tool_choice=required (#21259) · 582bbe6b
  bigmoyan authored Aug 21, 2025
```
Co-authored-by: wangzhengtao <wangzhengtao@msh.team>
```
  582bbe6b
- [Kernel/Quant] Remove the original marlin format and qqq (#23204) · 0cdbf5e6
  Michael Goin authored Aug 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0cdbf5e6
- Small fix for Command-A-Vision (#23268) · ebe56a00
  dongluw authored Aug 20, 2025
```
Signed-off-by: donglu <donglu@cohere.com>
```
  ebe56a00
- Limit HTTP header count and size (#23267) · f77a0802
  Russell Bryant authored Aug 20, 2025
```
Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com>
```
  f77a0802
- Migrate Mistral3ImagePixelInputs to TensorSchema (#21945) · c4477f55
  Benji Beck authored Aug 20, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c4477f55
- [torch.compile] Support conditional torch.compile per module (#22269) · dfd23820
  Yong Hoon Shin authored Aug 20, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  dfd23820
- [FIXBUG ] Allow disabling rocm_aiter_fa backend for ROCm GPUs not compatible with AITER (#22795) · 3b11b26b
  JartX authored Aug 20, 2025
```
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  3b11b26b
- [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) · d6d13bd4
  Woosuk Kwon authored Aug 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d6d13bd4
- [CLI][Doc] Formalize `--mm-encoder-tp-mode` (#23190) · 5efd6905
  Cyrus Leung authored Aug 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5efd6905
- [Kernel] CUTLASS MoE FP8: Integrate cuda moe permute/unpermute (#23045) · b17109be
  shixianc authored Aug 20, 2025
```
Signed-off-by: Shixian Cui <shixian@amazon.com>
```
  b17109be
- [Bugfix] Ensure correctness of HCXVision processing (#23254) · 44492358
  Cyrus Leung authored Aug 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  44492358
- [Fix] fix offline env use local mode path (#22526) · 38217877
  rongfu.leng authored Aug 20, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  38217877
- [Model] Improve olmo and olmo2 (#23228) · c6d80a7a
  Jee Jee Li authored Aug 20, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c6d80a7a
- [Model][V1] Support Ernie MTP (#22169) · 7cd17e22
  xyxinyang authored Aug 20, 2025
```
Signed-off-by: zhouchong <zhouchong03@baidu.com>
Co-authored-by: zhouchong <zhouchong03@baidu.com>
```
  7cd17e22
- Update to flashinfer-python==0.2.12 and disable AOT compile for non-release image (#23129) · 50df09fe
  Michael Goin authored Aug 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  50df09fe
- [Bugfix] Ensure correctness of Cohere2Vision processing (#23245) · 68fcd3fa
  Cyrus Leung authored Aug 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  68fcd3fa
- [Model] Support deepseek with eagle (#21086) · 83e69a09
  Xin Yang authored Aug 20, 2025
```
Signed-off-by: Xin Yang <xyangx@amazon.com>
```
  83e69a09
- Fix missing quotes (#23242) · 3aa8c100
  Shiming Zhang authored Aug 20, 2025
```
Signed-off-by: Shiming Zhang <wzshiming@hotmail.com>
```
  3aa8c100
- [Model] use autoWeightsLoader for gptoss (#22446) · 103f1ec8
  Calvin Chen authored Aug 20, 2025
```
Signed-off-by: calvin chen <wen.chen@dynamia.ai>
```
  103f1ec8
- fix cuda graph (#22721) · d983769c
  who who who authored Aug 20, 2025
```
Signed-off-by: fsx950223 <fsx950223@outlook.com>
```
  d983769c
- [BugFix] Fix stuck stats/metrics after requests are aborted (#22995) · 8fd92092
  Nick Hill authored Aug 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  8fd92092
- [CI/Build] Sync multimodal tests (#23181) · de7b67a0
  Cyrus Leung authored Aug 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  de7b67a0
- [CI/Build] Also check DP in benchmarks throughput script (#23038) · f7290232
  Zhewen Li authored Aug 19, 2025
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  f7290232
- chore: support pytorch format in lora (#22790) · 1a3079a1
  길재은 authored Aug 20, 2025
```
Signed-off-by: jaeeun.kil <rha3122@naver.com>
Signed-off-by: 길재은 <rha3122@naver.com>
```
  1a3079a1
- Fix a performance comparison issue in Benchmark Suite (#23047) · 941f5685
  Louie Tsai authored Aug 19, 2025
```
Signed-off-by: Tsai, Louie <louie.tsai@intel.com>
Signed-off-by: Louie Tsai <louie.tsai@intel.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Li, Jiang <bigpyj64@gmail.com>
```
  941f5685
- [Attention] Optimize make_local_attention_virtual_batches for Flash Attention (#23185) · a634733f
  Zebing Lin authored Aug 19, 2025
```
Signed-off-by: linzebing <linzebing1995@gmail.com>
```
  a634733f
- [Doc] Update V1 status of various pooling models (#23189) · 64ab3c72
  Cyrus Leung authored Aug 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  64ab3c72
- [Core] Add torch profiler CPU traces for AsyncLLM. (#21794) · e58c5a97
  Chenheli Hua authored Aug 19, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
  e58c5a97
- [CI Perf] Only test bfloat16 for tests/compile/test_fusion_all_reduce.py (#23132) · d46d417b
  Michael Goin authored Aug 19, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d46d417b
- [Core] Optimize scheduler request removal for single completions (#21917) · 0167efe2
  633WHU authored Aug 20, 2025
```
Signed-off-by: chiliu <chiliu@paypal.com>
Signed-off-by: chiliu <cliu_whu@yeah.net>
Co-authored-by: chiliu <chiliu@paypal.com>
```
  0167efe2
- [Quantization] Bump Compressed Tensors Version (#23202) · c32e6ad1
  Kyle Sayers authored Aug 19, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: Dipika Sikka <dipikasikka1@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  c32e6ad1
19 Aug, 2025 9 commits
- [Benchmarks] Add video inputs to ShareGPTDataset. (#23199) · 1630cc8d
  Chenheli Hua authored Aug 19, 2025
```
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
```
  1630cc8d
- [BugFix] fix CUTLASS MLA full cudagraph (#23200) · 14e2b073
  Lucas Wilkinson authored Aug 19, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  14e2b073
- [CI/Build] Replace lm-eval gsm8k tests with faster implementation (#23002) · 0f4f0191
  Michael Goin authored Aug 19, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0f4f0191
- [NVIDIA] Add SM100 Flashinfer Cutlass MoE fp8 backend (#22357) · a38b8af4
  amirkl94 authored Aug 20, 2025
```
Signed-off-by: Amir Klein <203507526+amirkl94@users.noreply.github.com>
```
  a38b8af4
- [CI/Build] Add support for Python 3.13 (#13164) · 21dce80e
  Michael Goin authored Aug 19, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  21dce80e
- [Misc] Minor refactoring for FlashInfer backend (#23147) · e61bac87
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  e61bac87
- fix: use cache_salt for gpt-oss (#23186) · 80141bbf
  Marko Rosenmueller authored Aug 19, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  80141bbf
- [Bugfix] Fix accuracy issue when using flashinfer cutlass moe, TP=1 and modelopt. (#23125) · b94faf9d
  bnellnm authored Aug 19, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b94faf9d
- [Misc] Enable yapf for FlashInfer backend (#23193) · 5b5f350d
  Woosuk Kwon authored Aug 19, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  5b5f350d