Commits · 577df69b26491aaa8f3fef2ea44d6ac256172032 · OpenDAS / vllm_cscc · GitLab

18 Mar, 2026 19 commits
- [Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype... · 577df69b
  Andy Lo authored Mar 18, 2026
```
[Bugfix] Fix KV scales inconsistency in fp8 MLA & FlashInfer kv_cache_dtype "auto" leading to gibberish (#37054)
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  577df69b
- [V0 Deprecation] Deprecate virtual engine (#37195) · 0d81a1fe
  Wentao Ye authored Mar 18, 2026
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  0d81a1fe
- [kv_offload+HMA][2/N]: Support multiple KV groups in GPULoadStoreSpec (#36642) · 5dd8df07
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  5dd8df07
- [Kernel] Add gpt-oss Router GEMM kernel (#37205) · b1169d7b
  Xin Yang authored Mar 18, 2026
```
Signed-off-by: Xin Yang <xyangx@amazon.com>
```
  b1169d7b
- [Perf] Eliminate padding and slicing op for GPT-OSS with Flashinfer MXFP4 MXFP8 MoE (#30647) · 296839a1
  elvischenv authored Mar 18, 2026
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  296839a1
- [2/3] Refactor InternVL-based processors (#37324) · 99267c23
  Cyrus Leung authored Mar 18, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  99267c23
- [kv_offload+HMA][6/N]: Split offloading_connector.py (#37405) · 525f2eeb
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  525f2eeb
- [Bugfix] Fix base64 JPEG video frames returning empty metadata (#37301) · 918b7890
  Yufeng He authored Mar 18, 2026
```
Signed-off-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Yufeng He <40085740+universeplayer@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  918b7890
- [NIXL][Bugfix] metrics & testing minor bug (#36051) · 98b09ddc
  Andy Lo authored Mar 18, 2026
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  98b09ddc
- [Build] Bump python openai version (#32316) · b322b197
  Chauncey authored Mar 18, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  b322b197
- [CI] Fix PaddleOCR-VL HF test failure due to create_causal_mask API rename (#37328) · eaf7c9b9
  Andreas Karatzas authored Mar 18, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  eaf7c9b9
- fix(glm47): improve tool call parsing and content normalization (#37386) · fad09e8a
  Karan Bansal authored Mar 18, 2026
```
Signed-off-by: karanb192 <karan@example.com>
Co-authored-by: karanb192 <karan@example.com>
```
  fad09e8a
- [kv_offload+HMA][0/N]: Support block-level preemption handling (#34805) · fcf0687b
  Or Ozeri authored Mar 18, 2026
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
```
  fcf0687b
- [XPU] skip unsupported ut and update test_nixl_connector (#37179) · 86b7e3c9
  liuzhenwei authored Mar 18, 2026
```
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
```
  86b7e3c9
- [CI] Stabilize test_cpu_offloading by waiting for async offload before cache reset (#37335) · ce2ef42f
  Andreas Karatzas authored Mar 18, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  ce2ef42f
- [ROCm][CI] Add ROCM_EXTRA_ARGS to audio_in_video test server fixture (#37349) · 8b632575
  Andreas Karatzas authored Mar 17, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  8b632575
- [Hardware][TPU] Add supports_async_scheduling() method to Executor interface... · a0dd1995
  gxd3 authored Mar 17, 2026
```
[Hardware][TPU] Add supports_async_scheduling() method to Executor interface so that it can be extended for Executor implementations. (#36924)
Signed-off-by: Guangxiang Du <gxd@google.com>
```
  a0dd1995
- [ROCm][CI] Skip trtllm kvfp8 dequant tests on ROCm (#37330) · 58cde5c0
  Andreas Karatzas authored Mar 17, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  58cde5c0
- [Kernel][Helion] [16/N] Refactor register_kernel API to be more Dynamo-friendly (#36705) · ff9fbc9a
  Yanan Cao authored Mar 17, 2026
```
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  ff9fbc9a
17 Mar, 2026 21 commits
- [Kernel] Add non-gated support for NVFP4 CUTLASS MoE (#37320) · 09e4576f
  Michael Goin authored Mar 17, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  09e4576f
- Make KV connector metadata build overridable via plugin (#37336) · de35c06c
  Yong Hoon Shin authored Mar 17, 2026
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  de35c06c
- [Model] Add ColQwen3.5 4.5B support (#36887) · c0745a85
  Athrael Soju authored Mar 17, 2026
```
Signed-off-by: Athrael Soju <athrael.soju@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
```
  c0745a85
- [Models] Cohere ASR (#35809) · b5ca9c35
  Ekagra Ranjan authored Mar 17, 2026
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
```
  b5ca9c35
- [Model] Remove unused `handle_oov_mm_token` (#37321) · 51f0acda
  Cyrus Leung authored Mar 18, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  51f0acda
- [Torch 2.11] Guard torch._C._cpu attribute checks for forward compatibility (#35673) · 68f783a7
  Andrey Talman authored Mar 17, 2026
```
Signed-off-by: atalman <atalman@fb.com>
```
  68f783a7
- [CI] Fix GPU memory leak when RemoteOpenAIServer fails to start in __init__ (#37230) · 4ed51308
  Andreas Karatzas authored Mar 17, 2026
```
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
```
  4ed51308
- [Chore] Replace all base64 usages with faster pybase64 package (#37290) · a836524d
  Isotr0py authored Mar 17, 2026
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  a836524d
- [Misc][LoRA] Add --lora-target-modules to restrict LoRA to specific modules (#34984) · 3717a4dd
  Bhoomit authored Mar 17, 2026
```
Signed-off-by: Bhoomit Vasani <bhoomit.2010@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3717a4dd
- Fix Phi3 test that fails with Transformers v5 (#37298) · ecfcdd2c
  Harry Mellor authored Mar 17, 2026
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  ecfcdd2c
- [Frontend] Complete OpenAI render delegation (#37287) · 59192dfd
  Sage authored Mar 17, 2026
```
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
```
  59192dfd
- [1/2] Move InternVL-based processors (#37260) · f3403243
  Cyrus Leung authored Mar 17, 2026
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f3403243
- Add gigachat 3.1 tool parser + fix gigachat3 tool parser (#36664) · 293f036e
  Viacheslav authored Mar 17, 2026
```
Signed-off-by: Viacheslav Barinov <viacheslav.teh@gmail.com>
```
  293f036e
- [Frontend] Delegate tokenization serving preprocessing to OpenAIServingRender (#37266) · 00f8e0d2
  Sage authored Mar 17, 2026
```
Signed-off-by: Sage Ahrac <sagiahrak@gmail.com>
```
  00f8e0d2
- [Feature]: Support for multiple embedding types in a single inference call (#35829) · 9c7cab5e
  Augusto Yao authored Mar 17, 2026
```
Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>
```
  9c7cab5e
- [Bugfix][ResponsesAPI] Fix crash when tool_choice=required exceeds max_output_tokens (#37258) · 132bfd45
  Chauncey authored Mar 17, 2026
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  132bfd45
- [Bugfix] Fix NemotronH MTP + Chunked Prefill (#35447) · 8a680463
  Benjamin Chislett authored Mar 17, 2026
  
  8a680463
- [Refactor] Relocate responses API tests (#37241) · 3e3d320c
  Flora Feng authored Mar 17, 2026
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  3e3d320c
- [Refactor] Relocate completion and chat completion tests (#37125) · 384dc7f7
  Flora Feng authored Mar 16, 2026
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  384dc7f7
- [CI] Fix flaky tool_use chat completion tests with deterministic seed (#37027) · f04d5226
  Flora Feng authored Mar 16, 2026
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  f04d5226
- Support non-contiguous KV cache in TRTLLM fp8 dequant kernel (#36867) · 6c1cfbad
  Vadim Gimpelson authored Mar 17, 2026
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com>
Co-authored-by: Pavani Majety <pavanimajety@gmail.com>
```
  6c1cfbad