Commits · ffb2cd6b5441af28da22694a19838a19507a15e1 · OpenDAS / vllm_cscc · GitLab

17 Jun, 2025 16 commits
- [Perf] Optimize `moe_align_block_size` CUDA kernel (#19572) · ffb2cd6b
  Wentao Ye authored Jun 17, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  ffb2cd6b
- [Bugfix] Update multimodel models mapping to fit new checkpoint after Transformers v4.52 (#19151) · ca94d7fa
  Isotr0py authored Jun 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  ca94d7fa
- [Mis] remove duplicate engine status checks (#19647) · 5a1c2e15
  CYJiang authored Jun 17, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  5a1c2e15
- [V1][Kernel] Flashinfer HND KV cache layout (#19280) · 4c8f64fa
  Nicolò Lucchesi authored Jun 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  4c8f64fa
- [doc] split "Other AI Accelerators" tabs (#19708) · 93aee29f
  David Xia authored Jun 17, 2025
  
  93aee29f
- [doc][mkdocs] Add edit button to documentation (#19637) · 154d063b
  Reid authored Jun 17, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  154d063b
- [Kernel] Add Split-KV Support to Unified Triton Attention Kernel (#19152) · ccd7c050
  jvlunteren authored Jun 17, 2025
```
Signed-off-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  ccd7c050
- Add a doc on how to update PyTorch version (#19705) · c48c6c40
  Huy Do authored Jun 17, 2025
  
  c48c6c40
- [Doc] Add missing llava family multi-image examples (#19698) · aed84686
  Isotr0py authored Jun 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  aed84686
- [Core] add remove_seq_from_computed_blocks_tracker to BlockSpaceManager (#19686) · 5c76b9cd
  quanliu authored Jun 17, 2025
```
Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
```
  5c76b9cd
- Fixes IMA for TP w/ flex-attention (#19712) · ddfed314
  Driss Guessous authored Jun 16, 2025
```
Signed-off-by: drisspg <drisspguessous@gmail.com>
```
  ddfed314
- [DOC] fix doc typos (#19600) · 5b3ad5ec
  Di Liu authored Jun 17, 2025
```
Signed-off-by: Di Liu <liu-di@sjtu.edu.cn>
```
  5b3ad5ec
- [Frontend] add chunking audio for > 30s audio (#19597) · ede5c4eb
  nguyenhoangthuan99 authored Jun 17, 2025
```
Signed-off-by: nguyenhoangthuan99 <thuanhppro12@gmail.com>
```
  ede5c4eb
- [Wheel Size] Only build FA2 8.0+PTX (#19336) · 07334959
  Lucas Wilkinson authored Jun 16, 2025
  
  07334959
- [doc] add project flag to gcloud TPU command (#19664) · 119f6839
  David Xia authored Jun 16, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  119f6839
- [Fix] Fall back to Gloo when NCCL backend is unavailable (#19641) · 0860087a
  Conroy Cheers authored Jun 17, 2025
```
Signed-off-by: conroy-cheers <conroy@corncheese.org>
```
  0860087a
16 Jun, 2025 20 commits
- [Quantization] Remove FP4 emulation; Fall-back to marlin for device < 100 (#19563) · 6bc7b573
  Dipika Sikka authored Jun 16, 2025
  
  6bc7b573
- [V1] Change return type on get_multimodal_embeddings() (#19446) · 90f9c2eb
  Russell Bryant authored Jun 16, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  90f9c2eb
- [Model] Add support for MiniMaxM1ForCausalLM (shares architecture with... · 387bdf0a
  qscqesze authored Jun 17, 2025
```
[Model] Add support for MiniMaxM1ForCausalLM (shares architecture with MiniMaxText01ForCausalLM) (#19677)
Signed-off-by: QscQ <qscqesze@gmail.com>
```
  387bdf0a
- [Kernels] Use empty for modular MoE workspaces (#19667) · 5e5baa91
  bnellnm authored Jun 16, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  5e5baa91
- [Bugfix] fix missing 'finish_reason': null in streaming chat (#19662) · 836d4ce1
  Chauncey authored Jun 16, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  836d4ce1
- [MISC] bump huggingface_hub pkg to 0.33.0 (#19547) · c3fec47b
  Ning Xie authored Jun 16, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  c3fec47b
- [Bugfix] Fix TP inference for Flex attention backend (#19657) · 1173804d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1173804d
- [Feature]:Allow for Granite MoE Hybrid models with _only_ shared experts. (#19652) · 4d542402
  Shawn Tan authored Jun 16, 2025
```
Signed-off-by: Shawn Tan <shawntan@ibm.com>
```
  4d542402
- [DOC] Add reasoning capability to vLLM streamlit code (#19557) · 3e750697
  Navanit Dubey authored Jun 16, 2025
  
  3e750697
- [BugFix] Don't catch BaseException when dumping execute_model errors (#19626) · ee35e96a
  Nick Hill authored Jun 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  ee35e96a
- [Kernel] GGUF MMVQ kernel for multiple input vectors (#18754) · dec66d25
  Szymon Ożóg authored Jun 16, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
```
  dec66d25
- [Docs] Move multiproc doc to v1 dir (#19651) · 8d120701
  Russell Bryant authored Jun 16, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  8d120701
- [CI] Add mteb testing for rerank models (#19344) · f40f763f
  wang.yuqi authored Jun 16, 2025
  
  f40f763f
- [MISC] typo fix (#19672) · 26bc46ef
  Ning Xie authored Jun 16, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  26bc46ef
- [TPU] support attention head dim smaller than 128 (#19620) · a77aea59
  Chengji Yao authored Jun 15, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  a77aea59
- [Misc] Fix skipped max-model-len validation when deriving max model length... · b692e9cd
  Ye (Charlotte) Qi authored Jun 15, 2025
```
[Misc] Fix skipped max-model-len validation when deriving max model length from tokenizer config (#19660)
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  b692e9cd
- [Misc][Frontend] passthrough `bad_words` (#19564) · 367871a4
  Francesco Bertolotti authored Jun 16, 2025
```
Signed-off-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai>
Co-authored-by: Francesco Bertolotti <francesco.bertolotti@igenius.ai>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
```
  367871a4
- [Bugfix][Core] Prefix caching causes incorrect outputs due to outdated... · 92183b41
  quanliu authored Jun 16, 2025
```
[Bugfix][Core] Prefix caching causes incorrect outputs due to outdated ComputedBlocksTracker (#18957)
Signed-off-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
Co-authored-by: 刘全 <quan.liu2@dbappsecurity.com.cn>
```
  92183b41
- [MISC] Remove unused variableds in C++ (#19609) · c6703d1e
  Lu Fang authored Jun 16, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  c6703d1e
- [Misc] Remove duplicate multiproc method setting for CPU platform (#19649) · a5e7242d
  Isotr0py authored Jun 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  a5e7242d
15 Jun, 2025 4 commits
- [CI/Build] Fix torch nightly CI dependencies part 2 (#19589) · 91b2c17a
  Richard Zou authored Jun 15, 2025
  
  91b2c17a
- Enable prefix caching with full cuda graphs (#19617) · 055915e6
  Woosuk Kwon authored Jun 15, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  055915e6
- [Benchmark] Refactor benchmark script for fp8 & int8 (#19627) · 3d330c4c
  Wentao Ye authored Jun 15, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  3d330c4c
- [Kernel] Raise verbose error and consolidate `num_heads/num_kv_heads` divisibility check (#19339) · 0b73736a
  22quinn authored Jun 14, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  0b73736a