Commits · 084bbac8cc4c29b7dcd2098418168c61d3d42e9b · OpenDAS / vllm_cscc · GitLab

28 Feb, 2025 14 commits
- [core] Bump ray to 2.43 (#13994) · 084bbac8
  Rui Qiao authored Feb 28, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  084bbac8
- [v1] Move block pool operations to a separate class (#13973) · 28943d36
  Chen Zhang authored Mar 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  28943d36
- [v1] Cleanup the BlockTable in InputBatch (#13977) · e7bd944e
  Chen Zhang authored Mar 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e7bd944e
- [V1][TPU] Integrate the new ragged paged attention kernel with vLLM v1 on TPU (#13379) · c3b6559a
  iefgnoix authored Feb 28, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  c3b6559a
- [Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series (#14031) · 9b61dd41
  Yang Liu authored Feb 28, 2025
  
  9b61dd41
- [VLM][Bugfix] Enable specifying prompt target via index (#14038) · f7bee5c8
  Cyrus Leung authored Feb 28, 2025
  
  f7bee5c8
- [Bugfix] Fix MoeWNA16Method activation (#14024) · e0734387
  Jee Jee Li authored Feb 28, 2025
  
  e0734387
- [V1][Minor] Restore V1 compatibility with LLMEngine class (#13090) · b3f7aacc
  Thibault Schueller authored Feb 28, 2025
  
  b3f7aacc
- [Hardware][Intel-Gaudi] Regional compilation support (#13213) · b91660dd
  Kacper Pietkun authored Feb 28, 2025
  
  b91660dd
- Use smaller embedding model when not testing model specifically (#13891) · 76c89fca
  Harry Mellor authored Feb 28, 2025
  
  76c89fca
- [Bugfix][Disaggregated] patch the inflight batching on the decode node in... · b9e41734
  Mathis Felardos authored Feb 28, 2025
```
[Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) (#13987)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>
```
  b9e41734
- [Doc] Move multimodal Embedding API example to Online Serving page (#14017) · 1088f062
  Cyrus Leung authored Feb 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1088f062
- [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#13911) · 73e0225e
  Travis Johnson authored Feb 27, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  73e0225e
- [V1]`SupportsV0Only` protocol for model definitions (#13959) · 6c85da3a
  Roger Wang authored Feb 27, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  6c85da3a
27 Feb, 2025 16 commits
- [Misc] Print FusedMoE detail info (#13974) · 67fc4268
  Jee Jee Li authored Feb 28, 2025
  
  67fc4268
- [Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626) · 9804145c
  Benjamin Chislett authored Feb 27, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  9804145c
- [Attention] Flash MLA for V1 (#13867) · 2e94b9cf
  Lucas Wilkinson authored Feb 27, 2025
```
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yang Chen <yangche@fb.com>
```
  2e94b9cf
- [core] Perf improvement for DSv3 on AMD GPUs (#13718) · 8294773e
  qli88 authored Feb 27, 2025
```
Signed-off-by: qli88 <qiang.li2@amd.com>
```
  8294773e
- [V1][Minor] Minor cleanup for GPU Model Runner (#13983) · cd813c6d
  Woosuk Kwon authored Feb 27, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  cd813c6d
- [VLM] Deprecate legacy input mapper for OOT multimodal models (#13979) · a2dd48c3
  Cyrus Leung authored Feb 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a2dd48c3
- [Attention] MLA support for V1 (#13789) · 58d1b2aa
  Yang Chen authored Feb 27, 2025
```
Signed-off-by: Yang Chen <yangche@fb.com>
```
  58d1b2aa
- [VLM] Generalized prompt updates for multi-modal processor (#13964) · f1579b22
  Cyrus Leung authored Feb 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f1579b22
- [Bugfix] Fix qwen2.5-vl overflow issue (#13968) · 78648758
  Isotr0py authored Feb 28, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  78648758
- [bugfix] Fix profiling for RayDistributedExecutor (#13945) · 06c8f8d8
  Rui Qiao authored Feb 27, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  06c8f8d8
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
- [VLM] Support multimodal inputs for Florence-2 models (#13320) · edf309eb
  Isotr0py authored Feb 27, 2025
  
  edf309eb
- [PP] Correct cache size check (#13873) · 4b1d141f
  Yang Zheng authored Feb 27, 2025
```
Signed-off-by: Yang Zheng <zhengy.gator@gmail.com>
```
  4b1d141f
- [V1][Metrics] Handle preemptions (#13169) · cd711c48
  Mark McLoughlin authored Feb 27, 2025
  
  cd711c48
- [misc] Rename Ray ADAG to Compiled Graph (#13928) · c9944acb
  Rui Qiao authored Feb 26, 2025
  
  c9944acb
- [Kernel] FlashMLA integration (#13747) · f9590390
  Lucas Wilkinson authored Feb 26, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f9590390
26 Feb, 2025 10 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Bugfix] Backend option to disable xgrammar any_whitespace (#12744) · 4cb6fa0a
  Wallas Henrique authored Feb 26, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4cb6fa0a
- [Bugfix] Do not crash V0 engine on input errors (#13101) · 3f808cc0
  Joe Runde authored Feb 26, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  3f808cc0
- [Bugfix] Handle None parameters in Mistral function calls. (#13786) · 215bf150
  Florian Greinacher authored Feb 26, 2025
  
  215bf150
- Add comments on accessing `kv_cache` and `attn_metadata` (#13887) · 0ecdd980
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0ecdd980
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [Misc] Improve LoRA spelling (#13831) · 5157338e
  Jee Jee Li authored Feb 26, 2025
  
  5157338e
- [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) · e206b543
  Seth Kimmel authored Feb 25, 2025
```
Signed-off-by: Seth Kimmel <seth.kimmel3@gmail.com>
```
  e206b543
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e
- Improve pipeline partitioning (#13839) · 145944cb
  Harry Mellor authored Feb 26, 2025
  
  145944cb