Commits · edf309ebbe25fcf55569c7fe94e3fe78428a8244 · OpenDAS / vllm_cscc · GitLab

27 Feb, 2025 5 commits
- [VLM] Support multimodal inputs for Florence-2 models (#13320) · edf309eb
  Isotr0py authored Feb 27, 2025
  
  edf309eb
- [PP] Correct cache size check (#13873) · 4b1d141f
  Yang Zheng authored Feb 27, 2025
```
Signed-off-by: Yang Zheng <zhengy.gator@gmail.com>
```
  4b1d141f
- [V1][Metrics] Handle preemptions (#13169) · cd711c48
  Mark McLoughlin authored Feb 27, 2025
  
  cd711c48
- [misc] Rename Ray ADAG to Compiled Graph (#13928) · c9944acb
  Rui Qiao authored Feb 26, 2025
  
  c9944acb
- [Kernel] FlashMLA integration (#13747) · f9590390
  Lucas Wilkinson authored Feb 26, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f9590390
26 Feb, 2025 15 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Bugfix] Backend option to disable xgrammar any_whitespace (#12744) · 4cb6fa0a
  Wallas Henrique authored Feb 26, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4cb6fa0a
- [Bugfix] Do not crash V0 engine on input errors (#13101) · 3f808cc0
  Joe Runde authored Feb 26, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  3f808cc0
- [Bugfix] Handle None parameters in Mistral function calls. (#13786) · 215bf150
  Florian Greinacher authored Feb 26, 2025
  
  215bf150
- Add comments on accessing `kv_cache` and `attn_metadata` (#13887) · 0ecdd980
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0ecdd980
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [Misc] Improve LoRA spelling (#13831) · 5157338e
  Jee Jee Li authored Feb 26, 2025
  
  5157338e
- [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) · e206b543
  Seth Kimmel authored Feb 25, 2025
```
Signed-off-by: Seth Kimmel <seth.kimmel3@gmail.com>
```
  e206b543
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e
- Improve pipeline partitioning (#13839) · 145944cb
  Harry Mellor authored Feb 26, 2025
  
  145944cb
- [Misc]Code Cleanup (#13859) · e1fe7591
  Chenguang Li authored Feb 26, 2025
```
Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
```
  e1fe7591
- [V1][Spec Decode] Change Spec Decode Rejection Sampling API (#13729) · 5629f26d
  Lily Liu authored Feb 25, 2025
  
  5629f26d
- [misc] Show driver IP info when Ray fails to allocate driver worker (#13858) · 9ba28043
  Rui Qiao authored Feb 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  9ba28043
- DeepSeek V2/V3/R1 only place `lm_head` on last pp rank (#13833) · 24679788
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  24679788
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 20 commits
- [Neuron] Add custom_ops for neuron backend (#13246) · f75aa727
  Liangfu Chen authored Feb 25, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
```
  f75aa727
- Fix string parsing error (#13825) · 340e39e3
  Chen1022 authored Feb 26, 2025
  
  340e39e3
- [Bugfix] Revert inspection code in #13743 (#13832) · f4133ce4
  Cyrus Leung authored Feb 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4133ce4
- Fix `/v1/audio/transcriptions ` Bad Request Error (#13811) · 6522d55b
  Wen Sun authored Feb 25, 2025
  
  6522d55b
- [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) · 6ff51862
  Isotr0py authored Feb 25, 2025
  
  6ff51862
- [Bugfix] Flush TunableOp results before worker processes are destroyed. (#13623) · fa820741
  Nichols A. Romero authored Feb 25, 2025
```
Signed-off-by: Nichols A. Romero <nick.romero@amd.com>
```
  fa820741
- [Bugfix] Initialize attention bias on the same device as Query/Key/Value (#13468) · 75e9d497
  Junlin Zhou authored Feb 25, 2025
  
  75e9d497
- [Misc]Clarify Error Handling for Non-existent Model Paths and HF Repo IDs (#13724) · 32c3b6bf
  Chen1022 authored Feb 25, 2025
```
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  32c3b6bf
- [Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953) · 2f42a488
  Jiayi Yao authored Feb 25, 2025
  
  2f42a488
- [misc] Clean up ray compiled graph type hints (#13731) · 3173c3b3
  Rui Qiao authored Feb 25, 2025
  
  3173c3b3
- [Bugfix] Modify modelscope api usage in transformer_utils (#13807) · 2d87d7d1
  Shanshan Shen authored Feb 25, 2025
  
  2d87d7d1
- [Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783) · aab39277
  Russell Bryant authored Feb 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  aab39277
- [Misc] Check that the model can be inspected upon registration (#13743) · 6724e791
  Cyrus Leung authored Feb 25, 2025
  
  6724e791
- [Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705) · 03f48b3d
  Varun Sundar Rabindranath authored Feb 25, 2025
  
  03f48b3d
- Fix CompressedTensorsWNA16MoE with grouped scales (#13769) · 4d251ad0
  Michael Goin authored Feb 25, 2025
  
  4d251ad0
- [Bugfix] Support MLA for CompressedTensorsWNA16 (#13725) · 18e50593
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  18e50593
- [Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802) · 4a8cfc75
  Lucas Wilkinson authored Feb 24, 2025
  
  4a8cfc75
- [V1][Metrics] Implement vllm:lora_requests_info metric (#13504) · bc32bc73
  Mark McLoughlin authored Feb 25, 2025
  
  bc32bc73
- [Misc][Attention][Quantization] init property earlier (#13733) · ab1091d5
  wangxiyuan authored Feb 25, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  ab1091d5
- [Bugfix][Quantization] Fix FP8 + EP (#13784) · 1e15aaef
  Tyler Michael Smith authored Feb 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1e15aaef