Commits · a31614e386edf86ea949dd7214b13f82f734b05c · OpenDAS / vllm_cscc · GitLab

27 Feb, 2025 2 commits
- [ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undefined (#13851) · a31614e3
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Feb 27, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  a31614e3
- [Kernel] FlashMLA integration (#13747) · f9590390
  Lucas Wilkinson authored Feb 26, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f9590390
26 Feb, 2025 22 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Bugfix] Backend option to disable xgrammar any_whitespace (#12744) · 4cb6fa0a
  Wallas Henrique authored Feb 26, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4cb6fa0a
- [Misc] fixed qwen_vl_utils parameter error (#13906) · d08b285a
  Chauncey authored Feb 27, 2025
  
  d08b285a
- [TPU] use torch2.6 with whl package (#13860) · b27122ac
  Chenyaaang authored Feb 26, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
```
  b27122ac
- [Bugfix] Update expected token counts for Ultravox tests (#13895) · 934bb99c
  Cyrus Leung authored Feb 26, 2025
  
  934bb99c
- [Bugfix] Do not crash V0 engine on input errors (#13101) · 3f808cc0
  Joe Runde authored Feb 26, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  3f808cc0
- [Misc]: Add support for goodput on guided benchmarking + TPOT calculation refactor (#13736) · ec8a5e53
  Brayden Zhong authored Feb 26, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  ec8a5e53
- [Bugfix] Handle None parameters in Mistral function calls. (#13786) · 215bf150
  Florian Greinacher authored Feb 26, 2025
  
  215bf150
- Add comments on accessing `kv_cache` and `attn_metadata` (#13887) · 0ecdd980
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0ecdd980
- [Bugfix] Add test example for Ultravox v0.5 (#13890) · 7b700ec8
  Cyrus Leung authored Feb 26, 2025
  
  7b700ec8
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [Misc] Improve LoRA spelling (#13831) · 5157338e
  Jee Jee Li authored Feb 26, 2025
  
  5157338e
- [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) · e206b543
  Seth Kimmel authored Feb 25, 2025
```
Signed-off-by: Seth Kimmel <seth.kimmel3@gmail.com>
```
  e206b543
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e
- [Doc] fix the incorrect module path of tensorize_vllm_model (#13863) · e656f638
  Albert authored Feb 26, 2025
  
  e656f638
- Improve pipeline partitioning (#13839) · 145944cb
  Harry Mellor authored Feb 26, 2025
  
  145944cb
- [Kernel][Build/CI] Bump CUTLASS to 3.8 and add initializers for cutlass epilogues (#13797) · 094b7d94
  Henry Tsang authored Feb 25, 2025
  
  094b7d94
- [Misc]Code Cleanup (#13859) · e1fe7591
  Chenguang Li authored Feb 26, 2025
```
Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
```
  e1fe7591
- [V1][Spec Decode] Change Spec Decode Rejection Sampling API (#13729) · 5629f26d
  Lily Liu authored Feb 25, 2025
  
  5629f26d
- [misc] Show driver IP info when Ray fails to allocate driver worker (#13858) · 9ba28043
  Rui Qiao authored Feb 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  9ba28043
- DeepSeek V2/V3/R1 only place `lm_head` on last pp rank (#13833) · 24679788
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  24679788
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 16 commits
- Fix failing `MyGemma2Embedding` test (#13820) · 34e3494e
  Harry Mellor authored Feb 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  34e3494e
- [Neuron] Add custom_ops for neuron backend (#13246) · f75aa727
  Liangfu Chen authored Feb 25, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
```
  f75aa727
- Fix string parsing error (#13825) · 340e39e3
  Chen1022 authored Feb 26, 2025
  
  340e39e3
- [Bugfix] Revert inspection code in #13743 (#13832) · f4133ce4
  Cyrus Leung authored Feb 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4133ce4
- Fix `/v1/audio/transcriptions ` Bad Request Error (#13811) · 6522d55b
  Wen Sun authored Feb 25, 2025
  
  6522d55b
- [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) · 6ff51862
  Isotr0py authored Feb 25, 2025
  
  6ff51862
- [Bugfix] Flush TunableOp results before worker processes are destroyed. (#13623) · fa820741
  Nichols A. Romero authored Feb 25, 2025
```
Signed-off-by: Nichols A. Romero <nick.romero@amd.com>
```
  fa820741
- [Bugfix] Initialize attention bias on the same device as Query/Key/Value (#13468) · 75e9d497
  Junlin Zhou authored Feb 25, 2025
  
  75e9d497
- [Misc]Clarify Error Handling for Non-existent Model Paths and HF Repo IDs (#13724) · 32c3b6bf
  Chen1022 authored Feb 25, 2025
```
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  32c3b6bf
- [CI/Build] Fix V1 LoRA failure (#13767) · 37b6cb49
  Jee Jee Li authored Feb 25, 2025
  
  37b6cb49
- [ROCm][Quantization][Kernel] Using HIP FP8 header (#12593) · aabeb268
  Gregory Shtrasberg authored Feb 25, 2025
  
  aabeb268
- [Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953) · 2f42a488
  Jiayi Yao authored Feb 25, 2025
  
  2f42a488
- [misc] Clean up ray compiled graph type hints (#13731) · 3173c3b3
  Rui Qiao authored Feb 25, 2025
  
  3173c3b3
- [Bugfix] Modify modelscope api usage in transformer_utils (#13807) · 2d87d7d1
  Shanshan Shen authored Feb 25, 2025
  
  2d87d7d1
- [Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783) · aab39277
  Russell Bryant authored Feb 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  aab39277
- [Misc] Check that the model can be inspected upon registration (#13743) · 6724e791
  Cyrus Leung authored Feb 25, 2025
  
  6724e791