Commits · d08b285adfb03f0b7952c8af4c2c83bb2dafcc2d · OpenDAS / vllm_cscc · GitLab

26 Feb, 2025 13 commits
- [Bugfix] Do not crash V0 engine on input errors (#13101) · 3f808cc0
  Joe Runde authored Feb 26, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  3f808cc0
- [Bugfix] Handle None parameters in Mistral function calls. (#13786) · 215bf150
  Florian Greinacher authored Feb 26, 2025
  
  215bf150
- Add comments on accessing `kv_cache` and `attn_metadata` (#13887) · 0ecdd980
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0ecdd980
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [Misc] Improve LoRA spelling (#13831) · 5157338e
  Jee Jee Li authored Feb 26, 2025
  
  5157338e
- [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) · e206b543
  Seth Kimmel authored Feb 25, 2025
```
Signed-off-by: Seth Kimmel <seth.kimmel3@gmail.com>
```
  e206b543
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e
- Improve pipeline partitioning (#13839) · 145944cb
  Harry Mellor authored Feb 26, 2025
  
  145944cb
- [Misc]Code Cleanup (#13859) · e1fe7591
  Chenguang Li authored Feb 26, 2025
```
Signed-off-by: noemotiovon <noemotiovon@gmail.com>
Co-authored-by: noemotiovon <noemotiovon@gmail.com>
```
  e1fe7591
- [V1][Spec Decode] Change Spec Decode Rejection Sampling API (#13729) · 5629f26d
  Lily Liu authored Feb 25, 2025
  
  5629f26d
- [misc] Show driver IP info when Ray fails to allocate driver worker (#13858) · 9ba28043
  Rui Qiao authored Feb 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  9ba28043
- DeepSeek V2/V3/R1 only place `lm_head` on last pp rank (#13833) · 24679788
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  24679788
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 24 commits
- [Neuron] Add custom_ops for neuron backend (#13246) · f75aa727
  Liangfu Chen authored Feb 25, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
```
  f75aa727
- Fix string parsing error (#13825) · 340e39e3
  Chen1022 authored Feb 26, 2025
  
  340e39e3
- [Bugfix] Revert inspection code in #13743 (#13832) · f4133ce4
  Cyrus Leung authored Feb 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4133ce4
- Fix `/v1/audio/transcriptions ` Bad Request Error (#13811) · 6522d55b
  Wen Sun authored Feb 25, 2025
  
  6522d55b
- [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) · 6ff51862
  Isotr0py authored Feb 25, 2025
  
  6ff51862
- [Bugfix] Flush TunableOp results before worker processes are destroyed. (#13623) · fa820741
  Nichols A. Romero authored Feb 25, 2025
```
Signed-off-by: Nichols A. Romero <nick.romero@amd.com>
```
  fa820741
- [Bugfix] Initialize attention bias on the same device as Query/Key/Value (#13468) · 75e9d497
  Junlin Zhou authored Feb 25, 2025
  
  75e9d497
- [Misc]Clarify Error Handling for Non-existent Model Paths and HF Repo IDs (#13724) · 32c3b6bf
  Chen1022 authored Feb 25, 2025
```
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  32c3b6bf
- [Feature] Support KV cache offloading and disagg prefill with LMCache connector. (#12953) · 2f42a488
  Jiayi Yao authored Feb 25, 2025
  
  2f42a488
- [misc] Clean up ray compiled graph type hints (#13731) · 3173c3b3
  Rui Qiao authored Feb 25, 2025
  
  3173c3b3
- [Bugfix] Modify modelscope api usage in transformer_utils (#13807) · 2d87d7d1
  Shanshan Shen authored Feb 25, 2025
  
  2d87d7d1
- [Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783) · aab39277
  Russell Bryant authored Feb 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  aab39277
- [Misc] Check that the model can be inspected upon registration (#13743) · 6724e791
  Cyrus Leung authored Feb 25, 2025
  
  6724e791
- [Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705) · 03f48b3d
  Varun Sundar Rabindranath authored Feb 25, 2025
  
  03f48b3d
- Fix CompressedTensorsWNA16MoE with grouped scales (#13769) · 4d251ad0
  Michael Goin authored Feb 25, 2025
  
  4d251ad0
- [Bugfix] Support MLA for CompressedTensorsWNA16 (#13725) · 18e50593
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  18e50593
- [Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802) · 4a8cfc75
  Lucas Wilkinson authored Feb 24, 2025
  
  4a8cfc75
- [V1][Metrics] Implement vllm:lora_requests_info metric (#13504) · bc32bc73
  Mark McLoughlin authored Feb 25, 2025
  
  bc32bc73
- [Misc][Attention][Quantization] init property earlier (#13733) · ab1091d5
  wangxiyuan authored Feb 25, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  ab1091d5
- [Bugfix][Quantization] Fix FP8 + EP (#13784) · 1e15aaef
  Tyler Michael Smith authored Feb 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1e15aaef
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
- [Doc] arg_utils.py: fixed a typo (#13785) · 7196a3b1
  Eli Boyarski authored Feb 25, 2025
  
  7196a3b1
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
- [Misc][Chore] Clean Up `AsyncOutputProcessing` Logs (#13780) · f61528d4
  Robert Shaw authored Feb 24, 2025
  
  f61528d4
24 Feb, 2025 3 commits
- [Misc] Clean Up `EngineArgs.create_engine_config` (#13734) · 1f0ae3ed
  Robert Shaw authored Feb 24, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  1f0ae3ed
- Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772) · db986c19
  Michael Goin authored Feb 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  db986c19
- Revert "[V1][Core] Fix memory issue with logits & sampling" (#13775) · 22757848
  Roger Wang authored Feb 24, 2025
  
  22757848