Commits · 7f0be2aa24dfacf6f7aaf92411fdcd2f55ff8478 · OpenDAS / vllm_cscc

"examples/offline_inference/vision_language_multi_image.py" did not exist on "2ca830dbaa1a7c30b8ff4d7c860c63f87dc18be3"

27 Feb, 2025 2 commits
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
- [VLM] Support multimodal inputs for Florence-2 models (#13320) · edf309eb
  Isotr0py authored Feb 27, 2025
  
  edf309eb
26 Feb, 2025 6 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Bugfix] Backend option to disable xgrammar any_whitespace (#12744) · 4cb6fa0a
  Wallas Henrique authored Feb 26, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4cb6fa0a
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) · e206b543
  Seth Kimmel authored Feb 25, 2025
```
Signed-off-by: Seth Kimmel <seth.kimmel3@gmail.com>
```
  e206b543
- DeepSeek V2/V3/R1 only place `lm_head` on last pp rank (#13833) · 24679788
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  24679788
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 10 commits
- [Neuron] Add custom_ops for neuron backend (#13246) · f75aa727
  Liangfu Chen authored Feb 25, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
```
  f75aa727
- [Bugfix] Revert inspection code in #13743 (#13832) · f4133ce4
  Cyrus Leung authored Feb 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4133ce4
- [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) · 6ff51862
  Isotr0py authored Feb 25, 2025
  
  6ff51862
- [Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783) · aab39277
  Russell Bryant authored Feb 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  aab39277
- [Misc] Check that the model can be inspected upon registration (#13743) · 6724e791
  Cyrus Leung authored Feb 25, 2025
  
  6724e791
- Fix CompressedTensorsWNA16MoE with grouped scales (#13769) · 4d251ad0
  Michael Goin authored Feb 25, 2025
  
  4d251ad0
- [Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802) · 4a8cfc75
  Lucas Wilkinson authored Feb 24, 2025
  
  4a8cfc75
- [Bugfix][Quantization] Fix FP8 + EP (#13784) · 1e15aaef
  Tyler Michael Smith authored Feb 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1e15aaef
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
24 Feb, 2025 4 commits
- Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772) · db986c19
  Michael Goin authored Feb 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  db986c19
- [BugFix] Illegal memory access for MoE On H20 (#13693) · ccc00515
  Zhonghua Deng authored Feb 24, 2025
  
  ccc00515
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
- [model][refactor] remove cuda hard code in models and layers (#13658) · 23eca9cf
  Mengqing Cao authored Feb 24, 2025
  
  23eca9cf
23 Feb, 2025 3 commits
- [LMM] Implement merged multimodal processor for whisper (#13278) · ba5106e5
  Isotr0py authored Feb 23, 2025
  
  ba5106e5
- [Quant] BaiChuan SupportsQuant (#13710) · d5ca2110
  Kyle Sayers authored Feb 22, 2025
  
  d5ca2110
- [ci] Use env var to control whether to use S3 bucket in CI (#13634) · 2c5e637b
  Kevin H. Luu authored Feb 22, 2025
  
  2c5e637b
22 Feb, 2025 6 commits
- [Bugfix] Fix boolean conversion for OpenVINO env variable (#13615) · 382f66fb
  Helena Kloosterman authored Feb 22, 2025
  
  382f66fb
- [ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231) · c904fddd
  Gregory Shtrasberg authored Feb 22, 2025
  
  c904fddd
- [CI/Build] Fix pre-commit errors (#13696) · 7f6bae56
  Cyrus Leung authored Feb 22, 2025
  
  7f6bae56
- [Misc] Reduce LoRA-related static variable (#13166) · 105b8ce4
  Jee Jee Li authored Feb 22, 2025
  
  105b8ce4
- Correction to TP logic for Mamba Mixer 2 when Num Groups not divisible by TP Size (#13660) · fca20841
  Yu Chin Fabian Lim authored Feb 22, 2025
  
  fca20841
- [Bugfix][Model] OLMo 2: split qkv correctly for GQA and MQA (#13687) · 9a1f1da5
  Shane A authored Feb 21, 2025
  
  9a1f1da5
21 Feb, 2025 1 commit

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

20 Feb, 2025 6 commits
- [Frontend] Add backend-specific options for guided decoding (#13505) · bfbc0b32
  Joe Runde authored Feb 20, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  bfbc0b32
- [Bugfix] Fix deepseekv3 grouped topk error (#13474) · ed6e9075
  chenxiaobing authored Feb 20, 2025
```
Signed-off-by: Chen-XiaoBing <chenxb002@whu.edu.cn>
```
  ed6e9075
- [Misc] add mm_processor_kwargs to extra_body for Qwen2.5-VL (#13533) · 041e2947
  燃 authored Feb 20, 2025
  
  041e2947
- [ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577) · 0d243f2a
  Divakar Verma authored Feb 19, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  0d243f2a
- [Misc] Qwen2.5 VL support LoRA (#13261) · 512368e3
  Jee Jee Li authored Feb 20, 2025
  
  512368e3
- [3/n][CI] Load Quantization test models with S3 (#13570) · 473f51cf
  Kevin H. Luu authored Feb 19, 2025
```
Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>
```
  473f51cf
19 Feb, 2025 2 commits
- [VLM][Bugfix] Pass processor kwargs properly on init (#13516) · 377d10bd
  Cyrus Leung authored Feb 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  377d10bd
- [Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) · f525c0be
  Lucia Fang authored Feb 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f525c0be