Commits · f64ffa8c2541ac5dabd1eed3edeeeed9c618f7a3 · OpenDAS / vllm_cscc · GitLab

28 Feb, 2025 7 commits
- [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename... · bd56c983
  Luka Govedič authored Feb 28, 2025
```
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
```
  bd56c983
- [Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series (#14031) · 9b61dd41
  Yang Liu authored Feb 28, 2025
  
  9b61dd41
- [VLM][Bugfix] Enable specifying prompt target via index (#14038) · f7bee5c8
  Cyrus Leung authored Feb 28, 2025
  
  f7bee5c8
- [Bugfix] Fix MoeWNA16Method activation (#14024) · e0734387
  Jee Jee Li authored Feb 28, 2025
  
  e0734387
- [Doc] Move multimodal Embedding API example to Online Serving page (#14017) · 1088f062
  Cyrus Leung authored Feb 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1088f062
- [Bugfix] Check that number of images matches number of <|image|> tokens with mllama (#13911) · 73e0225e
  Travis Johnson authored Feb 27, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  73e0225e
- [V1]`SupportsV0Only` protocol for model definitions (#13959) · 6c85da3a
  Roger Wang authored Feb 27, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  6c85da3a
27 Feb, 2025 8 commits
- [Misc] Print FusedMoE detail info (#13974) · 67fc4268
  Jee Jee Li authored Feb 28, 2025
  
  67fc4268
- [Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626) · 9804145c
  Benjamin Chislett authored Feb 27, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  9804145c
- [core] Perf improvement for DSv3 on AMD GPUs (#13718) · 8294773e
  qli88 authored Feb 27, 2025
```
Signed-off-by: qli88 <qiang.li2@amd.com>
```
  8294773e
- [Attention] MLA support for V1 (#13789) · 58d1b2aa
  Yang Chen authored Feb 27, 2025
```
Signed-off-by: Yang Chen <yangche@fb.com>
```
  58d1b2aa
- [VLM] Generalized prompt updates for multi-modal processor (#13964) · f1579b22
  Cyrus Leung authored Feb 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f1579b22
- [Bugfix] Fix qwen2.5-vl overflow issue (#13968) · 78648758
  Isotr0py authored Feb 28, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  78648758
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
- [VLM] Support multimodal inputs for Florence-2 models (#13320) · edf309eb
  Isotr0py authored Feb 27, 2025
  
  edf309eb
26 Feb, 2025 6 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Bugfix] Backend option to disable xgrammar any_whitespace (#12744) · 4cb6fa0a
  Wallas Henrique authored Feb 26, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
```
  4cb6fa0a
- [Misc] Fix input processing for Ultravox (#13871) · 7ca1da02
  Roger Wang authored Feb 25, 2025
  
  7ca1da02
- [v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine (#13837) · e206b543
  Seth Kimmel authored Feb 25, 2025
```
Signed-off-by: Seth Kimmel <seth.kimmel3@gmail.com>
```
  e206b543
- DeepSeek V2/V3/R1 only place `lm_head` on last pp rank (#13833) · 24679788
  Harry Mellor authored Feb 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  24679788
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 10 commits
- [Neuron] Add custom_ops for neuron backend (#13246) · f75aa727
  Liangfu Chen authored Feb 25, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
```
  f75aa727
- [Bugfix] Revert inspection code in #13743 (#13832) · f4133ce4
  Cyrus Leung authored Feb 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f4133ce4
- [Bugfix] Fix deepseek-vl2 inference with more than 2 images (#13818) · 6ff51862
  Isotr0py authored Feb 25, 2025
  
  6ff51862
- [Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783) · aab39277
  Russell Bryant authored Feb 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  aab39277
- [Misc] Check that the model can be inspected upon registration (#13743) · 6724e791
  Cyrus Leung authored Feb 25, 2025
  
  6724e791
- Fix CompressedTensorsWNA16MoE with grouped scales (#13769) · 4d251ad0
  Michael Goin authored Feb 25, 2025
  
  4d251ad0
- [Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802) · 4a8cfc75
  Lucas Wilkinson authored Feb 24, 2025
  
  4a8cfc75
- [Bugfix][Quantization] Fix FP8 + EP (#13784) · 1e15aaef
  Tyler Michael Smith authored Feb 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1e15aaef
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
24 Feb, 2025 4 commits
- Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772) · db986c19
  Michael Goin authored Feb 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  db986c19
- [BugFix] Illegal memory access for MoE On H20 (#13693) · ccc00515
  Zhonghua Deng authored Feb 24, 2025
  
  ccc00515
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
- [model][refactor] remove cuda hard code in models and layers (#13658) · 23eca9cf
  Mengqing Cao authored Feb 24, 2025
  
  23eca9cf
23 Feb, 2025 3 commits
- [LMM] Implement merged multimodal processor for whisper (#13278) · ba5106e5
  Isotr0py authored Feb 23, 2025
  
  ba5106e5
- [Quant] BaiChuan SupportsQuant (#13710) · d5ca2110
  Kyle Sayers authored Feb 22, 2025
  
  d5ca2110
- [ci] Use env var to control whether to use S3 bucket in CI (#13634) · 2c5e637b
  Kevin H. Luu authored Feb 22, 2025
  
  2c5e637b
22 Feb, 2025 2 commits
- [Bugfix] Fix boolean conversion for OpenVINO env variable (#13615) · 382f66fb
  Helena Kloosterman authored Feb 22, 2025
  
  382f66fb
- [ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231) · c904fddd
  Gregory Shtrasberg authored Feb 22, 2025
  
  c904fddd