Commits · 733e7c9e95f5b066ac420b00701eef7ea164a79e · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "346e56455a3b10787c7d6e951861317953723185"

26 Mar, 2025 14 commits
- [Refactor] Remove unnecessary backend parameter in structured output interface (#15317) · 733e7c9e
  Aaron Pham authored Mar 26, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  733e7c9e
- Fix weight loading for some models in Transformers backend (#15544) · 0af4d764
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0af4d764
- [Model] Add Reasoning Parser for Granite Models (#14202) · 1711b929
  Alex Brooks authored Mar 26, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
```
  1711b929
- Improve validation of TP in Transformers backend (#15540) · c091c0a5
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c091c0a5
- Apply torchfix (#15532) · 1aa162e0
  cyyever authored Mar 26, 2025
```
Signed-off-by: cyy <cyyever@outlook.com>
```
  1aa162e0
- Separate base model from `TransformersModel` (#15467) · cf5c8f16
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf5c8f16
- [Misc] Enhance warning information to user-defined chat template (#15408) · 99f536f8
  wwl2755 authored Mar 26, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  99f536f8
- [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) · 5ebf6674
  vllmellm authored Mar 26, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  5ebf6674
- [Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) · 781d0562
  Bryan Lu authored Mar 26, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  781d0562
- Fix raw_request extraction in load_aware_call decorator (#15382) · 5aefd6ac
  daniel-salib authored Mar 25, 2025
```
Signed-off-by: Daniel Salib <danielsalib@meta.com>
```
  5aefd6ac
- [misc] LoRA - Skip LoRA kernels when not required (#15152) · 6c663dfd
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  6c663dfd
- [BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1,... · 33437bc6
  Lucas Wilkinson authored Mar 25, 2025
```
[BugFix] Fix nightly MLA failure (FA2 + MLA chunked prefill, i.e. V1, producing bad results) (#15492)
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
  33437bc6
- [Model] Support multi-image for Molmo (#15438) · 997c8811
  Cyrus Leung authored Mar 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  997c8811
- Transformers backend already supports V1 (#15463) · e42389f9
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e42389f9
25 Mar, 2025 21 commits
- [Core] LoRA: V1 Scheduler optimization (#15422) · a5cfbab3
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a5cfbab3
- [core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8
  Chenyaaang authored Mar 25, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  ac3cd6e8
- [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) · 082ab86f
  Lu Fang authored Mar 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  082ab86f
- [V1][Minor] Use `SchedulerInterface` type for engine scheduler field (#15499) · 6aa196c8
  Nick Hill authored Mar 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  6aa196c8
- [TPU][V1] Fix Sampler recompilation (#15309) · a0dd7dcd
  Nicolò Lucchesi authored Mar 25, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a0dd7dcd
- Add workaround for shared field_names in pydantic model class (#13925) · e977c111
  Maximilien de Bayser authored Mar 25, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e977c111
- [bugfix] add supports_v1 platform interface (#15417) · 5f063a80
  Joe Runde authored Mar 25, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  5f063a80
- [Bugfix] Support triton==3.3.0+git95326d9f for RTX 5090 (Unsloth + vLLM compatibility) (#15471) · 5d8e1c92
  Antonio Gómez authored Mar 25, 2025
```
Co-authored-by: ServerAI <ai@exc-mad-ai.com>
```
  5d8e1c92
- [bugfix] fix inductor cache on max_position_embeddings (#15436) · d0cfec7a
  youkaichao authored Mar 25, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d0cfec7a
- [Misc] Remove redundant `num_embeds` (#15443) · 5994430b
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5994430b
- [Misc] Clean up MiniCPM-V/O code (#15337) · a9e879b3
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a9e879b3
- [Kernel][CPU] CPU MLA (#14744) · 4f044b1d
  Thien Tran authored Mar 25, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  4f044b1d
- [Hardware][TPU][Bugfix] Fix v1 mp profiler (#15409) · 4157f563
  Siyuan Liu authored Mar 25, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  4157f563
- [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) · 25f560a6
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25f560a6
- [V1] guidance backend for structured output + `auto` fallback mode (#14779) · a09ad90a
  Russell Bryant authored Mar 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
```
  a09ad90a
- [Bugfix] Fixed the issue of not being able to input video and image simultaneously (#15387) · 10b34e36
  Chauncey authored Mar 25, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  10b34e36
- Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" (#15398) · b5269db9
  Tyler Michael Smith authored Mar 24, 2025
  
  b5269db9
- [Misc] Remove LoRA log (#15388) · 6db94571
  Jee Jee Li authored Mar 25, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  6db94571
- Add pipeline parallel support to `TransformersModel` (#12832) · 97cfa65d
  Harry Mellor authored Mar 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  97cfa65d
- [Minor][Spec Decode] Remove compiled_softmax (#15416) · 911c8eb0
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  911c8eb0
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 5 commits

[ROCm][Kernel] MoE weights padding (#14454) · f533b583

Gregory Shtrasberg authored Mar 24, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>

f533b583

[Build] Cython compilation support fix (#14296) · 8279201c
Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
8279201c
[BugFix][V1] Quick fix for min_tokens with multiple EOS (#15407) · 623e2ed2
Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
623e2ed2

[V1][Perf] Simpler request output queues (#15156) · 9d72daf4

Nick Hill authored Mar 24, 2025


Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>

9d72daf4

[Doc] Update docs on handling OOM (#15357) · 6dd55af6

Cyrus Leung authored Mar 25, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

6dd55af6