Commits · 1fc973c0b53a151bba062576cc3ff489cc1ceeec · OpenDAS / vllm_cscc

11 Mar, 2025 1 commit

[V1][Core] Fix memory issue with logits & sampling (#14508) · 1fc973c0

Roger Wang authored Mar 10, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Varun Sundar Rabindranath <3337719+varun-sundar-rabindranath@users.noreply.github.com>

1fc973c0

10 Mar, 2025 1 commit
- Correct capitalisation: `VLLM` -> `vLLM` (#14562) · 3b352a2f
  Harry Mellor authored Mar 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3b352a2f
08 Mar, 2025 2 commits
- [Misc] add `use_tqdm_on_load` to reduce logs (#14407) · 0b7f06b4
  Aaron Pham authored Mar 08, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  0b7f06b4
- Default to `generation_config` from model (#12622) · 47512b32
  Harry Mellor authored Mar 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  47512b32
07 Mar, 2025 1 commit
- [Bugfix] Clean up multi-modal processors (#14417) · 05fb6718
  Cyrus Leung authored Mar 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  05fb6718
06 Mar, 2025 2 commits
- [Distributed] Add enable_expert_parallel arg (#14305) · cc2f9b32
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc2f9b32
- [RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185) · 151b08e0
  youkaichao authored Mar 07, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  151b08e0
05 Mar, 2025 1 commit
- [Model] New model support for Phi-4-multimodal-instruct (#14119) · 0a995d54
  Congcong Chen authored Mar 04, 2025
  
  0a995d54
03 Mar, 2025 1 commit
- Update deprecated Python 3.8 typing (#13971) · cf069aa8
  Harry Mellor authored Mar 03, 2025
  
  cf069aa8
02 Mar, 2025 1 commit
- [v0][structured output] Support reasoning output (#12955) · bf33700e
  Ce Gao authored Mar 03, 2025
```
Signed-off-by: Ce Gao <cegao@tensorchord.ai>
```
  bf33700e
28 Feb, 2025 2 commits
- [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename... · bd56c983
  Luka Govedič authored Feb 28, 2025
```
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
```
  bd56c983
- [V1]`SupportsV0Only` protocol for model definitions (#13959) · 6c85da3a
  Roger Wang authored Feb 27, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  6c85da3a
27 Feb, 2025 3 commits
- [Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626) · 9804145c
  Benjamin Chislett authored Feb 27, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
```
  9804145c
- [VLM] Deprecate legacy input mapper for OOT multimodal models (#13979) · a2dd48c3
  Cyrus Leung authored Feb 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a2dd48c3
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
26 Feb, 2025 1 commit
- [ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms (#13844) · 1d35662e
  Sage Moore authored Feb 25, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  1d35662e
25 Feb, 2025 2 commits
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
- [Misc][Chore] Clean Up `AsyncOutputProcessing` Logs (#13780) · f61528d4
  Robert Shaw authored Feb 24, 2025
  
  f61528d4
24 Feb, 2025 3 commits
- [Misc] Clean Up `EngineArgs.create_engine_config` (#13734) · 1f0ae3ed
  Robert Shaw authored Feb 24, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  1f0ae3ed
- [Misc][Docs] Raise error when flashinfer is not installed and... · 444b0f0f
  Nicolò Lucchesi authored Feb 24, 2025
```
[Misc][Docs] Raise error when flashinfer is not installed and `VLLM_ATTENTION_BACKEND` is set (#12513)
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  444b0f0f
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
23 Feb, 2025 1 commit
- [v1] torchrun compatibility (#13642) · eb24dc4a
  youkaichao authored Feb 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eb24dc4a
22 Feb, 2025 4 commits
- [ci] fix linter (#13701) · 2382ad29
  youkaichao authored Feb 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2382ad29
- [core] set up data parallel communication (#13591) · 3e472d88
  youkaichao authored Feb 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3e472d88
- [Metrics] Add `--show-hidden-metrics-for-version` CLI arg (#13295) · 2cb8c154
  Mark McLoughlin authored Feb 22, 2025
  
  2cb8c154
- [V1][Metrics] Support `vllm:cache_config_info` (#13299) · 1cd981da
  Mark McLoughlin authored Feb 22, 2025
  
  1cd981da
21 Feb, 2025 2 commits

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

[Bugfix] Fix max_num_batched_tokens for MLA (#13620) · 71face85
Michael Goin authored Feb 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
71face85

20 Feb, 2025 1 commit
- [Frontend] Add backend-specific options for guided decoding (#13505) · bfbc0b32
  Joe Runde authored Feb 20, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  bfbc0b32
19 Feb, 2025 3 commits

[Feature] Pluggable platform-specific scheduler (#13161) · 42333026

Yannick Schnider authored Feb 19, 2025


Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>

42333026

[Model][Speculative Decoding] DeepSeek MTP spec decode (#12755) · f525c0be
Lucia Fang authored Feb 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
Co-authored-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
f525c0be

[1/n][CI] Load models in CI from S3 instead of HF (#13205) · d5d214ac

Kevin H. Luu authored Feb 18, 2025



Signed-off-by: <>
Co-authored-by: EC2 Default User <ec2-user@ip-172-31-20-117.us-west-2.compute.internal>

d5d214ac

17 Feb, 2025 1 commit
- [Feature][Spec Decode] Simplify the use of Eagle Spec Decode (#12304) · 46cdd595
  shangmingc authored Feb 17, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  46cdd595
14 Feb, 2025 2 commits

[Core] Reduce TTFT with concurrent partial prefills (#10235) · 3bcb8c75

Joe Runde authored Feb 14, 2025


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

3bcb8c75

Expand MLA to support most types of quantization (#13181) · f0b2da72
Michael Goin authored Feb 14, 2025

f0b2da72

13 Feb, 2025 1 commit
- [Frontend] Add `/v1/audio/transcriptions` OpenAI API endpoint (#12909) · d84cef76
  Nicolò Lucchesi authored Feb 13, 2025
  
  d84cef76
12 Feb, 2025 1 commit
- [RFC][vllm-API] Support tokenizer registry for customized tokenizer in vLLM (#12518) · 3ee696a6
  Keyun Tong authored Feb 11, 2025
```
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
```
  3ee696a6
11 Feb, 2025 1 commit
- [Platform] add pre_register_and_update function (#12432) · 2e3b969e
  wangxiyuan authored Feb 11, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  2e3b969e
08 Feb, 2025 1 commit
- [bugfix] respect distributed_executor_backend in world_size=1 (#12934) · 91dd8f7a
  youkaichao authored Feb 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91dd8f7a
06 Feb, 2025 1 commit
- [torch.compile] PyTorch 2.6 and nightly compatibility (#12393) · 09b95e36
  youkaichao authored Feb 07, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  09b95e36