Commits · 690fe019f046c03978ebe728770e5200d4e9d5f2 · OpenDAS / vllm_cscc

27 Apr, 2025 2 commits

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

[Bugfix] Get a specific type of layer from forward context (#17222) · 838cedad
Chen Zhang authored Apr 27, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
838cedad

26 Apr, 2025 2 commits
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213) · 1cf0719e
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1cf0719e
25 Apr, 2025 4 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
423e9f1c

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131) · 6ca02344
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
6ca02344

24 Apr, 2025 3 commits
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
23 Apr, 2025 3 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- `CacheConfig.block_size` should always be `int` when used (#17052) · f3a21e9c
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f3a21e9c
22 Apr, 2025 8 commits
- Add assertion for no objects while hashing hf_config (#16930) · 7f58fb97
  Richard Zou authored Apr 22, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  7f58fb97
- [FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
```
  30bc3e0f
- Improve configs - `SpeculativeConfig` (#16971) · d0591104
  Harry Mellor authored Apr 22, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d0591104
- [Doc] Improve documentation for multimodal CLI args (#16960) · 8f7bace7
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f7bace7
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
- [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7
  Varun Sundar Rabindranath authored Apr 21, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7b8a2ab7
- [Misc] Remove the chunked prefill warning for LoRA (#16925) · c9acbf11
  Jee Jee Li authored Apr 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c9acbf11
21 Apr, 2025 1 commit
- [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) · bb3605db
  qizixi authored Apr 20, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  bb3605db
20 Apr, 2025 1 commit
- Improve configs - `CacheConfig` (#16835) · 4b07d368
  Harry Mellor authored Apr 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b07d368
18 Apr, 2025 2 commits
- Fix `nullable_kvs` fallback (#16837) · 686623c5
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  686623c5
- Improve-mm-and-pooler-and-decoding-configs (#16789) · e78587a6
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e78587a6
17 Apr, 2025 2 commits
- Improve configs - `TokenizerPoolConfig` + `DeviceConfig` (#16603) · d27ea940
  Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d27ea940
- [V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636) · 2cbd4d29
  Bryan Lu authored Apr 16, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  2cbd4d29
16 Apr, 2025 1 commit

[Model] Add PLaMo2 (#14323) · 3badb021

Shinichi Hemmi authored Apr 16, 2025


Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Signed-off-by: shemmi <shemmi@preferred.jp>
Co-authored-by: Kento Nozawa <nzw0301@preferred.jp>
Co-authored-by: Hiroaki Mikami <mhiroaki@preferred.jp>
Co-authored-by: Calvin Metzger <metzger@preferred.jp>

3badb021

15 Apr, 2025 1 commit
- Fix vLLM x torch.compile config caching (#16491) · b590adfd
  Richard Zou authored Apr 15, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b590adfd
14 Apr, 2025 2 commits
- config check sleep mode support oot platforms (#16562) · d2020aca
  Shuqiao Li authored Apr 15, 2025
  
  d2020aca
- Improve configs - `SchedulerConfig` (#16533) · e51929eb
  Harry Mellor authored Apr 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e51929eb
12 Apr, 2025 2 commits
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
11 Apr, 2025 3 commits
- Improve configs - `LoadConfig` (#16422) · cd77382a
  Harry Mellor authored Apr 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  cd77382a
- Fix erroneous "model doesn't support compile" warning (#16486) · 70de35a8
  Richard Zou authored Apr 11, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  70de35a8
- [Misc] Raise error for V1 not supporting Long LoRA. (#16415) · a26f59cc
  Jee Jee Li authored Apr 11, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a26f59cc
10 Apr, 2025 3 commits
- [V1] Set structured output backend to `auto` by default (#15724) · 9665313c
  Russell Bryant authored Apr 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9665313c
- Improve configs - `ParallelConfig` (#16332) · 0c54fc72
  Harry Mellor authored Apr 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0c54fc72
- Fix the torch version parsing logic (#15857) · 7678fcd5
  Lu Fang authored Apr 10, 2025
  
  7678fcd5