Commits · c20ef40fd0e8663e82911f53d00a64f53beb98aa · OpenDAS / vllm_cscc

07 May, 2025 3 commits

[Hardware][TPU][V1] Multi-LoRA implementation for the V1 TPU backend (#14238) · c20ef40f

Akshat Tripathi authored May 07, 2025


Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>

c20ef40f

Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49

Satyajith Chilappagari authored May 07, 2025


Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>

043e4c49

[Misc] Split model loader (#17712) · 822de7fb
Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
822de7fb

04 May, 2025 2 commits
- [Bugfix] Prioritize dtype in root config before checking text config (#17629) · 2858830c
  Cyrus Leung authored May 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2858830c
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
03 May, 2025 2 commits
- [Core] Gate `prompt_embeds` behind a feature flag (#17607) · 887d7af8
  Cyrus Leung authored May 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  887d7af8
- [Frontend][TPU] Add TPU default max-num-batched-tokens based on device name (#17508) · 87baebeb
  Chenyaaang authored May 02, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  87baebeb
02 May, 2025 2 commits
- Automatically tell users that dict args must be valid JSON in CLI (#17577) · 785d75a0
  Harry Mellor authored May 02, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  785d75a0
- Add `pt_load_map_location` to allow loading to cuda (#16869) · 109e15a3
  Jerry Zhang authored May 01, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  109e15a3
01 May, 2025 7 commits
- [Bug]change the position of cuda_graph_sizes in dataclasses (#17548) · 173daac1
  Chen Xia authored May 01, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
```
  173daac1
- [Bugfix] Fix lint error (#17547) · 9b1769dd
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9b1769dd
- [Misc]add configurable cuda graph size (#17201) · 61c299f8
  Chen Xia authored May 01, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  61c299f8
- Move the last arguments in `arg_utils.py` to be in their final groups (#17531) · 6768ff4a
  Harry Mellor authored May 01, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6768ff4a
- [Feature][Frontend]: Deprecate --enable-reasoning (#17452) · 98060b00
  Chauncey authored May 01, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  98060b00
- Improve configs - `ObservabilityConfig` (#17453) · a257d9bc
  Harry Mellor authored May 01, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a257d9bc
- [CI/Build] Reorganize models tests (#17459) · afb4429b
  Cyrus Leung authored May 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  afb4429b
30 Apr, 2025 3 commits

[Fix] Support passing args to logger (#17425) · da4e7687
Aaron Pham authored Apr 30, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
da4e7687

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

Improve configs - `ModelConfig` (#17130) · 13698db6
Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
13698db6

29 Apr, 2025 4 commits
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- Improve literal dataclass field conversion to argparse argument (#17391) · 2ef5d106
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2ef5d106
- [Misc] Move config fields to MultiModalConfig (#17343) · ebb3930d
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ebb3930d
- [V1][Spec Decode] Make Eagle model arch config driven (#17323) · e1360005
  Ekagra Ranjan authored Apr 28, 2025
  
  e1360005
28 Apr, 2025 3 commits
- Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) · c7941cca
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c7941cca
- Make name of `compressed-tensors` quant method consistent across vLLM (#17255) · b6dd32aa
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b6dd32aa
- Add missing class docstring for `PromptAdapterConfig` (#17302) · fb1c933a
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fb1c933a
27 Apr, 2025 2 commits

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

[Bugfix] Get a specific type of layer from forward context (#17222) · 838cedad
Chen Zhang authored Apr 27, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
838cedad

26 Apr, 2025 2 commits
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213) · 1cf0719e
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1cf0719e
25 Apr, 2025 4 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
423e9f1c

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131) · 6ca02344
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
6ca02344

24 Apr, 2025 3 commits
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
23 Apr, 2025 3 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- `CacheConfig.block_size` should always be `int` when used (#17052) · f3a21e9c
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f3a21e9c