Commits · e3d0a1d190678b49d541fea2bd3db8d3ce9f0430 · OpenDAS / vllm_cscc

02 May, 2025 2 commits
- Automatically tell users that dict args must be valid JSON in CLI (#17577) · 785d75a0
  Harry Mellor authored May 02, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  785d75a0
- Add `pt_load_map_location` to allow loading to cuda (#16869) · 109e15a3
  Jerry Zhang authored May 01, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  109e15a3
01 May, 2025 7 commits
- [Bug]change the position of cuda_graph_sizes in dataclasses (#17548) · 173daac1
  Chen Xia authored May 01, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
```
  173daac1
- [Bugfix] Fix lint error (#17547) · 9b1769dd
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9b1769dd
- [Misc]add configurable cuda graph size (#17201) · 61c299f8
  Chen Xia authored May 01, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  61c299f8
- Move the last arguments in `arg_utils.py` to be in their final groups (#17531) · 6768ff4a
  Harry Mellor authored May 01, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6768ff4a
- [Feature][Frontend]: Deprecate --enable-reasoning (#17452) · 98060b00
  Chauncey authored May 01, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  98060b00
- Improve configs - `ObservabilityConfig` (#17453) · a257d9bc
  Harry Mellor authored May 01, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a257d9bc
- [CI/Build] Reorganize models tests (#17459) · afb4429b
  Cyrus Leung authored May 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  afb4429b
30 Apr, 2025 3 commits

[Fix] Support passing args to logger (#17425) · da4e7687
Aaron Pham authored Apr 30, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
da4e7687

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

Improve configs - `ModelConfig` (#17130) · 13698db6
Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
13698db6

29 Apr, 2025 4 commits
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- Improve literal dataclass field conversion to argparse argument (#17391) · 2ef5d106
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2ef5d106
- [Misc] Move config fields to MultiModalConfig (#17343) · ebb3930d
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ebb3930d
- [V1][Spec Decode] Make Eagle model arch config driven (#17323) · e1360005
  Ekagra Ranjan authored Apr 28, 2025
  
  e1360005
28 Apr, 2025 3 commits
- Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) · c7941cca
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c7941cca
- Make name of `compressed-tensors` quant method consistent across vLLM (#17255) · b6dd32aa
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b6dd32aa
- Add missing class docstring for `PromptAdapterConfig` (#17302) · fb1c933a
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fb1c933a
27 Apr, 2025 2 commits

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

[Bugfix] Get a specific type of layer from forward context (#17222) · 838cedad
Chen Zhang authored Apr 27, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
838cedad

26 Apr, 2025 2 commits
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213) · 1cf0719e
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1cf0719e
25 Apr, 2025 4 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
423e9f1c

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131) · 6ca02344
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
6ca02344

24 Apr, 2025 3 commits
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
23 Apr, 2025 3 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- `CacheConfig.block_size` should always be `int` when used (#17052) · f3a21e9c
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f3a21e9c
22 Apr, 2025 7 commits
- Add assertion for no objects while hashing hf_config (#16930) · 7f58fb97
  Richard Zou authored Apr 22, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  7f58fb97
- [FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
```
  30bc3e0f
- Improve configs - `SpeculativeConfig` (#16971) · d0591104
  Harry Mellor authored Apr 22, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d0591104
- [Doc] Improve documentation for multimodal CLI args (#16960) · 8f7bace7
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f7bace7
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
- [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7
  Varun Sundar Rabindranath authored Apr 21, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7b8a2ab7