Commits · e14b43ff9d0b1671b71db9cff9d0c226946f32bf · OpenDAS / vllm_cscc

26 May, 2025 1 commit
- [fix]1.去掉lora kernel中的maxnreg参数，暂不支持；2.去掉merge-lora参数 · e14b43ff
  王敏 authored May 26, 2025
  
  e14b43ff
23 May, 2025 2 commits
- fix docstring · 95099cbc
  zhuwenwen authored May 23, 2025
  
  95099cbc
- fix run lora error · c7880912
  zhuwenwen authored May 23, 2025
  
  c7880912
20 May, 2025 2 commits
- support qiyuan-8b-v2 and FM9GForCausalLM · 8fb5dea5
  zhuwenwen authored May 20, 2025
  
  8fb5dea5
- support Z100L and K100 inference · a5aa55e8
  zhuwenwen authored May 20, 2025
  
  a5aa55e8
14 May, 2025 1 commit
- add kvint8 · 45273722
  xiabo authored May 14, 2025
  
  45273722
28 Apr, 2025 3 commits
- Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256) · c7941cca
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c7941cca
- Make name of `compressed-tensors` quant method consistent across vLLM (#17255) · b6dd32aa
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b6dd32aa
- Add missing class docstring for `PromptAdapterConfig` (#17302) · fb1c933a
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fb1c933a
27 Apr, 2025 2 commits

[Feature] support sequence parallelism using compilation pass (#16155) · 690fe019

cascade authored Apr 27, 2025


Signed-off-by: cascade812 <cascade812@outlook.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

690fe019

[Bugfix] Get a specific type of layer from forward context (#17222) · 838cedad
Chen Zhang authored Apr 27, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
838cedad

26 Apr, 2025 2 commits
- [CI/test] Fix Eagle Correctness Test (#17209) · 513f0747
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  513f0747
- [Minor][Spec Decode] Add use_eagle to SpeculativeConfig (#17213) · 1cf0719e
  Woosuk Kwon authored Apr 25, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1cf0719e
25 Apr, 2025 4 commits

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

Use Transformers helper `get_text_config()` instead of checking for `text_config` (#17105) · 423e9f1c
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
423e9f1c

[Quantization][FP8] Add support for FP8 models with input_scale for output... · a41351f3

rasmith authored Apr 25, 2025


[Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization (#15734)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

a41351f3

Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` (#17131) · 6ca02344
Harry Mellor authored Apr 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
6ca02344

24 Apr, 2025 3 commits
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
23 Apr, 2025 3 commits
- [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
  Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3cde34a4
- Use `@property` and private field for `data_parallel_rank_local` (#17053) · bdb36603
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bdb36603
- `CacheConfig.block_size` should always be `int` when used (#17052) · f3a21e9c
  Harry Mellor authored Apr 23, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f3a21e9c
22 Apr, 2025 8 commits
- Add assertion for no objects while hashing hf_config (#16930) · 7f58fb97
  Richard Zou authored Apr 22, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  7f58fb97
- [FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f
  vllmellm authored Apr 23, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
```
  30bc3e0f
- Improve configs - `SpeculativeConfig` (#16971) · d0591104
  Harry Mellor authored Apr 22, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d0591104
- [Doc] Improve documentation for multimodal CLI args (#16960) · 8f7bace7
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f7bace7
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
- [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7
  Varun Sundar Rabindranath authored Apr 21, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7b8a2ab7
- [Misc] Remove the chunked prefill warning for LoRA (#16925) · c9acbf11
  Jee Jee Li authored Apr 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c9acbf11
21 Apr, 2025 1 commit
- [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) · bb3605db
  qizixi authored Apr 20, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  bb3605db
20 Apr, 2025 1 commit
- Improve configs - `CacheConfig` (#16835) · 4b07d368
  Harry Mellor authored Apr 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b07d368
18 Apr, 2025 2 commits
- Fix `nullable_kvs` fallback (#16837) · 686623c5
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  686623c5
- Improve-mm-and-pooler-and-decoding-configs (#16789) · e78587a6
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e78587a6
17 Apr, 2025 2 commits
- Improve configs - `TokenizerPoolConfig` + `DeviceConfig` (#16603) · d27ea940
  Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d27ea940
- [V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636) · 2cbd4d29
  Bryan Lu authored Apr 16, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  2cbd4d29
16 Apr, 2025 1 commit

[Model] Add PLaMo2 (#14323) · 3badb021

Shinichi Hemmi authored Apr 16, 2025


Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Signed-off-by: shemmi <shemmi@preferred.jp>
Co-authored-by: Kento Nozawa <nzw0301@preferred.jp>
Co-authored-by: Hiroaki Mikami <mhiroaki@preferred.jp>
Co-authored-by: Calvin Metzger <metzger@preferred.jp>

3badb021

15 Apr, 2025 1 commit
- Fix vLLM x torch.compile config caching (#16491) · b590adfd
  Richard Zou authored Apr 15, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b590adfd
14 Apr, 2025 1 commit
- config check sleep mode support oot platforms (#16562) · d2020aca
  Shuqiao Li authored Apr 15, 2025
  
  d2020aca