Commits · d54af615d5cf63e111e564b3a2c25405fc16b4fd · OpenDAS / vllm_cscc · GitLab

30 May, 2025 1 commit
- [Bugfix] Fix PP default fallback behavior for V1 (#18915) · d54af615
  Michael Goin authored May 29, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d54af615
29 May, 2025 19 commits
- [TPU] remove transpose ops in moe kernel (#18923) · a1cc9f33
  Chengji Yao authored May 29, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a1cc9f33
- Use standalone_compile by default in torch >= 2.8.0 (#18846) · a521ef06
  Richard Zou authored May 29, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  a521ef06
- [P/D] NixlConnector DP fixes (#18903) · 64eaf5fe
  Will Eaton authored May 29, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  64eaf5fe
- [BugFix] Make DP work with connector-delayed new requests (#18559) · d1d61f33
  Nick Hill authored May 29, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Will Eaton <weaton@redhat.com>
```
  d1d61f33
- [V1] Allocate kv_cache with stride order for V1 (#18775) · 32ce3cf7
  Nicolò Lucchesi authored May 29, 2025
```
Signed-off-by: nicklucche <nlucches@redhat.com>
```
  32ce3cf7
- [Misc] Remove duplicate init for self.vllm_config (#18896) · d58f9c7f
  CYJiang authored May 30, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  d58f9c7f
- [Deprecation] Disallow pos-args other than `model` when initializing `LLM` (#18802) · c2903403
  Cyrus Leung authored May 30, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c2903403
- [ROCm][V0][Attention] Revert to the previous FA triton kernel (#18226) · 1b7cfd5a
  Gregory Shtrasberg authored May 29, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  1b7cfd5a
- [Attention][V1] Toggle for v1 attention backend (#18275) · da4b69d0
  Gregory Shtrasberg authored May 29, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  da4b69d0
- Fix an error in dummy weight loading for quantization models (#18855) · f8977c23
  Chenyaaang authored May 29, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  f8977c23
- [Bugfix] Ensure tensors are contiguous during serialisation (#18860) · 0b1447f8
  Lukas Geiger authored May 29, 2025
```
Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>
```
  0b1447f8
- [Misc] Replace TODO in serving transcription (#18895) · 24d0ef89
  Nicolò Lucchesi authored May 29, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  24d0ef89
- [LoRA] Add LoRA support for InternVL (#18842) · 34d6c447
  Jee Jee Li authored May 29, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  34d6c447
- [Neuron] Add multi-LoRA support for Neuron. (#18284) · 972eddf7
  Satyajith Chilappagari authored May 29, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
```
  972eddf7
- Skip device and quant Pydantic validation to make plugin device work (#18843) · 3c49dbdd
  Yikun Jiang authored May 29, 2025
```
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
```
  3c49dbdd
- [Doc][Neuron] Update documentation for Neuron (#18868) · 1661a9c2
  aws-elaineyz authored May 28, 2025
```
Signed-off-by: Elaine Zhao <elaineyz@amazon.com>
```
  1661a9c2
- [Bugfix][TPU] fix moe custom kernel import (#18853) · 8e882ffd
  Chengji Yao authored May 28, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  8e882ffd
- Add ability to use CUDAGraphs with use_inductor=False (#17345) · 26b4fa45
  Richard Zou authored May 28, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  26b4fa45
- Prevent the cross-encoder logic from being applied to classification tasks (#18838) · 515b413e
  Maximilien de Bayser authored May 28, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  515b413e
28 May, 2025 20 commits
- [Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py... · 269d9017
  Hongxia Yang authored May 28, 2025
```
[Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix (#18100)
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  269d9017
- [Core] Enable CUDA graphs for DP + All2All kernels (#18724) · 7951d787
  Varun Sundar Rabindranath authored May 28, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  7951d787
- Remove checks for `None` for fields which should never be `None` (#17985) · 6dbe5b5c
  Harry Mellor authored May 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6dbe5b5c
- [Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend (#15655) · 643622ba
  Akshat Tripathi authored May 28, 2025
```
Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Signed-off-by: xihajun <junfan@krai.ai>
Signed-off-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Signed-off-by: Jorge de Freitas <jorge@krai.ai>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: xihajun <junfan@krai.ai>
Co-authored-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Co-authored-by: Jorge de Freitas <jorge@krai.ai>
```
  643622ba
- [Chore][Spec Decode] Update check NoneType instead of assigning variables (#18836) · a09c7ca9
  Aaron Pham authored May 28, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  a09c7ca9
- [V1][Metrics] Remove metrics that were deprecated in 0.8 (#18837) · 0e98964e
  Mark McLoughlin authored May 28, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  0e98964e
- [Misc] fix olmoe model layer can't laod in tp gt 1 (#18828) · c68b5c63
  rongfu.leng authored May 29, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  c68b5c63
- [Core] Add Lora Support to Beam Search (#18346) · 321331b8
  Alex Brooks authored May 28, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  321331b8
- decrement server_load on listen for disconnect (#18784) · 6e4cea1c
  daniel-salib authored May 28, 2025
```
Signed-off-by: Daniel Salib <danielsalib@meta.com>
```
  6e4cea1c
- [Frontend] add run batch to CLI (#18804) · 435fa954
  Reid authored May 28, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  435fa954
- Enable Pydantic mypy checks and convert configs to Pydantic dataclasses (#17599) · 4c2b38ce
  Harry Mellor authored May 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4c2b38ce
- [Platform][Dist] Make torch distributed process group extendable (#18763) · d781930f
  Mengqing Cao authored May 28, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  d781930f
- [BugFix] FA2 MLA Accuracy Issue (#18807) · ce75efee
  Lucas Wilkinson authored May 28, 2025
```
Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>
```
  ce75efee
- Fix PiecewiseCompileInterpreter (#17338) · aa42561e
  Richard Zou authored May 28, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  aa42561e
- [CI] improve embed testing (#18747) · de65fc8e
  wang.yuqi authored May 28, 2025
  
  de65fc8e
- [Deprecation] Remove fallbacks for Embeddings API (#18795) · 0c492b78
  Cyrus Leung authored May 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0c492b78
- [Deprecation] Remove unused sync methods in `async_timeout` (#18792) · 0f0926b4
  Cyrus Leung authored May 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0f0926b4
- [Deprecation] Require overriding `get_dummy_text` and `get_dummy_mm_data` (#18796) · 7f2c1a87
  Cyrus Leung authored May 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7f2c1a87
- [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781) · 5e13c07d
  RonaldBXu authored May 27, 2025
```
Signed-off-by: Ronald Xu <ronaldxu@amazon.com>
```
  5e13c07d
- [V1] fix torch profiling for V1 offline scenarios (#18445) · 774c5fde
  Divakar Verma authored May 27, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  774c5fde