Commits · 5d477453f95450651d8a30a6c243b1a9fd3b0b37 · OpenDAS / vllm_cscc

31 Mar, 2025 3 commits
- remove repeated: use_int8_w8a8 · 5d477453
  zhuwenwen authored Mar 31, 2025
  
  5d477453
- update list and type · 14356d5d
  zhuwenwen authored Mar 31, 2025
  
  14356d5d
- skip fp8 kernels and paged_attention_rocm · 52675626
  zhuwenwen authored Mar 31, 2025
  
  52675626
28 Mar, 2025 2 commits
- [feat]添加VLLM_SPEC_DECODE_EAGER环境变量，用于选择draft model是否强制使用eager模式，在hygon cpu上ds3 mtp提升较大 · 87223113
  zhuwenwen authored Mar 28, 2025
  
  87223113
- Merge tag 'v0.8.2' into v0.8.2-dev · 469e903b
  zhuwenwen authored Mar 28, 2025
  
  469e903b
25 Mar, 2025 8 commits
- [V1][Spec Decode] Update target_logits in place for rejection sampling (#15427) · 25f560a6
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25f560a6
- [V1] guidance backend for structured output + `auto` fallback mode (#14779) · a09ad90a
  Russell Bryant authored Mar 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
```
  a09ad90a
- [Bugfix] Fixed the issue of not being able to input video and image simultaneously (#15387) · 10b34e36
  Chauncey authored Mar 25, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  10b34e36
- Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" (#15398) · b5269db9
  Tyler Michael Smith authored Mar 24, 2025
  
  b5269db9
- [Misc] Remove LoRA log (#15388) · 6db94571
  Jee Jee Li authored Mar 25, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  6db94571
- Add pipeline parallel support to `TransformersModel` (#12832) · 97cfa65d
  Harry Mellor authored Mar 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  97cfa65d
- [Minor][Spec Decode] Remove compiled_softmax (#15416) · 911c8eb0
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  911c8eb0
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 26 commits
- [ROCm][Kernel] MoE weights padding (#14454) · f533b583
  Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  f533b583
- [Build] Cython compilation support fix (#14296) · 8279201c
  Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  8279201c
- [Hardware][TPU] Skip failed compilation test (#15421) · 23fdab00
  Siyuan Liu authored Mar 24, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  23fdab00
- [BugFix][V1] Quick fix for min_tokens with multiple EOS (#15407) · 623e2ed2
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  623e2ed2
- [V1][Perf] Simpler request output queues (#15156) · 9d72daf4
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  9d72daf4
- [Doc] Update docs on handling OOM (#15357) · 6dd55af6
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  6dd55af6
- [DOC] Add Kubernetes deployment guide with CPUs (#14865) · 3eb08ed9
  Yuan Tang authored Mar 24, 2025
  
  3eb08ed9
- [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral (#12303) · 5eeadc26
  liuzhenwei authored Mar 25, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  5eeadc26
- [V1] Aggregate chunked prompt logprobs in model runner (#14875) · 3aee6573
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  3aee6573
- [MISC] Refine no available block debug msg (#15076) · 9cc64514
  Yi Liu authored Mar 25, 2025
```
Signed-off-by: Yi Liu <yiliu4@habana.ai>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Co-authored-by: Yi Liu <yiliu4@habana.ai>
```
  9cc64514
- [V1][Minor] fix comments (#15392) · 0893567d
  Chen1022 authored Mar 24, 2025
```
Signed-off-by: chenjincong <chenjincong@baidu.com>
Signed-off-by: Chen-0210 <chenjincong11@gmail.com>
Co-authored-by: chenjincong <chenjincong@baidu.com>
```
  0893567d
- [Core] Don't force uppercase for VLLM_LOGGING_LEVEL (#15306) · 8abe69b4
  Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  8abe69b4
- [Core] Integrate `fastsafetensors` loader for loading model weights (#10647) · 761702fd
  Manish Sethi authored Mar 24, 2025
```
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
```
  761702fd
- [distributed] fix dp group (#15355) · 9606d572
  youkaichao authored Mar 24, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  9606d572
- [Bugfix] Fix chat template loading (#15143) · cbcdf2c6
  Cyrus Leung authored Mar 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  cbcdf2c6
- Fix zmq IPv6 URL format error (#15341) · 038de04d
  Russell Bryant authored Mar 24, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  038de04d
- [Kernel] allow non-contiguous input for marlin kernel (#14658) · 6b3cc75b
  Jinzhen Lin authored Mar 24, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  6b3cc75b
- Revert "[CI/Build] Use uv python for docker rather than ppa:deadsnakess/ppa (#13569)" (#15377) · 7ffcccfa
  Simon Mo authored Mar 24, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  7ffcccfa
- [Misc] Update guided decoding logs to debug (#15310) · cc8accfd
  sfbemerk authored Mar 24, 2025
```
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
```
  cc8accfd
- [fix]修复fused_moe.py中fused_moe接口未初始化moe_ep_size导致的deekseek等模型报错 · 389ebcf7
  zhuwenwen authored Mar 24, 2025
  
  389ebcf7
- [Bugfix][V1] Avoid importing PreTrainedModel (#15366) · 948ab03e
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Mar 24, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  948ab03e
- add mla tuning script and configs · 18c2244b
  zhuwenwen authored Mar 24, 2025
  
  18c2244b
- [Misc] Remove ignore_reinit_error for ray.init() (#15373) · 5797fb97
  Rui Qiao authored Mar 24, 2025
  
  5797fb97
- [Misc] Upgrade BNB version (#15183) · 3892e58a
  Jee Jee Li authored Mar 24, 2025
  
  3892e58a
- Fix non-contiguous input passed to Marlin kernel (#15319) · d20e2611
  Qubitium-ModelCloud authored Mar 24, 2025
  
  d20e2611
- [Fix] [torch.compile] Improve UUID system for custom passes (#15249) · f622dbcf
  Luka Govedič authored Mar 23, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  f622dbcf
23 Mar, 2025 1 commit

[V1] Enable V1 Fp8 cache for FA3 in the oracle (#15191) · dccf535f

Lucas Wilkinson authored Mar 23, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

dccf535f