Commits · d07efb31c5efdb16eb386493b326cf3e90047978 · OpenDAS / vllm_cscc · GitLab

23 Jan, 2025 10 commits
- [Doc] Troubleshooting errors during model inspection (#12351) · d07efb31
  Cyrus Leung authored Jan 23, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d07efb31
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
- [Bugfix] Fix k_proj's bias for whisper self attention (#12342) · c5b4b11d
  Isotr0py authored Jan 23, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c5b4b11d
- [Hardware][Gaudi][BugFix] Fix dataclass error due to triton package update (#12338) · 8ae5ff20
  liuzhenwei authored Jan 23, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  8ae5ff20
- [doc] explain common errors around torch.compile (#12340) · 51162744
  youkaichao authored Jan 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51162744
- [V1] Add `uncache_blocks` (#12333) · f0ef3723
  Cody Yu authored Jan 22, 2025
  
  f0ef3723
- [Docs] Document vulnerability disclosure process (#12326) · 7551a340
  Russell Bryant authored Jan 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  7551a340
- [Docs] Update FP8 KV Cache documentation (#12238) · 01a55941
  Michael Goin authored Jan 22, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  01a55941
- [Bugfix] Fixing AMD LoRA CI test. (#12329) · 8d7aa9de
  Alexei-V-Ivanov-AMD authored Jan 22, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
```
  8d7aa9de
- [AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) · 68c4421b
  rasmith authored Jan 22, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  68c4421b
22 Jan, 2025 17 commits
- [Frontend][V1] Online serving performance improvements (#12287) · aea94362
  Nick Hill authored Jan 22, 2025
  
  aea94362
- [Core] Support `reset_prefix_cache` (#12284) · 7206ce4c
  Cody Yu authored Jan 22, 2025
  
  7206ce4c
- [Bugfix] Fix HPU multiprocessing executor (#12167) · 96f6a759
  Konrad Zawora authored Jan 22, 2025
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  96f6a759
- [Misc] Improve the readability of BNB error messages (#12320) · 84bee4bd
  Jee Jee Li authored Jan 23, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  84bee4bd
- [Misc] Fix the error in the tip for the --lora-modules parameter (#12319) · fc66dee7
  Robin authored Jan 23, 2025
```
Signed-off-by: wangerxiao <863579016@qq.com>
```
  fc66dee7
- [Doc] Add docs for prompt replacement (#12318) · 6609cdf0
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6609cdf0
- [Bugfix][VLM] Fix mixed-modality inference backward compatibility for V0 (#12313) · 16366ee8
  Roger Wang authored Jan 22, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  16366ee8
- [Model][Bugfix]: correct Aria model output (#12309) · 528dbcac
  zhou fan authored Jan 22, 2025
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  528dbcac
- [VLM] Avoid unnecessary tokenization (#12310) · cd7b6f08
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cd7b6f08
- [Core] Support fully transparent sleep mode (#11743) · 68ad4e3a
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  68ad4e3a
- [Build] update requirements of no-device (#12299) · 4004f144
  Mengqing Cao authored Jan 22, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  4004f144
- [core] separate builder init and builder prepare for each batch (#12253) · 66818e5b
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  66818e5b
- [Benchmark] More accurate TPOT calc in `benchmark_serving.py` (#12288) · 222a9dc3
  Nick Hill authored Jan 21, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  222a9dc3
- [Ci/Build] Fix mypy errors on main (#12296) · cbdc4ad5
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cbdc4ad5
- [CI] add docker volume prune to neuron CI (#12291) · 016e3676
  Liangfu Chen authored Jan 21, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  016e3676
- [ci/lint] Add back default arg for pre-commit (#12279) · 64ea24d0
  Kevin H. Luu authored Jan 21, 2025
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  64ea24d0
- [VLM] Simplify post-processing of replacement info (#12269) · df76e5af
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  df76e5af
21 Jan, 2025 13 commits
- [Documentation][AMD] Add information about prebuilt ROCm vLLM docker for perf... · 09ccc9c8
  Hongxia Yang authored Jan 21, 2025
```
[Documentation][AMD] Add information about prebuilt ROCm vLLM docker for perf validation purpose (#12281)
Signed-off-by: Hongxia Yang <hongxyan@amd.com>
```
  09ccc9c8
- [BUGFIX] When skip_tokenize_init and multistep are set, execution crashes (#12277) · 69196a9b
  Aleksandr Malyshev authored Jan 21, 2025
```
Signed-off-by: maleksan85 <maleksan@amd.com>
Co-authored-by: maleksan85 <maleksan@amd.com>
```
  69196a9b
- [bugfix] moe tuning. rm is_navi() (#12273) · 2acba47d
  Divakar Verma authored Jan 21, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  2acba47d
- [Core] Free CPU pinned memory on environment cleanup (#10477) · 9c485d9e
  Jani Monoses authored Jan 21, 2025
  
  9c485d9e
- [Misc] Set default backend to SDPA for get_vit_attn_backend (#12235) · fa9ee081
  wangxiyuan authored Jan 22, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  fa9ee081
- [Misc] Remove experimental dep from tracing.py (#12007) · 347eeebe
  Adrian Cole authored Jan 21, 2025
```
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
```
  347eeebe
- [Bugfix] Multi-sequence broken (#11898) · 18fd4a83
  Andy Lo authored Jan 21, 2025
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  18fd4a83
- [v1][stats][1/n] Add RequestStatsUpdate and RequestStats types (#10907) · 132a1321
  Ricky Xu authored Jan 21, 2025
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  132a1321
- [Kernel] fix moe_align_block_size error condition (#12239) · 1e60f87b
  Jinzhen Lin authored Jan 22, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  1e60f87b
- [Bugfix] fix race condition that leads to wrong order of token returned (#10802) · 9705b90b
  Jannis Schönleber authored Jan 21, 2025
```
Signed-off-by: Jannis Schönleber <joennlae@gmail.com>
```
  9705b90b
- [ci/build] update nightly torch for gh200 test (#12270) · 3aec49e5
  youkaichao authored Jan 21, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3aec49e5
- [Platform] improve platforms getattr (#12264) · c6461280
  Mengqing Cao authored Jan 21, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  c6461280
- Remove pytorch comments for outlines + compressed-tensors (#12260) · 9a7c3a00
  Thomas Parnell authored Jan 21, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  9a7c3a00