Commits · e4b8713380cebc6a336d20ee32800d8be3e28877 · OpenDAS / vllm_cscc · GitLab

11 May, 2025 10 commits
- [New Model]: nomic-embed-text-v2-moe (#17785) · e4b87133
  wang.yuqi authored May 11, 2025
  
  e4b87133
- [FP8][ROCm][Attention] Enable FP8 KV cache on ROCm for V1 (#17870) · 06c0922a
  Gregory Shtrasberg authored May 11, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  06c0922a
- [Misc] Add compressed-tensors NVFP4A16 emulation support (#17914) · cd3edfc9
  Dipika Sikka authored May 11, 2025
```
Signed-off-by: Dipika Sikka <dipikasikka1@gmail.com>
Signed-off-by: Dipika <dipikasikka1@gmail.com>
```
  cd3edfc9
- [Frontend] Add /classify endpoint (#17032) · 9cea90ea
  Frieda Huang authored May 11, 2025
```
Signed-off-by: Frieda (Jingying) Huang <jingyingfhuang@gmail.com>
```
  9cea90ea
- [doc] update lora doc (#17936) · d1110f5b
  Reid authored May 11, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d1110f5b
- [Bugfix]: v1 engine - consider lora adapters in allowed_token_ids (#17855) · 8132365b
  Ben Browning authored May 11, 2025
```
Signed-off-by: Ben Browning <bbrownin@redhat.com>
```
  8132365b
- fix amd triton mla path (#17871) · eea22a56
  Shiyan Deng authored May 11, 2025
  
  eea22a56
- [Perf] Use small max_num_batched_tokens for A100 (#17885) · 91121552
  Kuntai Du authored May 11, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  91121552
- [Bugfix] Add revision to `transformers.Auto*.from_pretrained` processors (#17948) · 90d0a74b
  xinli-centml authored May 11, 2025
```
Signed-off-by: Xin Li <xin@centml.ai>
```
  90d0a74b
- [Kernel] fp4 marlin kernel (#17687) · d74e5f37
  Jinzhen Lin authored May 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d74e5f37
10 May, 2025 7 commits
- [v1] Rename specialized_manager.py to single_type_kv_cache_manager.py (#17946) · ca66a167
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  ca66a167
- [v1] Pass BlockTable and KVCacheSpec to AttentionMetadataBuilders (#17483) · 950751a9
  Chen Zhang authored May 11, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  950751a9
- [Misc] remove --model from vllm serve usage (#17944) · 4c31218f
  Reid authored May 10, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  4c31218f
- Don't default construct `ModelConfig` when default constructing `VllmConfig` (#17943) · 68311891
  Harry Mellor authored May 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  68311891
- Add missing content type headers to /ping and /health (#17036) (#17786) · fc4441a4
  Ximo Guanter authored May 10, 2025
```
Signed-off-by: Ximo Guanter <ximo.guanter@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  fc4441a4
- fix broken test vllm:test_kernels - test_attention_selector.py::test_flash_attn (#17873) · 246e3e0a
  tracelogfb authored May 09, 2025
```
Co-authored-by: Stephen Chen <tracelog@meta.com>
```
  246e3e0a
- [V1][Spec Decoding] Log accumulated metrics after system goes idle (#17913) · 7042cc96
  Mark McLoughlin authored May 10, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7042cc96
09 May, 2025 23 commits
- [Hardware/NVIDIA/Kernel] Enable nvidia/DeepSeek-R1-FP4 Model (#16362) · 0c0fdae8
  Pavani Majety authored May 09, 2025
  
  0c0fdae8
- AMD conditional all test execution // new test groups (#17556) · 3b602cde
  Alexei-V-Ivanov-AMD authored May 09, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
```
  3b602cde
- Improve configs - the rest! (#17562) · 4b2ed792
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  4b2ed792
- [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) · 7e357113
  Mark McLoughlin authored May 09, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7e357113
- Add option to use torch._inductor.standalone_compile (#17057) · ea2236bf
  Richard Zou authored May 09, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  ea2236bf
- Handle error when `str` passed to `/v1/audio/transcriptions` (#17909) · 7d4aedae
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7d4aedae
- Update CT WNA16MarlinMoE integration (#16666) · 22481fbf
  Michael Goin authored May 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  22481fbf
- [Misc] Auto fallback to float16 for pre-Ampere GPUs when detected bfloat16 config (#17265) · 5c4c08f6
  Isotr0py authored May 10, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  5c4c08f6
- [Misc] Add references in ray_serve_deepseek example (#17907) · c44c384b
  Rui Qiao authored May 09, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c44c384b
- Revert "[BugFix][AMD] Compatible patch for latest AITER(05/07/2025)" (#17910) · 85b72cb7
  Michael Goin authored May 09, 2025
  
  85b72cb7
- [CI/Build] Automatically retry flaky tests (#17856) · 6e5595ca
  Cyrus Leung authored May 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6e5595ca
- [v1] Move block management logic from KVCacheManager to SpecializedManager (#17474) · 200da9a5
  Chen Zhang authored May 09, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  200da9a5
- [BugFix][AMD] Compatible patch for latest AITER(05/07/2025) (#17864) · 9f64e934
  qli88 authored May 09, 2025
```
Signed-off-by: Qiang Li <qiang.li2@amd.com>
```
  9f64e934
- [Misc] add dify integration (#17895) · ec61ea20
  Reid authored May 09, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  ec61ea20
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
- Fix Whisper crash caused by invalid``` max_num_batched_tokens``` config (#17853) · 5b2dcbf0
  inkcherry authored May 09, 2025
```
Signed-off-by: inkcherry <mingzhi.liu@intel.com>
```
  5b2dcbf0
- [Bugfix][CPU] Fix broken AVX2 CPU TP support (#17252) · 6e4a93e3
  Isotr0py authored May 09, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  6e4a93e3
- [Bugfix][ROCm] Fix AITER MLA V1 (#17880) · 217db4ba
  vllmellm authored May 09, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  217db4ba
- [Doc] remove visible token in doc (#17884) · ff8c4005
  Yan Ma authored May 09, 2025
```
Signed-off-by: yan <yanma1@habana.ai>
```
  ff8c4005
- [Doc] Update several links in reasoning_outputs.md (#17846) · 89a0315f
  Michael Yao authored May 09, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  89a0315f
- [Docs] Add Slides from NYC Meetup (#17879) · 3d1e3876
  Simon Mo authored May 08, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  3d1e3876
- [BUGFIX]: return fast when request requires prompt logprobs (#17251) · d310e6de
  Ning Xie authored May 09, 2025
  
  d310e6de
- [Attention] MLA move rotary embedding to cuda-graph region (#17668) · 5e6f9394
  Lucas Wilkinson authored May 08, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  5e6f9394