Commits · dae68969774e41b93b01cd31171ca033a92b574a · OpenDAS / vllm_cscc · GitLab

07 Mar, 2025 3 commits
- [Perf] Reduce MLA CPU overheads in V1 (#14384) · dae68969
  Lucas Wilkinson authored Mar 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dae68969
- [Bugfix] Correctly call `cudaProfilerStop` in benchmarks script (#14183) · c34eeec5
  Brayden Zhong authored Mar 06, 2025
```
Signed-off-by: Brayden Zhong <b8zhong@uwaterloo.ca>
```
  c34eeec5
- [Doc] Fix a typo (#14385) · ad60bbb2
  Daniel Li authored Mar 06, 2025
  
  ad60bbb2
06 Mar, 2025 35 commits
- [Hardware][TPU]Enable ragged paged attention kernel and resolve recompilation issue (#14310) · 0578e5a4
  Chengji Yao authored Mar 06, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  0578e5a4
- [Docs] Add nsight guide to profiling docs (#14298) · 04222984
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  04222984
- [V1][Bugfix] Standardize quantized kv cache rejection for attention backends (#14221) · 6832707e
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6832707e
- [Bug] Fix Attention when ignored in by quant_method (#14313) · 6b2ef5cd
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  6b2ef5cd
- [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) · 958adce4
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  958adce4
- [Kernel] Add needs_fixed_stride_order tag to most GEMMs (#14306) · 99b0915d
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  99b0915d
- [CI] Disable spawn when running V1 Test (#14345) · 8ca2b21c
  Thomas Parnell authored Mar 06, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  8ca2b21c
- [CI/Build] Use uv python for docker rather than ppa:deadsnakes/ppa (#13569) · d9292786
  Michael Goin authored Mar 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d9292786
- [Distributed] Add enable_expert_parallel arg (#14305) · cc2f9b32
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc2f9b32
- [V1] Do not detokenize if sampling param detokenize is False (#14224) · cd579352
  Himanshu Jaju authored Mar 06, 2025
```
Signed-off-by: Himanshu Jaju <hj@mistral.ai>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  cd579352
- Fix mla prefill context performance (#13897) · 9f1710f1
  Ying Zhong authored Mar 07, 2025
```
Signed-off-by: ZhongYingMatrix <zhongyingmatrix@gmail.com>
```
  9f1710f1
- Add authors to license header. (#14371) · e642ec96
  Thomas Parnell authored Mar 06, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Burkhard Ringlein <ngl@zurich.ibm.com>
Co-authored-by: Jan van Lunteren <jvl@zurich.ibm.com>
```
  e642ec96
- Adding cpu inference with VXE ISA for s390x architecture (#12613) · ada19210
  Dilip Gowda Bhagavan authored Mar 06, 2025
```
Signed-off-by: Dilip Gowda Bhagavan <dilip.bhagavan@ibm.com>
Signed-off-by: Rishika Kedia <rishika.kedia@in.ibm.com>
Co-authored-by: Rishika Kedia <rishika.kedia@in.ibm.com>
```
  ada19210
- Reinstate `best_of` for V0 (#14356) · bf0560bd
  Harry Mellor authored Mar 06, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  bf0560bd
- [RLHF] use worker_extension_cls for compatibility with V0 and V1 (#14185) · 151b08e0
  youkaichao authored Mar 07, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  151b08e0
- [Doc] Fix date typo in README.md (#14366) · 81b2f4a4
  Jitse Klomp authored Mar 06, 2025
```
Signed-off-by: Jitse Klomp <jitse.klomp@conclusionxforce.nl>
```
  81b2f4a4
- [Core] Don't use cache during multi-modal profiling (#14336) · 82551ad6
  Cyrus Leung authored Mar 07, 2025
  
  82551ad6
- [Bugfix][Core] fix abort_seq_group and memory leak when n>1 (#14326) · caac5c2e
  courage17340 authored Mar 06, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  caac5c2e
- [Kernel] [V1] Improved performance for V1 Triton (ROCm) backend (#14152) · 6bd1dd9d
  Thomas Parnell authored Mar 06, 2025
  
  6bd1dd9d
- [Doc] Correct beam_search using in generative_models.md (#14363) · 4f27044a
  Irina Yuryeva authored Mar 06, 2025
  
  4f27044a
- [Doc] Update reasoning with stream example to use OpenAI library (#14077) · 0ddc991f
  Yanyi Liu authored Mar 06, 2025
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
```
  0ddc991f
- [Frontend][Docs] Transcription API streaming (#13301) · fa82b938
  Nicolò Lucchesi authored Mar 06, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  fa82b938
- [Core] Optimizing cross-attention `QKVParallelLinear` computation (#12325) · 69ff99fd
  Nicolò Lucchesi authored Mar 06, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
```
  69ff99fd
- [V1][VLM][Pixtral-HF] Support Pixtral-HF on V1 (#14275) · 5d802522
  lkchen authored Mar 06, 2025
```
Signed-off-by: Linkun Chen <github@lkchen.net>
```
  5d802522
- [Model] Update Paligemma multimodal processing with PromptUpdate (#14015) · 17699280
  kYLe authored Mar 06, 2025
```
Signed-off-by: Kyle Huang <kylhuang@nvidia.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  17699280
- [Hardware] Update the flash attn tag to support Blackwell (#14244) · ed6ea065
  Pavani Majety authored Mar 05, 2025
  
  ed6ea065
- [Bugfix][CI] ALiBi test case in xformers multi_query_kv_attention (#11301) · 5ee10e99
  Nicolò Lucchesi authored Mar 06, 2025
  
  5ee10e99
- [V1] LoRA - Enable more V1 tests (#14315) · 3dbd2d81
  Varun Sundar Rabindranath authored Mar 05, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  3dbd2d81
- [Bugfix][Structured Output] Support outlines engine with reasoning outputs for DeepSeek R1 (#14114) · f5f7f00c
  Ce Gao authored Mar 06, 2025
  
  f5f7f00c
- [misc] Mention `ray list nodes` command to troubleshoot ray issues (#14318) · abcc61e0
  Rui Qiao authored Mar 05, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  abcc61e0
- [BugFix] MLA + V1, illegal memory access and accuracy issues (#14253) · f6bb18fd
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  f6bb18fd
- [Build] Add UV_HTTP_TIMEOUT to avoid timeout during installation (#13850) · 71eaf896
  Yuan Tang authored Mar 05, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  71eaf896
- Add benchmark for DeepGEMM and vLLM Block FP8 Dense GEMM (#13917) · ca100c90
  Michael Goin authored Mar 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  ca100c90
- [CI/Build] Use spawn multiprocessing mode for V1 test pipeline (#14243) · ffad9439
  Russell Bryant authored Mar 05, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ffad9439
- [BugFix] Fix prefix caching V0 MLA (#14255) · 4dacaa4a
  Lucas Wilkinson authored Mar 05, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Ying Zhong <zhongyingmatrix@gmail.com>
```
  4dacaa4a
05 Mar, 2025 2 commits
- [Bugfix] Remove num_tokens_across_dp (#14302) · a7ea35aa
  Tyler Michael Smith authored Mar 05, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  a7ea35aa
- [Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due... · 1e3e76b6
  pyc96 authored Mar 05, 2025
```
[Bugfix] Fix DeepSeek MTP crash when using TP1ModelRunner with CUDA graph due to shape mismatch (#14237)
Signed-off-by: pyc96 <pychen96@gmail.com>
```
  1e3e76b6