Commits · 9a1f1da5d1f2e94adb49e0d82b464dc3c1318cc7 · OpenDAS / vllm_cscc · GitLab

22 Feb, 2025 7 commits
- [Bugfix][Model] OLMo 2: split qkv correctly for GQA and MQA (#13687) · 9a1f1da5
  Shane A authored Feb 21, 2025
  
  9a1f1da5
- [ROCM] fix native attention function call (#13650) · 68d630a0
  Gordon Wong authored Feb 22, 2025
  
  68d630a0
- [Misc] Capture and log the time of loading weights (#13666) · 68d535ef
  Jun Duan authored Feb 22, 2025
  
  68d535ef
- [Bugfix][API Server] Fix invalid usage of 'ge' and 'le' in port valid… (#13672) · c6ed9386
  Robin authored Feb 22, 2025
  
  c6ed9386
- [HTTP Server] Make model param optional in request (#13568) · 0ffdf8ce
  Keyun Tong authored Feb 21, 2025
  
  0ffdf8ce
- docs: Add a note on full CI run in contributing guide (#13646) · 8c0dd3d4
  Yuan Tang authored Feb 22, 2025
  
  8c0dd3d4
- [Misc] Fix yapf linting tools etc not running on pre-commit (#13695) · ada7c780
  Isotr0py authored Feb 22, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  ada7c780
21 Feb, 2025 15 commits
- [Attention] MLA with chunked prefill (#12639) · 288cc6c2
  Lucas Wilkinson authored Feb 21, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  288cc6c2
- fix typo of grafana dashboard, with correct datasource (#13668) · 900edbfa
  John Zheng authored Feb 22, 2025
```
Signed-off-by: John Zheng <john.zheng@hp.com>
```
  900edbfa
- [Bugfix][CPU] Fix cpu all-reduce using native pytorch implementation (#13586) · b2c3fc5d
  Isotr0py authored Feb 21, 2025
  
  b2c3fc5d
- [Kernel]Add streamK for block-quantized CUTLASS kernels (#12978) · 839b27c6
  leoneo authored Feb 21, 2025
  
  839b27c6
- [ci] Fix metrics test model path (#13635) · 34ad27fe
  Kevin H. Luu authored Feb 20, 2025
  
  34ad27fe
- [FEATURE] Enables /score endpoint for embedding models (#12846) · 1c3c9757
  Gabriel Marinho authored Feb 21, 2025
  
  1c3c9757
- Missing comment explaining VDR variable in GGUF kernels (#13290) · 1cdc8861
  Szymon Ożóg authored Feb 21, 2025
  
  1cdc8861
- [V1][Sampler] Avoid an operation during temperature application (#13587) · 31aa045c
  Nick Hill authored Feb 20, 2025
  
  31aa045c
- [Bugfix] Add `mm_processor_kwargs` to chat-related protocols (#13644) · a30c0935
  Roger Wang authored Feb 20, 2025
  
  a30c0935
- Use pre-commit to update `requirements-test.txt` (#13617) · c7b07a95
  Harry Mellor authored Feb 21, 2025
  
  c7b07a95
- [NVIDIA] Fix an issue to use current stream for the nvfp4 quant (#13632) · 27a09dc5
  Kaixi Hou authored Feb 20, 2025
  
  27a09dc5
- [Misc] Adding script to setup ray for multi-node vllm deployments (#12913) · 981f3c83
  Edwin Hernandez authored Feb 20, 2025
  
  981f3c83
- Add llmaz as another integration (#13643) · 44c33f01
  Kante Yin authored Feb 21, 2025
```
Signed-off-by: kerthcet <kerthcet@gmail.com>
```
  44c33f01
- [Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245) · 33170081
  Lingfan Yu authored Feb 20, 2025
```
Signed-off-by: Lingfan Yu <lingfany@amazon.com>
```
  33170081
- [Bugfix] Fix max_num_batched_tokens for MLA (#13620) · 71face85
  Michael Goin authored Feb 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  71face85
20 Feb, 2025 18 commits
- [Frontend] Add backend-specific options for guided decoding (#13505) · bfbc0b32
  Joe Runde authored Feb 20, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  bfbc0b32
- fix neuron performance issue (#13589) · 6a417b86
  ajayvohra2005 authored Feb 20, 2025
  
  6a417b86
- [V1][Minor] Print KV cache size in token counts (#13596) · d3ea5011
  Woosuk Kwon authored Feb 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d3ea5011
- Update `pre-commit`'s `isort` version to remove warnings (#13614) · 34aad515
  Harry Mellor authored Feb 20, 2025
  
  34aad515
- [Bugfix] Fix deepseekv3 grouped topk error (#13474) · ed6e9075
  chenxiaobing authored Feb 20, 2025
```
Signed-off-by: Chen-XiaoBing <chenxb002@whu.edu.cn>
```
  ed6e9075
- Merge similar examples in `offline_inference` into single `basic` example (#12737) · 992e5c3d
  Harry Mellor authored Feb 20, 2025
  
  992e5c3d
- [Kernel] LoRA - Refactor sgmv kernels (#13110) · b69692a2
  Varun Sundar Rabindranath authored Feb 20, 2025
  
  b69692a2
- [2/n][ci] S3: Use full model path (#13564) · a64a8443
  Kevin H. Luu authored Feb 20, 2025
```
Signed-off-by: <>
```
  a64a8443
- [ci] Fix spec decode test (#13600) · aa1e62d0
  Kevin H. Luu authored Feb 20, 2025
  
  aa1e62d0
- [CI/Build] Use uv in the Dockerfile (#13566) · 497bc831
  Michael Goin authored Feb 20, 2025
  
  497bc831
- [API Server] Add port number range validation (#13506) · 3738e6fa
  Yuan Tang authored Feb 20, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  3738e6fa
- [ROCm] MI300A compile targets deprecation (#13560) · 0023cd2b
  Gregory Shtrasberg authored Feb 20, 2025
  
  0023cd2b
- [Misc] add mm_processor_kwargs to extra_body for Qwen2.5-VL (#13533) · 041e2947
  燃 authored Feb 20, 2025
  
  041e2947
- [Misc] Warn if the vLLM version can't be retrieved (#13501) · 96216678
  Alex Brooks authored Feb 19, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
```
  96216678
- [bugfix] spec decode worker get tp group only when initialized (#13578) · 8c755c3b
  Simon Mo authored Feb 19, 2025
  
  8c755c3b
- [core] add sleep and wake up endpoint and v1 support (#12987) · ba811639
  youkaichao authored Feb 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: cennn <2523403608@qq.com>
Co-authored-by: cennn <2523403608@qq.com>
```
  ba811639
- [ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577) · 0d243f2a
  Divakar Verma authored Feb 19, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  0d243f2a
- [ci] Add AWS creds for AMD (#13572) · 88f6ba32
  Kevin H. Luu authored Feb 19, 2025
  
  88f6ba32