Commits · 5c9121203cc34e781f1f249b69cb789244e861f0 · OpenDAS / vllm_cscc · GitLab

19 Apr, 2025 1 commit
- [release] Publish neuron docker image (#16733) · 5c912120
  omrishiv authored Apr 19, 2025
```
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
```
  5c912120
18 Apr, 2025 18 commits
- [Doc] Updated Llama section in tool calling docs to have llama 3.2 config info (#16857) · 490b1698
  Justin Ho authored Apr 18, 2025
```
Signed-off-by: jmho <jaylenho734@gmail.com>
```
  490b1698
- [Misc] refactor examples series - Chat Completion Client With Tools (#16829) · 5a5e29de
  Reid authored Apr 19, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  5a5e29de
- [New Model]: Snowflake Arctic Embed (Family) (#16649) · 3d3ab368
  wang.yuqi authored Apr 18, 2025
  
  3d3ab368
- Fix `nullable_kvs` fallback (#16837) · 686623c5
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  686623c5
- [Misc] Clean up Kimi-VL (#16833) · aadb6565
  Cyrus Leung authored Apr 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aadb6565
- [Model] use AutoWeightsLoader for BigCode, GPT-J (#16823) · 87e067de
  Jonghyun Choe authored Apr 18, 2025
```
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
```
  87e067de
- [Docs] Fix a link and grammar issue in production-stack.md (#16809) · 26507f89
  Michael Yao authored Apr 18, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  26507f89
- [Doc] add podman setup instructions for official image (#16796) · 9c1d5b45
  Nathan Weinberg authored Apr 18, 2025
```
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
```
  9c1d5b45
- [Bugfix] fix pp for llama4 (#16746) · e31045f9
  Lucia Fang authored Apr 17, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  e31045f9
- [ROCm] [Attention] Cleanup ROCm output passing (#16431) · aaec845f
  Luka Govedič authored Apr 18, 2025
```
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
```
  aaec845f
- [Misc] add collect_env to cli and docker image (#16759) · 7bdfd29a
  rongfu.leng authored Apr 18, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  7bdfd29a
- Improve-mm-and-pooler-and-decoding-configs (#16789) · e78587a6
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e78587a6
- [BugFix] Accuracy fix for llama4 int4 - improperly casted scales (#16801) · 7eb42556
  Lucas Wilkinson authored Apr 18, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  7eb42556
- Add hardware print to TPU V1 test (#16792) · 6a0f5475
  Michael Goin authored Apr 17, 2025
  
  6a0f5475
- [V1][Structured Output] Minor modification to `_validate_structured_output()` (#16748) · 30ed81b7
  Shanshan Shen authored Apr 18, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  30ed81b7
- [Misc] Update outdated note: LMCache now supports chunked prefill (#16697) · 7a4a5de7
  Chauncey authored Apr 18, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  7a4a5de7
- [Doc] Improve help examples for `--compilation-config` (#16729) · c16fb5da
  Cyrus Leung authored Apr 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c16fb5da
- Add property-based testing for vLLM endpoints using an API defined by an... · e37073ef
  Tarun Kumar authored Apr 18, 2025
```
Add property-based testing for vLLM endpoints using an API defined by an OpenAPI 3.1 schema (#16721)
Signed-off-by: Tarun Kumar <takumar@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e37073ef
17 Apr, 2025 21 commits
- [Attention] Update to lastest FA3 code (#13111) · 183dad7a
  Lucas Wilkinson authored Apr 17, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  183dad7a
- [P/D][V1] KV Connector API V1 (#15960) · 3408e471
  Yihua Cheng authored Apr 17, 2025
```
Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
  3408e471
- [MLA] Simplification to batch P/D reordering (#16673) · 0377b831
  Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  0377b831
- [V1][Metrics] Fix http metrics middleware (#15894) · e4755f7f
  Mark McLoughlin authored Apr 17, 2025
  
  e4755f7f
- [ROCM] enable aiter fused moe kernel for llama4 bf16 checkpoints (#16674) · 92edf358
  Sijia(Jackson) Chen authored Apr 17, 2025
  
  92edf358
- [V1][TPU] Enable Top K (#15489) · eb5819b2
  Nicolò Lucchesi authored Apr 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>
```
  eb5819b2
- [TPU][V1] Fix padding recompilation when `max-num-batched-tokens` is not even (#16726) · 5989f468
  Nicolò Lucchesi authored Apr 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5989f468
- [Model] use AutoWeightsLoader for olmoe,opt,orion,persimmon,phi3_small (#16548) · 5125d72f
  rongfu.leng authored Apr 18, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  5125d72f
- [Kernel] Add fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 on NVIDIA H20 (#16753) · a018e555
  Ximingwang-09 authored Apr 18, 2025
```
Signed-off-by: ximing.wxm <ximing.wxm@antgroup.com>
Co-authored-by: ximing.wxm <ximing.wxm@antgroup.com>
```
  a018e555
- [Bugfix]Fix index out of range error in api server log (#16787) · 6211b922
  Robin authored Apr 18, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
  6211b922
- [V1][Perf] Faster incremental detokenization (#15137) · 05fcd1b4
  Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  05fcd1b4
- [Doc] Changed explanation of generation_tokens_total and prompt_tokens_total... · 7c02d6a1
  Insu Kim authored Apr 17, 2025
```
[Doc] Changed explanation of generation_tokens_total and prompt_tokens_total counter type metrics to avoid confusion (#16784)
Signed-off-by: insukim1994 <insu.kim@moreh.io>
```
  7c02d6a1
- [Doc] Document Matryoshka Representation Learning support (#16770) · 11c3b984
  wang.yuqi authored Apr 17, 2025
  
  11c3b984
- [Doc] Make sure to update vLLM when installing latest code (#16781) · dbe7f070
  Cyrus Leung authored Apr 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  dbe7f070
- fix: hyperlink (#16778) · c69bf4ee
  Reid authored Apr 17, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  c69bf4ee
- Improve configs - `TokenizerPoolConfig` + `DeviceConfig` (#16603) · d27ea940
  Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d27ea940
- [Misc] refactor examples series - lmcache (#16758) · 99ed5261
  Reid authored Apr 17, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  99ed5261
- [Doc] Fix a 404 link in installation/cpu.md (#16773) · 207da281
  Michael Yao authored Apr 17, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  207da281
- [Bugfix] Fix GLM4 model (#16618) · 5b1aca2a
  intervitens authored Apr 17, 2025
```
Signed-off-by: intervitens <intervitens@tutanota.com>
```
  5b1aca2a
- [doc] add open-webui example (#16747) · d8e557b5
  Reid authored Apr 17, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d8e557b5
- [Doc] Add more tips to avoid OOM (#16765) · 61a44a0b
  Cyrus Leung authored Apr 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61a44a0b