Commits · 0cc6b383d73eb662dfeec671d3b47cda301b2f47 · OpenDAS / vllm_cscc

27 Jan, 2025 2 commits
- [Frontend] Support scores endpoint in run_batch (#12430) · 0cc6b383
  Pooya Davoodi authored Jan 26, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  0cc6b383
- [Frontend] Rerank API (Jina- and Cohere-compatible API) (#12376) · 0034b09c
  Kyle Mistele authored Jan 26, 2025
```
Signed-off-by: Kyle Mistele <kyle@mistele.com>
```
  0034b09c
26 Jan, 2025 3 commits

[Bugfix/CI] Fix broken kernels/test_mha.py (#12450) · 72f48804
Tyler Michael Smith authored Jan 26, 2025

72f48804
[Bugfix] Disable w16a16 2of4 sparse CompressedTensors24 (#12417) · aa2cd2c4
Tyler Michael Smith authored Jan 26, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
aa2cd2c4

[Frontend] generation_config.json for maximum tokens(#12242) · 9ddc3522

Matthew Hendrey authored Jan 26, 2025


Signed-off-by: Matthew Hendrey <matthew.hendrey@gmail.com>
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Co-authored-by: shangmingc <caishangming@linux.alibaba.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Yuan Tang <terrytangyuan@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com>

9ddc3522

25 Jan, 2025 1 commit
- [Misc] Add FA2 support to ViT MHA layer (#12355) · f1fc0510
  Isotr0py authored Jan 25, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f1fc0510
24 Jan, 2025 3 commits
- [Misc] Remove deprecated code (#12383) · df5dafaa
  Cyrus Leung authored Jan 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  df5dafaa
- [Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) · ab5bbf5a
  Lucas Wilkinson authored Jan 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  ab5bbf5a
- [V1][Frontend] Coalesce bunched `RequestOutput`s (#12298) · 24b0205f
  Nick Hill authored Jan 23, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  24b0205f
23 Jan, 2025 4 commits
- [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b
  Gregory Shtrasberg authored Jan 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  e97f802b
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
- [V1] Add `uncache_blocks` (#12333) · f0ef3723
  Cody Yu authored Jan 22, 2025
  
  f0ef3723
- [AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) · 68c4421b
  rasmith authored Jan 22, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  68c4421b
22 Jan, 2025 4 commits
- [Core] Support `reset_prefix_cache` (#12284) · 7206ce4c
  Cody Yu authored Jan 22, 2025
  
  7206ce4c
- [Core] Support fully transparent sleep mode (#11743) · 68ad4e3a
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  68ad4e3a
- [ci/lint] Add back default arg for pre-commit (#12279) · 64ea24d0
  Kevin H. Luu authored Jan 21, 2025
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  64ea24d0
- [VLM] Simplify post-processing of replacement info (#12269) · df76e5af
  Cyrus Leung authored Jan 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  df76e5af
21 Jan, 2025 4 commits
- [Misc] Remove experimental dep from tracing.py (#12007) · 347eeebe
  Adrian Cole authored Jan 21, 2025
```
Signed-off-by: Adrian Cole <adrian.cole@elastic.co>
```
  347eeebe
- [Bugfix] Multi-sequence broken (#11898) · 18fd4a83
  Andy Lo authored Jan 21, 2025
```
Signed-off-by: Andy Lo <andy@mistral.ai>
```
  18fd4a83
- [v1][stats][1/n] Add RequestStatsUpdate and RequestStats types (#10907) · 132a1321
  Ricky Xu authored Jan 21, 2025
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  132a1321
- [BugFix] Fix GGUF tp>1 when vocab_size is not divisible by 64 (#12230) · 5fe6bf29
  Nicolò Lucchesi authored Jan 21, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  5fe6bf29
20 Jan, 2025 4 commits
- [Bugfix] Fix `HfExampleModels.find_hf_info` (#12223) · 18572e33
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  18572e33
- [Model] Upgrade Aria to transformers 4.48 (#12203) · b37d8279
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b37d8279
- [Core] Interface for accessing model from `VllmRunner` (#10353) · 59a0192f
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a0192f
- [Model] Add Qwen2 PRM model support (#12202) · 83609791
  Isotr0py authored Jan 20, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  83609791
19 Jan, 2025 3 commits

[Model] Support for fairseq2 Llama (#11442) · bbe5f9de

Martin Gleize authored Jan 19, 2025


Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>

bbe5f9de

[V1] Add V1 support of Qwen2-VL (#12128) · 81763c58

Roger Wang authored Jan 19, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

81763c58

[Misc] Support register quantization method out-of-tree (#11969) · 32eb0da8
yancong authored Jan 19, 2025

32eb0da8

18 Jan, 2025 2 commits
- [Model] Port deepseek-vl2 processor, remove dependency (#12169) · 02798eca
  Isotr0py authored Jan 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  02798eca
- [core] further polish memory profiling (#12126) · da02cb4b
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  da02cb4b
17 Jan, 2025 5 commits
- [Bugfix] Fix score api for missing max_model_len validation (#12119) · 58fd57ff
  Wallas Henrique authored Jan 17, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  58fd57ff
- [core] allow callable in collective_rpc (#12151) · 87a0c076
  youkaichao authored Jan 17, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  87a0c076
- [Misc][LoRA] Improve the readability of LoRA error messages (#12102) · 07934cc2
  Jee Jee Li authored Jan 17, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  07934cc2
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
- [Misc] Add deepseek_vl2 chat template (#12143) · d75ab55f
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  d75ab55f
16 Jan, 2025 4 commits
- [Model] Add support for deepseek-vl2-tiny model (#12068) · 62b06ba2
  Isotr0py authored Jan 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  62b06ba2
- [Bugfix] Fix max image feature size for Llava-one-vision (#12104) · 874f7c29
  Roger Wang authored Jan 16, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  874f7c29
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
- [Bugfix] Remove hardcoded `head_size=256` for Deepseek v2 and v3 (#12067) · dd7c9ad8
  Isotr0py authored Jan 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dd7c9ad8
15 Jan, 2025 1 commit
- [Bugfix] use right truncation for non-generative tasks (#12050) · edce722e
  Joe Runde authored Jan 15, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  edce722e