Commits · 56a955e7748e497d8c24c79a76c75f3f982fab4a · OpenDAS / vllm_cscc · GitLab

13 Nov, 2024 4 commits
- Bump to compressed-tensors v0.8.0 (#10279) · 56a955e7
  Dipika Sikka authored Nov 13, 2024
```
Signed-off-by: Dipika <dipikasikka1@gmail.com>
```
  56a955e7
- [V1] Support VLMs with fine-grained scheduling (#9871) · bbd3e869
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  bbd3e869
- [core][distributed] use tcp store directly (#10275) · 0d4ea3fb
  youkaichao authored Nov 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0d4ea3fb
- [V1] Fix CI tests on V1 engine (#10272) · 112fa0bb
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  112fa0bb
12 Nov, 2024 17 commits
- Revert "[ci][build] limit cmake version" (#10271) · 377b74fe
  youkaichao authored Nov 12, 2024
  
  377b74fe
- [doc] improve debugging doc (#10270) · 18081451
  youkaichao authored Nov 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  18081451
- [doc] fix location of runllm widget (#10266) · 96ae0eae
  youkaichao authored Nov 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  96ae0eae
- [V1] Enable Inductor when using piecewise CUDA graphs (#10268) · 1f55e057
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1f55e057
- [LoRA] Adds support for bias in LoRA (#5733) · 8a06428c
  Umesh authored Nov 12, 2024
```
Signed-off-by: Umesh Deshpande <udeshpa@us.ibm.com>
Co-authored-by: Umesh Deshpande <udeshpa@us.ibm.com>
```
  8a06428c
- [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) · b41fb9d3
  sroy745 authored Nov 12, 2024
```
Signed-off-by: Sourashis Roy <sroy@roblox.com>
```
  b41fb9d3
- [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to... · 7c655279
  Woosuk Kwon authored Nov 12, 2024
```
[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7c655279
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
- [Bugfix] Fix QwenModel argument (#10262) · 176fcb1c
  Jie Fu (傅杰) authored Nov 13, 2024
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  176fcb1c
- [Misc]Fix Idefics3Model argument (#10255) · a838ba72
  Jee Jee Li authored Nov 12, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a838ba72
- [BugFix] Do not raise a `ValueError` when `tool_choice` is set to the... · 36c513a0
  Guillaume Calmettes authored Nov 12, 2024
```
[BugFix] Do not raise a `ValueError` when `tool_choice` is set to the supported `none` option and `tools` are not defined. (#10000)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  36c513a0
- [CI][CPU]refactor CPU tests to allow to bind with different cores (#10222) · d201d419
  Yuan authored Nov 12, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  d201d419
- [doc] explain the class hierarchy in vLLM (#10240) · 3a28f18b
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3a28f18b
- Splitting attention kernel file (#10091) · 812c981f
  Aleksandr Malyshev authored Nov 11, 2024
```
Signed-off-by: maleksan85 <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
```
  812c981f
- [Misc][LoRA] Replace hardcoded cuda device with configurable argument (#10223) · 7f5edb59
  Jee Jee Li authored Nov 12, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7f5edb59
- [1/N] torch.compile user interface design (#10237) · eea55cca
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eea55cca
- [Doc] Update help text for `--distributed-executor-backend` (#10231) · 9cdba966
  Russell Bryant authored Nov 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9cdba966
11 Nov, 2024 19 commits
- [doc] update debugging guide (#10236) · d1c6799b
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  d1c6799b
- [V1] `AsyncLLM` Implementation (#9826) · 6ace6fba
  Robert Shaw authored Nov 11, 2024
```
Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  6ace6fba
- Make shutil rename in python_only_dev (#10233) · 08f93e74
  Nikolai Shcheglov authored Nov 11, 2024
```
Signed-off-by: shcheglovnd <shcheglovnd@avride.ai>
```
  08f93e74
- [V1] Enable custom ops with piecewise CUDA graphs (#10228) · 9d5b4e4d
  Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  9d5b4e4d
- [misc][distributed] auto port selection and disable tests (#10226) · 8a7fe47d
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8a7fe47d
- Add docs on serving with Llama Stack (#10183) · 4800339c
  Yuan Tang authored Nov 11, 2024
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  4800339c
- [V1] Use custom ops for piecewise CUDA graphs (#10227) · fe15729a
  Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  fe15729a
- [v1][torch.compile] support managing cudagraph buffer (#10203) · 330e82d3
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  330e82d3
- [V1] Do not use inductor for piecewise CUDA graphs (#10225) · d7a4f220
  Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d7a4f220
- [V1] Fix detokenizer ports (#10224) · f9dadfbe
  Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f9dadfbe
- Bump actions/setup-python from 5.2.0 to 5.3.0 (#10209) · 25144cee
  dependabot[bot] authored Nov 11, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  25144cee
- [core][distributed] add stateless process group (#10216) · e6de9784
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e6de9784
- [Doc] fix doc string typo in block_manager `swap_out` function (#10212) · 36fc439d
  Yangcheng Li authored Nov 12, 2024
  
  36fc439d
- [Metrics] add more metrics (#4464) · 874f551b
  harrywu authored Nov 12, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  874f551b
- [Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (#10218) · 2cebda42
  Isotr0py authored Nov 11, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2cebda42
- [V1] Allow `tokenizer_mode` and `trust_remote_code` for Detokenizer (#10211) · 5fb1f935
  Roger Wang authored Nov 11, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  5fb1f935
- [LoRA][Kernel] Remove the unused libentry module (#10214) · 36e4acd0
  Jee Jee Li authored Nov 11, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  36e4acd0
- [Hardware][CPU] Add embedding models support for CPU backend (#10193) · 58170d65
  Isotr0py authored Nov 11, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  58170d65
- Bump the patch-update group with 5 updates (#10210) · 9804ac7c
  dependabot[bot] authored Nov 11, 2024
```
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
```
  9804ac7c