Commits · 76a5e13270f32216bb28cfe185bada5e88e407d7 · OpenDAS / vllm_cscc · GitLab

22 Oct, 2024 2 commits
- [core] move parallel sampling out from vllm core (#9302) · 76a5e132
  youkaichao authored Oct 21, 2024
  
  76a5e132
- 🐛 Fixup more test failures from memory profiling (#9563) · ef7faad1
  Joe Runde authored Oct 21, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  ef7faad1
21 Oct, 2024 11 commits
- [CI] Make format checker error message more user-friendly by using emoji (#9564) · 575dcebe
  Kuntai Du authored Oct 21, 2024
```
This PR makes format checker error message more user-friendly by adding emojis.
```
  575dcebe
- [Frontend] Don't log duplicate error stacktrace for every request in the batch (#9023) · 711f3a78
  Wallas Henrique authored Oct 21, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  711f3a78
- [BugFix] Update draft model TP size check to allow matching target TP size (#9394) · 15713e3b
  Nick Hill authored Oct 21, 2024
```
Co-authored-by: Baoyuan Qi <qibaoyuan@126.com>
```
  15713e3b
- [doc] fix format (#9562) · d621c43d
  youkaichao authored Oct 21, 2024
  
  d621c43d
- [Frontend] Reduce frequency of client cancellation checking (#7959) · 9d9186be
  Nick Hill authored Oct 21, 2024
  
  9d9186be
- [Model][Bugfix] Fix batching with multi-image in PixtralHF (#9518) · 5241aa14
  Michael Goin authored Oct 21, 2024
  
  5241aa14
- [BugFix] Use correct python3 binary in Docker.ppc64le entrypoint (#9492) · ec6bd6c4
  Varad Ahirwadkar authored Oct 21, 2024
```
Signed-off-by: Varad Ahirwadkar <varad.ahirwadkar1@ibm.com>
```
  ec6bd6c4
- [Bugfix][Misc]: fix graph capture for decoder (#9549) · 8ca89548
  yudian0504 authored Oct 22, 2024
  
  8ca89548
- [Model] FalconMamba Support (#9325) · f6b97293
  Dhia Eddine Rhaiem authored Oct 21, 2024
  
  f6b97293
- [Doc] Consistent naming of attention backends (#9498) · 496e991d
  Thomas Parnell authored Oct 21, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  496e991d
- [CI/Build] Split up decoder-only LM tests (#9488) · 696b01af
  Cyrus Leung authored Oct 21, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  696b01af
20 Oct, 2024 4 commits
- [Frontend][Misc] Goodput metric support (#9338) · 855e0e6f
  Andy Dai authored Oct 20, 2024
  
  855e0e6f
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
- [Model][Pixtral] Use memory_efficient_attention for PixtralHFVision (#9520) · 962d2c63
  Michael Goin authored Oct 20, 2024
  
  962d2c63
- [Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530) · 5b59fe0f
  Chen Zhang authored Oct 19, 2024
  
  5b59fe0f
19 Oct, 2024 10 commits
- [Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (#9514) · 8e3e7f27
  Michael Goin authored Oct 19, 2024
  
  8e3e7f27
- [Bugfix] Fix missing task for speculative decoding (#9524) · 263d8ee1
  Cyrus Leung authored Oct 19, 2024
  
  263d8ee1
- [Frontend] Support simpler image input format (#9478) · c5eea3c8
  Yue Zhang authored Oct 18, 2024
  
  c5eea3c8
- [CI/Build] Configure matcher for actionlint workflow (#9511) · 85dc92fc
  Russell Bryant authored Oct 19, 2024
```
Signed-off-by: Russell Bryant <russell.bryant@gmail.com>
```
  85dc92fc
- [CI/Build] Add error matching for ruff output (#9513) · dfd951ed
  Russell Bryant authored Oct 19, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dfd951ed
- [Doc] update gpu-memory-utilization flag docs (#9507) · 82c25151
  Joe Runde authored Oct 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  82c25151
- [Frontend] Avoid creating guided decoding LogitsProcessor unnecessarily (#9521) · 1325872e
  Nick Hill authored Oct 19, 2024
  
  1325872e
- 🐛 fix torch memory profiling (#9516) · 380e1863
  Joe Runde authored Oct 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  380e1863
- [Bugfix] Fix offline mode when using `mistral_common` (#9457) · 337ed766
  sasha0552 authored Oct 19, 2024
  
  337ed766
- [Kernel] Add env variable to force flashinfer backend to enable tensor cores (#9497) · 0c9a5258
  Thomas Parnell authored Oct 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  0c9a5258
18 Oct, 2024 13 commits
- [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) · d11bf435
  Cody Yu authored Oct 18, 2024
  
  d11bf435
- [MISC] Add lora requests to metrics (#9477) · 9bb10a7d
  Kunjan authored Oct 18, 2024
```
Co-authored-by: Kunjan Patel <kunjanp_google_com@vllm.us-central1-a.c.kunjanp-gke-dev-2.internal>
```
  9bb10a7d
- [Model] Support Pixtral models in the HF Transformers format (#9036) · 3921a2f2
  Michael Goin authored Oct 18, 2024
  
  3921a2f2
- [CI/Build] Add error matching config for mypy (#9512) · 67a7e5ef
  Russell Bryant authored Oct 18, 2024
  
  67a7e5ef
- [Model] Add user-configurable task for models that support both generation and embedding (#9424) · 051eaf6d
  Cyrus Leung authored Oct 19, 2024
  
  051eaf6d
- [Misc] benchmark: Add option to set max concurrency (#9390) · 7dbe738d
  Russell Bryant authored Oct 18, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  7dbe738d
- [Bugfix] Fix offline_inference_with_prefix.py (#9505) · ae8b633b
  Tyler Michael Smith authored Oct 18, 2024
  
  ae8b633b
- [CI/Build] Fix lint errors in mistral tokenizer (#9504) · 1bbbcc0b
  Cyrus Leung authored Oct 19, 2024
  
  1bbbcc0b
- [BugFix] Fix and simplify completion API usage streaming (#9475) · 25aeb7d4
  Nick Hill authored Oct 18, 2024
  
  25aeb7d4
- [Frontend][Feature] Add jamba tool parser (#9154) · d2b1bf55
  tomeras91 authored Oct 18, 2024
  
  d2b1bf55
- [BugFix] Typing fixes to RequestOutput.prompt and beam search (#9473) · 1ffc8a73
  Nick Hill authored Oct 18, 2024
  
  1ffc8a73
- [CI/Build] Use commit hash references for github actions (#9430) · 944dd8ed
  Russell Bryant authored Oct 18, 2024
  
  944dd8ed
- [Qwen2.5] Support bnb quant for Qwen2.5 (#9467) · 154a8ae8
  Haoyu Wang authored Oct 18, 2024
  
  154a8ae8