Commits · 0d02747f2ed5f65bd7100b6dcf1805cefb458f5d · OpenDAS / vllm_cscc · GitLab

22 Oct, 2024 5 commits
- support TP in qwen2 bnb (#9574) · 0d02747f
  chenqianfzh authored Oct 22, 2024
  
  0d02747f
- [Core] Remove evictor_v1 (#9572) · ca30c3c8
  Kuntai Du authored Oct 21, 2024
  
  ca30c3c8
- [Bugfix]: phi.py get rope_theta from config file (#9503) · 74692421
  Falko1 authored Oct 22, 2024
```
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  74692421
- [Bugfix]: serialize config by value for --trust-remote-code (#6751) · b7299011
  Travis Johnson authored Oct 21, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b7299011
- [core] move parallel sampling out from vllm core (#9302) · 76a5e132
  youkaichao authored Oct 21, 2024
  
  76a5e132
21 Oct, 2024 7 commits
- [Frontend] Don't log duplicate error stacktrace for every request in the batch (#9023) · 711f3a78
  Wallas Henrique authored Oct 21, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  711f3a78
- [BugFix] Update draft model TP size check to allow matching target TP size (#9394) · 15713e3b
  Nick Hill authored Oct 21, 2024
```
Co-authored-by: Baoyuan Qi <qibaoyuan@126.com>
```
  15713e3b
- [Frontend] Reduce frequency of client cancellation checking (#7959) · 9d9186be
  Nick Hill authored Oct 21, 2024
  
  9d9186be
- [Model][Bugfix] Fix batching with multi-image in PixtralHF (#9518) · 5241aa14
  Michael Goin authored Oct 21, 2024
  
  5241aa14
- [Bugfix][Misc]: fix graph capture for decoder (#9549) · 8ca89548
  yudian0504 authored Oct 22, 2024
  
  8ca89548
- [Model] FalconMamba Support (#9325) · f6b97293
  Dhia Eddine Rhaiem authored Oct 21, 2024
  
  f6b97293
- [Doc] Consistent naming of attention backends (#9498) · 496e991d
  Thomas Parnell authored Oct 21, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  496e991d
20 Oct, 2024 3 commits
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
- [Model][Pixtral] Use memory_efficient_attention for PixtralHFVision (#9520) · 962d2c63
  Michael Goin authored Oct 20, 2024
  
  962d2c63
- [Bugfix] Pass json-schema to GuidedDecodingParams and make test stronger (#9530) · 5b59fe0f
  Chen Zhang authored Oct 19, 2024
  
  5b59fe0f
19 Oct, 2024 8 commits
- [Model][Pixtral] Optimizations for input_processor_for_pixtral_hf (#9514) · 8e3e7f27
  Michael Goin authored Oct 19, 2024
  
  8e3e7f27
- [Bugfix] Fix missing task for speculative decoding (#9524) · 263d8ee1
  Cyrus Leung authored Oct 19, 2024
  
  263d8ee1
- [Frontend] Support simpler image input format (#9478) · c5eea3c8
  Yue Zhang authored Oct 18, 2024
  
  c5eea3c8
- [Doc] update gpu-memory-utilization flag docs (#9507) · 82c25151
  Joe Runde authored Oct 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  82c25151
- [Frontend] Avoid creating guided decoding LogitsProcessor unnecessarily (#9521) · 1325872e
  Nick Hill authored Oct 19, 2024
  
  1325872e
- 🐛 fix torch memory profiling (#9516) · 380e1863
  Joe Runde authored Oct 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  380e1863
- [Bugfix] Fix offline mode when using `mistral_common` (#9457) · 337ed766
  sasha0552 authored Oct 19, 2024
  
  337ed766
- [Kernel] Add env variable to force flashinfer backend to enable tensor cores (#9497) · 0c9a5258
  Thomas Parnell authored Oct 19, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh Yang <chih.chieh.yang@ibm.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  0c9a5258
18 Oct, 2024 11 commits
- [MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510) · d11bf435
  Cody Yu authored Oct 18, 2024
  
  d11bf435
- [MISC] Add lora requests to metrics (#9477) · 9bb10a7d
  Kunjan authored Oct 18, 2024
```
Co-authored-by: Kunjan Patel <kunjanp_google_com@vllm.us-central1-a.c.kunjanp-gke-dev-2.internal>
```
  9bb10a7d
- [Model] Support Pixtral models in the HF Transformers format (#9036) · 3921a2f2
  Michael Goin authored Oct 18, 2024
  
  3921a2f2
- [Model] Add user-configurable task for models that support both generation and embedding (#9424) · 051eaf6d
  Cyrus Leung authored Oct 19, 2024
  
  051eaf6d
- [CI/Build] Fix lint errors in mistral tokenizer (#9504) · 1bbbcc0b
  Cyrus Leung authored Oct 19, 2024
  
  1bbbcc0b
- [BugFix] Fix and simplify completion API usage streaming (#9475) · 25aeb7d4
  Nick Hill authored Oct 18, 2024
  
  25aeb7d4
- [Frontend][Feature] Add jamba tool parser (#9154) · d2b1bf55
  tomeras91 authored Oct 18, 2024
  
  d2b1bf55
- [BugFix] Typing fixes to RequestOutput.prompt and beam search (#9473) · 1ffc8a73
  Nick Hill authored Oct 18, 2024
  
  1ffc8a73
- [Qwen2.5] Support bnb quant for Qwen2.5 (#9467) · 154a8ae8
  Haoyu Wang authored Oct 18, 2024
  
  154a8ae8
- [Bugfix][Core] Use torch.cuda.memory_stats() to profile peak memory usage (#9352) · de4008e2
  Joe Runde authored Oct 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  de4008e2
- [BugFix] Stop silent failures on compressed-tensors parsing (#9381) · 48138a84
  Dipika Sikka authored Oct 17, 2024
  
  48138a84
17 Oct, 2024 6 commits
- Support `BERTModel` (first `encoder-only` embedding model) (#9056) · 343f8e09
  Robert Shaw authored Oct 17, 2024
```
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: laishzh <laishengzhang@gmail.com>
Co-authored-by: Max de Bayser <maxdebayser@gmail.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  343f8e09
- [Hardwware][Neuron] Simplify model load for transformers-neuronx library (#9380) · bb76538b
  Shashwat Srijan authored Oct 17, 2024
  
  bb76538b
- [Bugfix] Print warnings related to `mistral_common` tokenizer only once (#9468) · d615b5c9
  sasha0552 authored Oct 17, 2024
  
  d615b5c9
- [Bugfix] Fix support for dimension like integers and ScalarType (#9299) · eca2c5f7
  bnellnm authored Oct 17, 2024
  
  eca2c5f7
- [torch.compile] Fine-grained CustomOp enabling mechanism (#9300) · 0f41fbe5
  Luka Govedič authored Oct 17, 2024
  
  0f41fbe5
- [Misc] Remove commit id file (#9470) · 7871659a
  Cyrus Leung authored Oct 18, 2024
  
  7871659a