Commits · 1d65ec7eeb35f03eb87ed080094f1aa5ff2ae3d3 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "d1b5e7afbf8a4ff45904c191748594167dd34e78"

15 Nov, 2024 13 commits
- [Bugfix] Fix fully sharded LoRA bug (#10352) · 1d65ec7e
  Jee Jee Li authored Nov 15, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1d65ec7e
- [Doc] Remove float32 choice from --lora-dtype (#10348) · 26908554
  Xin Yang authored Nov 15, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  26908554
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
- Add default value to avoid Falcon crash (#5363) (#10347) · 3d158cdc
  wchen61 authored Nov 15, 2024
```
Signed-off-by: wchen61 <wchen61@foxmail.com>
```
  3d158cdc
- [Misc] Consolidate pooler config overrides (#10351) · 2ac6d0e7
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  2ac6d0e7
- [Bugfix] Qwen-vl output is inconsistent in speculative decoding (#10350) · 2ec88272
  Sky Lee authored Nov 15, 2024
  
  2ec88272
- [Model] Support Qwen2 embeddings and use tags to select model tests (#10184) · b40cf640
  Cyrus Leung authored Nov 15, 2024
  
  b40cf640
- [Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (#10308) · 2885ba0e
  Tyler Michael Smith authored Nov 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2885ba0e
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
- [Bugfix] Fix unable to load some models (#10312) · 972112d8
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  972112d8
- [Tool parsing] Improve / correct mistral tool parsing (#10333) · 11cd1ae6
  Patrick von Platen authored Nov 15, 2024
  
  11cd1ae6
- [Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (#9583) · 554af922
  Zijin Xiao authored Nov 15, 2024
```
Signed-off-by: xiaozijin <xiaozijin@bytedance.com>
```
  554af922
- [Perf] Reduce peak memory usage of llama (#10339) · b2e0ad3b
  Murali Andoorveedu authored Nov 14, 2024
```
Signed-off-by: andoorve <37849411+andoorve@users.noreply.github.com>
```
  b2e0ad3b
14 Nov, 2024 6 commits
- Support Roberta embedding models (#9387) · 4a18fd14
  Maximilien de Bayser authored Nov 14, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>
```
  4a18fd14
- [Model] Add BNB quantization support for Idefics3 (#10310) · 294bf467
  B-201 authored Nov 14, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  294bf467
- [BugFix]: properly deserialize `tool_calls` iterator before processing by... · 52b48c1e
  Guillaume Calmettes authored Nov 14, 2024
```
[BugFix]: properly deserialize `tool_calls` iterator before processing by mistral-common when MistralTokenizer is used (#9951)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  52b48c1e
- [Frontend] Pythonic tool parser (#9859) · f67ce05d
  Mike Depinet authored Nov 13, 2024
```
Signed-off-by: Mike Depinet <mike@fixie.ai>
```
  f67ce05d
- [misc] error early for old-style class (#10304) · 504ac53d
  youkaichao authored Nov 13, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  504ac53d
- [Bugfix] Fix tensor parallel for qwen2 classification model (#10297) · 15bb8330
  Isotr0py authored Nov 14, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  15bb8330
13 Nov, 2024 10 commits
- [Bugfix] bitsandbytes models fail to run pipeline parallel (#10200) · ac49b59d
  HoangCongDuc authored Nov 14, 2024
```
Signed-off-by: Hoang Cong Duc <hoangcongducltt@gmail.com>
```
  ac49b59d
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
- [V1] Add missing tokenizer options for `Detokenizer` (#10288) · bb7991aa
  Roger Wang authored Nov 13, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  bb7991aa
- [Model][LoRA]LoRA support added for idefics3 (#10281) · d909acf9
  B-201 authored Nov 13, 2024
```
Signed-off-by: B-201 <Joy25810@foxmail.com>
```
  d909acf9
- [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (#9944) · 1b886aa1
  Austin Veselka authored Nov 13, 2024
```
Signed-off-by: FurtherAI <austin.veselka@lighton.ai>
Co-authored-by: FurtherAI <austin.veselka@lighton.ai>
```
  1b886aa1
- [Model] Add support for Qwen2-VL video embeddings input & multiple image... · 3945c823
  电脑星人 authored Nov 13, 2024
```
[Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (#10221)
Signed-off-by: imkero <kerorek@outlook.com>
```
  3945c823
- [Doc] Fix typo in arg_utils.py (#10264) · 032fcf16
  Xin Yang authored Nov 12, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  032fcf16
- [V1] Support VLMs with fine-grained scheduling (#9871) · bbd3e869
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  bbd3e869
- [core][distributed] use tcp store directly (#10275) · 0d4ea3fb
  youkaichao authored Nov 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  0d4ea3fb
- [V1] Fix CI tests on V1 engine (#10272) · 112fa0bb
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  112fa0bb
12 Nov, 2024 11 commits
- [V1] Enable Inductor when using piecewise CUDA graphs (#10268) · 1f55e057
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1f55e057
- [LoRA] Adds support for bias in LoRA (#5733) · 8a06428c
  Umesh authored Nov 12, 2024
```
Signed-off-by: Umesh Deshpande <udeshpa@us.ibm.com>
Co-authored-by: Umesh Deshpande <udeshpa@us.ibm.com>
```
  8a06428c
- [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (#9982) · b41fb9d3
  sroy745 authored Nov 12, 2024
```
Signed-off-by: Sourashis Roy <sroy@roblox.com>
```
  b41fb9d3
- [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to... · 7c655279
  Woosuk Kwon authored Nov 12, 2024
```
[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7c655279
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
- [Bugfix] Fix QwenModel argument (#10262) · 176fcb1c
  Jie Fu (傅杰) authored Nov 13, 2024
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  176fcb1c
- [Misc]Fix Idefics3Model argument (#10255) · a838ba72
  Jee Jee Li authored Nov 12, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  a838ba72
- [BugFix] Do not raise a `ValueError` when `tool_choice` is set to the... · 36c513a0
  Guillaume Calmettes authored Nov 12, 2024
```
[BugFix] Do not raise a `ValueError` when `tool_choice` is set to the supported `none` option and `tools` are not defined. (#10000)
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  36c513a0
- [Misc][LoRA] Replace hardcoded cuda device with configurable argument (#10223) · 7f5edb59
  Jee Jee Li authored Nov 12, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  7f5edb59
- [1/N] torch.compile user interface design (#10237) · eea55cca
  youkaichao authored Nov 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eea55cca
- [Doc] Update help text for `--distributed-executor-backend` (#10231) · 9cdba966
  Russell Bryant authored Nov 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9cdba966