Commits · 5223199e03ac3729eb60043a1ef57156c8af1bc9 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "682789d4026429a04cc32acb88064265441080dd"

07 Aug, 2024 11 commits
- [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) · 5223199e
  Michael Goin authored Aug 07, 2024
  
  5223199e
- [BugFix] Fix frontend multiprocessing hang (#7217) · fde47d3b
  Maximilien de Bayser authored Aug 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  fde47d3b
- [Bugfix] Fix input processor for InternVL2 model (#7164) · b7645476
  Isotr0py authored Aug 08, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b7645476
- Fixes typo in function name (#7275) · ab0f5e28
  Rafael Vasquez authored Aug 07, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  ab0f5e28
- [ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222) · 56498572
  Robert Shaw authored Aug 07, 2024
  
  56498572
- [Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and... · 0f7052bc
  Dipika Sikka authored Aug 07, 2024
```
[Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and `weight_loader_v2` (#5874)
```
  0f7052bc
- [distributed][misc] add specialized method for cuda platform (#7249) · 639159b2
  youkaichao authored Aug 07, 2024
  
  639159b2
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
- [BUGFIX]: top_k is expected to be an integer. (#7227) · 7b261092
  Atilla Akkuş authored Aug 07, 2024
  
  7b261092
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
- [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225) · f9a56006
  Michael Goin authored Aug 06, 2024
  
  f9a56006
06 Aug, 2024 7 commits
- [Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026
  afeldman-nm authored Aug 06, 2024
```
[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  fd95e026
- [Core] Optimize evictor-v2 performance (#7193) · 660470e5
  xiaobochen123 authored Aug 07, 2024
  
  660470e5
- [Kernel] Add per-tensor and per-token AZP epilogues (#5941) · 8d59dbb0
  Luka Govedič authored Aug 06, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8d59dbb0
- [SpecDecode] [Minor] Fix spec decode sampler tests (#7183) · 5c60c8c4
  Lily Liu authored Aug 06, 2024
  
  5c60c8c4
- [ BugFix ] Fix ZMQ when `VLLM_PORT` is set (#7205) · 541c1852
  Robert Shaw authored Aug 06, 2024
  
  541c1852
- [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) · 1f26efbb
  Cyrus Leung authored Aug 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  1f26efbb
- [LoRA] Relax LoRA condition (#7146) · 9118217f
  Jee Jee Li authored Aug 06, 2024
  
  9118217f
05 Aug, 2024 13 commits
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
- [MISC] Use non-blocking transfer in prepare_input (#7172) · ef527be0
  Cody Yu authored Aug 05, 2024
  
  ef527be0
- [Bugfix] Specify device when loading LoRA and embedding tensors (#7129) · 89b8db6b
  Jacob Schein authored Aug 05, 2024
```
Co-authored-by: Jacob Schein <jacobschein@Jacobs-MacBook-Pro-2.local>
```
  89b8db6b
- [Doc] [SpecDecode] Update MLPSpeculator documentation (#7100) · 789937af
  Thomas Parnell authored Aug 06, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  789937af
- bump version to v0.5.4 (#7139) · 4db5176d
  Simon Mo authored Aug 05, 2024
  
  4db5176d
- [Misc] Fix typo in GroupCoordinator.recv() (#7167) · 997cf783
  Rui Qiao authored Aug 05, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  997cf783
- [BugFix] Use args.trust_remote_code (#7121) · 57f560aa
  Aditya Paliwal authored Aug 05, 2024
  
  57f560aa
- [BugFix] Use IP4 localhost form for zmq bind (#7163) · 003f8ee1
  Nick Hill authored Aug 05, 2024
  
  003f8ee1
- [SpecDecode] Support FlashInfer in DraftModelRunner (#6926) · e9630458
  Bongwon Jang authored Aug 06, 2024
  
  e9630458
- [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963) · 82a1b1a8
  Cade Daniel authored Aug 05, 2024
  
  82a1b1a8
- [Model] SiglipVisionModel ported from transformers (#6942) · c0d8f163
  Jungho Christopher Cho authored Aug 05, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  c0d8f163
- [Frontend] Reapply "Factor out code for running uvicorn" (#7095) · cc08fc72
  Cyrus Leung authored Aug 05, 2024
  
  cc08fc72
- [Model] Add multi-image support for minicpmv (#7122) · 7b86e7c9
  Alphi authored Aug 05, 2024
```
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  7b86e7c9
04 Aug, 2024 6 commits
- Clean up remaining Punica C information (#7027) · f80ab352
  Jee Jee Li authored Aug 05, 2024
  
  f80ab352
- [misc][distributed] improve libcudart.so finding (#7127) · 16a1cc9b
  youkaichao authored Aug 04, 2024
  
  16a1cc9b
- [Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when... · b1c9aa3d
  Thomas Parnell authored Aug 04, 2024
```
[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  b1c9aa3d
- [Model]Refactor MiniCPMV (#7020) · 179a6a36
  Jee Jee Li authored Aug 04, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  179a6a36
- [core][misc] simply output processing with shortcut code path (#7117) · 83c644fe
  youkaichao authored Aug 04, 2024
  
  83c644fe
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca
03 Aug, 2024 3 commits
- [Frontend] Warn if user `max_model_len` is greater than derived `max_model_len` (#7080) · 825b0448
  Jeff Fialho authored Aug 03, 2024
```
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  825b0448
- [LoRA] ReplicatedLinear support LoRA (#7081) · 99d7cabd
  Jee Jee Li authored Aug 03, 2024
  
  99d7cabd
- [Bugfix] Fix block table for seqs that have prefix cache hits (#7018) · fb2c1c86
  Zach Zheng authored Aug 02, 2024
  
  fb2c1c86