Commits · 639159b2a665173bc7a81625887b8e76f85e2e32 · OpenDAS / vllm_cscc

"docs/vscode:/vscode.git/clone" did not exist on "e24336b5a772ab3aa6ad83527b880f9e5050ea2a"

07 Aug, 2024 6 commits
- [distributed][misc] add specialized method for cuda platform (#7249) · 639159b2
  youkaichao authored Aug 07, 2024
  
  639159b2
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
- [BUGFIX]: top_k is expected to be an integer. (#7227) · 7b261092
  Atilla Akkuş authored Aug 07, 2024
  
  7b261092
- [Doc] Mock new dependencies for documentation (#7245) · 2385c8f3
  Roger Wang authored Aug 06, 2024
  
  2385c8f3
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
- [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225) · f9a56006
  Michael Goin authored Aug 06, 2024
  
  f9a56006
06 Aug, 2024 10 commits
- [Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026
  afeldman-nm authored Aug 06, 2024
```
[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  fd95e026
- [Core] Optimize evictor-v2 performance (#7193) · 660470e5
  xiaobochen123 authored Aug 07, 2024
  
  660470e5
- [Kernel] Add per-tensor and per-token AZP epilogues (#5941) · 8d59dbb0
  Luka Govedič authored Aug 06, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  8d59dbb0
- [SpecDecode] [Minor] Fix spec decode sampler tests (#7183) · 5c60c8c4
  Lily Liu authored Aug 06, 2024
  
  5c60c8c4
- [Bugfix] add gguf dependency (#7198) · 00afc785
  Katarzyna Papis authored Aug 06, 2024
```
Co-authored-by: katarzyna.papis <kpapis@kpapis-u20.sclab.intel.com>
```
  00afc785
- [ BugFix ] Fix ZMQ when `VLLM_PORT` is set (#7205) · 541c1852
  Robert Shaw authored Aug 06, 2024
  
  541c1852
- [BugFix] Fix DeepSeek remote code (#7178) · a3bbbfa1
  Dipika Sikka authored Aug 06, 2024
  
  a3bbbfa1
- [Model] Support SigLIP encoder and alternative decoders for LLaVA models (#7153) · 1f26efbb
  Cyrus Leung authored Aug 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  1f26efbb
- [LoRA] Relax LoRA condition (#7146) · 9118217f
  Jee Jee Li authored Aug 06, 2024
  
  9118217f
- [Build] Add initial conditional testing spec (#6841) · e3c664bf
  Simon Mo authored Aug 05, 2024
  
  e3c664bf
05 Aug, 2024 17 commits
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
- [MISC] Use non-blocking transfer in prepare_input (#7172) · ef527be0
  Cody Yu authored Aug 05, 2024
  
  ef527be0
- [Bugfix] Specify device when loading LoRA and embedding tensors (#7129) · 89b8db6b
  Jacob Schein authored Aug 05, 2024
```
Co-authored-by: Jacob Schein <jacobschein@Jacobs-MacBook-Pro-2.local>
```
  89b8db6b
- [Doc] [SpecDecode] Update MLPSpeculator documentation (#7100) · 789937af
  Thomas Parnell authored Aug 06, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  789937af
- [ci][frontend] deduplicate tests (#7101) · dfb1a15d
  youkaichao authored Aug 05, 2024
  
  dfb1a15d
- bump version to v0.5.4 (#7139) · 4db5176d
  Simon Mo authored Aug 05, 2024
  
  4db5176d
- [Bugfix][CI/Build] Fix CUTLASS FetchContent (#7171) · 4cf1dc39
  Tyler Michael Smith authored Aug 05, 2024
  
  4cf1dc39
- [CI/Build] Suppress divide-by-zero and missing return statement warnings (#7001) · 6e4852ce
  Tyler Michael Smith authored Aug 05, 2024
  
  6e4852ce
- [Kernel] Update CUTLASS to 3.5.1 (#7085) · 8571ac46
  Tyler Michael Smith authored Aug 05, 2024
  
  8571ac46
- [Misc] Fix typo in GroupCoordinator.recv() (#7167) · 997cf783
  Rui Qiao authored Aug 05, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  997cf783
- [BugFix] Use args.trust_remote_code (#7121) · 57f560aa
  Aditya Paliwal authored Aug 05, 2024
  
  57f560aa
- [BugFix] Use IP4 localhost form for zmq bind (#7163) · 003f8ee1
  Nick Hill authored Aug 05, 2024
  
  003f8ee1
- [SpecDecode] Support FlashInfer in DraftModelRunner (#6926) · e9630458
  Bongwon Jang authored Aug 06, 2024
  
  e9630458
- [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963) · 82a1b1a8
  Cade Daniel authored Aug 05, 2024
  
  82a1b1a8
- [Model] SiglipVisionModel ported from transformers (#6942) · c0d8f163
  Jungho Christopher Cho authored Aug 05, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  c0d8f163
- [Frontend] Reapply "Factor out code for running uvicorn" (#7095) · cc08fc72
  Cyrus Leung authored Aug 05, 2024
  
  cc08fc72
- [Model] Add multi-image support for minicpmv (#7122) · 7b86e7c9
  Alphi authored Aug 05, 2024
```
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  7b86e7c9
04 Aug, 2024 7 commits
- Clean up remaining Punica C information (#7027) · f80ab352
  Jee Jee Li authored Aug 05, 2024
  
  f80ab352
- [misc][distributed] improve libcudart.so finding (#7127) · 16a1cc9b
  youkaichao authored Aug 04, 2024
  
  16a1cc9b
- [Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when... · b1c9aa3d
  Thomas Parnell authored Aug 04, 2024
```
[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  b1c9aa3d
- [Model]Refactor MiniCPMV (#7020) · 179a6a36
  Jee Jee Li authored Aug 04, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  179a6a36
- [core][misc] simply output processing with shortcut code path (#7117) · 83c644fe
  youkaichao authored Aug 04, 2024
  
  83c644fe
- [misc] add zmq in collect env (#7119) · 9fadc7b7
  youkaichao authored Aug 03, 2024
  
  9fadc7b7
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca