Commits · 3f674a49b5033a6ed778ab960e86e03cfa64aa1f · OpenDAS / vllm_cscc · GitLab

14 Aug, 2024 10 commits
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
- [Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef
  Wallas Henrique authored Aug 14, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  70b746ef
- [Bugfix][Frontend] Disable embedding API for chat models (#7504) · 67d115db
  jack authored Aug 15, 2024
```
Co-authored-by: jack <jack@alex>
```
  67d115db
- [ci] fix model tests (#7507) · d3d9cb6e
  youkaichao authored Aug 14, 2024
  
  d3d9cb6e
- Fix empty output when temp is too low (#2937) · c134a464
  Chang Su authored Aug 13, 2024
```
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c134a464
- [doc] update test script to include cudagraph (#7501) · 199adbb7
  youkaichao authored Aug 13, 2024
  
  199adbb7
- [Bugfix][Docs] Update list of mock imports (#7493) · dd164d72
  Cyrus Leung authored Aug 14, 2024
  
  dd164d72
- [misc][ci] fix cpu test with plugins (#7489) · ea49e6a3
  youkaichao authored Aug 13, 2024
  
  ea49e6a3
- [CI/Build]Reduce the time consumption for LoRA tests (#7396) · 97992802
  Jee Jee Li authored Aug 14, 2024
  
  97992802
- [Bugfix][CI] Import ray under guard (#7486) · 59edd0f1
  Woosuk Kwon authored Aug 13, 2024
  
  59edd0f1
13 Aug, 2024 18 commits
- [TPU] Support multi-host inference (#7457) · a08df832
  Woosuk Kwon authored Aug 13, 2024
  
  a08df832
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Misc] `compressed-tensors` code reuse (#7277) · 373538f9
  Kyle Sayers authored Aug 13, 2024
  
  373538f9
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- Announce NVIDIA Meetup (#7483) · c5c77682
  Simon Mo authored Aug 13, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c5c77682
- [Misc] Update `awq` and `awq_marlin` to use `vLLMParameters` (#7422) · b1e5afc3
  Dipika Sikka authored Aug 13, 2024
  
  b1e5afc3
- [Misc] Update Fused MoE weight loading (#7334) · d3bdfd3a
  Dipika Sikka authored Aug 13, 2024
  
  d3bdfd3a
- [Misc] Update `gptq_marlin` to use new vLLMParameters (#7281) · fb377d7e
  Dipika Sikka authored Aug 13, 2024
  
  fb377d7e
- [Misc] Update LM Eval Tolerance (#7473) · 181abbc2
  Dipika Sikka authored Aug 13, 2024
  
  181abbc2
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- Revert "[Doc] Update supported_hardware.rst (#7276)" (#7467) · e20233d3
  Woosuk Kwon authored Aug 13, 2024
  
  e20233d3
- [TPU] Suppress import custom_ops warning (#7458) · d6e634f3
  Woosuk Kwon authored Aug 13, 2024
  
  d6e634f3
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
- [ci] Remove fast check cancel workflow (#7455) · 5469146b
  Kevin H. Luu authored Aug 12, 2024
  
  5469146b
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
- [mypy] Misc. typing improvements (#7417) · 9ba85bc1
  Cyrus Leung authored Aug 13, 2024
  
  9ba85bc1
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
12 Aug, 2024 12 commits
- [CI/Build] bump minimum cmake version (#6999) · 774cd1d3
  Daniele authored Aug 13, 2024
  
  774cd1d3
- [Bugfix] Handle PackageNotFoundError when checking for xpu version (#7398) · 91294d56
  sasha0552 authored Aug 12, 2024
  
  91294d56
- [Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefill Triton Kernel (#7208) · a046f863
  jon-chuang authored Aug 12, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  a046f863
- [Core] Consolidate `GB` constant and enable float GB arguments (#7416) · 4ddc4743
  Cyrus Leung authored Aug 13, 2024
  
  4ddc4743
- [Misc] Use scalar type to dispatch to different `gptq_marlin` kernels (#7323) · 6aa33cb2
  Lucas Wilkinson authored Aug 12, 2024
  
  6aa33cb2
- [ci] Cancel fastcheck when PR is ready (#7433) · 1137f343
  Kevin H. Luu authored Aug 12, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  1137f343
- [ci] Cancel fastcheck run when PR is marked ready (#7427) · 9b3e2edd
  Kevin H. Luu authored Aug 12, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  9b3e2edd
- [ci] Entrypoints run upon changes in vllm/ (#7423) · 65950e8f
  Kevin H. Luu authored Aug 12, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  65950e8f
- [Bugfix] Fix logit soft cap in flash-attn backend (#7425) · cfba4def
  Woosuk Kwon authored Aug 12, 2024
  
  cfba4def
- [CI/Build] bump Dockerfile.neuron image base, use public ECR (#6832) · d2bc4510
  Daniele authored Aug 12, 2024
  
  d2bc4510
- [Frontend] Disallow passing `model` as both argument and option (#7347) · 24154f86
  Cyrus Leung authored Aug 12, 2024
  
  24154f86
- [Core][VLM] Support image embeddings as input (#6613) · e6e42e4b
  Roger Wang authored Aug 12, 2024
  
  e6e42e4b