Commits · e16fa99a6ad5bae4aedfb76121d4e622d27f81c3 · OpenDAS / vllm_cscc · GitLab

04 Sep, 2024 3 commits
- [Misc] Update fbgemmfp8 to use `vLLMParameters` (#7972) · e16fa99a
  Dipika Sikka authored Sep 03, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e16fa99a
- [TPU][Bugfix] Use XLA rank for persistent cache path (#8137) · 61f4a93d
  Woosuk Kwon authored Sep 03, 2024
  
  61f4a93d
- [Benchmark] Add `--async-engine` option to benchmark_throughput.py (#7964) · d4db9f53
  Nick Hill authored Sep 03, 2024
  
  d4db9f53
03 Sep, 2024 11 commits
- [Misc] Update `GPTQ` to use `vLLMParameters` (#7976) · 2188a60c
  Dipika Sikka authored Sep 03, 2024
  
  2188a60c
- [CI] Change PR remainder to avoid at-mentions (#8134) · dc0b6066
  Simon Mo authored Sep 03, 2024
  
  dc0b6066
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
- [ci] Fix GHA workflow (#8129) · f1575dc9
  Kevin H. Luu authored Sep 03, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  f1575dc9
- [CI/Build] make pip install vllm work in macos (for import only) (#8118) · c02638ef
  tomeras91 authored Sep 03, 2024
  
  c02638ef
- [Misc] Raise a more informative exception in add/remove_logger (#7750) · 652c83b6
  Antoni Baum authored Sep 03, 2024
  
  652c83b6
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
- [CI] Only PR reviewers/committers can trigger CI on PR (#8124) · 95a178f8
  Kevin H. Luu authored Sep 03, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
```
  95a178f8
- [Performance] Enable chunked prefill and prefix caching together (#8120) · bd852f2a
  Cody Yu authored Sep 03, 2024
```
Co-authored-by: Tao He <sighingnow@gmail.com>
Co-authored-by: Juelianqvq <Juelianqvq@noreply.github.com>
```
  bd852f2a
- [Bugfix][VLM] Add fallback to SDPA for ViT model running on CPU backend (#8061) · ec266536
  Isotr0py authored Sep 03, 2024
  
  ec266536
- [Bugfix] Fix single output condition in output processor (#7881) · 0fbc6696
  Woosuk Kwon authored Sep 02, 2024
  
  0fbc6696
02 Sep, 2024 6 commits
- improve chunked prefill performance · 6e36f4fa
  wang.yuqi authored Sep 03, 2024
```
[Bugfix] Fix #7592 vllm 0.5.4 enable_chunked_prefill throughput is slightly lower than 0.5.3~0.5.0. (#7874)
```
  6e36f4fa
- [Bugfix] Fix internlm2 tensor parallel inference (#8055) · dd2a6a82
  Isotr0py authored Sep 02, 2024
  
  dd2a6a82
- [Core][Bugfix] Accept GGUF model without .gguf extension (#8056) · 4ca65a97
  Isotr0py authored Sep 02, 2024
  
  4ca65a97
- [TPU] Align worker index with node boundary (#7932) · e2b2aa5a
  Woosuk Kwon authored Sep 01, 2024
  
  e2b2aa5a
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
- [Model] Add Granite model (#7436) · f8d60145
  Shawn Tan authored Sep 01, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f8d60145
01 Sep, 2024 1 commit
- [Misc] Optional installation of audio related packages (#8063) · 5b86b199
  Roger Wang authored Sep 01, 2024
  
  5b86b199
31 Aug, 2024 5 commits
- [Frontend][VLM] Add support for multiple multi-modal items (#8049) · 5231f089
  Roger Wang authored Aug 31, 2024
  
  5231f089
- [BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059) · 8423aef4
  Robert Shaw authored Aug 31, 2024
  
  8423aef4
- [Bugfix] Fix ModelScope models in v0.5.5 (#8037) · 4f5d8446
  Nicolò Lucchesi authored Aug 31, 2024
  
  4f5d8446
- [Bugfix] Fix import error in Phi-3.5-MoE (#8052) · d05f0a9d
  Cyrus Leung authored Aug 31, 2024
  
  d05f0a9d
- [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) · 622f8abf
  Pavani Majety authored Aug 30, 2024
  
  622f8abf
30 Aug, 2024 12 commits
- [Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850
  Wenxiang authored Aug 31, 2024
```
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>
```
  1248e850
- [TPU][Bugfix] Fix tpu type api (#8035) · 2684efc4
  Woosuk Kwon authored Aug 30, 2024
  
  2684efc4
- [Frontend]-config-cli-args (#7737) · 058344f8
  Kaunil Dhruv authored Aug 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Kaunil Dhruv <kaunil_dhruv@intuit.com>
```
  058344f8
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [VLM][Model] TP support for ViTs (#7186) · f97be32d
  Jungho Christopher Cho authored Aug 31, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f97be32d
- [Bugfix] Fix import error in Exaone model (#8034) · afd39a45
  Cyrus Leung authored Aug 30, 2024
  
  afd39a45
- [TPU] Support single and multi-host TPUs on GKE (#7613) · 2148441f
  Richard Liu authored Aug 30, 2024
  
  2148441f
- [MODEL] add Exaone model support (#7819) · dc13e993
  Yohan Na authored Aug 30, 2024
  
  dc13e993
- [Kernel] changing fused moe kernel chunk size default to 32k (#7995) · 34a0e96d
  Avshalom Manevich authored Aug 30, 2024
  
  34a0e96d
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
- [VLM] Disallow overflowing `max_model_len` for multimodal models (#7998) · 4abed65c
  Cyrus Leung authored Aug 30, 2024
  
  4abed65c
29 Aug, 2024 2 commits
- Add more percentiles and latencies (#7759) · 0c785d34
  Wei-Sheng Chin authored Aug 29, 2024
  
  0c785d34
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa