Commits · 8685ba1a1ec08d2c14df924b6e2b499be14405e7 · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "5f71c3f807819e255ffef71fba80d29f68810769"

05 Sep, 2024 3 commits
- Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) (#7860) · 8685ba1a
  manikandan.tm@zucisystems.com authored Sep 05, 2024
  
  8685ba1a
- [Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173) · e39ebf5c
  Elfie Guo authored Sep 04, 2024
  
  e39ebf5c
- [Misc] Clean up RoPE forward_native (#8076) · 4624d98d
  Woosuk Kwon authored Sep 04, 2024
  
  4624d98d
04 Sep, 2024 9 commits
- Bump version to v0.6.0 (#8166) · 32e7db25
  Simon Mo authored Sep 04, 2024
  
  32e7db25
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
- [Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649) · e02ce498
  Kyle Mistele authored Sep 04, 2024
```
Co-authored-by: constellate <constellate@1-ai-appserver-staging.codereach.com>
Co-authored-by: Kyle Mistele <kyle@constellate.ai>
```
  e02ce498
- [Bugfix] remove post_layernorm in siglip (#8106) · d3311562
  wnma authored Sep 04, 2024
  
  d3311562
- [Frontend] Multimodal support in offline chat (#8098) · 855c262a
  Cyrus Leung authored Sep 04, 2024
  
  855c262a
- [Model] Add Ultravox support for multiple audio chunks (#7963) · 2be8ec6e
  Peter Salas authored Sep 03, 2024
  
  2be8ec6e
- [Misc] Update fbgemmfp8 to use `vLLMParameters` (#7972) · e16fa99a
  Dipika Sikka authored Sep 03, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e16fa99a
- [TPU][Bugfix] Use XLA rank for persistent cache path (#8137) · 61f4a93d
  Woosuk Kwon authored Sep 03, 2024
  
  61f4a93d
- [Benchmark] Add `--async-engine` option to benchmark_throughput.py (#7964) · d4db9f53
  Nick Hill authored Sep 03, 2024
  
  d4db9f53
03 Sep, 2024 6 commits
- [Misc] Update `GPTQ` to use `vLLMParameters` (#7976) · 2188a60c
  Dipika Sikka authored Sep 03, 2024
  
  2188a60c
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
- [Misc] Raise a more informative exception in add/remove_logger (#7750) · 652c83b6
  Antoni Baum authored Sep 03, 2024
  
  652c83b6
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
- [Bugfix][VLM] Add fallback to SDPA for ViT model running on CPU backend (#8061) · ec266536
  Isotr0py authored Sep 03, 2024
  
  ec266536
- [Bugfix] Fix single output condition in output processor (#7881) · 0fbc6696
  Woosuk Kwon authored Sep 02, 2024
  
  0fbc6696
02 Sep, 2024 6 commits
- improve chunked prefill performance · 6e36f4fa
  wang.yuqi authored Sep 03, 2024
```
[Bugfix] Fix #7592 vllm 0.5.4 enable_chunked_prefill throughput is slightly lower than 0.5.3~0.5.0. (#7874)
```
  6e36f4fa
- [Bugfix] Fix internlm2 tensor parallel inference (#8055) · dd2a6a82
  Isotr0py authored Sep 02, 2024
  
  dd2a6a82
- [Core][Bugfix] Accept GGUF model without .gguf extension (#8056) · 4ca65a97
  Isotr0py authored Sep 02, 2024
  
  4ca65a97
- [TPU] Align worker index with node boundary (#7932) · e2b2aa5a
  Woosuk Kwon authored Sep 01, 2024
  
  e2b2aa5a
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
- [Model] Add Granite model (#7436) · f8d60145
  Shawn Tan authored Sep 01, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f8d60145
01 Sep, 2024 1 commit
- [Misc] Optional installation of audio related packages (#8063) · 5b86b199
  Roger Wang authored Sep 01, 2024
  
  5b86b199
31 Aug, 2024 5 commits
- [Frontend][VLM] Add support for multiple multi-modal items (#8049) · 5231f089
  Roger Wang authored Aug 31, 2024
  
  5231f089
- [BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059) · 8423aef4
  Robert Shaw authored Aug 31, 2024
  
  8423aef4
- [Bugfix] Fix ModelScope models in v0.5.5 (#8037) · 4f5d8446
  Nicolò Lucchesi authored Aug 31, 2024
  
  4f5d8446
- [Bugfix] Fix import error in Phi-3.5-MoE (#8052) · d05f0a9d
  Cyrus Leung authored Aug 31, 2024
  
  d05f0a9d
- [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) · 622f8abf
  Pavani Majety authored Aug 30, 2024
  
  622f8abf
30 Aug, 2024 10 commits
- [Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850
  Wenxiang authored Aug 31, 2024
```
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>
```
  1248e850
- [TPU][Bugfix] Fix tpu type api (#8035) · 2684efc4
  Woosuk Kwon authored Aug 30, 2024
  
  2684efc4
- [Frontend]-config-cli-args (#7737) · 058344f8
  Kaunil Dhruv authored Aug 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Kaunil Dhruv <kaunil_dhruv@intuit.com>
```
  058344f8
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [VLM][Model] TP support for ViTs (#7186) · f97be32d
  Jungho Christopher Cho authored Aug 31, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f97be32d
- [Bugfix] Fix import error in Exaone model (#8034) · afd39a45
  Cyrus Leung authored Aug 30, 2024
  
  afd39a45
- [TPU] Support single and multi-host TPUs on GKE (#7613) · 2148441f
  Richard Liu authored Aug 30, 2024
  
  2148441f
- [MODEL] add Exaone model support (#7819) · dc13e993
  Yohan Na authored Aug 30, 2024
  
  dc13e993
- [Kernel] changing fused moe kernel chunk size default to 32k (#7995) · 34a0e96d
  Avshalom Manevich authored Aug 30, 2024
  
  34a0e96d
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089