Commits · 6e36f4fa6ce64619b9ea94c88a157f5783a63a65 · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "d29483b58a036526307669c5d98bbb35a9e45b98"

02 Sep, 2024 6 commits
- improve chunked prefill performance · 6e36f4fa
  wang.yuqi authored Sep 03, 2024
```
[Bugfix] Fix #7592 vllm 0.5.4 enable_chunked_prefill throughput is slightly lower than 0.5.3~0.5.0. (#7874)
```
  6e36f4fa
- [Bugfix] Fix internlm2 tensor parallel inference (#8055) · dd2a6a82
  Isotr0py authored Sep 02, 2024
  
  dd2a6a82
- [Core][Bugfix] Accept GGUF model without .gguf extension (#8056) · 4ca65a97
  Isotr0py authored Sep 02, 2024
  
  4ca65a97
- [TPU] Align worker index with node boundary (#7932) · e2b2aa5a
  Woosuk Kwon authored Sep 01, 2024
  
  e2b2aa5a
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
- [Model] Add Granite model (#7436) · f8d60145
  Shawn Tan authored Sep 01, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f8d60145
01 Sep, 2024 1 commit
- [Misc] Optional installation of audio related packages (#8063) · 5b86b199
  Roger Wang authored Sep 01, 2024
  
  5b86b199
31 Aug, 2024 5 commits
- [Frontend][VLM] Add support for multiple multi-modal items (#8049) · 5231f089
  Roger Wang authored Aug 31, 2024
  
  5231f089
- [BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059) · 8423aef4
  Robert Shaw authored Aug 31, 2024
  
  8423aef4
- [Bugfix] Fix ModelScope models in v0.5.5 (#8037) · 4f5d8446
  Nicolò Lucchesi authored Aug 31, 2024
  
  4f5d8446
- [Bugfix] Fix import error in Phi-3.5-MoE (#8052) · d05f0a9d
  Cyrus Leung authored Aug 31, 2024
  
  d05f0a9d
- [Bugfix] bugfix and add model test for flashinfer fp8 kv cache. (#8013) · 622f8abf
  Pavani Majety authored Aug 30, 2024
  
  622f8abf
30 Aug, 2024 12 commits
- [Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850
  Wenxiang authored Aug 31, 2024
```
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>
```
  1248e850
- [TPU][Bugfix] Fix tpu type api (#8035) · 2684efc4
  Woosuk Kwon authored Aug 30, 2024
  
  2684efc4
- [Frontend]-config-cli-args (#7737) · 058344f8
  Kaunil Dhruv authored Aug 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Kaunil Dhruv <kaunil_dhruv@intuit.com>
```
  058344f8
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [VLM][Model] TP support for ViTs (#7186) · f97be32d
  Jungho Christopher Cho authored Aug 31, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f97be32d
- [Bugfix] Fix import error in Exaone model (#8034) · afd39a45
  Cyrus Leung authored Aug 30, 2024
  
  afd39a45
- [TPU] Support single and multi-host TPUs on GKE (#7613) · 2148441f
  Richard Liu authored Aug 30, 2024
  
  2148441f
- [MODEL] add Exaone model support (#7819) · dc13e993
  Yohan Na authored Aug 30, 2024
  
  dc13e993
- [Kernel] changing fused moe kernel chunk size default to 32k (#7995) · 34a0e96d
  Avshalom Manevich authored Aug 30, 2024
  
  34a0e96d
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
- [VLM] Disallow overflowing `max_model_len` for multimodal models (#7998) · 4abed65c
  Cyrus Leung authored Aug 30, 2024
  
  4abed65c
29 Aug, 2024 13 commits
- Add more percentiles and latencies (#7759) · 0c785d34
  Wei-Sheng Chin authored Aug 29, 2024
  
  0c785d34
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa
- [Neuron] Adding support for context-lenght, token-gen buckets. (#7885) · 257afc37
  Harsha vardhan manoj Bikki authored Aug 29, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  257afc37
- [misc] update tpu int8 to use new vLLM Parameters (#7973) · 86a677de
  Dipika Sikka authored Aug 29, 2024
  
  86a677de
- [Bugfix] Fix incorrect vocal embedding shards for GGUF model in tensor parallelism (#7954) · d78789ac
  Isotr0py authored Aug 30, 2024
  
  d78789ac
- extend cuda graph size for H200 (#7894) · c334b189
  kushanam authored Aug 29, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  c334b189
- [Core][Kernels] Enable FP8 KV Cache with Flashinfer backend. + BugFix for... · 6b342156
  Pavani Majety authored Aug 29, 2024
```
[Core][Kernels] Enable FP8 KV Cache with Flashinfer backend.  + BugFix for kv_cache_dtype=auto (#7985)
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  6b342156
- [Core] Combine async postprocessor and multi-step (#7921) · 3f60f224
  Alexander Matveev authored Aug 29, 2024
  
  3f60f224
- [Bugfix] Unify rank computation across regular decoding and speculative decoding (#7899) · f205c098
  Jonas M. Kübler authored Aug 29, 2024
  
  f205c098
- Revert "[Core][Kernels] Use FlashInfer backend for FP8 KV Cache when available." (#7982) · ef99a787
  youkaichao authored Aug 28, 2024
  
  ef99a787
- [VLM][Core] Fix exceptions on ragged NestedTensors (#7974) · 74d5543e
  Peter Salas authored Aug 28, 2024
  
  74d5543e
- [torch.compile] remove reset (#7975) · a7f65c2b
  youkaichao authored Aug 28, 2024
  
  a7f65c2b
- [Frontend] Minor optimizations to zmq decoupled front-end (#7957) · 4289cad3
  Nick Hill authored Aug 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  4289cad3
28 Aug, 2024 3 commits
- Remove faulty Meta-Llama-3-8B-Instruct-FP8.yaml lm-eval test (#7961) · af59df0a
  Michael Goin authored Aug 28, 2024
  
  af59df0a
- [torch.compile] avoid Dynamo guard evaluation overhead (#7898) · ce6bf3a2
  youkaichao authored Aug 28, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ce6bf3a2
- [Bugfix] Make torch registration of punica ops optional (#7970) · 3cdfe1f3
  bnellnm authored Aug 28, 2024
  
  3cdfe1f3