Commits · f366f6339b1dbd9176470489b99dae4fb454f5ce · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "eab2f3980cd30132dd99e23ea8d6fc0274365757"

16 Aug, 2024 12 commits
- [spec decode] [4/N] Move update_flash_attn_metadata to attn backend (#7571) · f366f633
  William Lin authored Aug 16, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  f366f633
- [Kernel] Add tuned triton configs for ExpertsInt8 (#7601) · 855866ca
  Michael Goin authored Aug 16, 2024
  
  855866ca
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
- support tqdm in notebooks (#7510) · ec724a72
  fzyzcjy authored Aug 17, 2024
  
  ec724a72
- [Bugfix][Hardware][AMD][Frontend] add quantization param to embedding checking method (#7513) · 0e39a33c
  Gordon Wong authored Aug 17, 2024
  
  0e39a33c
- [Core] Use uvloop with zmq-decoupled front-end (#7570) · 9587b050
  Nick Hill authored Aug 15, 2024
  
  9587b050
- register custom op for flash attn and use from torch.ops (#7536) · 54bd9a03
  youkaichao authored Aug 15, 2024
  
  54bd9a03
- Chat method for offline llm (#5049) · 3b19e39d
  nunjunj authored Aug 16, 2024
```
Co-authored-by: nunjunj <ray@g-3ff9f30f2ed650001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-1df6075697c3f0001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-c5a2c23abc49e0001.c.vllm-405802.internal>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b19e39d
- [Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) · f878c8fe
  Grant Pinkert authored Aug 16, 2024
  
  f878c8fe
- [Misc] Add quantization config support for speculative model. (#7343) · b67ae00c
  shangmingc authored Aug 16, 2024
  
  b67ae00c
- [Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566) · 9c8e2d11
  Michael Goin authored Aug 15, 2024
  
  9c8e2d11
15 Aug, 2024 3 commits
- [Bugfix] Fix default weight loading for scalars (#7534) · 21313e09
  Michael Goin authored Aug 15, 2024
  
  21313e09
- [Bugfix] update neuron for version > 0.5.0 (#7175) · 9c1f78d5
  omrishiv authored Aug 15, 2024
```
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9c1f78d5
- [Bugfix][TPU] Correct env variable for XLA cache path (#7544) · fc93e561
  Woosuk Kwon authored Aug 15, 2024
  
  fc93e561
14 Aug, 2024 8 commits
- [Misc] Revert `compressed-tensors` code reuse (#7521) · f55a9aea
  Kyle Sayers authored Aug 14, 2024
  
  f55a9aea
- [TPU] Set per-rank XLA cache (#7533) · 951fdd66
  Woosuk Kwon authored Aug 14, 2024
  
  951fdd66
- [core] [3/N] multi-step args and sequence.py (#7452) · 2ecf7b17
  William Lin authored Aug 14, 2024
  
  2ecf7b17
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
- [Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef
  Wallas Henrique authored Aug 14, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  70b746ef
- [Bugfix][Frontend] Disable embedding API for chat models (#7504) · 67d115db
  jack authored Aug 15, 2024
```
Co-authored-by: jack <jack@alex>
```
  67d115db
- Fix empty output when temp is too low (#2937) · c134a464
  Chang Su authored Aug 13, 2024
```
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c134a464
- [Bugfix][CI] Import ray under guard (#7486) · 59edd0f1
  Woosuk Kwon authored Aug 13, 2024
  
  59edd0f1
13 Aug, 2024 14 commits
- [TPU] Support multi-host inference (#7457) · a08df832
  Woosuk Kwon authored Aug 13, 2024
  
  a08df832
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Misc] `compressed-tensors` code reuse (#7277) · 373538f9
  Kyle Sayers authored Aug 13, 2024
  
  373538f9
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- [Misc] Update `awq` and `awq_marlin` to use `vLLMParameters` (#7422) · b1e5afc3
  Dipika Sikka authored Aug 13, 2024
  
  b1e5afc3
- [Misc] Update Fused MoE weight loading (#7334) · d3bdfd3a
  Dipika Sikka authored Aug 13, 2024
  
  d3bdfd3a
- [Misc] Update `gptq_marlin` to use new vLLMParameters (#7281) · fb377d7e
  Dipika Sikka authored Aug 13, 2024
  
  fb377d7e
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [TPU] Suppress import custom_ops warning (#7458) · d6e634f3
  Woosuk Kwon authored Aug 13, 2024
  
  d6e634f3
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
- [mypy] Misc. typing improvements (#7417) · 9ba85bc1
  Cyrus Leung authored Aug 13, 2024
  
  9ba85bc1
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
12 Aug, 2024 3 commits
- [Bugfix] Handle PackageNotFoundError when checking for xpu version (#7398) · 91294d56
  sasha0552 authored Aug 12, 2024
  
  91294d56
- [Core/Bugfix] Add FP8 K/V Scale and dtype conversion for prefix/prefill Triton Kernel (#7208) · a046f863
  jon-chuang authored Aug 12, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  a046f863
- [Core] Consolidate `GB` constant and enable float GB arguments (#7416) · 4ddc4743
  Cyrus Leung authored Aug 13, 2024
  
  4ddc4743