Commits · df1a21131d951ba8ee65363aeb9b9486f569aa4f · OpenDAS / vllm_cscc

21 Aug, 2024 2 commits
- [distributed][misc] error on same VLLM_HOST_IP setting (#7756) · 7eebe8cc
  youkaichao authored Aug 21, 2024
  
  7eebe8cc
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
19 Aug, 2024 1 commit
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
15 Aug, 2024 1 commit
- [Bugfix][TPU] Correct env variable for XLA cache path (#7544) · fc93e561
  Woosuk Kwon authored Aug 15, 2024
  
  fc93e561
14 Aug, 2024 1 commit

[Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef

Wallas Henrique authored Aug 14, 2024


Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>

70b746ef

13 Aug, 2024 2 commits
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 2 commits
- [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) · 5223199e
  Michael Goin authored Aug 07, 2024
  
  5223199e
- [ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222) · 56498572
  Robert Shaw authored Aug 07, 2024
  
  56498572
06 Aug, 2024 1 commit
- [ BugFix ] Fix ZMQ when `VLLM_PORT` is set (#7205) · 541c1852
  Robert Shaw authored Aug 06, 2024
  
  541c1852
03 Aug, 2024 2 commits

[Frontend] Warn if user `max_model_len` is greater than derived `max_model_len` (#7080) · 825b0448
Jeff Fialho authored Aug 03, 2024
```
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
825b0448

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

02 Aug, 2024 2 commits
- [misc] add a flag to enable compile (#7092) · 70898934
  youkaichao authored Aug 02, 2024
  
  70898934
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
01 Aug, 2024 1 commit
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343
31 Jul, 2024 1 commit
- [MISC] Introduce pipeline parallelism partition strategies (#6920) · bd700134
  Cody Yu authored Jul 31, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  bd700134
26 Jul, 2024 1 commit
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936
18 Jul, 2024 1 commit
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
16 Jul, 2024 1 commit
- [CI/Build] vLLM cache directory for images (#6444) · d9701151
  Cyrus Leung authored Jul 16, 2024
  
  d9701151
15 Jul, 2024 1 commit
- [Doc] add env docs for flashinfer backend (#6437) · 44874a0b
  DefTruth authored Jul 15, 2024
  
  44874a0b
13 Jul, 2024 1 commit
- [Misc] Add deprecation warning for beam search (#6402) · eeceadae
  Woosuk Kwon authored Jul 13, 2024
  
  eeceadae
01 Jul, 2024 1 commit
- [Bugfix] adding chunking mechanism to fused_moe to handle large inputs (#6029) · 12a59959
  Avshalom Manevich authored Jul 02, 2024
  
  12a59959
28 Jun, 2024 1 commit
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
21 Jun, 2024 2 commits
- [Core][Distributed] add shm broadcast (#5399) · d9a252bc
  youkaichao authored Jun 20, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  d9a252bc
- [distributed][misc] use fork by default for mp (#5669) · 6c5b7af1
  youkaichao authored Jun 20, 2024
  
  6c5b7af1
12 Jun, 2024 1 commit
- [Hardware] Initial TPU integration (#5292) · 1a8bfd92
  Woosuk Kwon authored Jun 12, 2024
  
  1a8bfd92
11 Jun, 2024 1 commit
- [Misc] Remove VLLM_BUILD_WITH_NEURON env variable (#5389) · 8bab4959
  Woosuk Kwon authored Jun 11, 2024
  
  8bab4959
07 Jun, 2024 2 commits
- [Frontend] Add OpenAI Vision API Support (#5237) · 7a9cb294
  Roger Wang authored Jun 07, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7a9cb294
- [Misc][Utils] allow get_open_port to be called for multiple times (#5333) · 388596c9
  youkaichao authored Jun 06, 2024
  
  388596c9
25 May, 2024 1 commit
- [Misc] add logging level env var (#5045) · 325c1199
  youkaichao authored May 24, 2024
  
  325c1199
20 May, 2024 1 commit
- [Misc]: allow user to specify port in distributed setting (#4914) · 546a97ef
  Wenwei Zhang authored May 21, 2024
  
  546a97ef
13 May, 2024 1 commit
- [Frontend] [Core] perf: Automatically detect vLLM-tensorized model, update... · 8bc68e19
  Sanger Steel authored May 13, 2024
```
[Frontend] [Core] perf: Automatically detect vLLM-tensorized model, update `tensorizer` to version 2.9.0 (#4208)
```
  8bc68e19
03 May, 2024 2 commits
- [Misc] add installation time env vars (#4574) · 344bf7cd
  youkaichao authored May 03, 2024
  
  344bf7cd
- [Doc] add env vars to the doc (#4572) · 2d7bce9c
  youkaichao authored May 02, 2024
  
  2d7bce9c
02 May, 2024 1 commit
- [Misc] centralize all usage of environment variables (#4548) · 5b8a7c1c
  youkaichao authored May 02, 2024
  
  5b8a7c1c