Commits · 0640f227c9a5fb1e7193792eb36ff48d152af099 · OpenDAS / vllm_cscc · GitLab

"docs/pages/fault-tolerance/request-rejection.md" did not exist on "7ca6a562f4d5b5926b1d1299425e90033cb725c9"

09 Sep, 2024 1 commit
- add cutlass fa and support bloom nn layout · 82f1ffdf
  zhuwenwen authored Sep 09, 2024
  
  82f1ffdf
02 Sep, 2024 1 commit
- [SpecDecode][Kernel] Flashinfer Rejection Sampling (#7244) · e6a26ed0
  Lily Liu authored Sep 01, 2024
  
  e6a26ed0
30 Aug, 2024 1 commit
- [Kernel] changing fused moe kernel chunk size default to 32k (#7995) · 34a0e96d
  Avshalom Manevich authored Aug 30, 2024
  
  34a0e96d
28 Aug, 2024 2 commits
- [torch.compile] avoid Dynamo guard evaluation overhead (#7898) · ce6bf3a2
  youkaichao authored Aug 28, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ce6bf3a2
- [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and... · e5697d16
  rasmith authored Aug 28, 2024
```
[Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386)
```
  e5697d16
22 Aug, 2024 1 commit
- [Bug][Frontend] Improve ZMQ client robustness (#7443) · cde9183b
  Joe Runde authored Aug 21, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cde9183b
21 Aug, 2024 4 commits
- [distributed][misc] error on same VLLM_HOST_IP setting (#7756) · 7eebe8cc
  youkaichao authored Aug 21, 2024
  
  7eebe8cc
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
- Add VLLM_USE_PA_PRINT_PARAM flag to print pa size · 2f9e0bad
  zhuwenwen authored Aug 21, 2024
  
  2f9e0bad
19 Aug, 2024 2 commits
- [Core] Use flashinfer sampling kernel when available (#7137) · f710fb52
  Peng Guanwen authored Aug 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  f710fb52
- Add fa pad conditions and automatic switching strategy · b40f2ffc
  zhuwenwen authored Aug 19, 2024
  
  b40f2ffc
15 Aug, 2024 1 commit
- [Bugfix][TPU] Correct env variable for XLA cache path (#7544) · fc93e561
  Woosuk Kwon authored Aug 15, 2024
  
  fc93e561
14 Aug, 2024 1 commit

[Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef

Wallas Henrique authored Aug 14, 2024


Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>

70b746ef

13 Aug, 2024 2 commits
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
10 Aug, 2024 1 commit
- add benchmarks to vllm and add env of VLLM_USE_FLASH_ATTN_AUTO · efb2f75f
  zhuwenwen authored Aug 10, 2024
  
  efb2f75f
08 Aug, 2024 2 commits
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
- The timeout for each iteration in the engine is set to 120 seconds by default · 3d087876
  zhuwenwen authored Aug 08, 2024
  
  3d087876
07 Aug, 2024 2 commits
- [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) · 5223199e
  Michael Goin authored Aug 07, 2024
  
  5223199e
- [ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222) · 56498572
  Robert Shaw authored Aug 07, 2024
  
  56498572
06 Aug, 2024 1 commit
- [ BugFix ] Fix ZMQ when `VLLM_PORT` is set (#7205) · 541c1852
  Robert Shaw authored Aug 06, 2024
  
  541c1852
03 Aug, 2024 2 commits

[Frontend] Warn if user `max_model_len` is greater than derived `max_model_len` (#7080) · 825b0448
Jeff Fialho authored Aug 03, 2024
```
Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
825b0448

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

02 Aug, 2024 2 commits
- [misc] add a flag to enable compile (#7092) · 70898934
  youkaichao authored Aug 02, 2024
  
  70898934
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
01 Aug, 2024 1 commit
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343
31 Jul, 2024 1 commit
- [MISC] Introduce pipeline parallelism partition strategies (#6920) · bd700134
  Cody Yu authored Jul 31, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  bd700134
26 Jul, 2024 1 commit
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936
18 Jul, 2024 2 commits
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
- default use of ck fa · e661266e
  zhuwenwen authored Jul 18, 2024
  
  e661266e
16 Jul, 2024 1 commit
- [CI/Build] vLLM cache directory for images (#6444) · d9701151
  Cyrus Leung authored Jul 16, 2024
  
  d9701151
15 Jul, 2024 1 commit
- [Doc] add env docs for flashinfer backend (#6437) · 44874a0b
  DefTruth authored Jul 15, 2024
  
  44874a0b
13 Jul, 2024 1 commit
- [Misc] Add deprecation warning for beam search (#6402) · eeceadae
  Woosuk Kwon authored Jul 13, 2024
  
  eeceadae
06 Jul, 2024 1 commit
- Use triton fa by default · 1863c926
  zhuwenwen authored Jul 06, 2024
  
  1863c926
01 Jul, 2024 1 commit
- [Bugfix] adding chunking mechanism to fused_moe to handle large inputs (#6029) · 12a59959
  Avshalom Manevich authored Jul 02, 2024
  
  12a59959
28 Jun, 2024 1 commit
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
21 Jun, 2024 2 commits
- [Core][Distributed] add shm broadcast (#5399) · d9a252bc
  youkaichao authored Jun 20, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  d9a252bc
- [distributed][misc] use fork by default for mp (#5669) · 6c5b7af1
  youkaichao authored Jun 20, 2024
  
  6c5b7af1
20 Jun, 2024 1 commit
- update triton fa · e2df3544
  zhuwenwen authored Jun 20, 2024
  
  e2df3544