Commits · 373538f973ac4ea93a4b675655b893bf495ac050 · OpenDAS / vllm_cscc

"lib/bindings/kvbm/src/block_manager/vllm.rs" did not exist on "a68c2f8f12c296529345804a7eb6d8d5e9bd8608"

13 Aug, 2024 1 commit
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
12 Aug, 2024 1 commit
- [Core] Consolidate `GB` constant and enable float GB arguments (#7416) · 4ddc4743
  Cyrus Leung authored Aug 13, 2024
  
  4ddc4743
11 Aug, 2024 2 commits
- [core] [2/N] refactor worker_base input preparation for multi-step (#7387) · c08e2b30
  William Lin authored Aug 11, 2024
  
  c08e2b30
- [TPU] Use mark_dynamic to reduce compilation time (#7340) · 90bab18f
  Woosuk Kwon authored Aug 10, 2024
  
  90bab18f
09 Aug, 2024 5 commits
- [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) · 933790c2
  Mahesh Keralapura authored Aug 09, 2024
  
  933790c2
- [Bugfix] Fix reinit procedure in ModelInputForGPUBuilder (#7360) · 74af2bbd
  Alexander Matveev authored Aug 09, 2024
  
  74af2bbd
- [Model][Jamba] Mamba cache single buffer (#6739) · 07ab1607
  Mor Zusman authored Aug 09, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  07ab1607
- [Performance] Optimize e2e overheads: Reduce python allocations (#7162) · e02ac556
  Alexander Matveev authored Aug 09, 2024
  
  e02ac556
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

05 Aug, 2024 1 commit
- [MISC] Use non-blocking transfer in prepare_input (#7172) · ef527be0
  Cody Yu authored Aug 05, 2024
  
  ef527be0
02 Aug, 2024 3 commits
- [misc] add a flag to enable compile (#7092) · 70898934
  youkaichao authored Aug 02, 2024
  
  70898934
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
- [ci] set timeout for test_oot_registration.py (#7082) · 80694951
  youkaichao authored Aug 02, 2024
  
  80694951
01 Aug, 2024 2 commits
- PP comm optimization: replace send with partial send + allgather (#6695) · 0437492e
  Aurick Qiao authored Jul 31, 2024
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  0437492e
- [Kernel][RFC] Refactor the punica kernel based on Triton (#5036) · 7ecee343
  Jee Jee Li authored Aug 01, 2024
  
  7ecee343
31 Jul, 2024 2 commits
- [Bugfix][TPU] Set readonly=True for non-root devices (#6980) · 533d1932
  Woosuk Kwon authored Jul 31, 2024
  
  533d1932
- [Bugfix] Fix broadcasting logic for `multi_modal_kwargs` (#6836) · f230cc2c
  Cyrus Leung authored Jul 31, 2024
  
  f230cc2c
30 Jul, 2024 2 commits
- [BugFix] Fix use of per-request seed with pipeline parallel (#6698) · 5cf9254a
  Nick Hill authored Jul 30, 2024
  
  5cf9254a
- [TPU] Fix greedy decoding (#6933) · 6e063ea3
  Woosuk Kwon authored Jul 30, 2024
  
  6e063ea3
27 Jul, 2024 2 commits
- [TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856) · fad5576c
  Woosuk Kwon authored Jul 27, 2024
  
  fad5576c
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
26 Jul, 2024 1 commit
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936
24 Jul, 2024 2 commits
- [Core] Tweaks to model runner/input builder developer APIs (#6712) · 5448f676
  Antoni Baum authored Jul 24, 2024
  
  5448f676
- [Bugfix] fix flashinfer cudagraph capture for PP (#6708) · 5e8ca973
  William Lin authored Jul 23, 2024
  
  5e8ca973
23 Jul, 2024 2 commits
- [misc] add start loading models for users information (#6670) · 7c2749a4
  youkaichao authored Jul 22, 2024
  
  7c2749a4
- [Core] Modulize prepare input and attention metadata builder (#6596) · e0c15758
  Cody Yu authored Jul 22, 2024
  
  e0c15758
22 Jul, 2024 2 commits
- [Core] Support dynamically loading Lora adapter from HuggingFace (#6234) · 42c7f66a
  Jiaxin Shan authored Jul 22, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  42c7f66a
- [Misc] Add a wrapper for torch.inference_mode (#6618) · 42de2cef
  Woosuk Kwon authored Jul 21, 2024
  
  42de2cef
20 Jul, 2024 2 commits
- [Misc] Consolidate and optimize logic for building padded tensors (#6541) · 9042d683
  Cyrus Leung authored Jul 20, 2024
  
  9042d683
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
19 Jul, 2024 1 commit
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
18 Jul, 2024 3 commits
- [TPU] Refactor TPU worker & model runner (#6506) · 4634c872
  Woosuk Kwon authored Jul 18, 2024
  
  4634c872
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
17 Jul, 2024 3 commits
- [Core] Refactor _prepare_model_input_tensors - take 2 (#6164) · 2fa4623d
  Cody Yu authored Jul 17, 2024
  
  2fa4623d
- [Misc] Use `torch.Tensor` for type annotation (#6505) · a9a2e74d
  Woosuk Kwon authored Jul 17, 2024
  
  a9a2e74d
- [TPU] Remove multi-modal args in TPU backend (#6504) · e09ce759
  Woosuk Kwon authored Jul 17, 2024
  
  e09ce759
16 Jul, 2024 2 commits
- [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) · c467dff2
  Woosuk Kwon authored Jul 16, 2024
  
  c467dff2
- [CI/Build] vLLM cache directory for images (#6444) · d9701151
  Cyrus Leung authored Jul 16, 2024
  
  d9701151