Commits · 473e7b3606e9b95b39c7da46cce00a33c069dc00 · OpenDAS / vllm_cscc

14 Oct, 2024 2 commits
- [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (#9350) · 473e7b36
  Woosuk Kwon authored Oct 14, 2024
  
  473e7b36
- [Hardware][intel GPU] add async output process for xpu (#8897) · 4141608c
  Kunshang Ji authored Oct 15, 2024
  
  4141608c
13 Oct, 2024 1 commit
- [Bugfix] Bandaid fix for speculative decoding tests (#9327) · 16b24e7d
  Tyler Michael Smith authored Oct 13, 2024
  
  16b24e7d
11 Oct, 2024 4 commits
- [Bugfix] Sets `is_first_step_output` for TPUModelRunner (#9202) · c6cf9295
  Allen Wang authored Oct 11, 2024
  
  c6cf9295
- [Doc] Compatibility matrix for mutual exclusive features (#8512) · 8baf85e4
  Wallas Henrique authored Oct 11, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  8baf85e4
- [Model] Support Mamba (#6484) · 7342a7d7
  Tyler Michael Smith authored Oct 11, 2024
  
  7342a7d7
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
10 Oct, 2024 1 commit
- [torch.compile] integration with compilation control (#9058) · e4d652ea
  youkaichao authored Oct 10, 2024
  
  e4d652ea
09 Oct, 2024 1 commit
- [Hardware][CPU] Support AWQ for CPU backend (#7515) · ca77dd7a
  Li, Jiang authored Oct 10, 2024
  
  ca77dd7a
08 Oct, 2024 2 commits
- [Core][Frontend] Add Support for Inference Time mm_processor_kwargs (#9131) · a3691b6b
  Alex Brooks authored Oct 08, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  a3691b6b
- [Intel GPU] Fix xpu decode input (#9145) · 80b57f00
  Kunshang Ji authored Oct 08, 2024
  
  80b57f00
07 Oct, 2024 3 commits
- [Hardware][CPU] Cross-attention and Encoder-Decoder models support on CPU backend (#9089) · 4f95ffee
  Isotr0py authored Oct 07, 2024
  
  4f95ffee
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
- [Bugfix][Hardware][CPU] Fix CPU model input for decode (#9044) · 487678d0
  Isotr0py authored Oct 07, 2024
  
  487678d0
06 Oct, 2024 1 commit
- [Model] PP support for embedding models and update docs (#9090) · b22b7984
  Cyrus Leung authored Oct 06, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  b22b7984
04 Oct, 2024 2 commits
- [Hardware][Neuron] Add on-device sampling support for Neuron (#8746) · cc90419e
  Chongming Ni authored Oct 04, 2024
```
Co-authored-by: Ashraf Mahgoub <ashymahg@amazon.com>
```
  cc90419e
- [Misc] Move registry to its own file (#9064) · 0e36fd49
  Cyrus Leung authored Oct 04, 2024
  
  0e36fd49
03 Oct, 2024 1 commit
- [misc] add forward context for attention (#9029) · 9aaf14c6
  youkaichao authored Oct 03, 2024
  
  9aaf14c6
02 Oct, 2024 2 commits
- [OpenVINO] Enable GPU support for OpenVINO vLLM backend (#8192) · f58d4fcc
  Sergey Shlyapnikov authored Oct 03, 2024
  
  f58d4fcc
- [Core] CUDA Graphs for Multi-Step + Chunked-Prefill (#8645) · afb050b2
  Varun Sundar Rabindranath authored Oct 02, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  afb050b2
01 Oct, 2024 2 commits
- [Spec Decode] (1/2) Remove batch expansion (#8839) · 15702038
  Lily Liu authored Oct 01, 2024
  
  15702038
- [torch.compile] fix tensor alias (#8982) · 7da24875
  youkaichao authored Sep 30, 2024
  
  7da24875
30 Sep, 2024 1 commit
- [Misc] Adjust max_position_embeddings for LoRA compatibility (#8957) · 1cabfcef
  Jee Jee Li authored Sep 30, 2024
  
  1cabfcef
29 Sep, 2024 2 commits
- [BugFix] Fix seeded random sampling with encoder-decoder models (#8870) · 31f46a0d
  Nick Hill authored Sep 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  31f46a0d
- [Model][LoRA]LoRA support added for MiniCPMV2.5 (#7199) · 3d49776b
  Jee Jee Li authored Sep 29, 2024
  
  3d49776b
28 Sep, 2024 1 commit
- [Bugfix] Fix PP for Multi-Step (#8887) · 19d02ff9
  Varun Sundar Rabindranath authored Sep 28, 2024
  
  19d02ff9
27 Sep, 2024 2 commits
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
25 Sep, 2024 2 commits

[Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602

Chen Zhang authored Sep 25, 2024


Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

770ec602

[Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) · c2395367
Isotr0py authored Sep 25, 2024

c2395367

24 Sep, 2024 1 commit
- [MISC] Skip dumping inputs when unpicklable (#8744) · b8747e8a
  Cody Yu authored Sep 23, 2024
  
  b8747e8a
23 Sep, 2024 2 commits
- [Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (#8733) · 3e83c12b
  Li, Jiang authored Sep 23, 2024
  
  3e83c12b
- [Hardware][CPU] Refactor CPU model runner (#8729) · e551ca15
  Isotr0py authored Sep 23, 2024
  
  e551ca15
22 Sep, 2024 1 commit
- [MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (#8703) · ca2b628b
  Huazhong Ji authored Sep 23, 2024
  
  ca2b628b
20 Sep, 2024 1 commit
- [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) · 9e5ec35b
  William Lin authored Sep 19, 2024
  
  9e5ec35b
19 Sep, 2024 1 commit

[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction... · 3118f633

sroy745 authored Sep 18, 2024

[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models.  (#8545)

3118f633

18 Sep, 2024 2 commits
- [Core] *Prompt* logprobs support in Multi-step (#8199) · a8c1d161
  afeldman-nm authored Sep 18, 2024
  
  a8c1d161
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 2 commits
- [Misc] Don't dump contents of kvcache tensors on errors (#8527) · 56c3de01
  Nick Hill authored Sep 17, 2024
  
  56c3de01
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c