Commits · 39d3f8d94fd2691b70ee809e7565402f8a061c6b · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "757eafcf37ba1db654bf9abb96e39054af0dc69d"

27 Sep, 2024 2 commits
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
25 Sep, 2024 2 commits

[Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602

Chen Zhang authored Sep 25, 2024


Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

770ec602

[Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) · c2395367
Isotr0py authored Sep 25, 2024

c2395367

24 Sep, 2024 1 commit
- [MISC] Skip dumping inputs when unpicklable (#8744) · b8747e8a
  Cody Yu authored Sep 23, 2024
  
  b8747e8a
23 Sep, 2024 2 commits
- [Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (#8733) · 3e83c12b
  Li, Jiang authored Sep 23, 2024
  
  3e83c12b
- [Hardware][CPU] Refactor CPU model runner (#8729) · e551ca15
  Isotr0py authored Sep 23, 2024
  
  e551ca15
22 Sep, 2024 1 commit
- [MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (#8703) · ca2b628b
  Huazhong Ji authored Sep 23, 2024
  
  ca2b628b
20 Sep, 2024 1 commit
- [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) · 9e5ec35b
  William Lin authored Sep 19, 2024
  
  9e5ec35b
19 Sep, 2024 1 commit

[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction... · 3118f633

sroy745 authored Sep 18, 2024

[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models.  (#8545)

3118f633

18 Sep, 2024 2 commits
- [Core] *Prompt* logprobs support in Multi-step (#8199) · a8c1d161
  afeldman-nm authored Sep 18, 2024
  
  a8c1d161
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 2 commits
- [Misc] Don't dump contents of kvcache tensors on errors (#8527) · 56c3de01
  Nick Hill authored Sep 17, 2024
  
  56c3de01
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
14 Sep, 2024 2 commits
- [TPU] Implement multi-step scheduling (#8489) · 50e9ec41
  Woosuk Kwon authored Sep 14, 2024
  
  50e9ec41
- [torch.compile] fix functionalization (#8480) · a36e070d
  youkaichao authored Sep 14, 2024
  
  a36e070d
13 Sep, 2024 1 commit
- [plugin][torch.compile] allow to add custom compile backend (#8445) · 0a4806f0
  youkaichao authored Sep 13, 2024
  
  0a4806f0
12 Sep, 2024 4 commits
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [BugFix] lazy init _copy_stream to avoid torch init wrong gpu instance (#8403) · 8a23e933
  WANGWEI authored Sep 13, 2024
  
  8a23e933
- [Misc] Raise error when using encoder/decoder model with cpu backend (#8355) · 295c4730
  Kevin Lin authored Sep 12, 2024
  
  295c4730
- [MISC] Dump model runner inputs when crashing (#8305) · a65cb160
  Cody Yu authored Sep 11, 2024
  
  a65cb160
11 Sep, 2024 3 commits
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
10 Sep, 2024 1 commit
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
08 Sep, 2024 1 commit
- [Bugfix] Fix async postprocessor in case of preemption (#8267) · 4ef41b84
  Alexander Matveev authored Sep 08, 2024
  
  4ef41b84
07 Sep, 2024 1 commit
- [tpu][misc] fix typo (#8260) · ce2702a9
  youkaichao authored Sep 06, 2024
  
  ce2702a9
04 Sep, 2024 2 commits
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
- [TPU][Bugfix] Use XLA rank for persistent cache path (#8137) · 61f4a93d
  Woosuk Kwon authored Sep 03, 2024
  
  61f4a93d
03 Sep, 2024 2 commits
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
30 Aug, 2024 3 commits
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 3 commits
- extend cuda graph size for H200 (#7894) · c334b189
  kushanam authored Aug 29, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  c334b189
- [Core] Combine async postprocessor and multi-step (#7921) · 3f60f224
  Alexander Matveev authored Aug 29, 2024
  
  3f60f224
- [torch.compile] remove reset (#7975) · a7f65c2b
  youkaichao authored Aug 28, 2024
  
  a7f65c2b
28 Aug, 2024 3 commits
- [torch.compile] avoid Dynamo guard evaluation overhead (#7898) · ce6bf3a2
  youkaichao authored Aug 28, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ce6bf3a2
- [Performance] Enable chunked prefill and prefix caching together (#7753) · e3580537
  Cody Yu authored Aug 28, 2024
  
  e3580537
- [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) · f508e03e
  Alexander Matveev authored Aug 28, 2024
  
  f508e03e