Commits · d4db9f53c8a50a2b0788cf1e03b5b91f20de4313 · OpenDAS / vllm_cscc

03 Sep, 2024 2 commits
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
30 Aug, 2024 3 commits
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 3 commits
- extend cuda graph size for H200 (#7894) · c334b189
  kushanam authored Aug 29, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  c334b189
- [Core] Combine async postprocessor and multi-step (#7921) · 3f60f224
  Alexander Matveev authored Aug 29, 2024
  
  3f60f224
- [torch.compile] remove reset (#7975) · a7f65c2b
  youkaichao authored Aug 28, 2024
  
  a7f65c2b
28 Aug, 2024 4 commits
- [torch.compile] avoid Dynamo guard evaluation overhead (#7898) · ce6bf3a2
  youkaichao authored Aug 28, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ce6bf3a2
- [Performance] Enable chunked prefill and prefix caching together (#7753) · e3580537
  Cody Yu authored Aug 28, 2024
  
  e3580537
- [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) · f508e03e
  Alexander Matveev authored Aug 28, 2024
  
  f508e03e
- [Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for... · c166e7e4
  bnellnm authored Aug 27, 2024
```
[Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for registering FakeScalarType and dynamo support. (#7886)
```
  c166e7e4
27 Aug, 2024 3 commits
- [Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810) · 076169f6
  Kunshang Ji authored Aug 28, 2024
  
  076169f6
- [core][torch.compile] discard the compile for profiling (#7796) · 64cc6444
  youkaichao authored Aug 26, 2024
  
  64cc6444
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
26 Aug, 2024 1 commit
- [Bugfix] neuron: enable tensor parallelism (#7562) · 760e9f71
  omrishiv authored Aug 26, 2024
```
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
```
  760e9f71
23 Aug, 2024 2 commits
- [misc] Add Torch profiler support for CPU-only devices (#7806) · faeddb56
  Jie Fu (傅杰) authored Aug 23, 2024
  
  faeddb56
- [Hardware][Intel GPU] refactor xpu_model_runner for tp (#7712) · fc5ebbd1
  Kunshang Ji authored Aug 23, 2024
  
  fc5ebbd1
22 Aug, 2024 1 commit
- [Speculative Decoding] EAGLE Implementation with Top-1 proposer (#6830) · a3fce56b
  Abhinav Goyal authored Aug 22, 2024
  
  a3fce56b
21 Aug, 2024 2 commits
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
- [Bugfix][Hardware][CPU] Fix `mm_limits` initialization for CPU backend (#7735) · 6925cdbe
  Isotr0py authored Aug 22, 2024
  
  6925cdbe
20 Aug, 2024 2 commits
- [Core] Add `AttentionState` abstraction (#7663) · 3b682179
  Antoni Baum authored Aug 20, 2024
  
  3b682179
- [Hardware] [Intel GPU] refactor xpu worker/executor (#7686) · c42590f9
  Kunshang Ji authored Aug 21, 2024
  
  c42590f9
19 Aug, 2024 3 commits
- [TPU] Remove redundant input tensor cloning (#7660) · 43735bf5
  Woosuk Kwon authored Aug 19, 2024
  
  43735bf5
- [core] Multi Step Scheduling (#7000) · 47b65a55
  William Lin authored Aug 19, 2024
```
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
```
  47b65a55
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
18 Aug, 2024 1 commit
- [TPU] Use mark_dynamic only for dummy run (#7634) · 0c2fa50b
  Woosuk Kwon authored Aug 18, 2024
  
  0c2fa50b
17 Aug, 2024 3 commits
- [TPU] Skip creating empty tensor (#7630) · ce143353
  Woosuk Kwon authored Aug 17, 2024
  
  ce143353
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
- [misc] use nvml to get consistent device name (#7582) · eed020f6
  youkaichao authored Aug 16, 2024
  
  eed020f6
16 Aug, 2024 1 commit
- [Core] Fix tracking of model forward time in case of PP>1 (#7440) · 93478b63
  Mahesh Keralapura authored Aug 16, 2024
```
[Core] Fix tracking of model forward time to the span traces in case of PP>1 (#7440)
```
  93478b63
15 Aug, 2024 1 commit

[Bugfix] update neuron for version > 0.5.0 (#7175) · 9c1f78d5

omrishiv authored Aug 15, 2024


Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9c1f78d5

14 Aug, 2024 2 commits
- [TPU] Set per-rank XLA cache (#7533) · 951fdd66
  Woosuk Kwon authored Aug 14, 2024
  
  951fdd66
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
13 Aug, 2024 2 commits
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
12 Aug, 2024 1 commit
- [Core] Consolidate `GB` constant and enable float GB arguments (#7416) · 4ddc4743
  Cyrus Leung authored Aug 13, 2024
  
  4ddc4743
11 Aug, 2024 2 commits
- [core] [2/N] refactor worker_base input preparation for multi-step (#7387) · c08e2b30
  William Lin authored Aug 11, 2024
  
  c08e2b30
- [TPU] Use mark_dynamic to reduce compilation time (#7340) · 90bab18f
  Woosuk Kwon authored Aug 10, 2024
  
  90bab18f
09 Aug, 2024 1 commit
- [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) · 933790c2
  Mahesh Keralapura authored Aug 09, 2024
  
  933790c2