Commits · 2f7d31f16e5cbb9c0d55e316d0bb3c877da0f6fc · OpenDAS / vllm_cscc

20 Nov, 2024 2 commits
- [fix]修复单测test_medusa_correctness.py中的错误 · 215f33b0
  王敏 authored Nov 20, 2024
  
  215f33b0
- [fix]修复单测test_encoder_decoder_model_runner报错问题 · 93089fb2
  王敏 authored Nov 20, 2024
  
  93089fb2
06 Nov, 2024 1 commit

王敏 authored Nov 06, 2024

2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题

f9060e6b

24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
25 Sep, 2024 2 commits

[Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602

Chen Zhang authored Sep 25, 2024


Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

770ec602

[Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770) · c2395367
Isotr0py authored Sep 25, 2024

c2395367

24 Sep, 2024 1 commit
- [MISC] Skip dumping inputs when unpicklable (#8744) · b8747e8a
  Cody Yu authored Sep 23, 2024
  
  b8747e8a
23 Sep, 2024 2 commits
- [Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (#8733) · 3e83c12b
  Li, Jiang authored Sep 23, 2024
  
  3e83c12b
- [Hardware][CPU] Refactor CPU model runner (#8729) · e551ca15
  Isotr0py authored Sep 23, 2024
  
  e551ca15
22 Sep, 2024 1 commit
- [MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (#8703) · ca2b628b
  Huazhong Ji authored Sep 23, 2024
  
  ca2b628b
20 Sep, 2024 1 commit
- [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) · 9e5ec35b
  William Lin authored Sep 19, 2024
  
  9e5ec35b
19 Sep, 2024 1 commit

[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction... · 3118f633

sroy745 authored Sep 18, 2024

[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models.  (#8545)

3118f633

18 Sep, 2024 2 commits
- [Core] *Prompt* logprobs support in Multi-step (#8199) · a8c1d161
  afeldman-nm authored Sep 18, 2024
  
  a8c1d161
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 2 commits
- [Misc] Don't dump contents of kvcache tensors on errors (#8527) · 56c3de01
  Nick Hill authored Sep 17, 2024
  
  56c3de01
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
14 Sep, 2024 2 commits
- [TPU] Implement multi-step scheduling (#8489) · 50e9ec41
  Woosuk Kwon authored Sep 14, 2024
  
  50e9ec41
- [torch.compile] fix functionalization (#8480) · a36e070d
  youkaichao authored Sep 14, 2024
  
  a36e070d
13 Sep, 2024 2 commits
- [plugin][torch.compile] allow to add custom compile backend (#8445) · 0a4806f0
  youkaichao authored Sep 13, 2024
  
  0a4806f0
- remove the automatic switching strategy of fa · 408f663a
  zhuwenwen authored Sep 13, 2024
  
  408f663a
12 Sep, 2024 4 commits
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [BugFix] lazy init _copy_stream to avoid torch init wrong gpu instance (#8403) · 8a23e933
  WANGWEI authored Sep 13, 2024
  
  8a23e933
- [Misc] Raise error when using encoder/decoder model with cpu backend (#8355) · 295c4730
  Kevin Lin authored Sep 12, 2024
  
  295c4730
- [MISC] Dump model runner inputs when crashing (#8305) · a65cb160
  Cody Yu authored Sep 11, 2024
  
  a65cb160
11 Sep, 2024 4 commits
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- fix run error · 9b902f9e
  zhuwenwen authored Sep 11, 2024
  
  9b902f9e
10 Sep, 2024 1 commit
- [Spec Decode] Move ops.advance_step to flash attn advance_step (#8224) · 5faedf1b
  Kevin Lin authored Sep 10, 2024
  
  5faedf1b
08 Sep, 2024 1 commit
- [Bugfix] Fix async postprocessor in case of preemption (#8267) · 4ef41b84
  Alexander Matveev authored Sep 08, 2024
  
  4ef41b84
07 Sep, 2024 1 commit
- [tpu][misc] fix typo (#8260) · ce2702a9
  youkaichao authored Sep 06, 2024
  
  ce2702a9
04 Sep, 2024 2 commits
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
- [TPU][Bugfix] Use XLA rank for persistent cache path (#8137) · 61f4a93d
  Woosuk Kwon authored Sep 03, 2024
  
  61f4a93d
03 Sep, 2024 2 commits
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
02 Sep, 2024 1 commit
- fix error in test_long_context.py::test_batched_rope_kernel and add a demo · 5e19cdef
  zhuwenwen authored Sep 02, 2024
  
  5e19cdef
30 Aug, 2024 3 commits
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 1 commit
- extend cuda graph size for H200 (#7894) · c334b189
  kushanam authored Aug 29, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  c334b189