Commits · 641fc5b7f64b57ab020df81bc7d360cbd7e76344 · OpenDAS / vllm_cscc · GitLab

14 Apr, 2025 3 commits
- remove unused code · 641fc5b7
  zhuwenwen authored Apr 14, 2025
  
  641fc5b7
- remove unused mla utils.py · b01efa0b
  zhuwenwen authored Apr 14, 2025
  
  b01efa0b
- update vdim padding strategy · 4a19cdf5
  zhuwenwen authored Apr 14, 2025
  
  4a19cdf5
12 Apr, 2025 1 commit
- set MOE_NN to default value of 0 · 155d41d1
  zhuwenwen authored Apr 12, 2025
  
  155d41d1
11 Apr, 2025 2 commits
- support v1 engine · e19ddc39
  zhuwenwen authored Apr 11, 2025
  
  e19ddc39
- add VLLM_USE_FLASH_MLA to use flashmla backend · a0e22db9
  zhuwenwen authored Apr 11, 2025
  
  a0e22db9
10 Apr, 2025 2 commits
- support vdim 128 · c91e1a7c
  zhuwenwen authored Apr 10, 2025
  
  c91e1a7c
- support flashmla backend · bb1d0df8
  zhuwenwen authored Apr 10, 2025
  
  bb1d0df8
09 Apr, 2025 6 commits
- add benchmark_utils.py · 23607ca0
  zhuwenwen authored Apr 09, 2025
  
  23607ca0
- add benchmark_dataset.py · 47763a73
  zhuwenwen authored Apr 09, 2025
  
  47763a73
- add latency · 39171c5f
  zhuwenwen authored Apr 09, 2025
  
  39171c5f
- update fused_moe.py · c588349c
  zhuwenwen authored Apr 09, 2025
  
  c588349c
- update baichuan.py · 277271f8
  zhuwenwen authored Apr 09, 2025
  
  277271f8
- update arg_utils.py · 746d9b40
  zhuwenwen authored Apr 09, 2025
  
  746d9b40
07 Apr, 2025 1 commit
- 增加fused-moe int4/int8的支持，以及deepseek精度问题的修复 · bb94d2e5
  yangql authored Apr 07, 2025
  
  bb94d2e5
06 Apr, 2025 2 commits
- Revert "[V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906)" · 296c6572
  simon-mo authored Apr 05, 2025
```
This reverts commit 651cf0fe.
```
  296c6572
- [Model] Support Llama4 in vLLM (#16104) · c5752323
  Lu Fang authored Apr 05, 2025
  
  c5752323
04 Apr, 2025 16 commits
- [V1][Spec Decode] Update N-gram Proposer Interface (#15750) · 63375f0c
  Woosuk Kwon authored Apr 04, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  63375f0c
- [Bugfix][TPU] Fix V1 TPU worker for sliding window (#16059) · 70ad3f9e
  Michael Goin authored Apr 04, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  70ad3f9e
- [Kernel][Minor] Re-fuse triton moe weight application (#16071) · d6fc629f
  bnellnm authored Apr 04, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  d6fc629f
- Revert "[V1] Scatter and gather placeholders in the model runner" (#16075) · af51d80f
  Roger Wang authored Apr 04, 2025
  
  af51d80f
- [V1] Scatter and gather placeholders in the model runner (#15712) · f5722a50
  Cyrus Leung authored Apr 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f5722a50
- [V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (#15906) · 651cf0fe
  Nick Hill authored Apr 04, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  651cf0fe
- [Bugfix] Fix default behavior/fallback for pp in v1 (#16057) · 4708f13a
  Michael Goin authored Apr 04, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  4708f13a
- [ROCm][Bugfix] Bring back fallback to eager mode removed in #14917, but for ROCm only (#15413) · a6d042df
  Gregory Shtrasberg authored Apr 04, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a6d042df
- [ROCm][Bugfix] Use platform specific FP8 dtype (#15717) · 40a36ccf
  Gregory Shtrasberg authored Apr 04, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  40a36ccf
- [Distributed] [ROCM] Fix custom allreduce enable checks (#16010) · ef608c37
  Ilya Markov authored Apr 04, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
```
  ef608c37
- [CPU] Change default block_size for CPU backend (#16002) · 2386803f
  Li, Jiang authored Apr 05, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  2386803f
- [Hardware][Gaudi][BugFix] fix arguments of hpu fused moe (#15945) · 0812d8dd
  liuzhenwei authored Apr 05, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  0812d8dd
- [Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (#15939) · bf7e3c51
  Jonghyun Choe authored Apr 05, 2025
```
Signed-off-by: Jonghyun Choe <andy.choe729@gmail.com>
```
  bf7e3c51
- [V1][Spec Decode] Avoid logging useless nan metrics (#16023) · a35a8a83
  Mark McLoughlin authored Apr 04, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  a35a8a83
- [TPU][V1] Remove ragged attention kernel parameter hard coding (#16041) · fadc59c0
  Chengji Yao authored Apr 04, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  fadc59c0
- [Misc] improve gguf check (#15974) · 86cbd2ee
  Reid authored Apr 04, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  86cbd2ee
03 Apr, 2025 7 commits
- [TPU] Support sliding window and logit soft capping in the paged attention kernel for TPU. (#15732) · b6be6f8d
  iefgnoix authored Apr 03, 2025
```
Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
```
  b6be6f8d
- [Minor] Fused experts refactor (#15914) · 15ba07ef
  bnellnm authored Apr 03, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  15ba07ef
- [Neuron][kernel] Fuse kv cache into a single tensor (#15911) · d2b58ca2
  Liangfu Chen authored Apr 03, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  d2b58ca2
- [SupportsQuant] Chameleon, Chatglm, Commandr (#15952) · 82e7e19a
  Kyle Sayers authored Apr 03, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  82e7e19a
- [SupportsQuant] Bert, Blip, Blip2, Bloom (#15573) · 421c4629
  Kyle Sayers authored Apr 03, 2025
```
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
```
  421c4629
- [Bugfix][V1] Fix bug from putting llm_engine.model_executor in a background process (#15367) · 463bbb18
  wwl2755 authored Apr 03, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  463bbb18
- [misc] improve error message for "Failed to infer device type" (#15994) · 5e125e74
  youkaichao authored Apr 03, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  5e125e74