Commits · a92842454ca824ce6fcf356f31e3bf0daf53629b · OpenDAS / vllm_cscc · GitLab

03 May, 2025 8 commits
- [Bugfix][ROCm] Using device_type because on ROCm the API is still torch.cuda (#17601) · a9284245
  Gregory Shtrasberg authored May 03, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a9284245
- [Build/CI] Upgrade CUTLASS to 3.9.1 (#17602) · c8386fa6
  Tyler Michael Smith authored May 03, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c8386fa6
- [Frontend][TPU] Add TPU default max-num-batched-tokens based on device name (#17508) · 87baebeb
  Chenyaaang authored May 02, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  87baebeb
- [Quantizaton] [AMD] Add support for running DeepSeek int8 w8a8 MoE on ROCm (#17558) · e3d0a1d1
  rasmith authored May 02, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  e3d0a1d1
- Update test requirements to CUDA 12.8 (#17576) · d47b605e
  22quinn authored May 02, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  d47b605e
- [Neuron][Build] Require setuptools >= 77.0.3 for PEP 639 (#17603) · 22c6f639
  Liangfu Chen authored May 02, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  22c6f639
- [release] Add command to clean up Docker containers/images in TPU release machine (#17606) · 3ec97e2c
  Kevin H. Luu authored May 02, 2025
  
  3ec97e2c
- fix typo in logging (#17605) · 9b103a1d
  Eric Hartford authored May 02, 2025
  
  9b103a1d
02 May, 2025 25 commits
- [easy] Print number of needed GPUs in skip message (#17594) · b90b0852
  Richard Zou authored May 02, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b90b0852
- [Hardware][AMD] Improve OAM device ID + llama4 Maverick MOE tuning (#16263) · 9352cdb5
  Xiaodong Wang authored May 02, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
Co-authored-by: Lu Fang <lufang@fb.com>
```
  9352cdb5
- Add NVIDIA TensorRT Model Optimizer in vLLM documentation (#17561) · 182f40ea
  Zhiyu authored May 02, 2025
  
  182f40ea
- permute/unpermute kernel for moe optimization (#14568) · 3e887d2e
  Caleb_Du authored May 03, 2025
```
Signed-off-by: Caleb_Du <Caleb_Du@zju.edu.cn>
```
  3e887d2e
- [BugFix][Attention] Fix sliding window attention in V1 giving incorrect results (#17574) · 0f87d8f7
  Lucas Wilkinson authored May 02, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  0f87d8f7
- [Bugfix] fix tmp_out and exp_sums dimensions (#17438) · 4c33d673
  Hui Liu authored May 02, 2025
```
Signed-off-by: Hui Liu <96135754+hliuca@users.noreply.github.com>
```
  4c33d673
- [Misc] Clean up input processing (#17582) · cb234955
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cb234955
- [doc] miss result (#17589) · 3a500cd0
  Reid authored May 02, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  3a500cd0
- Support W8A8 INT8 MoE for compressed-tensors (#16745) · 868c546d
  Michael Goin authored May 02, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  868c546d
- [Security] Fix image hash collision (#17378) · 99404f53
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  99404f53
- Automatically tell users that dict args must be valid JSON in CLI (#17577) · 785d75a0
  Harry Mellor authored May 02, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  785d75a0
- [doc] add the print result (#17584) · 6d1479ca
  Reid authored May 02, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  6d1479ca
- add more pytorch related tests for torch nightly (#17422) · b8b0859b
  Yang Wang authored May 02, 2025
```
Signed-off-by: Yang Wang <elainewy@meta.com>
```
  b8b0859b
- [Misc] Rename assets for testing (#17575) · d7543862
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d7543862
- [BugFix] Fix Memory Leak (#17567) · c777df79
  Robert Shaw authored May 02, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  c777df79
- [Core] [Bugfix] Add Input Embeddings (#15428) · cc2a77d7
  Andrew Sansom authored May 02, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: 临景 <linjing.yx@alibaba-inc.com>
Co-authored-by: Bryce1010 <bryceyx@gmail.com>
Co-authored-by: Nan2018 <nan@protopia.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cc2a77d7
- [Bugifx] Remove TritonPlaceholder from sys.modules (#17317) · 9e2de9b9
  Isotr0py authored May 02, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  9e2de9b9
- Add `pt_load_map_location` to allow loading to cuda (#16869) · 109e15a3
  Jerry Zhang authored May 01, 2025
```
Signed-off-by: Jerry Zhang <jerryzh168@gmail.com>
```
  109e15a3
- Fix PixtralHF missing spatial_merge_size (#17571) · f192ca90
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f192ca90
- [Misc] Continue refactoring model tests (#17573) · f89d0e11
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f89d0e11
- Check if bitblas is installed during support check (#17572) · b4003d11
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b4003d11
- [CI] Actually run tests/kv_transfer/test_disagg.py in CI (#17555) · 292fc59d
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  292fc59d
- [Attention] MLA move o_proj q_proj into cuda-graph region (#17484) · afcb3f88
  Lucas Wilkinson authored May 01, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  afcb3f88
- [Doc] note that not all unit tests pass on CPU platforms (#17554) · afb12e42
  David Xia authored May 01, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  afb12e42
- [Bugfix] Disable gptq_bitblas for <SM80 to fix GPTQ on V100/T4 (#17541) · 24aebae1
  Michael Goin authored May 01, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  24aebae1
01 May, 2025 7 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE3 (#17504) · 39c0813a
  qizixi authored May 01, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  39c0813a
- [Misc][Tools][Benchmark] Publish script to auto tune server parameters (#17207) · 9b70e2b4
  Chenyaaang authored May 01, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  9b70e2b4
- [Bug]change the position of cuda_graph_sizes in dataclasses (#17548) · 173daac1
  Chen Xia authored May 01, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
```
  173daac1
- Remove duplicate code from dbrx.py (#17550) · 04f2cfc8
  sstamenk authored May 01, 2025
  
  04f2cfc8
- [ROCM] Add gfx950 to the custom attention archs (#16034) · 811a6c09
  Juan Villamizar authored May 01, 2025
```
Signed-off-by: jpvillam <Juan.Villamizar@amd.com>
Signed-off-by: seungrokjung <seungrok.jung@amd.com>
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: seungrokjung <seungrok.jung@amd.com>
Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  811a6c09
- [Bugfix] Fix lint error (#17547) · 9b1769dd
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9b1769dd
- [Misc]add configurable cuda graph size (#17201) · 61c299f8
  Chen Xia authored May 01, 2025
```
Signed-off-by: CXIAAAAA <cxia0209@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  61c299f8