Commits · ecf67814f1a9e31e9802d93e8bd8b11a1c2810e7 · OpenDAS / vllm_cscc

21 Jan, 2025 4 commits

Add quantization and guided decoding CODEOWNERS (#12228) · ecf67814
Michael Goin authored Jan 20, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
ecf67814

[Kernel] optimize moe_align_block_size for cuda graph and large num_experts... · 750f4cab

Jinzhen Lin authored Jan 21, 2025


[Kernel] optimize moe_align_block_size for cuda graph and large num_experts (e.g. DeepSeek-V3) (#12222)
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

750f4cab

[bugfix] catch xgrammar unsupported array constraints (#12210) · 06a760d6
Cheng Kuan Yong Jason authored Jan 21, 2025
```
Signed-off-by: Jason Cheng <jasoncky96@gmail.com>
```
06a760d6
[misc] add cuda runtime version to usage data (#12190) · da751221
youkaichao authored Jan 21, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
da751221

20 Jan, 2025 20 commits
- fix: update platform detection for M-series arm based MacBook processors (#12227) · af69a6ad
  Işık authored Jan 20, 2025
```
Signed-off-by: isikhi <huseyin.isik000@gmail.com>
```
  af69a6ad
- [Misc] Update CODEOWNERS (#12229) · 7bd36300
  Roger Wang authored Jan 20, 2025
  
  7bd36300
- [CI] Pass local python version explicitly to pre-commit mypy.sh (#12224) · 96663699
  Chen Zhang authored Jan 20, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  96663699
- [Bugfix] Fix `HfExampleModels.find_hf_info` (#12223) · 18572e33
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  18572e33
- [Misc] Pass `attention` to impl backend (#12218) · 86bfb6db
  wangxiyuan authored Jan 20, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  86bfb6db
- [V1] Remove `_get_cache_block_size` (#12214) · 5f0ec393
  Chen Zhang authored Jan 20, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  5f0ec393
- [core][bugfix] configure env var during import vllm (#12209) · c222f479
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c222f479
- [misc] print a message to suggest how to bypass commit hooks (#12217) · 170eb350
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  170eb350
- [Model] Upgrade Aria to transformers 4.48 (#12203) · b37d8279
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b37d8279
- [CI/Build] Make pre-commit faster (#12212) · 3127e975
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3127e975
- [CI/Build] Remove dummy CI steps (#12208) · 4001ea12
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  4001ea12
- [misc] add placeholder format.sh (#12206) · 5c89a29c
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  5c89a29c
- [Core] Interface for accessing model from `VllmRunner` (#10353) · 59a0192f
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a0192f
- [Model] Add Qwen2 PRM model support (#12202) · 83609791
  Isotr0py authored Jan 20, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  83609791
- [Bugfix] Fix incorrect types in LayerwiseProfileResults (#12196) · 0974c9bc
  Yuan Tang authored Jan 20, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  0974c9bc
- [DOC] Add missing docstring in LLMEngine.add_request() (#12195) · d2643128
  Yuan Tang authored Jan 20, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  d2643128
- [DOC] Fix typo in docstring and assert message (#12194) · c5c06209
  Yuan Tang authored Jan 20, 2025
```
Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>
```
  c5c06209
- Move linting to `pre-commit` (#11975) · 3ea7b945
  Harry Mellor authored Jan 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3ea7b945
- [torch.compile] fix sym_tensor_indices (#12191) · 51ef828f
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51ef828f
- [Bugfix] Fix num_heads value for simple connector when tp enabled (#12074) · df450aa5
  shangmingc authored Jan 20, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  df450aa5
19 Jan, 2025 9 commits
- [Model] Support for fairseq2 Llama (#11442) · bbe5f9de
  Martin Gleize authored Jan 19, 2025
```
Signed-off-by: Martin Gleize <mgleize@meta.com>
Co-authored-by: mgleize user <mgleize@a100-st-p4de24xlarge-4.fair-a100.hpcaas>
```
  bbe5f9de
- [V1] Add V1 support of Qwen2-VL (#12128) · 81763c58
  Roger Wang authored Jan 19, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  81763c58
- [Misc] Add BNB support to GLM4-V model (#12184) · edaae198
  Isotr0py authored Jan 19, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  edaae198
- benchmark_serving support --served-model-name param (#12109) · 936db119
  gujing authored Jan 19, 2025
```
Signed-off-by: zibai <zibai.gj@alibaba-inc.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  936db119
- [torch.compile] store inductor compiled Python file (#12182) · e66faf48
  youkaichao authored Jan 19, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e66faf48
- [Bugfix] Fix multi-modal processors for transformers 4.48 (#12187) · 630eb5b5
  Cyrus Leung authored Jan 19, 2025
  
  630eb5b5
- [BUGFIX] Move scores to float32 in case of running xgrammar on cpu (#12152) · 4e94951b
  Michal Adamczyk authored Jan 19, 2025
```
Signed-off-by: Michal Adamczyk <madamczyk@habana.ai>
```
  4e94951b
- [V1] Collect env var for usage stats (#12115) · 7a8a48d5
  Simon Mo authored Jan 18, 2025
  
  7a8a48d5
- [Misc] Support register quantization method out-of-tree (#11969) · 32eb0da8
  yancong authored Jan 19, 2025
  
  32eb0da8
18 Jan, 2025 6 commits
- [core] clean up executor class hierarchy between v1 and v0 (#12171) · 6d0e3d37
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6d0e3d37
- [Model] Port deepseek-vl2 processor, remove dependency (#12169) · 02798eca
  Isotr0py authored Jan 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  02798eca
- [Docs] Fix broken link in SECURITY.md (#12175) · 813f249f
  Russell Bryant authored Jan 17, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  813f249f
- [core] further polish memory profiling (#12126) · da02cb4b
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  da02cb4b
- [AMD][CI/Build][Bugfix] use pytorch stale wheel (#12172) · c09503dd
  Hongxia Yang authored Jan 17, 2025
```
Signed-off-by: hongxyan <hongxyan@amd.com>
```
  c09503dd
- [misc] fix cross-node TP (#12166) · 2b835032
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  2b835032
17 Jan, 2025 1 commit
- [torch.compile] disable logging when cache is disabled (#12043) · 7b98a65a
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7b98a65a