Commits · a94eee4456b05458bafacc17377de4701ac598a0 · OpenDAS / vllm_cscc

21 Jan, 2025 1 commit

[Kernel] optimize moe_align_block_size for cuda graph and large num_experts... · 750f4cab

Jinzhen Lin authored Jan 21, 2025


[Kernel] optimize moe_align_block_size for cuda graph and large num_experts (e.g. DeepSeek-V3) (#12222)
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

750f4cab

19 Jan, 2025 1 commit
- [torch.compile] store inductor compiled Python file (#12182) · e66faf48
  youkaichao authored Jan 19, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  e66faf48
17 Jan, 2025 1 commit
- [BugFix] add more `is not None` check in VllmConfig.__post_init__ (#12138) · d1adb9b4
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  d1adb9b4
16 Jan, 2025 3 commits
- [Bugfix] Set enforce_eager automatically for mllama (#12127) · d06e8240
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  d06e8240
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
- [Bugfix] Remove hardcoded `head_size=256` for Deepseek v2 and v3 (#12067) · dd7c9ad8
  Isotr0py authored Jan 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dd7c9ad8
15 Jan, 2025 4 commits
- [V1][Core] Autotune encoder cache budget (#11895) · 70755e81
  Roger Wang authored Jan 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  70755e81
- [Bugfix] use right truncation for non-generative tasks (#12050) · edce722e
  Joe Runde authored Jan 15, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  edce722e
- [Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6
  kewang-xlnx authored Jan 16, 2025
```
Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  de0526f6
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
10 Jan, 2025 2 commits
- [Doc] [1/N] Initial guide for merged multi-modal processor (#11925) · 12664ddd
  Cyrus Leung authored Jan 10, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  12664ddd
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
09 Jan, 2025 1 commit

[Misc] Move `print_*_once` from utils to logger (#11298) · d848800e

Cyrus Leung authored Jan 09, 2025

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>

d848800e

08 Jan, 2025 4 commits
- [torch.compile] consider relevant code in compilation cache (#11614) · f1214117
  youkaichao authored Jan 08, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  f1214117
- [Hardware][Apple] Native support for macOS Apple Silicon (#11696) · cfd3219f
  Wallas Henrique authored Jan 08, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  cfd3219f
- [Bug] Fix pickling of `ModelConfig` when RunAI Model Streamer is used (#11825) · ef68eb28
  Cyrus Leung authored Jan 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ef68eb28
- [Bugfix] Add checks for LoRA and CPU offload (#11810) · f645eb69
  Jee Jee Li authored Jan 08, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  f645eb69
06 Jan, 2025 2 commits
- [Doc][2/N] Reorganize Models and Usage sections (#11755) · ee77fdb5
  Cyrus Leung authored Jan 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ee77fdb5
- [Bugfix] Remove block size constraint (#11723) · 408e5600
  Cody Yu authored Jan 05, 2025
  
  408e5600
03 Jan, 2025 1 commit
- [Model] Whisper model implementation (#11280) · e1a5c2f0
  Aurick Qiao authored Jan 03, 2025
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  e1a5c2f0
30 Dec, 2024 2 commits
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
29 Dec, 2024 1 commit
- [Misc] KV cache transfer connector registry (#11481) · faef77c0
  Kuntai Du authored Dec 29, 2024
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  faef77c0
28 Dec, 2024 1 commit
- [bugfix] interleaving sliding window for cohere2 model (#11583) · 328841d0
  youkaichao authored Dec 29, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  328841d0
27 Dec, 2024 1 commit

Deepseek v3 (#11502) · f49777ba

Simon Mo authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>

f49777ba

26 Dec, 2024 2 commits

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

[Misc] Add placeholder module (#11501) · eec906d8
Cyrus Leung authored Dec 26, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
eec906d8

23 Dec, 2024 1 commit
- [Docs] Convert rST to MyST (Markdown) (#11145) · 32aa2059
  Rafael Vasquez authored Dec 23, 2024
```
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
```
  32aa2059
20 Dec, 2024 2 commits
- [Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) · 995f5623
  omer-dayan authored Dec 20, 2024
```
Signed-off-by: OmerD <omer@run.ai>
```
  995f5623
- [Bugfix][Hardware][POWERPC] Fix auto dtype failure in case of POWER10 (#11331) · 48edab80
  Akash kaothalkar authored Dec 20, 2024
```
Signed-off-by: Akash Kaothalkar <0052v2@linux.vnet.ibm.com>
```
  48edab80
19 Dec, 2024 1 commit

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

18 Dec, 2024 2 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [Bugfix] Restore support for larger block sizes (#11259) · 866fa455
  Konrad Zawora authored Dec 18, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  866fa455
17 Dec, 2024 2 commits

[V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
59c9b6eb

[torch.compile] fast inductor (#11108) · 88a412ed

youkaichao authored Dec 16, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

88a412ed

15 Dec, 2024 1 commit
- [Core] Support disaggregated prefill with Mooncake Transfer Engine (#10884) · d263bd9d
  shangmingc authored Dec 16, 2024
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  d263bd9d
14 Dec, 2024 1 commit
- [Frontend] Add `logits_processors` as an extra completion argument (#11150) · 9c3dadd1
  Brad Hilton authored Dec 14, 2024
```
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
```
  9c3dadd1
13 Dec, 2024 1 commit
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
12 Dec, 2024 1 commit

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 1 commit
- [torch.compile] use depyf to dump torch.compile internals (#10972) · 91642db9
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91642db9