Commits · 4078052f09f42f898b542e18d60d15a43db67a8b · OpenDAS / vllm_cscc

28 Jan, 2025 1 commit
- Update `pre-commit` hooks (#12475) · 823ab796
  Harry Mellor authored Jan 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  823ab796
25 Jan, 2025 1 commit
- [Bugfix][Kernel] Fix moe align block issue for mixtral (#12413) · 221d388c
  ElizaWszola authored Jan 24, 2025
  
  221d388c
23 Jan, 2025 1 commit

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

22 Jan, 2025 1 commit
- [Core] Support fully transparent sleep mode (#11743) · 68ad4e3a
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  68ad4e3a
21 Jan, 2025 2 commits

[Kernel] fix moe_align_block_size error condition (#12239) · 1e60f87b
Jinzhen Lin authored Jan 22, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
1e60f87b

[Kernel] optimize moe_align_block_size for cuda graph and large num_experts... · 750f4cab

Jinzhen Lin authored Jan 21, 2025


[Kernel] optimize moe_align_block_size for cuda graph and large num_experts (e.g. DeepSeek-V3) (#12222)
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

750f4cab

20 Jan, 2025 1 commit
- Move linting to `pre-commit` (#11975) · 3ea7b945
  Harry Mellor authored Jan 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3ea7b945
15 Jan, 2025 2 commits
- [Misc] Add multipstep chunked-prefill support for FlashInfer (#10467) · 0794e744
  Elfie Guo authored Jan 14, 2025
  
  0794e744
- [Kernel] Support MulAndSilu (#11624) · 42f5e7c5
  Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  42f5e7c5
08 Jan, 2025 1 commit
- [Hardware][Apple] Native support for macOS Apple Silicon (#11696) · cfd3219f
  Wallas Henrique authored Jan 08, 2025
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  cfd3219f
05 Jan, 2025 1 commit
- [MISC] Replace c10::optional with std::optional (#11730) · 4068f4b5
  Lu Fang authored Jan 04, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  4068f4b5
02 Jan, 2025 1 commit
- Resolve race conditions in Marlin kernel (#11493) · 5dba2575
  wchen61 authored Jan 03, 2025
```
Signed-off-by: wchen61 <wchen61@foxmail.com>
```
  5dba2575
30 Dec, 2024 1 commit
- [Build][Kernel] Update CUTLASS to v3.6.0 (#11607) · 970d6d07
  Tyler Michael Smith authored Dec 30, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  970d6d07
27 Dec, 2024 1 commit

Deepseek v3 (#11502) · f49777ba

Simon Mo authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>

f49777ba

19 Dec, 2024 2 commits
- [Kernel] Refactor Cutlass c3x (#10049) · 8936316d
  Varun Sundar Rabindranath authored Dec 19, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8936316d
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

13 Dec, 2024 1 commit

[torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f

Luka Govedič authored Dec 12, 2024


Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

30870b4f

09 Dec, 2024 1 commit
- [V1] Further reduce CPU overheads in flash-attn (#10989) · 3b61cb45
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3b61cb45
07 Dec, 2024 2 commits
- [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) · 78029b34
  zhou fan authored Dec 08, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  78029b34
- [Build] Fix for the Wswitch-bool clang warning (#10060) · f13cf9ad
  Gregory Shtrasberg authored Dec 07, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  f13cf9ad
27 Nov, 2024 1 commit
- [Kernel] Remove if-else with identical branches in marlin 2:4 (#10687) · e2251109
  Tyler Michael Smith authored Nov 27, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  e2251109
26 Nov, 2024 1 commit

[Feature] vLLM ARM Enablement for AARCH64 CPUs (#9228) · a6760f64

Sanket Kale authored Nov 26, 2024


Signed-off-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: Sanket Kale <sanketk.kale@fujitsu.com>
Co-authored-by: mgoin <michael@neuralmagic.com>

a6760f64

23 Nov, 2024 1 commit
- [AMD] Add support for GGUF quantization on ROCm (#10254) · 7c25fe45
  kliuae authored Nov 23, 2024
  
  7c25fe45
20 Nov, 2024 1 commit
- [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) · d200972e
  Lucas Wilkinson authored Nov 19, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d200972e
19 Nov, 2024 2 commits

[Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
b00b33d7

Fix: Build error seen on Power Architecture (#10421) · 1ea291a4

Manjul Mohan authored Nov 19, 2024


Signed-off-by: Manjul Mohan <manjul.mohan@ibm.com>
Signed-off-by: B-201 <Joy25810@foxmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: ismael-dm <ismaeldm99@gmail.com>
Signed-off-by: Andrew Nesbitt <andrewnez@gmail.com>
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: yan ma <yan.ma@intel.com>
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Manjul Mohan manjul.mohan@ibm.com <manjulmohan@ltcd97-lp2.aus.stglabs.ibm.com>
Co-authored-by: B-201 <Joy25810@foxmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: ismael-dm <ismaeldm99@gmail.com>
Co-authored-by: Andrew Nesbitt <andrewnez@gmail.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Angus Wang <wangjadehao@gmail.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Ricky Xu <rickyx@anyscale.com>
Co-authored-by: Kevin H. Luu <kevin@anyscale.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

1ea291a4

18 Nov, 2024 1 commit
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
14 Nov, 2024 1 commit

Support Roberta embedding models (#9387) · 4a18fd14

Maximilien de Bayser authored Nov 14, 2024


Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

4a18fd14

13 Nov, 2024 1 commit
- [Core] Flashinfer - Remove advance step size restriction (#10282) · b6dde330
  Pavani Majety authored Nov 13, 2024
  
  b6dde330
12 Nov, 2024 1 commit

Splitting attention kernel file (#10091) · 812c981f

Aleksandr Malyshev authored Nov 11, 2024


Signed-off-by: maleksan85 <maleksan@amd.com>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>

812c981f

08 Nov, 2024 1 commit
- [torch.compile] Fuse RMSNorm with quant (#9138) · 4f93dfe9
  Luka Govedič authored Nov 08, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  4f93dfe9
07 Nov, 2024 3 commits
- [Hardware][CPU][bugfix] Fix half dtype support on AVX2-only target (#10108) · a6f332d0
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a6f332d0
- [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (#10030) · 6192e9b8
  Hanzhi Zhou authored Nov 06, 2024
```
Signed-off-by: Hanzhi Zhou <hanzhi713@gmail.com>
```
  6192e9b8
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
06 Nov, 2024 1 commit
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
31 Oct, 2024 1 commit
- [BugFix][Kernel] Fix Illegal memory access in causal_conv1d in H100 (#9838) · 9fb12f78
  Mor Zusman authored Oct 31, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  9fb12f78
27 Oct, 2024 1 commit
- [core] cudagraph output with tensor weak reference (#9724) · 8549c826
  youkaichao authored Oct 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8549c826
24 Oct, 2024 2 commits
- [Performance][Kernel] Fused_moe Performance Improvement (#9384) · 59449095
  Charlie Fu authored Oct 24, 2024
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  59449095
- [Kernel] add kernel for FATReLU (#9610) · 295a061f
  Jee Jee Li authored Oct 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  295a061f