Commits · aa2cd2c43d1d19ece0f3b36ad716c3a9b8a2def0 · OpenDAS / vllm_cscc

"examples/offline_inference/simple_profiling.py" did not exist on "dd0c8f490868762e6671bd01c558b9413fb3caa1"

26 Jan, 2025 1 commit
- [Bugfix] Disable w16a16 2of4 sparse CompressedTensors24 (#12417) · aa2cd2c4
  Tyler Michael Smith authored Jan 26, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  aa2cd2c4
25 Jan, 2025 1 commit
- [Misc] Add FA2 support to ViT MHA layer (#12355) · f1fc0510
  Isotr0py authored Jan 25, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  f1fc0510
24 Jan, 2025 1 commit
- [Bugfix][Kernel] Fix CUDA 11.8 being broken by FA3 build (#12375) · ab5bbf5a
  Lucas Wilkinson authored Jan 24, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  ab5bbf5a
23 Jan, 2025 3 commits
- [FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b
  Gregory Shtrasberg authored Jan 23, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
```
  e97f802b
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
- [AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) · 68c4421b
  rasmith authored Jan 22, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  68c4421b
16 Jan, 2025 1 commit
- [Bugfix] Remove hardcoded `head_size=256` for Deepseek v2 and v3 (#12067) · dd7c9ad8
  Isotr0py authored Jan 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  dd7c9ad8
15 Jan, 2025 2 commits

[Platform] Do not raise error if _Backend is not found (#12023) · 3adf0ffd

wangxiyuan authored Jan 15, 2025


Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

3adf0ffd

[Kernel] Support MulAndSilu (#11624) · 42f5e7c5
Jee Jee Li authored Jan 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
42f5e7c5

12 Jan, 2025 1 commit
- [Hardware][TPU] workaround fix for MoE on TPU (#11764) · 263a870e
  Avshalom Manevich authored Jan 12, 2025
  
  263a870e
10 Jan, 2025 1 commit
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
09 Jan, 2025 1 commit

[platform] Allow platform specify attention backend (#11609) · 405eb8e3

wangxiyuan authored Jan 09, 2025


Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Co-authored-by: Mengqing Cao <cmq0113@163.com>

405eb8e3

06 Jan, 2025 1 commit
- [Kernel] Move attn_type to Attention.__init__() (#11690) · e20c92bb
  Chen Zhang authored Jan 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e20c92bb
01 Jan, 2025 1 commit
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
30 Dec, 2024 1 commit
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
26 Dec, 2024 1 commit

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

19 Dec, 2024 1 commit
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

13 Dec, 2024 1 commit

[torch.compile] Dynamic fp8 + rms_norm fusion (#10906) · 30870b4f

Luka Govedič authored Dec 12, 2024


Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

30870b4f

07 Dec, 2024 1 commit
- [BugFix][Kernel]: fix illegal memory access in causal_conv1d when conv_states is None (#10928) · 78029b34
  zhou fan authored Dec 08, 2024
```
Signed-off-by: xffxff <1247714429@qq.com>
```
  78029b34
02 Dec, 2024 1 commit
- [Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
  Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  073a4bd1
25 Nov, 2024 2 commits
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [misc] move functions to config.py (#10624) · 05d1f8c9
  youkaichao authored Nov 25, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  05d1f8c9
22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
20 Nov, 2024 1 commit
- [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) · d200972e
  Lucas Wilkinson authored Nov 19, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d200972e
19 Nov, 2024 2 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 1 commit
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
15 Nov, 2024 2 commits
- [Misc] Bump up test_fused_moe tolerance (#10364) · 79ee45b4
  ElizaWszola authored Nov 15, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  79ee45b4
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
09 Nov, 2024 1 commit

[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8... · 127c0748

rasmith authored Nov 08, 2024


[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>

127c0748

08 Nov, 2024 1 commit
- [torch.compile] Fuse RMSNorm with quant (#9138) · 4f93dfe9
  Luka Govedič authored Nov 08, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  4f93dfe9
06 Nov, 2024 2 commits
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
05 Nov, 2024 1 commit
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
02 Nov, 2024 1 commit
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 2 commits
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
- [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) · 598b6d7b
  Pavani Majety authored Nov 01, 2024
  
  598b6d7b
31 Oct, 2024 1 commit
- [BugFix][Kernel] Fix Illegal memory access in causal_conv1d in H100 (#9838) · 9fb12f78
  Mor Zusman authored Oct 31, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  9fb12f78
29 Oct, 2024 1 commit
- [Hardware] using current_platform.seed_everything (#9785) · 622b7ab9
  wangshuai09 authored Oct 29, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  622b7ab9