Commits · eebad39f265606cfe35af4d1e0bea678516648a3 · OpenDAS / vllm_cscc

22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
20 Nov, 2024 1 commit
- [Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464) · d200972e
  Lucas Wilkinson authored Nov 19, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  d200972e
19 Nov, 2024 2 commits
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
- [Platform][Refactor] Extract func `get_default_attn_backend` to `Platform` (#10358) · 8c1fb507
  Mengqing Cao authored Nov 19, 2024
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  8c1fb507
18 Nov, 2024 1 commit
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
15 Nov, 2024 2 commits
- [Misc] Bump up test_fused_moe tolerance (#10364) · 79ee45b4
  ElizaWszola authored Nov 15, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  79ee45b4
- [bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66
  Luka Govedič authored Nov 14, 2024
```
Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>
```
  bf2ddc66
09 Nov, 2024 1 commit

[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8... · 127c0748

rasmith authored Nov 08, 2024


[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>

127c0748

08 Nov, 2024 1 commit
- [torch.compile] Fuse RMSNorm with quant (#9138) · 4f93dfe9
  Luka Govedič authored Nov 08, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@126.com>
```
  4f93dfe9
06 Nov, 2024 2 commits
- [V1] Make v1 more testable (#9888) · d58268c5
  Joe Runde authored Nov 06, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  d58268c5
- [CI/Build] drop support for Python 3.8 EOL (#8464) · 21063c11
  Aaron Pham authored Nov 06, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  21063c11
05 Nov, 2024 1 commit
- [CI] Prune back the number of tests in tests/kernels/* (#9932) · 235366fe
  Michael Goin authored Nov 05, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  235366fe
02 Nov, 2024 1 commit
- [Encoder Decoder] Add flash_attn kernel support for encoder-decoder models (#9559) · a78dd330
  sroy745 authored Nov 01, 2024
  
  a78dd330
01 Nov, 2024 2 commits
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
- [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) · 598b6d7b
  Pavani Majety authored Nov 01, 2024
  
  598b6d7b
31 Oct, 2024 1 commit
- [BugFix][Kernel] Fix Illegal memory access in causal_conv1d in H100 (#9838) · 9fb12f78
  Mor Zusman authored Oct 31, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  9fb12f78
29 Oct, 2024 1 commit
- [Hardware] using current_platform.seed_everything (#9785) · 622b7ab9
  wangshuai09 authored Oct 29, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  622b7ab9
28 Oct, 2024 2 commits
- [torch.compile] support moe models (#9632) · 32176fee
  youkaichao authored Oct 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  32176fee
- [Hardware][ROCM] using current_platform.is_rocm (#9642) · 4e2d95e3
  wangshuai09 authored Oct 28, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  4e2d95e3
26 Oct, 2024 1 commit
- [Hardware][openvino] is_openvino --> current_platform.is_openvino (#9716) · 5cbdccd1
  Mengqing Cao authored Oct 26, 2024
  
  5cbdccd1
24 Oct, 2024 2 commits
- [Performance][Kernel] Fused_moe Performance Improvement (#9384) · 59449095
  Charlie Fu authored Oct 24, 2024
```
Signed-off-by: charlifu <charlifu@amd.com>
```
  59449095
- [Kernel] add kernel for FATReLU (#9610) · 295a061f
  Jee Jee Li authored Oct 24, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  295a061f
22 Oct, 2024 1 commit
- [Hardware][CPU] using current_platform.is_cpu (#9536) · 3ddbe255
  wangshuai09 authored Oct 22, 2024
  
  3ddbe255
20 Oct, 2024 1 commit
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
17 Oct, 2024 1 commit
- [Bugfix] Fix support for dimension like integers and ScalarType (#9299) · eca2c5f7
  bnellnm authored Oct 17, 2024
  
  eca2c5f7
16 Oct, 2024 2 commits
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
- [Misc] Standardize RoPE handling for Qwen2-VL (#9250) · 7e7eae33
  Cyrus Leung authored Oct 16, 2024
  
  7e7eae33
11 Oct, 2024 1 commit
- [Model] Support Mamba (#6484) · 7342a7d7
  Tyler Michael Smith authored Oct 11, 2024
  
  7342a7d7
10 Oct, 2024 1 commit
- [Bugfix] Machete garbage results for some models (large K dim) (#9212) · a64e7b94
  Lucas Wilkinson authored Oct 10, 2024
  
  a64e7b94
08 Oct, 2024 1 commit
- [Bugfix] Try to handle older versions of pytorch (#9086) · bd37b9fb
  bnellnm authored Oct 08, 2024
  
  bd37b9fb
04 Oct, 2024 1 commit
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
03 Oct, 2024 1 commit
- [misc] add forward context for attention (#9029) · 9aaf14c6
  youkaichao authored Oct 03, 2024
  
  9aaf14c6
29 Sep, 2024 2 commits
- [Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533) · f13a07b1
  Mor Zusman authored Sep 30, 2024
  
  f13a07b1
- [Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741) · d081da00
  ElizaWszola authored Sep 29, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d081da00
27 Sep, 2024 1 commit
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
25 Sep, 2024 1 commit
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
23 Sep, 2024 1 commit

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df

Lucas Wilkinson authored Sep 23, 2024

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

86e9c8df

19 Sep, 2024 1 commit
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
18 Sep, 2024 2 commits
- [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) · db9120cd
  Tyler Michael Smith authored Sep 18, 2024
  
  db9120cd
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31