Commits · 8e1cddcd44da6bc58d4201e2a388ed9afd5adfb8 · OpenDAS / vllm_cscc · GitLab

17 Oct, 2024 1 commit
- [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) · 92d86da2
  rasmith authored Oct 16, 2024
  
  92d86da2
16 Oct, 2024 2 commits
- [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) · c3fab5f7
  Tyler Michael Smith authored Oct 16, 2024
  
  c3fab5f7
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
10 Oct, 2024 2 commits
- [Bugfix] Fix Machete unittests failing with `NotImplementedError` (#9218) · 18511aed
  Lucas Wilkinson authored Oct 10, 2024
  
  18511aed
- [Bugfix] Machete garbage results for some models (large K dim) (#9212) · a64e7b94
  Lucas Wilkinson authored Oct 10, 2024
  
  a64e7b94
04 Oct, 2024 2 commits
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
- [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) (#8845) · aeb37c2a
  Lucas Wilkinson authored Oct 03, 2024
  
  aeb37c2a
02 Oct, 2024 1 commit
- [Core] CUDA Graphs for Multi-Step + Chunked-Prefill (#8645) · afb050b2
  Varun Sundar Rabindranath authored Oct 02, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  afb050b2
01 Oct, 2024 1 commit
- [CI/Build] Fix machete generated kernel files ordering (#8976) · aaccca2b
  Kevin H. Luu authored Sep 30, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  aaccca2b
29 Sep, 2024 2 commits
- [Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533) · f13a07b1
  Mor Zusman authored Sep 30, 2024
  
  f13a07b1
- [Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741) · d081da00
  ElizaWszola authored Sep 29, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d081da00
27 Sep, 2024 1 commit
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
26 Sep, 2024 1 commit
- [Bugfix] Fixup advance_step.cu warning (#8815) · 71d21c73
  Tyler Michael Smith authored Sep 26, 2024
  
  71d21c73
25 Sep, 2024 2 commits
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
- [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776) · b4522474
  sasha0552 authored Sep 25, 2024
  
  b4522474
24 Sep, 2024 2 commits
- [Kernel] Split Marlin MoE kernels into multiple files (#8661) · a928ded9
  ElizaWszola authored Sep 24, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  a928ded9
- [Bugfix] Fix potentially unsafe custom allreduce synchronization (#8558) · cc4325b6
  Hanzhi Zhou authored Sep 24, 2024
  
  cc4325b6
23 Sep, 2024 1 commit

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df

Lucas Wilkinson authored Sep 23, 2024

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

86e9c8df

21 Sep, 2024 1 commit
- [Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (#8643) · d66ac628
  Tyler Michael Smith authored Sep 21, 2024
  
  d66ac628
19 Sep, 2024 2 commits
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
- [Kernel] Remove marlin moe templating on thread_m_blocks (#8573) · 4c34ce89
  Tyler Michael Smith authored Sep 18, 2024
```
Co-authored-by: lwilkinson@neuralmagic.com
```
  4c34ce89
17 Sep, 2024 2 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
16 Sep, 2024 3 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506) · 781e3b9a
  sasha0552 authored Sep 16, 2024
  
  781e3b9a
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
14 Sep, 2024 1 commit
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef
12 Sep, 2024 1 commit
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
11 Sep, 2024 2 commits
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
10 Sep, 2024 1 commit
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
06 Sep, 2024 1 commit
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
28 Aug, 2024 2 commits
- [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) · fdd9daaf
  Mor Zusman authored Aug 29, 2024
  
  fdd9daaf
- [Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for... · c166e7e4
  bnellnm authored Aug 27, 2024
```
[Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for registering FakeScalarType and dynamo support. (#7886)
```
  c166e7e4
27 Aug, 2024 1 commit
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
22 Aug, 2024 3 commits
- [Bugfix] Don't build machete on cuda <12.0 (#7757) · 55d63b12
  Lucas Wilkinson authored Aug 22, 2024
  
  55d63b12
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
- [Kernel] Replaced `blockReduce[...]` functions with `cub::BlockReduce` (#7233) · 7937009a
  Luka Govedič authored Aug 21, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7937009a
21 Aug, 2024 1 commit
- [AMD][CI/Build] Disambiguation of the function call for ROCm 6.2 headers compatibility (#7477) · 99846054
  Gregory Shtrasberg authored Aug 21, 2024
```
Co-authored-by: Charlie Fu <Charlie.Fu@amd.com>
```
  99846054