Commits · 6d2051cc498f78cd60b4c1da054a499d6a7594ff · OpenDAS / vllm_cscc · GitLab

18 Oct, 2024 1 commit
- 解决pa v2 bug · e2bd7e16
  zhangshao authored Oct 18, 2024
  
  e2bd7e16
17 Oct, 2024 6 commits
- [Hardware][CPU] compressed-tensor INT8 W8A8 AZP support (#9344) · 5eda21e7
  Li, Jiang authored Oct 18, 2024
  
  5eda21e7
- update pa tc interfaces · 304e2bab
  zhuwenwen authored Oct 17, 2024
  
  304e2bab
- update attention_kernels_opt_tc.cu · 1c2aa04c
  zhuwenwen authored Oct 17, 2024
  
  1c2aa04c
- update attention_kernels_opt_tc.cu · 83bb8f5a
  zhuwenwen authored Oct 17, 2024
  
  83bb8f5a
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
- [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels (#9391) · 92d86da2
  rasmith authored Oct 16, 2024
  
  92d86da2
16 Oct, 2024 2 commits
- [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel (#9425) · c3fab5f7
  Tyler Michael Smith authored Oct 16, 2024
  
  c3fab5f7
- [Kernel][Model] Improve continuous batching for Jamba and Mamba (#9189) · fb60ae9b
  Mor Zusman authored Oct 17, 2024
  
  fb60ae9b
10 Oct, 2024 2 commits
- [Bugfix] Fix Machete unittests failing with `NotImplementedError` (#9218) · 18511aed
  Lucas Wilkinson authored Oct 10, 2024
  
  18511aed
- [Bugfix] Machete garbage results for some models (large K dim) (#9212) · a64e7b94
  Lucas Wilkinson authored Oct 10, 2024
  
  a64e7b94
04 Oct, 2024 2 commits
- [Kernel] Zero point support in fused MarlinMoE kernel + AWQ Fused MoE (#8973) · 05d68643
  ElizaWszola authored Oct 04, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
Co-authored-by: Dipika Sikka <ds3822@columbia.edu>
```
  05d68643
- [CI/Build] Per file CUDA Archs (improve wheel size and dev build times) (#8845) · aeb37c2a
  Lucas Wilkinson authored Oct 03, 2024
  
  aeb37c2a
02 Oct, 2024 1 commit
- [Core] CUDA Graphs for Multi-Step + Chunked-Prefill (#8645) · afb050b2
  Varun Sundar Rabindranath authored Oct 02, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  afb050b2
01 Oct, 2024 1 commit
- [CI/Build] Fix machete generated kernel files ordering (#8976) · aaccca2b
  Kevin H. Luu authored Sep 30, 2024
```
Signed-off-by: kevin <kevin@anyscale.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  aaccca2b
29 Sep, 2024 2 commits
- [Kernel][Model] Varlen prefill + Prefill chunking support for mamba kernels and Jamba model (#8533) · f13a07b1
  Mor Zusman authored Sep 30, 2024
  
  f13a07b1
- [Bugfix] Fix Marlin MoE act order when is_k_full == False (#8741) · d081da00
  ElizaWszola authored Sep 29, 2024
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  d081da00
27 Sep, 2024 1 commit
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
26 Sep, 2024 2 commits
- [Bugfix] Fixup advance_step.cu warning (#8815) · 71d21c73
  Tyler Michael Smith authored Sep 26, 2024
  
  71d21c73
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
25 Sep, 2024 2 commits
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
- [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776) · b4522474
  sasha0552 authored Sep 25, 2024
  
  b4522474
24 Sep, 2024 3 commits
- [Kernel] Split Marlin MoE kernels into multiple files (#8661) · a928ded9
  ElizaWszola authored Sep 24, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  a928ded9
- [Bugfix] Fix potentially unsafe custom allreduce synchronization (#8558) · cc4325b6
  Hanzhi Zhou authored Sep 24, 2024
  
  cc4325b6
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
23 Sep, 2024 1 commit

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df

Lucas Wilkinson authored Sep 23, 2024

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

86e9c8df

21 Sep, 2024 1 commit
- [Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (#8643) · d66ac628
  Tyler Michael Smith authored Sep 21, 2024
  
  d66ac628
20 Sep, 2024 1 commit
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
19 Sep, 2024 2 commits
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
- [Kernel] Remove marlin moe templating on thread_m_blocks (#8573) · 4c34ce89
  Tyler Michael Smith authored Sep 18, 2024
```
Co-authored-by: lwilkinson@neuralmagic.com
```
  4c34ce89
17 Sep, 2024 2 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
16 Sep, 2024 3 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506) · 781e3b9a
  sasha0552 authored Sep 16, 2024
  
  781e3b9a
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
14 Sep, 2024 1 commit
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef
12 Sep, 2024 1 commit
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
11 Sep, 2024 2 commits
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4