Commits · f164171b1729d7eafcfa2a028e67a6d29c9b1b8f · OpenDAS / vllm_cscc · GitLab

17 Oct, 2024 1 commit
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
26 Sep, 2024 1 commit
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
25 Sep, 2024 2 commits
- [Kernel] Fullgraph and opcheck tests (#8479) · 300da091
  bnellnm authored Sep 25, 2024
  
  300da091
- [Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776) · b4522474
  sasha0552 authored Sep 25, 2024
  
  b4522474
24 Sep, 2024 3 commits
- [Kernel] Split Marlin MoE kernels into multiple files (#8661) · a928ded9
  ElizaWszola authored Sep 24, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  a928ded9
- [Bugfix] Fix potentially unsafe custom allreduce synchronization (#8558) · cc4325b6
  Hanzhi Zhou authored Sep 24, 2024
  
  cc4325b6
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
23 Sep, 2024 1 commit

[Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df

Lucas Wilkinson authored Sep 23, 2024

Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

86e9c8df

21 Sep, 2024 1 commit
- [Kernel][Bugfix] Delete some more useless code in marlin_moe_ops.cu (#8643) · d66ac628
  Tyler Michael Smith authored Sep 21, 2024
  
  d66ac628
20 Sep, 2024 1 commit
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
19 Sep, 2024 2 commits
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
- [Kernel] Remove marlin moe templating on thread_m_blocks (#8573) · 4c34ce89
  Tyler Michael Smith authored Sep 18, 2024
```
Co-authored-by: lwilkinson@neuralmagic.com
```
  4c34ce89
17 Sep, 2024 2 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
16 Sep, 2024 3 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506) · 781e3b9a
  sasha0552 authored Sep 16, 2024
  
  781e3b9a
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
14 Sep, 2024 1 commit
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef
12 Sep, 2024 1 commit
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
11 Sep, 2024 2 commits
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
10 Sep, 2024 2 commits
- update version and requirements-rocm.txt · 6897826e
  zhuwenwen authored Sep 10, 2024
  
  6897826e
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
06 Sep, 2024 1 commit
- [Misc] Remove `SqueezeLLM` (#8220) · 23f32229
  Dipika Sikka authored Sep 06, 2024
  
  23f32229
03 Sep, 2024 1 commit
- fix build error · 6bb59e94
  zhuwenwen authored Sep 03, 2024
  
  6bb59e94
28 Aug, 2024 2 commits
- [Kernel/Model] Migrate mamba_ssm and causal_conv1d kernels to vLLM (#7651) · fdd9daaf
  Mor Zusman authored Aug 29, 2024
  
  fdd9daaf
- [Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for... · c166e7e4
  bnellnm authored Aug 27, 2024
```
[Bugfix] Allow ScalarType to be compiled with pytorch 2.3 and add checks for registering FakeScalarType and dynamo support. (#7886)
```
  c166e7e4
27 Aug, 2024 1 commit
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
23 Aug, 2024 1 commit
- Update layernorm_kernels_opt.cu · 35068264
  zhuwenwen authored Aug 23, 2024
  
  35068264
22 Aug, 2024 5 commits
- [Bugfix] Don't build machete on cuda <12.0 (#7757) · 55d63b12
  Lucas Wilkinson authored Aug 22, 2024
  
  55d63b12
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
- Update refactoring operation of pa binding · f1077dda
  zhuwenwen authored Aug 22, 2024
  
  f1077dda
- Update refactoring operation of pa · e06b5899
  zhuwenwen authored Aug 22, 2024
  
  e06b5899
- [Kernel] Replaced `blockReduce[...]` functions with `cub::BlockReduce` (#7233) · 7937009a
  Luka Govedič authored Aug 21, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  7937009a
21 Aug, 2024 4 commits
- [AMD][CI/Build] Disambiguation of the function call for ROCm 6.2 headers compatibility (#7477) · 99846054
  Gregory Shtrasberg authored Aug 21, 2024
```
Co-authored-by: Charlie Fu <Charlie.Fu@amd.com>
```
  99846054
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a
- Update refactoring operation · 421310ba
  zhuwenwen authored Aug 21, 2024
  
  421310ba
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a