Commits · dd0c8f490868762e6671bd01c558b9413fb3caa1 · OpenDAS / vllm_cscc

06 Aug, 2024 2 commits
- fix deepseek_v2 236b fused_moe_kernel expert_ids_ptr value error · b2068609
  wangmin6 authored Aug 06, 2024
  
  b2068609
- 恢复对bf16的支持 · 9f9f3796
  zhangshao authored Aug 06, 2024
  
  9f9f3796
05 Aug, 2024 1 commit
- pa add v prefetch for gemm1 · f38bd872
  flyingdown authored Aug 05, 2024
  
  f38bd872
29 Jul, 2024 1 commit
- 增加head size 64-256支持 · 69185c0b
  zhangshao authored Jul 29, 2024
  
  69185c0b
24 Jul, 2024 1 commit
- update gptq relative path · 2d0a73a3
  zhuwenwen authored Jul 24, 2024
  
  2d0a73a3
23 Jul, 2024 2 commits
- optimize rmsnorm kernel · c62f8e9a
  zhuwenwen authored Jul 23, 2024
  
  c62f8e9a
- refactoring the transpose kernel and update supported model · b2dd1743
  zhuwenwen authored Jul 23, 2024
  
  b2dd1743
22 Jul, 2024 2 commits
- pa优化，编译选项优化 · 1be9a629
  zhangshao authored Jul 22, 2024
  
  1be9a629
- fix gptq performance degradation when batch size>4 issue · f423ad60
  huangwb authored Jul 22, 2024
  
  f423ad60
20 Jul, 2024 2 commits
- support nn layout · 1e0cb1f4
  zhuwenwen authored Jul 20, 2024
  
  1e0cb1f4
- 修改nn支持方式 · 835bd9fc
  gaoqiong authored Jul 20, 2024
  
  835bd9fc
18 Jul, 2024 1 commit
- back to pa and rn · 71b1be50
  zhuwenwen authored Jul 18, 2024
  
  71b1be50
17 Jul, 2024 1 commit
- add rotary_embedding for tgi · e499f96c
  huangwb authored Jul 17, 2024
  
  e499f96c
10 Jul, 2024 2 commits
- pa_v1用原始代码pa_v2用新代码 · deeb9cb8
  zhangshao authored Jul 10, 2024
  
  deeb9cb8
- 优化rmsnorm和page_attn · 9e10e8f7
  zhangshao authored Jul 10, 2024
  
  9e10e8f7
02 Jul, 2024 2 commits
- change pa v1 to 128 · bbf9488b
  zhuwenwen authored Jul 02, 2024
  
  bbf9488b
- change num_thread to 256 · 8ee4ae1f
  zhuwenwen authored Jul 02, 2024
  
  8ee4ae1f
12 Jun, 2024 1 commit
- skip fp8 · 103f3110
  zhuwenwen authored Jun 12, 2024
  
  103f3110
09 Jun, 2024 1 commit
- [Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047) · 5467ac31
  bnellnm authored Jun 09, 2024
  
  5467ac31
07 Jun, 2024 2 commits
- [Misc] Remove unused cuda_utils.h in CPU backend (#5345) · 6840a716
  Jie Fu (傅杰) authored Jun 08, 2024
  
  6840a716
- [Kernel] Dynamic Per-Token Activation Quantization (#5037) · ca3ea51b
  Dipika Sikka authored Jun 07, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  ca3ea51b
05 Jun, 2024 1 commit
- [Kernel] Add GPU architecture guards to the CUTLASS w8a8 kernels to reduce binary size (#5157) · ccd4f129
  Tyler Michael Smith authored Jun 05, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  ccd4f129
03 Jun, 2024 2 commits
- [CI/BUILD] enable intel queue for longer CPU tests (#4113) · cafb8e06
  Yuan authored Jun 04, 2024
  
  cafb8e06
- [Kernel] Pass a device pointer into the quantize kernel for the scales (#5159) · cbb2f59c
  Tyler Michael Smith authored Jun 03, 2024
  
  cbb2f59c
02 Jun, 2024 1 commit
- [Kernel][ROCm][AMD] enable fused topk_softmax kernel for moe layer (#4927) · a66cf40b
  Divakar Verma authored Jun 02, 2024
```
This PR enables the fused topk_softmax kernel used in moe layer for HIP
```
  a66cf40b
01 Jun, 2024 3 commits
- [Kernel] Update Cutlass fp8 configs (#5144) · f081c3ce
  Varun Sundar Rabindranath authored Jun 01, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  f081c3ce
- [Kernel] Refactor CUTLASS kernels to always take scales that reside on the GPU (#5137) · 260d119e
  Tyler Michael Smith authored Jun 01, 2024
  
  260d119e
- [Build] Guard against older CUDA versions when building CUTLASS 3.x kernels (#5168) · 1197e021
  Tyler Michael Smith authored May 31, 2024
  
  1197e021
31 May, 2024 4 commits
- add int8 · 0de4f1dc
  zhuwenwen authored May 31, 2024
  
  0de4f1dc
- Revert "[Kernel] Marlin_24: Ensure the mma.sp instruction is using the... · e9d3aa04
  Simon Mo authored May 31, 2024
```
Revert "[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5)" (#5149)
```
  e9d3aa04
- [Model] Support MAP-NEO model (#5081) · a22dea54
  SnowDist authored May 31, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  a22dea54
- [Kernel] Marlin_24: Ensure the mma.sp instruction is using the... · 6d21fa1c
  Alexander Matveev authored May 30, 2024
```
[Kernel] Marlin_24: Ensure the mma.sp instruction is using the ::ordered_metadata modifier (introduced with PTX 8.5) (#5136)
```
  6d21fa1c
29 May, 2024 1 commit
- support bf16 infer · fc92ed40
  zhuwenwen authored May 29, 2024
  
  fc92ed40
25 May, 2024 2 commits

skip fp8 · f09d77ac
zhuwenwen authored May 25, 2024

f09d77ac

[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) · 8e192ff9

Eric Xihui Lin authored May 25, 2024


Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

8e192ff9

23 May, 2024 2 commits
- [Kernel] Initial Activation Quantization Support (#4525) · a1242324
  Dipika Sikka authored May 23, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  a1242324
- Marlin 24 prefill performance improvement (about 25% better on average) (#4983) · 60662532
  Alexander Matveev authored May 23, 2024
  
  60662532
22 May, 2024 3 commits
- [Model] LoRA gptbigcode implementation (#3949) · 97b03000
  raywanb authored May 23, 2024
  
  97b03000
- [Kernel] Fixup for CUTLASS kernels in CUDA graphs (#4954) · 8674f988
  Tyler Michael Smith authored May 22, 2024
```
Pass the CUDA stream into the CUTLASS GEMMs, to avoid future issues with CUDA graphs
```
  8674f988
- [CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722) · 5f6d10c1
  Michael Goin authored May 22, 2024
  
  5f6d10c1