Commits · fa5b0b39ff8ad2a3e712a20ff21092cfc5d259f4 · OpenDAS / vllm_cscc

25 Oct, 2024 1 commit
- support telechat-12b · fa5b0b39
  zhuwenwen authored Oct 25, 2024
  
  fa5b0b39
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
22 Oct, 2024 1 commit
- 解决pa v1 tc 部分 size bug · bac73e5d
  zhangshao authored Oct 22, 2024
  
  bac73e5d
18 Oct, 2024 1 commit
- 解决pa v2 bug · e2bd7e16
  zhangshao authored Oct 18, 2024
  
  e2bd7e16
17 Oct, 2024 3 commits
- update attention_kernels_opt_tc.cu · 1c2aa04c
  zhuwenwen authored Oct 17, 2024
  
  1c2aa04c
- update attention_kernels_opt_tc.cu · 83bb8f5a
  zhuwenwen authored Oct 17, 2024
  
  83bb8f5a
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
26 Sep, 2024 1 commit
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
24 Sep, 2024 1 commit
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
20 Sep, 2024 1 commit
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
22 Aug, 2024 1 commit
- Update refactoring operation of pa · e06b5899
  zhuwenwen authored Aug 22, 2024
  
  e06b5899
21 Aug, 2024 2 commits
- [AMD][CI/Build] Disambiguation of the function call for ROCm 6.2 headers compatibility (#7477) · 99846054
  Gregory Shtrasberg authored Aug 21, 2024
```
Co-authored-by: Charlie Fu <Charlie.Fu@amd.com>
```
  99846054
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
13 Aug, 2024 1 commit
- restore support for block_size 8 and 32 · 675bceed
  zhuwenwen authored Aug 13, 2024
  
  675bceed
09 Aug, 2024 1 commit
- Revert "pa add v prefetch for gemm1" · 749242a0
  flyingdown authored Aug 09, 2024
```
This reverts commit f38bd872.
```
  749242a0
06 Aug, 2024 1 commit
- 恢复对bf16的支持 · 9f9f3796
  zhangshao authored Aug 06, 2024
  
  9f9f3796
05 Aug, 2024 2 commits
- [CI/Build] Suppress divide-by-zero and missing return statement warnings (#7001) · 6e4852ce
  Tyler Michael Smith authored Aug 05, 2024
  
  6e4852ce
- pa add v prefetch for gemm1 · f38bd872
  flyingdown authored Aug 05, 2024
  
  f38bd872
30 Jul, 2024 1 commit
- [Kernel] Squash a few more warnings (#6914) · cbbc9044
  Tyler Michael Smith authored Jul 30, 2024
  
  cbbc9044
29 Jul, 2024 1 commit
- 增加head size 64-256支持 · 69185c0b
  zhangshao authored Jul 29, 2024
  
  69185c0b
27 Jul, 2024 1 commit
- [Model] H2O Danube3-4b (#6451) · 14dbd5a7
  Joe authored Jul 26, 2024
  
  14dbd5a7
25 Jul, 2024 1 commit
- update attention kernels and version · 0e640807
  zhuwenwen authored Jul 25, 2024
  
  0e640807
23 Jul, 2024 1 commit
- optimize rmsnorm kernel · c62f8e9a
  zhuwenwen authored Jul 23, 2024
  
  c62f8e9a
22 Jul, 2024 1 commit
- pa优化，编译选项优化 · 1be9a629
  zhangshao authored Jul 22, 2024
  
  1be9a629
18 Jul, 2024 1 commit
- back to pa and rn · 71b1be50
  zhuwenwen authored Jul 18, 2024
  
  71b1be50
16 Jul, 2024 2 commits
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- back to pa and fused rn · af837396
  zhuwenwen authored Jul 16, 2024
  
  af837396
10 Jul, 2024 2 commits
- pa_v1用原始代码pa_v2用新代码 · deeb9cb8
  zhangshao authored Jul 10, 2024
  
  deeb9cb8
- 优化rmsnorm和page_attn · 9e10e8f7
  zhangshao authored Jul 10, 2024
  
  9e10e8f7
02 Jul, 2024 2 commits
- change pa v1 to 128 · bbf9488b
  zhuwenwen authored Jul 02, 2024
  
  bbf9488b
- change num_thread to 256 · 8ee4ae1f
  zhuwenwen authored Jul 02, 2024
  
  8ee4ae1f
09 Jun, 2024 1 commit
- [Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047) · 5467ac31
  bnellnm authored Jun 09, 2024
  
  5467ac31
31 May, 2024 1 commit
- [Model] Support MAP-NEO model (#5081) · a22dea54
  SnowDist authored May 31, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  a22dea54
29 May, 2024 1 commit
- support bf16 infer · fc92ed40
  zhuwenwen authored May 29, 2024
  
  fc92ed40
25 May, 2024 1 commit

[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) · 8e192ff9

Eric Xihui Lin authored May 25, 2024


Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

8e192ff9

22 May, 2024 1 commit
- [CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722) · 5f6d10c1
  Michael Goin authored May 22, 2024
  
  5f6d10c1
10 May, 2024 1 commit
- [Kernel] Refactor FP8 kv-cache with NVIDIA float8_e4m3 support (#4535) · c8331017
  Cody Yu authored May 09, 2024
  
  c8331017
03 May, 2024 1 commit
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
03 Apr, 2024 1 commit

Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290) · 2ff767b5

Adrian Abeyta authored Apr 03, 2024


Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: AdrianAbeyta <Adrian.Abeyta@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: root <root@gt-pla-u18-08.pla.dcgpu>
Co-authored-by: mawong-amd <156021403+mawong-amd@users.noreply.github.com>
Co-authored-by: ttbachyinsda <ttbachyinsda@outlook.com>
Co-authored-by: guofangze <guofangze@kuaishou.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: jacobthebanana <50071502+jacobthebanana@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2ff767b5

25 Mar, 2024 1 commit
- add fp8 · 73b1705e
  zhuwenwen authored Mar 25, 2024
  
  73b1705e