Commits · 40083064bfce2d385107cb3aee97cf728c116b4e · OpenDAS / vllm_cscc

26 Feb, 2025 1 commit
- 解决cudagraph计算错误的问题 · d365012a
  zhangshao authored Feb 26, 2025
  
  d365012a
13 Jan, 2025 1 commit
- update alibi · dc5f1906
  zhuwenwen authored Jan 13, 2025
  
  dc5f1906
18 Dec, 2024 1 commit
- 增加bw pa tc优化 · 8af5263f
  zhangshao authored Dec 18, 2024
  
  8af5263f
11 Dec, 2024 1 commit
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
19 Nov, 2024 1 commit
- 增加bw支持 · c5877810
  zhangshao authored Nov 19, 2024
  
  c5877810
06 Nov, 2024 1 commit

王敏 authored Nov 06, 2024

2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题

f9060e6b

05 Nov, 2024 1 commit
- 解决pa tc 版读取Q矩阵时数组越界的问题 · 76058dcd
  zhangshao authored Nov 05, 2024
  
  76058dcd
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
22 Oct, 2024 1 commit
- 解决pa v1 tc 部分 size bug · bac73e5d
  zhangshao authored Oct 22, 2024
  
  bac73e5d
18 Oct, 2024 1 commit
- 解决pa v2 bug · e2bd7e16
  zhangshao authored Oct 18, 2024
  
  e2bd7e16
17 Oct, 2024 3 commits
- update attention_kernels_opt_tc.cu · 1c2aa04c
  zhuwenwen authored Oct 17, 2024
  
  1c2aa04c
- update attention_kernels_opt_tc.cu · 83bb8f5a
  zhuwenwen authored Oct 17, 2024
  
  83bb8f5a
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
24 Sep, 2024 1 commit
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
20 Sep, 2024 1 commit
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
22 Aug, 2024 1 commit
- Update refactoring operation of pa · e06b5899
  zhuwenwen authored Aug 22, 2024
  
  e06b5899
21 Aug, 2024 1 commit
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
13 Aug, 2024 1 commit
- restore support for block_size 8 and 32 · 675bceed
  zhuwenwen authored Aug 13, 2024
  
  675bceed
09 Aug, 2024 1 commit
- Revert "pa add v prefetch for gemm1" · 749242a0
  flyingdown authored Aug 09, 2024
```
This reverts commit f38bd872.
```
  749242a0
06 Aug, 2024 1 commit
- 恢复对bf16的支持 · 9f9f3796
  zhangshao authored Aug 06, 2024
  
  9f9f3796
05 Aug, 2024 1 commit
- pa add v prefetch for gemm1 · f38bd872
  flyingdown authored Aug 05, 2024
  
  f38bd872
30 Jul, 2024 1 commit
- [Kernel] Squash a few more warnings (#6914) · cbbc9044
  Tyler Michael Smith authored Jul 30, 2024
  
  cbbc9044
27 Jul, 2024 1 commit
- [Model] H2O Danube3-4b (#6451) · 14dbd5a7
  Joe authored Jul 26, 2024
  
  14dbd5a7
25 Jul, 2024 1 commit
- update attention kernels and version · 0e640807
  zhuwenwen authored Jul 25, 2024
  
  0e640807
22 Jul, 2024 1 commit
- pa优化，编译选项优化 · 1be9a629
  zhangshao authored Jul 22, 2024
  
  1be9a629
18 Jul, 2024 1 commit
- back to pa and rn · 71b1be50
  zhuwenwen authored Jul 18, 2024
  
  71b1be50
16 Jul, 2024 2 commits
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- back to pa and fused rn · af837396
  zhuwenwen authored Jul 16, 2024
  
  af837396
10 Jul, 2024 2 commits
- pa_v1用原始代码pa_v2用新代码 · deeb9cb8
  zhangshao authored Jul 10, 2024
  
  deeb9cb8
- 优化rmsnorm和page_attn · 9e10e8f7
  zhangshao authored Jul 10, 2024
  
  9e10e8f7
02 Jul, 2024 2 commits
- change pa v1 to 128 · bbf9488b
  zhuwenwen authored Jul 02, 2024
  
  bbf9488b
- change num_thread to 256 · 8ee4ae1f
  zhuwenwen authored Jul 02, 2024
  
  8ee4ae1f
09 Jun, 2024 1 commit
- [Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047) · 5467ac31
  bnellnm authored Jun 09, 2024
  
  5467ac31
31 May, 2024 1 commit
- [Model] Support MAP-NEO model (#5081) · a22dea54
  SnowDist authored May 31, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  a22dea54
25 May, 2024 1 commit

[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) · 8e192ff9

Eric Xihui Lin authored May 25, 2024


Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

8e192ff9

22 May, 2024 1 commit
- [CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722) · 5f6d10c1
  Michael Goin authored May 22, 2024
  
  5f6d10c1
10 May, 2024 1 commit
- [Kernel] Refactor FP8 kv-cache with NVIDIA float8_e4m3 support (#4535) · c8331017
  Cody Yu authored May 09, 2024
  
  c8331017
03 May, 2024 1 commit
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
03 Apr, 2024 1 commit

Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290) · 2ff767b5

Adrian Abeyta authored Apr 03, 2024


Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: AdrianAbeyta <Adrian.Abeyta@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: root <root@gt-pla-u18-08.pla.dcgpu>
Co-authored-by: mawong-amd <156021403+mawong-amd@users.noreply.github.com>
Co-authored-by: ttbachyinsda <ttbachyinsda@outlook.com>
Co-authored-by: guofangze <guofangze@kuaishou.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: jacobthebanana <50071502+jacobthebanana@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2ff767b5

25 Mar, 2024 1 commit
- add fp8 · 73b1705e
  zhuwenwen authored Mar 25, 2024
  
  73b1705e