Commits · 83dd9e1a098a3239c32c96d58b27806623628eaf · OpenDAS / vllm_cscc

"deploy/inference-gateway/epp-patches/v0.8.0/gaie.patch" did not exist on "fa1d651735b3ec64c4bff0609f720a2444ae60d2"

14 May, 2025 1 commit
- add kvint8 · 45273722
  xiabo authored May 14, 2025
  
  45273722
26 Feb, 2025 1 commit
- 解决cudagraph计算错误的问题 · d365012a
  zhangshao authored Feb 26, 2025
  
  d365012a
11 Dec, 2024 1 commit
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
06 Nov, 2024 1 commit

王敏 authored Nov 06, 2024

2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题

f9060e6b

24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
17 Oct, 2024 1 commit
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
24 Sep, 2024 1 commit
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
20 Sep, 2024 1 commit
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
22 Aug, 2024 1 commit
- Update refactoring operation of pa · e06b5899
  zhuwenwen authored Aug 22, 2024
  
  e06b5899
21 Aug, 2024 1 commit
- Refactoring the optimized kernel · 0aa2480f
  zhuwenwen authored Aug 21, 2024
  
  0aa2480f
13 Aug, 2024 1 commit
- restore support for block_size 8 and 32 · 675bceed
  zhuwenwen authored Aug 13, 2024
  
  675bceed
09 Aug, 2024 1 commit
- Revert "pa add v prefetch for gemm1" · 749242a0
  flyingdown authored Aug 09, 2024
```
This reverts commit f38bd872.
```
  749242a0
06 Aug, 2024 1 commit
- 恢复对bf16的支持 · 9f9f3796
  zhangshao authored Aug 06, 2024
  
  9f9f3796
05 Aug, 2024 1 commit
- pa add v prefetch for gemm1 · f38bd872
  flyingdown authored Aug 05, 2024
  
  f38bd872
30 Jul, 2024 1 commit
- [Kernel] Squash a few more warnings (#6914) · cbbc9044
  Tyler Michael Smith authored Jul 30, 2024
  
  cbbc9044
27 Jul, 2024 1 commit
- [Model] H2O Danube3-4b (#6451) · 14dbd5a7
  Joe authored Jul 26, 2024
  
  14dbd5a7
25 Jul, 2024 1 commit
- update attention kernels and version · 0e640807
  zhuwenwen authored Jul 25, 2024
  
  0e640807
22 Jul, 2024 1 commit
- pa优化，编译选项优化 · 1be9a629
  zhangshao authored Jul 22, 2024
  
  1be9a629
18 Jul, 2024 1 commit
- back to pa and rn · 71b1be50
  zhuwenwen authored Jul 18, 2024
  
  71b1be50
16 Jul, 2024 2 commits
- [Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081) · 978aed53
  Michael Goin authored Jul 16, 2024
  
  978aed53
- back to pa and fused rn · af837396
  zhuwenwen authored Jul 16, 2024
  
  af837396
10 Jul, 2024 2 commits
- pa_v1用原始代码pa_v2用新代码 · deeb9cb8
  zhangshao authored Jul 10, 2024
  
  deeb9cb8
- 优化rmsnorm和page_attn · 9e10e8f7
  zhangshao authored Jul 10, 2024
  
  9e10e8f7
02 Jul, 2024 2 commits
- change pa v1 to 128 · bbf9488b
  zhuwenwen authored Jul 02, 2024
  
  bbf9488b
- change num_thread to 256 · 8ee4ae1f
  zhuwenwen authored Jul 02, 2024
  
  8ee4ae1f
09 Jun, 2024 1 commit
- [Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047) · 5467ac31
  bnellnm authored Jun 09, 2024
  
  5467ac31
31 May, 2024 1 commit
- [Model] Support MAP-NEO model (#5081) · a22dea54
  SnowDist authored May 31, 2024
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  a22dea54
25 May, 2024 1 commit

[Kernel][Backend][Model] Blocksparse flash attention kernel and Phi-3-Small model (#4799) · 8e192ff9

Eric Xihui Lin authored May 25, 2024


Co-authored-by: beagleski <yunanzhang@microsoft.com>
Co-authored-by: bapatra <bapatra@microsoft.com>
Co-authored-by: Barun Patra <codedecde@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

8e192ff9

22 May, 2024 1 commit
- [CI/Build] Enforce style for C++ and CUDA code with `clang-format` (#4722) · 5f6d10c1
  Michael Goin authored May 22, 2024
  
  5f6d10c1
10 May, 2024 1 commit
- [Kernel] Refactor FP8 kv-cache with NVIDIA float8_e4m3 support (#4535) · c8331017
  Cody Yu authored May 09, 2024
  
  c8331017
03 May, 2024 1 commit
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
03 Apr, 2024 1 commit

Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) (#3290) · 2ff767b5

Adrian Abeyta authored Apr 03, 2024


Co-authored-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: HaiShaw <hixiao@gmail.com>
Co-authored-by: AdrianAbeyta <Adrian.Abeyta@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
Co-authored-by: root <root@gt-pla-u18-08.pla.dcgpu>
Co-authored-by: mawong-amd <156021403+mawong-amd@users.noreply.github.com>
Co-authored-by: ttbachyinsda <ttbachyinsda@outlook.com>
Co-authored-by: guofangze <guofangze@kuaishou.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: jacobthebanana <50071502+jacobthebanana@users.noreply.github.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

2ff767b5

25 Mar, 2024 1 commit
- add fp8 · 73b1705e
  zhuwenwen authored Mar 25, 2024
  
  73b1705e
23 Mar, 2024 1 commit
- add bf16 · 9fed1f5d
  zhuwenwen authored Mar 23, 2024
  
  9fed1f5d
10 Mar, 2024 1 commit
- [ROCM] Fix blockReduceSum to use correct warp counts for ROCm and CUDA (#3262) · e4a28e53
  Douglas Lehr authored Mar 10, 2024
  
  e4a28e53
01 Feb, 2024 1 commit
- Fix compile error when using rocm (#2648) · 923797fe
  zhaoyang-star authored Feb 02, 2024
  
  923797fe
29 Jan, 2024 1 commit

Support FP8-E5M2 KV Cache (#2279) · 9090bf02

zhaoyang-star authored Jan 29, 2024


Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9090bf02

04 Jan, 2024 1 commit
- support dtk2310 · 4095d0db
  zhuwenwen authored Jan 04, 2024
  
  4095d0db
03 Jan, 2024 1 commit
- [FIX] Support non-zero CUDA devices in custom kernels (#1959) · 77af974b
  Jee Li authored Jan 03, 2024
  
  77af974b
26 Dec, 2023 1 commit
- support dtk23.10 · eb181638
  zhuwenwen authored Dec 26, 2023
  
  eb181638