Commits · 7c4f76e3482cc0dcaef72dcc2011db7a001bfe99 · OpenDAS / vllm_cscc

27 Mar, 2024 1 commit
- [Kernel] support non-zero cuda devices in punica kernels (#3636) · 566b57c5
  Jee Li authored Mar 27, 2024
  
  566b57c5
26 Mar, 2024 1 commit
- Enable more models to inference based on LoRA (#3382) · 8af890a8
  Jee Li authored Mar 26, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  8af890a8
25 Mar, 2024 2 commits
- add fp8 · 73b1705e
  zhuwenwen authored Mar 25, 2024
  
  73b1705e
- add bf16 · dd823f7f
  zhuwenwen authored Mar 25, 2024
  
  dd823f7f
23 Mar, 2024 3 commits
- add hip_bf16.h · 7477e8f3
  zhuwenwen authored Mar 23, 2024
  
  7477e8f3
- add bf16 · 9fed1f5d
  zhuwenwen authored Mar 23, 2024
  
  9fed1f5d
- use half atomicAdd of dtk24.04 · 3f1166ab
  zhuwenwen authored Mar 23, 2024
  
  3f1166ab
22 Mar, 2024 1 commit
- [BugFix] Some fixes for custom allreduce kernels (#2760) · f721096d
  Hanzhi Zhou authored Mar 21, 2024
  
  f721096d
18 Mar, 2024 1 commit
- [Bugfix] Make moe_align_block_size AMD-compatible (#3470) · 9101d832
  Woosuk Kwon authored Mar 18, 2024
  
  9101d832
16 Mar, 2024 1 commit
- [Misc] fix line length for entire codebase (#3444) · 8e67598a
  Simon Mo authored Mar 16, 2024
  
  8e67598a
15 Mar, 2024 1 commit
- Dynamically configure shared memory size for moe_align_block_size_kernel (#3376) · 78b6c484
  akhoroshev authored Mar 15, 2024
  
  78b6c484
13 Mar, 2024 3 commits
- Add batched RoPE kernel (#3095) · 7e9bd08f
  Terry authored Mar 13, 2024
  
  7e9bd08f
- Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when... · ae0ccb40
  Or Sharir authored Mar 13, 2024
```
Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when using Multi-LoRA. (#3350)
```
  ae0ccb40
- Add kernel for GeGLU with approximate GELU (#3337) · 602358f8
  Woosuk Kwon authored Mar 12, 2024
  
  602358f8
11 Mar, 2024 1 commit
- [ROCm] Fix warp and lane calculation in blockReduceSum (#3321) · c9415c19
  kliuae authored Mar 12, 2024
  
  c9415c19
10 Mar, 2024 2 commits
- [ROCM] Fix blockReduceSum to use correct warp counts for ROCm and CUDA (#3262) · e4a28e53
  Douglas Lehr authored Mar 10, 2024
  
  e4a28e53
- Enhance lora tests with more layer and rank variations (#3243) · 0bba88df
  Terry authored Mar 09, 2024
  
  0bba88df
08 Mar, 2024 1 commit
- Feature add lora support for Qwen2 (#3177) · c59e120c
  whyiug authored Mar 08, 2024
  
  c59e120c
01 Mar, 2024 1 commit

Integrate Marlin Kernels for Int4 GPTQ inference (#2497) · c0c2335c

Robert Shaw authored Mar 01, 2024


Co-authored-by: Robert Shaw <114415538+rib-2@users.noreply.github.com>
Co-authored-by: alexm <alexm@neuralmagic.com>

c0c2335c

29 Feb, 2024 1 commit
- Add Support for 2/3/8-bit GPTQ Quantization Models (#2330) · 01a5d18a
  CHU Tianxiang authored Feb 29, 2024
  
  01a5d18a
28 Feb, 2024 1 commit
- Add LoRA support for Gemma (#3050) · 929b4f29
  Woosuk Kwon authored Feb 28, 2024
  
  929b4f29
26 Feb, 2024 1 commit
- [Minor] Remove gather_cached_kv kernel (#3043) · d6e4a130
  Woosuk Kwon authored Feb 26, 2024
  
  d6e4a130
22 Feb, 2024 1 commit
- Optimize GeGLU layer in Gemma (#2975) · fd5dcc5c
  Woosuk Kwon authored Feb 21, 2024
  
  fd5dcc5c
19 Feb, 2024 1 commit
- skip bf16 · 5e5b497d
  zhuwenwen authored Feb 19, 2024
  
  5e5b497d
12 Feb, 2024 1 commit
- Refactor 2 awq gemm kernels into m16nXk32 (#2723) · 56383649
  Rex authored Feb 12, 2024
```
Co-authored-by: Chunan Zeng <chunanzeng@Chunans-Air.attlocal.net>
```
  56383649
06 Feb, 2024 1 commit
- Add fused top-K softmax kernel for MoE (#2769) · f0d4e145
  Woosuk Kwon authored Feb 05, 2024
  
  f0d4e145
01 Feb, 2024 2 commits
- Fix compile error when using rocm (#2648) · 923797fe
  zhaoyang-star authored Feb 02, 2024
  
  923797fe
- skip fp8 · 318e2b5a
  zhuwenwen authored Feb 01, 2024
  
  318e2b5a
30 Jan, 2024 2 commits
- Fused MOE for Mixtral (#2542) · ab406446
  Philipp Moritz authored Jan 29, 2024
```
Co-authored-by: chen shen <scv119@gmail.com>
```
  ab406446
- DeepseekMoE support with Fused MoE kernel (#2453) · 5d60def0
  wangding zeng authored Jan 30, 2024
```
Co-authored-by: roy <jasonailu87@gmail.com>
```
  5d60def0
29 Jan, 2024 2 commits
- No repeated IPC open (#2642) · 1b20639a
  Hanzhi Zhou authored Jan 30, 2024
  
  1b20639a
- Support FP8-E5M2 KV Cache (#2279) · 9090bf02
  zhaoyang-star authored Jan 29, 2024
```
Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  9090bf02
28 Jan, 2024 1 commit
- Speed up Punica compilation (#2632) · f8ecb84c
  Woosuk Kwon authored Jan 27, 2024
  
  f8ecb84c
27 Jan, 2024 2 commits
- Implement custom all reduce kernels (#2192) · 38017003
  Hanzhi Zhou authored Jan 28, 2024
  
  38017003
- AWQ: Up to 2.66x higher throughput (#2566) · beb89f68
  Casper authored Jan 27, 2024
  
  beb89f68
26 Jan, 2024 2 commits
- [ROCm] add support to ROCm 6.0 and MI300 (#2274) · 6b7de1a0
  Hongxia Yang authored Jan 26, 2024
  
  6b7de1a0
- use a correct device when creating OptionalCUDAGuard (#2583) · 5265631d
  Vladimir authored Jan 26, 2024
  
  5265631d
23 Jan, 2024 1 commit

[Experimental] Add multi-LoRA support (#1804) · 9b945daa

Antoni Baum authored Jan 24, 2024


Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Shreyas Krishnaswamy <shrekris@anyscale.com>
Co-authored-by: Avnish Narayan <avnish@anyscale.com>

9b945daa

04 Jan, 2024 1 commit
- support dtk2310 · 4095d0db
  zhuwenwen authored Jan 04, 2024
  
  4095d0db
03 Jan, 2024 1 commit
- Enable CUDA graph for GPTQ & SqueezeLLM (#2318) · 6ef00b03
  Woosuk Kwon authored Jan 03, 2024
  
  6ef00b03