Commits · 1be9a6290bb90585b7cdc70b422b102b3a0f7b18 · OpenDAS / vllm_cscc

22 Jul, 2024 3 commits
- pa优化，编译选项优化 · 1be9a629
  zhangshao authored Jul 22, 2024
  
  1be9a629
- Merge branch 'gptq_fix' into 'v0.5.0-dtk24.04.1' · d4c0015a
  huangwb authored Jul 22, 2024
```
fix gptq performance degradation when batch size>4 issue

See merge request dcutoolkit/deeplearing/vllm!5
```
  d4c0015a
- fix gptq performance degradation when batch size>4 issue · f423ad60
  huangwb authored Jul 22, 2024
  
  f423ad60
20 Jul, 2024 7 commits
- use two method to add bias · 4caf1539
  zhuwenwen authored Jul 20, 2024
  
  4caf1539
- update ray version · 767bb050
  zhuwenwen authored Jul 20, 2024
  
  767bb050
- support nn layout · 1e0cb1f4
  zhuwenwen authored Jul 20, 2024
  
  1e0cb1f4
- Merge branch 'v0.5.0-dtk24.04.1_nn_gaoq' into 'v0.5.0-dtk24.04.1' · 281ca6c1
  gaoqiong authored Jul 20, 2024
```
V0.5.0 Modify nn support methods

See merge request dcutoolkit/deeplearing/vllm!4
```
  281ca6c1
- 去掉调试信息 · 9653385f
  gaoqiong authored Jul 20, 2024
  
  9653385f
- 修改nn支持方式 · 835bd9fc
  gaoqiong authored Jul 20, 2024
  
  835bd9fc
- modify gemm pad strategy · 7fe40ced
  zhuwenwen authored Jul 20, 2024
  
  7fe40ced
18 Jul, 2024 2 commits
- default use of ck fa · e661266e
  zhuwenwen authored Jul 18, 2024
  
  e661266e
- back to pa and rn · 71b1be50
  zhuwenwen authored Jul 18, 2024
  
  71b1be50
17 Jul, 2024 3 commits
- Merge branch 'vllm-v0.5.0-dtk24.04.1-rotary_embedding' into 'vllm-v0.5.0-dtk24.04.1' · c628c6ec
  huangwb authored Jul 17, 2024
```
add rotary_embedding for tgi

See merge request dcutoolkit/deeplearing/vllm!3
```
  c628c6ec
- add rotary_embedding for tgi · e499f96c
  huangwb authored Jul 17, 2024
  
  e499f96c
- set default layout · 22839191
  zhuwenwen authored Jul 17, 2024
  
  22839191
12 Jul, 2024 1 commit
- fix bf16 error · bb73b5a6
  zhuwenwen authored Jul 12, 2024
  
  bb73b5a6
10 Jul, 2024 3 commits
- pa_v1用原始代码pa_v2用新代码 · deeb9cb8
  zhangshao authored Jul 10, 2024
  
  deeb9cb8
- Merge branch 'vllm-0.5-zhagnshao' into 'vllm-v0.5.0-dtk24.04.1' · c4b56490
  zhangshao authored Jul 10, 2024
```
优化rmsnorm和page_attn

See merge request dcutoolkit/deeplearing/vllm!2
```
  c4b56490
- 优化rmsnorm和page_attn · 9e10e8f7
  zhangshao authored Jul 10, 2024
  
  9e10e8f7
09 Jul, 2024 2 commits
- Merge branch 'vllm-v0.5.0-dtk24.04.1-support-deepseekv2' into 'vllm-v0.5.0-dtk24.04.1' · c6e8cf73
  huangwb authored Jul 09, 2024
```
Support Deepseek-V2 (#4650)

See merge request dcutoolkit/deeplearing/vllm!1
```
  c6e8cf73
- Support Deepseek-V2 (#4650) · b1b95055
  huangwb authored Jul 09, 2024
  
  b1b95055
08 Jul, 2024 1 commit
- add 7b pad dim · 5cdabd7b
  zhuwenwen authored Jul 08, 2024
  
  5cdabd7b
06 Jul, 2024 3 commits
- add fa pad · 371b1251
  zhuwenwen authored Jul 06, 2024
  
  371b1251
- Use triton fa by default · 1863c926
  zhuwenwen authored Jul 06, 2024
  
  1863c926
- fix ray error when tp>=2 · b6247705
  zhuwenwen authored Jul 06, 2024
  
  b6247705
05 Jul, 2024 1 commit
- skip cutlass fwop and xformers cuda backend · 045b5ad2
  zhuwenwen authored Jul 05, 2024
  
  045b5ad2
02 Jul, 2024 2 commits
- change pa v1 to 128 · bbf9488b
  zhuwenwen authored Jul 02, 2024
  
  bbf9488b
- change num_thread to 256 · 8ee4ae1f
  zhuwenwen authored Jul 02, 2024
  
  8ee4ae1f
01 Jul, 2024 1 commit
- add qwen2 arch · e9aa4ff0
  zhuwenwen authored Jul 01, 2024
  
  e9aa4ff0
28 Jun, 2024 1 commit
- add gemm paddig · e58014d7
  zhuwenwen authored Jun 28, 2024
  
  e58014d7
20 Jun, 2024 1 commit
- update triton fa · e2df3544
  zhuwenwen authored Jun 20, 2024
  
  e2df3544
15 Jun, 2024 1 commit
- update kernel test · 21c06ecb
  zhuwenwen authored Jun 15, 2024
  
  21c06ecb
12 Jun, 2024 2 commits
- skip fp8 · 103f3110
  zhuwenwen authored Jun 12, 2024
  
  103f3110
- merge v0.5.0 · f48954a4
  zhuwenwen authored Jun 12, 2024
  
  f48954a4
11 Jun, 2024 6 commits
- [Doc] add common case for long waiting time (#5430) · 8f89d720
  youkaichao authored Jun 11, 2024
  
  8f89d720
- [Core][Doc] Default to multiprocessing for single-node distributed case (#5230) · 99dac099
  Nick Hill authored Jun 11, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  99dac099
- [Core][Distributed] add same-node detection (#5369) · c4bd03c7
  youkaichao authored Jun 11, 2024
  
  c4bd03c7
- [Frontend] Customizable RoPE theta (#5197) · dcbf4286
  sasha0552 authored Jun 11, 2024
  
  dcbf4286
- [Bugfix] fix lora_dtype value type in arg_utils.py (#5398) · 00e6a2dc
  Ali Panahi authored Jun 11, 2024
  
  00e6a2dc
- [Bugfix] Fix `MultiprocessingGPUExecutor.check_health` when world_size == 1 (#5254) · 2e02311a
  Junichi Sato authored Jun 12, 2024
  
  2e02311a