Commits · dd0c8f490868762e6671bd01c558b9413fb3caa1 · OpenDAS / vllm_cscc

08 Aug, 2024 5 commits
- update offline_inference.py · dd0c8f49
  zhuwenwen authored Aug 08, 2024
  
  dd0c8f49
- add git version for internal testing · 52c3dd59
  zhuwenwen authored Aug 08, 2024
  
  52c3dd59
- fix awq index of qwen · 04633c47
  zhuwenwen authored Aug 08, 2024
  
  04633c47
- Fix multiprocessing shutdown errors · 708d897e
  zhuwenwen authored Aug 08, 2024
  
  708d897e
- The timeout for each iteration in the engine is set to 120 seconds by default · 3d087876
  zhuwenwen authored Aug 08, 2024
  
  3d087876
06 Aug, 2024 4 commits
- update version and add gptq_kernels info · c7e7ca7b
  zhuwenwen authored Aug 06, 2024
  
  c7e7ca7b
- update version and skip install gptq_kernels · 58fb0c33
  zhuwenwen authored Aug 06, 2024
  
  58fb0c33
- fix deepseek_v2 236b fused_moe_kernel expert_ids_ptr value error · b2068609
  wangmin6 authored Aug 06, 2024
  
  b2068609
- 恢复对bf16的支持 · 9f9f3796
  zhangshao authored Aug 06, 2024
  
  9f9f3796
05 Aug, 2024 1 commit
- pa add v prefetch for gemm1 · f38bd872
  flyingdown authored Aug 05, 2024
  
  f38bd872
04 Aug, 2024 4 commits
- Merge branch 'v0.5.0-dtk24.04.1_awq' into 'v0.5.0-dtk24.04.1' · 8ebc32aa
  gaoqiong authored Aug 04, 2024
```
V0.5.0 dtk24.04.1 awq

See merge request dcutoolkit/deeplearing/vllm!6
```
  8ebc32aa
- add qwen awq support · 982c1545
  gaoqiong authored Aug 05, 2024
  
  982c1545
- add qwen2 awq support · ec98d390
  gaoqiong authored Aug 04, 2024
  
  ec98d390
- add llama model awq support · 5f5ddc3d
  gaoqiong authored Aug 04, 2024
  
  5f5ddc3d
03 Aug, 2024 1 commit
- update benchmraks and tests · bdac8f06
  zhuwenwen authored Aug 03, 2024
  
  bdac8f06
01 Aug, 2024 1 commit
- support fa pad · ffbef65c
  zhuwenwen authored Aug 01, 2024
  
  ffbef65c
31 Jul, 2024 1 commit
- update benchmarks and examples · 1190e964
  zhuwenwen authored Jul 31, 2024
  
  1190e964
29 Jul, 2024 1 commit
- 增加head size 64-256支持 · 69185c0b
  zhangshao authored Jul 29, 2024
  
  69185c0b
25 Jul, 2024 1 commit
- update prefix_prefill params · d69fb7b2
  zhuwenwen authored Jul 25, 2024
  
  d69fb7b2
24 Jul, 2024 3 commits
- add gemm pad and fa pad for 7b model · 0b5e4e11
  zhuwenwen authored Jul 24, 2024
  
  0b5e4e11
- update gptq relative path · 2d0a73a3
  zhuwenwen authored Jul 24, 2024
  
  2d0a73a3
- add a list that supports nn model architectures · 0e7df36b
  zhuwenwen authored Jul 24, 2024
  
  0e7df36b
23 Jul, 2024 2 commits
- optimize rmsnorm kernel · c62f8e9a
  zhuwenwen authored Jul 23, 2024
  
  c62f8e9a
- refactoring the transpose kernel and update supported model · b2dd1743
  zhuwenwen authored Jul 23, 2024
  
  b2dd1743
22 Jul, 2024 4 commits
- fix index error of baichuan2 · 795ce518
  zhuwenwen authored Jul 22, 2024
  
  795ce518
- pa优化，编译选项优化 · 1be9a629
  zhangshao authored Jul 22, 2024
  
  1be9a629
- Merge branch 'gptq_fix' into 'v0.5.0-dtk24.04.1' · d4c0015a
  huangwb authored Jul 22, 2024
```
fix gptq performance degradation when batch size>4 issue

See merge request dcutoolkit/deeplearing/vllm!5
```
  d4c0015a
- fix gptq performance degradation when batch size>4 issue · f423ad60
  huangwb authored Jul 22, 2024
  
  f423ad60
20 Jul, 2024 7 commits
- use two method to add bias · 4caf1539
  zhuwenwen authored Jul 20, 2024
  
  4caf1539
- update ray version · 767bb050
  zhuwenwen authored Jul 20, 2024
  
  767bb050
- support nn layout · 1e0cb1f4
  zhuwenwen authored Jul 20, 2024
  
  1e0cb1f4
- Merge branch 'v0.5.0-dtk24.04.1_nn_gaoq' into 'v0.5.0-dtk24.04.1' · 281ca6c1
  gaoqiong authored Jul 20, 2024
```
V0.5.0 Modify nn support methods

See merge request dcutoolkit/deeplearing/vllm!4
```
  281ca6c1
- 去掉调试信息 · 9653385f
  gaoqiong authored Jul 20, 2024
  
  9653385f
- 修改nn支持方式 · 835bd9fc
  gaoqiong authored Jul 20, 2024
  
  835bd9fc
- modify gemm pad strategy · 7fe40ced
  zhuwenwen authored Jul 20, 2024
  
  7fe40ced
18 Jul, 2024 2 commits
- default use of ck fa · e661266e
  zhuwenwen authored Jul 18, 2024
  
  e661266e
- back to pa and rn · 71b1be50
  zhuwenwen authored Jul 18, 2024
  
  71b1be50
17 Jul, 2024 3 commits
- Merge branch 'vllm-v0.5.0-dtk24.04.1-rotary_embedding' into 'vllm-v0.5.0-dtk24.04.1' · c628c6ec
  huangwb authored Jul 17, 2024
```
add rotary_embedding for tgi

See merge request dcutoolkit/deeplearing/vllm!3
```
  c628c6ec
- add rotary_embedding for tgi · e499f96c
  huangwb authored Jul 17, 2024
  
  e499f96c
- set default layout · 22839191
  zhuwenwen authored Jul 17, 2024
  
  22839191