Commits · f51086deccecb9d2e77b3633e868dea9f3af6347 · OpenDAS / vllm_cscc

24 Apr, 2025 2 commits
- update utils.py · f51086de
  zhuwenwen authored Apr 24, 2025
  
  f51086de
- fix llama and qwen layout · db94f061
  zhuwenwen authored Apr 24, 2025
  
  db94f061
18 Apr, 2025 1 commit
- deepseek_v3/r1 int8 量化首字调优 · 8fc15e04
  gaoqiong authored Apr 18, 2025
  
  8fc15e04
07 Apr, 2025 1 commit
- update layout · f6044f1a
  zhuwenwen authored Apr 07, 2025
  
  f6044f1a
03 Apr, 2025 1 commit
- debug and fix error · 20e9c3eb
  lizhigong authored Apr 03, 2025
  
  20e9c3eb
27 Mar, 2025 2 commits
- fix error when cpu is slower than dcu · 2c3f0e14
  lizhigong authored Mar 27, 2025
  
  2c3f0e14
- 更新bw以及k500sm_ai的支持 · 18ec9eaa
  yangql authored Mar 27, 2025
  
  18ec9eaa
26 Mar, 2025 5 commits
- support serving in use zero overhead · d4b6b8cc
  lizhigong authored Mar 26, 2025
  
  d4b6b8cc
- 增加blockint8支持优化 · 5f2801b1
  gaoqiong authored Mar 26, 2025
  
  5f2801b1
- 增加blockint8支持优化 · 5f15bdb5
  gaoqiong authored Mar 26, 2025
  
  5f15bdb5
- 增加blockint8支持优化 · f3deca99
  gaoqiong authored Mar 26, 2025
  
  f3deca99
- add server zero overhead engine · 024e595d
  lizhigong authored Mar 26, 2025
  
  024e595d
24 Mar, 2025 2 commits
- [fix]修复fused_moe.py中fused_moe接口未初始化moe_ep_size导致的deekseek等模型报错 · 4b7f382f
  王敏 authored Mar 24, 2025
  
  4b7f382f
- use two thread in step too improve first tokens · 2344d22e
  lizhigong authored Mar 24, 2025
  
  2344d22e
19 Mar, 2025 1 commit
- debug zero overhead · b78549c2
  lizhigong authored Mar 19, 2025
  
  b78549c2
18 Mar, 2025 1 commit
- fix llm_engine to zero_overhead · 7d867671
  lizhigong authored Mar 18, 2025
  
  7d867671
17 Mar, 2025 2 commits
- sampler修改 · 08c2298a
  guanyu1 authored Mar 17, 2025
  
  08c2298a
- zero overhead engine update · 9bd32639
  lizhigong authored Mar 17, 2025
  
  9bd32639
15 Mar, 2025 5 commits
- update fused_moe.py · 8a7c4451
  zhuwenwen authored Mar 15, 2025
  
  8a7c4451
- update qwen2-vl and qwen2.5-vl layout · 37ef8dd3
  zhuwenwen authored Mar 15, 2025
  
  37ef8dd3
- set llama and qwen tn layout · b845811e
  zhuwenwen authored Mar 15, 2025
  
  b845811e
- 支持fusemoe对int4的scale合zero合并读取操作 · 50424a23
  yangql authored Mar 15, 2025
  
  50424a23
- 支持fusemoe对int4的scale合zero合并读取操作 · f78a0510
  yangql authored Mar 15, 2025
  
  f78a0510
14 Mar, 2025 2 commits
- 增加线性int8 gemm配置 · 268d8a77
  gaoqiong authored Mar 14, 2025
  
  268d8a77
- 增加deepseek v3 block-int8量化支持 · 92545504
  gaoqiong authored Mar 14, 2025
  
  92545504
13 Mar, 2025 1 commit
- update mla to obtain the optimal configuration from config · 146eb9d3
  zhuwenwen authored Mar 13, 2025
  
  146eb9d3
12 Mar, 2025 1 commit
- [feat]添加BW deepseek_v3 fused_moe configs · ef27eec0
  王敏 authored Mar 12, 2025
  
  ef27eec0
11 Mar, 2025 1 commit
- [feat]优化fused_moe tuning，上传K100_AI的configs · 79a9bc29
  王敏 authored Mar 11, 2025
  
  79a9bc29
28 Feb, 2025 2 commits
- 删除vllm的w8a8 config · 6fd8c21b
  gaoqiong authored Feb 28, 2025
  
  6fd8c21b
- 修复w8a8找不到对应config的bug · 044076a1
  gaoqiong authored Feb 28, 2025
  
  044076a1
26 Feb, 2025 3 commits
- [fix]修复ds3量化代码引入的报错 · bcce8974
  王敏 authored Feb 26, 2025
  
  bcce8974
- [feat]合入ds3 MTP功能 · 1fbdf957
  王敏 authored Feb 26, 2025
  
  1fbdf957
- update deepseek-v2 · 96802ca0
  zhuwenwen authored Feb 26, 2025
  
  96802ca0
25 Feb, 2025 1 commit
- update version · 76621631
  zhuwenwen authored Feb 25, 2025
  
  76621631
22 Feb, 2025 2 commits
- update qwen2_vl nn lyaout · 437e6aef
  zhuwenwen authored Feb 22, 2025
  
  437e6aef
- support qwen2_5-vl · 0a89b8a3
  zhuwenwen authored Feb 22, 2025
  
  0a89b8a3
20 Feb, 2025 1 commit
- 适配deepseekv3\v2 moe awq的推理支持 · 47bd229c
  yangql authored Feb 20, 2025
  
  47bd229c
19 Feb, 2025 2 commits
- add deepseek-v2 tp2 fused_moe_config · 177520a9
  zhuwenwen authored Feb 19, 2025
  
  177520a9
- update lm_head tn layout for awq · f0e7d72d
  zhuwenwen authored Feb 19, 2025
  
  f0e7d72d
18 Feb, 2025 1 commit
- add deepseek-v2 tp1 fused_moe_config · f0ac5f23
  zhuwenwen authored Feb 18, 2025
  
  f0ac5f23