Commits · 3bda040583e2a3884b1105438596d417b28ac36e · OpenDAS / vllm_cscc

24 Oct, 2024 2 commits
- [Bugfix] TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding_side' · 3bda0405
  zhuwenwen authored Oct 24, 2024
  
  3bda0405
- set VLLM_USE_TC_PAGED_ATTN=1 · 91577443
  zhuwenwen authored Oct 24, 2024
  
  91577443
23 Oct, 2024 2 commits
- update llama.py · 905411f1
  zhuwenwen authored Oct 23, 2024
  
  905411f1
- update lm_head weight to support llama3.2 · 4e0b233d
  zhuwenwen authored Oct 23, 2024
  
  4e0b233d
22 Oct, 2024 5 commits
- add os · aba40fda
  zhuwenwen authored Oct 22, 2024
  
  aba40fda
- Merge remote-tracking branch 'origin/vllm-0.6-zhagnshao' into v0.6.2-dev · ee105627
  zhuwenwen authored Oct 22, 2024
  
  ee105627
- Merge branch '0.6.2_w8a8_triton2' into 'v0.6.2-dev' · d329c4db
  zhuwenwen authored Oct 22, 2024
```
增加w8a8的triton环境变量控制

See merge request dcutoolkit/deeplearing/vllm!28
```
  d329c4db
- 增加w8a8的triton环境变量控制 · daa7f7d7
  gaoqiong authored Oct 22, 2024
  
  daa7f7d7
- 解决pa v1 tc 部分 size bug · bac73e5d
  zhangshao authored Oct 22, 2024
  
  bac73e5d
19 Oct, 2024 2 commits
- update _custom_ops.py · 2c7f740a
  zhuwenwen authored Oct 19, 2024
  
  2c7f740a
- update _custom_ops.py · 5289f7c4
  zhuwenwen authored Oct 19, 2024
  
  5289f7c4
18 Oct, 2024 2 commits
- 删除v2 8192限制 · a7e3c9d6
  zhangshao authored Oct 18, 2024
  
  a7e3c9d6
- 解决pa v2 bug · e2bd7e16
  zhangshao authored Oct 18, 2024
  
  e2bd7e16
17 Oct, 2024 4 commits
- update pa tc interfaces · 304e2bab
  zhuwenwen authored Oct 17, 2024
  
  304e2bab
- update attention_kernels_opt_tc.cu · 1c2aa04c
  zhuwenwen authored Oct 17, 2024
  
  1c2aa04c
- update attention_kernels_opt_tc.cu · 83bb8f5a
  zhuwenwen authored Oct 17, 2024
  
  83bb8f5a
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
16 Oct, 2024 2 commits
- add LM_TN for bloom lm_head weight · 2a9c497e
  zhuwenwen authored Oct 16, 2024
  
  2a9c497e
- update paged_attn.py · 85e8224c
  zhuwenwen authored Oct 16, 2024
  
  85e8224c
15 Oct, 2024 1 commit
- support baichuan awq and skip _rocm_C · b3ab1cdc
  zhuwenwen authored Oct 15, 2024
  
  b3ab1cdc
12 Oct, 2024 1 commit
- restore qwen-vl to default layout · 422af727
  zhuwenwen authored Oct 12, 2024
  
  422af727
11 Oct, 2024 2 commits
- [Model] support telechat_12B · 6d4b207a
  zhuwenwen authored Oct 11, 2024
  
  6d4b207a
- [Misc] Update config loading for Qwen2-VL and remove Granite · a7e72c36
  zhuwenwen authored Oct 11, 2024
  
  a7e72c36
10 Oct, 2024 2 commits
- update benchmarks · 00d3d196
  zhuwenwen authored Oct 10, 2024
  
  00d3d196
- remove torch version info · 2f5f98bb
  zhuwenwen authored Oct 10, 2024
  
  2f5f98bb
09 Oct, 2024 4 commits
- update version · 7a8923fb
  zhuwenwen authored Oct 09, 2024
  
  7a8923fb
- update test_prefix_prefill.py · ec417173
  zhuwenwen authored Oct 09, 2024
  
  ec417173
- 增加AWQ相关环境变量控制 · cc4b902f
  zhuwenwen authored Oct 09, 2024
  
  cc4b902f
- 添加w8a8 rocblas非融合支持 · 35a8304d
  zhuwenwen authored Oct 09, 2024
  
  35a8304d
27 Sep, 2024 2 commits
- update version and deps · 5a9c236d
  zhuwenwen authored Sep 27, 2024
  
  5a9c236d
- Merge tag 'v0.6.2' into v0.6.2-dev · 539aa992
  zhuwenwen authored Sep 27, 2024
  
  539aa992
26 Sep, 2024 1 commit
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
25 Sep, 2024 8 commits
- [Misc] Support quantization of MllamaForCausalLM (#8822) · 7193774b
  Michael Goin authored Sep 25, 2024
  
  7193774b
- [Doc] Update doc for Transformers 4.45 (#8817) · e2c6e0a8
  Roger Wang authored Sep 25, 2024
  
  e2c6e0a8
- [Model] Add support for the multi-modal Llama 3.2 model (#8811) · 770ec602
  Chen Zhang authored Sep 25, 2024
```
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Chang Su <chang.s.su@oracle.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  770ec602
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- [Misc] Support FP8 MoE for compressed-tensors (#8588) · 873edda6
  Michael Goin authored Sep 25, 2024
  
  873edda6
- [Frontend] MQLLMEngine supports profiling. (#8761) · 64840dfa
  科英 authored Sep 26, 2024
  
  64840dfa
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
- [VLM][Bugfix] internvl with num_scheduler_steps > 1 (#8614) · 0c4d2ad5
  DefTruth authored Sep 26, 2024
  
  0c4d2ad5