Commits · 201768d595388bb3cf24c2bf9f2519c136068f85 · OpenDAS / vllm_cscc

27 Nov, 2024 3 commits
- update tests · 87a2e37f
  zhuwenwen authored Nov 27, 2024
  
  87a2e37f
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
- add VLLM_OPTEST_MODELS_PATH/OPTEST_MODELS_PATH to load models from local path... · 3c9817d2
  zhuwenwen authored Nov 27, 2024
```
add VLLM_OPTEST_MODELS_PATH/OPTEST_MODELS_PATH  to load models from local path instead of Hugging Face Hub
```
  3c9817d2
25 Nov, 2024 1 commit
- update the lm-head layout of llama and qwen series models · 122d7146
  zhuwenwen authored Nov 25, 2024
  
  122d7146
20 Nov, 2024 3 commits
- [fix]修复单测test_mlp_correctness.py运行时的崩溃问题 · 96dcaff9
  王敏 authored Nov 20, 2024
  
  96dcaff9
- [fix]修复单测test_medusa_correctness.py中的错误 · 215f33b0
  王敏 authored Nov 20, 2024
  
  215f33b0
- [fix]修复单测test_encoder_decoder_model_runner报错问题 · 93089fb2
  王敏 authored Nov 20, 2024
  
  93089fb2
19 Nov, 2024 1 commit
- [fix]解决单测test_encode报错问题 · 4e23f2ad
  王敏 authored Nov 19, 2024
  
  4e23f2ad
18 Nov, 2024 2 commits
- [fix]解决test_model_input中报的找不到move_cache错误 · 4d29e0a8
  王敏 authored Nov 18, 2024
  
  4d29e0a8
- [fix]修复test_spec_decode_worker中的错误 · dc2ce261
  王敏 authored Nov 18, 2024
  
  dc2ce261
15 Nov, 2024 1 commit
- [fix]修复test_long_context中报错问题，单测依然无法通过，nv也是同样的问题 · 1d6cfb11
  王敏 authored Nov 15, 2024
  
  1d6cfb11
12 Nov, 2024 1 commit
- update idefics2_vision_model.py · 9a68d3a8
  zhuwenwen authored Nov 12, 2024
  
  9a68d3a8
07 Nov, 2024 2 commits
- remove xformers deps · 55c5f16f
  zhuwenwen authored Nov 07, 2024
  
  55c5f16f
- 1.Mixtral模型的gemm由TN改为NN · e7d022fd
  王敏 authored Nov 07, 2024
  
  e7d022fd
06 Nov, 2024 2 commits
- optimize qwen2-moe layout · 82aee745
  zhuwenwen authored Nov 06, 2024
  
  82aee745
- 1.解决medusa模型多卡推理结果异常问题 · f9060e6b
  王敏 authored Nov 06, 2024
```
2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题
```
  f9060e6b
04 Nov, 2024 1 commit
- optimize qwen2-vl layout · a4167889
  zhuwenwen authored Nov 04, 2024
  
  a4167889
01 Nov, 2024 1 commit
- support multi image input of local files · a89ac72c
  zhuwenwen authored Nov 01, 2024
  
  a89ac72c
30 Oct, 2024 2 commits
- remove unused code · 04d33670
  zhuwenwen authored Oct 30, 2024
  
  04d33670
- auto convert lm_head layout of llama · 1428c17d
  zhuwenwen authored Oct 30, 2024
  
  1428c17d
29 Oct, 2024 3 commits
- update llama.py · 85def94c
  zhuwenwen authored Oct 29, 2024
  
  85def94c
- [Model] Add telechat-12b and GLM-4v support · 1e4ba896
  zhuwenwen authored Oct 29, 2024
  
  1e4ba896
- [Model] Add GLM-4v support · cbdc3a13
  zhuwenwen authored Oct 29, 2024
  
  cbdc3a13
28 Oct, 2024 2 commits
- update utils.py · 86d634c3
  zhuwenwen authored Oct 28, 2024
  
  86d634c3
- update lm_head of llama · 112588c2
  zhuwenwen authored Oct 28, 2024
  
  112588c2
25 Oct, 2024 1 commit
- support telechat-12b · fa5b0b39
  zhuwenwen authored Oct 25, 2024
  
  fa5b0b39
24 Oct, 2024 3 commits
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
- [Bugfix] TypeError: ChatGLMTokenizer._pad() got an unexpected keyword argument 'padding_side' · 3bda0405
  zhuwenwen authored Oct 24, 2024
  
  3bda0405
- set VLLM_USE_TC_PAGED_ATTN=1 · 91577443
  zhuwenwen authored Oct 24, 2024
  
  91577443
23 Oct, 2024 2 commits
- update llama.py · 905411f1
  zhuwenwen authored Oct 23, 2024
  
  905411f1
- update lm_head weight to support llama3.2 · 4e0b233d
  zhuwenwen authored Oct 23, 2024
  
  4e0b233d
22 Oct, 2024 2 commits
- add os · aba40fda
  zhuwenwen authored Oct 22, 2024
  
  aba40fda
- 增加w8a8的triton环境变量控制 · daa7f7d7
  gaoqiong authored Oct 22, 2024
  
  daa7f7d7
19 Oct, 2024 2 commits
- update _custom_ops.py · 2c7f740a
  zhuwenwen authored Oct 19, 2024
  
  2c7f740a
- update _custom_ops.py · 5289f7c4
  zhuwenwen authored Oct 19, 2024
  
  5289f7c4
18 Oct, 2024 1 commit
- 删除v2 8192限制 · a7e3c9d6
  zhangshao authored Oct 18, 2024
  
  a7e3c9d6
17 Oct, 2024 1 commit
- add VLLM_USE_TC_PAGED_ATTN to convert tc pa · f164171b
  zhuwenwen authored Oct 17, 2024
  
  f164171b
16 Oct, 2024 2 commits
- add LM_TN for bloom lm_head weight · 2a9c497e
  zhuwenwen authored Oct 16, 2024
  
  2a9c497e
- update paged_attn.py · 85e8224c
  zhuwenwen authored Oct 16, 2024
  
  85e8224c
15 Oct, 2024 1 commit
- support baichuan awq and skip _rocm_C · b3ab1cdc
  zhuwenwen authored Oct 15, 2024
  
  b3ab1cdc