GitLab

[feat]添加qwen的lora单测

See merge request dcutoolkit/deeplearing/vllm!57

增加bw pa tc优化

See merge request dcutoolkit/deeplearing/vllm!54

解决qwen lora模型推理结果异常问题

See merge request dcutoolkit/deeplearing/vllm!56

2.修复medusa推理时内存访问越界问题
3.benchmark_moe添加deepseek_v2模型

0.6.2 dev w8a8

See merge request dcutoolkit/deeplearing/vllm!55

添加numa绑定能力

See merge request dcutoolkit/deeplearing/vllm!52

[feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑与主干隔离

See merge request dcutoolkit/deeplearing/vllm!51

test_audio文件将TEST_AUDIO_URLS改为本地；testchat文件360行路径改为本地；test_metrics文件197行将base_...

See merge request dcutoolkit/deeplearing/vllm!50

test_audio文件将TEST_AUDIO_URLS改为本地；testchat文件360行路径改为本地；test_metrics文件197行将base_url从0.0.0.0改为localhost；test_run_batch将INPUT_BATCH，INVALID_INPUT_BATCH,INPUT_EMBEDDING_BATCH改为原来的格式；test_tokenizer_group将18行的gpt路径改为修改后的路径；test_braodcast将model的判断改为if llava-hf/llava-1.5-7b-hf in model

Update offline_streaming_inference_chat_demo.py

See merge request dcutoolkit/deeplearing/vllm!49

[feat]并行解码支持多卡推理

See merge request dcutoolkit/deeplearing/vllm!48

fix

See merge request dcutoolkit/deeplearing/vllm!47

[fix]修复单测test_mlp_correctness失败问题

See merge request dcutoolkit/deeplearing/vllm!45