GitLab

[feat]添加qwen的lora单测

See merge request dcutoolkit/deeplearing/vllm!57

增加bw pa tc优化

See merge request dcutoolkit/deeplearing/vllm!54

解决qwen lora模型推理结果异常问题

See merge request dcutoolkit/deeplearing/vllm!56

2.修复medusa推理时内存访问越界问题
3.benchmark_moe添加deepseek_v2模型

0.6.2 dev w8a8

See merge request dcutoolkit/deeplearing/vllm!55

Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiaoyu Zhang <BBuf@users.noreply.github.com>

Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

[CI] Add test case with JSON schema using references + use xgrammar by default with OpenAI parse (#10935)
Signed-off-by: mgoin <michael@neuralmagic.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

[Docs] hint to enable use of GPU performance counters in profiling tools for multi-node distributed serving (#11235)
Co-authored-by: Michael Goin <michael@neuralmagic.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Signed-off-by: drikster80 <ed.sealing@gmail.com>
Co-authored-by: drikster80 <ed.sealing@gmail.com>

添加numa绑定能力

See merge request dcutoolkit/deeplearing/vllm!52

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>

Signed-off-by: Kuntai Du <kuntai@uchicago.edu>