- 11 Feb, 2026 1 commit
-
-
zhuwenwen authored
-
- 10 Feb, 2026 1 commit
-
-
wanglong3 authored
-
- 28 Jan, 2026 1 commit
-
-
laibao authored
增加环境变量开关,禁用 Marlin W16A16 MoE 路径 强制 Triton 且权重已是 Marlin packed 时给出明确报错 Marlin 支持探测改为 best-effort(不再依赖 VLLM_USE_LIGHTOP)
-
- 21 Jan, 2026 1 commit
-
-
zhuwenwen authored
for qwen3, VLLM_USE_FUSED_RMS_ROPE=1 (default)
-
- 20 Jan, 2026 3 commits
-
-
laibao authored
- 使用 get_moe_cuda_marlin_config_w16a16(status) 判断 W16A16 Marlin MoE 是否可用 - 在 FusedMoE 初始化阶段计算并缓存 _marlin_w16a16_moe_enabled,满足条件时强制 use_nn_moe=False - 权重加载后按缓存结果进行一次性 Marlin pack;运行时按 packed 标记走 Marlin fast path - 删除 envs.py 中 VLLM_USE_MARLIN_W16A16_MOE 环境变量定义与解析逻辑
-
zhuwenwen authored
-
laibao authored
-
- 16 Jan, 2026 2 commits
- 15 Jan, 2026 1 commit
-
-
zhuwenwen authored
todo: add VLLM_USE_QUERY_QUANT to not use q quant
-
- 14 Jan, 2026 2 commits
- 10 Jan, 2026 3 commits
- 09 Jan, 2026 1 commit
-
-
zhuwenwen authored
-
- 08 Jan, 2026 1 commit
-
-
laibao authored
-
- 07 Jan, 2026 3 commits
-
-
laibao authored
- fused_experts_impl 增加 VLLM_USE_MARLIN_W16A16_MOE fast path:首次对 w1/w2 做 Marlin pack 后缓存,避免重复 reorder;并将原始 权重 offload 到 CPU,降低 GPU 双份驻留 - envs 补齐环境变量 VLLM_USE_LIGHTOP_MOE_SUM_MUL_ADD - 更新 fuse_moe_w16a16_marlin.py 的报错提示为 VLLM_USE_LIGHTOP=1 -
zhuwenwen authored
-
laibao authored
新增环境变量开关 VLLM_V1_FAST_TOKEN_ID_COPY(默认关闭)。开启后在 CachedRequestState 中缓存 prompt_token_ids 的 np.int32,并在 add_request 里用 np.copyto 写入 token_ids_cpu,避免长 prompt 场景反复 list->NumPy 转换开销(尤其是抢占/反复进出 batch 时)
-
- 24 Dec, 2025 2 commits
- 16 Dec, 2025 1 commit
-
-
laibao authored
新增环境变量 VLLM_USE_MARLIN_W16A16_MOE,用于显式启用 Marlin W16A16 MoE experts 在 fused_moe 中当开关开启且实现可用时,调用 fused_experts_impl_w16a16_marlin 增加 Marlin W16A16 MoE 实现与 reduce 路径
-
- 03 Dec, 2025 1 commit
-
-
zhuwenwen authored
add VLLM_USE_OPT_RESHAPE_AND_CACHE、VLLM_USE_FUSE_SILU_AND_MUL and VLLM_USE_TOPK_RENORM for qwen3-30b
-
- 02 Dec, 2025 1 commit
-
-
zhuwenwen authored
-
- 26 Nov, 2025 1 commit
-
-
zhuwenwen authored
-
- 20 Nov, 2025 2 commits
- 13 Nov, 2025 2 commits
- 20 Oct, 2025 1 commit
-
-
zhuwenwen authored
-
- 13 Oct, 2025 1 commit
-
-
zhuwenwen authored
-
- 02 Oct, 2025 1 commit
-
-
Chenheli Hua authored
Signed-off-by:
Chenheli Hua <huachenheli@outlook.com> Signed-off-by:
simon-mo <simon.mo@hey.com>
-
- 26 Sep, 2025 1 commit
-
-
Aleksandr Malyshev authored
Signed-off-by:
Aleksandr Malyshev <maleksan@amd.com> Co-authored-by:
Aleksandr Malyshev <maleksan@amd.com> Co-authored-by:
Doug Lehr <douglehr@amd.com>
-
- 25 Sep, 2025 2 commits
-
-
zhuwenwen authored
-
Saman A. Pour authored
Signed-off-by:Saman Keon <samanamp@outlook.com>
-
- 24 Sep, 2025 3 commits
-
-
Duncan Moss authored
Signed-off-by:Duncan Moss <djm.moss@gmail.com>
-
Shu Wang authored
Signed-off-by:
Shu Wang <shuw@nvidia.com> Signed-off-by:
Shu Wang. <shuw@nvidia.com> Signed-off-by:
Tyler Michael Smith <tyler@neuralmagic.com> Signed-off-by:
Tyler Michael Smith <tlrmchlsmth@gmail.com> Co-authored-by:
Tyler Michael Smith <tyler@neuralmagic.com> Co-authored-by:
Tyler Michael Smith <tlrmchlsmth@gmail.com>
-
rongfu.leng authored
Signed-off-by:rongfu.leng <rongfu.leng@daocloud.io>
-
- 23 Sep, 2025 1 commit
-
-
Lucas Wilkinson authored
Signed-off-by:Lucas Wilkinson <lwilkins@redhat.com>
-