1. 07 Jan, 2026 3 commits
    • laibao's avatar
      perf(fused-moe): 接入 W16A16 Marlin MoE 并缓存 pack 权重 · 2b0c9835
      laibao authored
       - fused_experts_impl 增加 VLLM_USE_MARLIN_W16A16_MOE fast path:首次对 w1/w2 做 Marlin pack 后缓存,避免重复 reorder;并将原始
          权重 offload 到 CPU,降低 GPU 双份驻留
        - envs 补齐环境变量 VLLM_USE_LIGHTOP_MOE_SUM_MUL_ADD
        - 更新 fuse_moe_w16a16_marlin.py 的报错提示为 VLLM_USE_LIGHTOP=1
      2b0c9835
    • laibao's avatar
      perf: 加速 v1 InputBatch.add_request 的 token_ids 拷贝 · 8da572a9
      laibao authored
      新增环境变量开关 VLLM_V1_FAST_TOKEN_ID_COPY(默认关闭)。开启后在 CachedRequestState 中缓存 prompt_token_ids 的 np.int32,并在 add_request 里用 np.copyto 写入 token_ids_cpu,避免长 prompt 场景反复 list->NumPy 转换开销(尤其是抢占/反复进出 batch 时)
      8da572a9
    • zhuwenwen's avatar
      skip indexer_k_cache · 383f2ce8
      zhuwenwen authored
      383f2ce8
  2. 06 Jan, 2026 4 commits
  3. 05 Jan, 2026 13 commits
  4. 04 Jan, 2026 1 commit
  5. 24 Dec, 2025 6 commits
  6. 23 Dec, 2025 2 commits
  7. 22 Dec, 2025 4 commits
  8. 20 Dec, 2025 1 commit
  9. 18 Dec, 2025 2 commits
  10. 17 Dec, 2025 4 commits