Skip to content

GitLab

Unverified Commit fe15729a authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

[V1] Use custom ops for piecewise CUDA graphs (#10227) 


Signed-off-by: default avatarWoosuk Kwon <woosuk.kwon@berkeley.edu>
parent 330e82d3
Hide whitespace changes
Inline Side-by-side
Showing with 0 additions and 2 deletions
vllm/v1/worker/gpu_model_runner.py
View file @ fe15729a
import os
import time import time
from dataclasses import dataclass from dataclasses import dataclass
from typing import TYPE_CHECKING, Dict, List, Optional, Set from typing import TYPE_CHECKING, Dict, List, Optional, Set
...@@ -406,7 +405,6 @@ class GPUModelRunner: ...@@ -406,7 +405,6 @@ class GPUModelRunner:
if self.use_cuda_graph: if self.use_cuda_graph:
# FIXME(woosuk): Currently, we do not use inductor to reduce the # FIXME(woosuk): Currently, we do not use inductor to reduce the
# compilation time and any potential issues with the inductor. # compilation time and any potential issues with the inductor.
os.environ["VLLM_CUSTOM_OPS"] = "none"
set_compilation_config( set_compilation_config(
CompilationConfig( CompilationConfig(
use_cudagraph=True, use_cudagraph=True,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment