Skip to content

GitLab

Commit b40f13ab authored by qiyuxinlin's avatar qiyuxinlin
Browse files

fix deduplicate_and_sort cudagraphs

parent d35d61f6
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
ktransformers/operators/experts.py
View file @ b40f13ab
...@@ -45,7 +45,7 @@ def generate_cuda_graphs(chunk_size: int) -> list: ...@@ -45,7 +45,7 @@ def generate_cuda_graphs(chunk_size: int) -> list:
base_list = [1, 2, 3, Config().max_batch_size, 64, 256, 512, chunk_size] base_list = [1, 2, 3, Config().max_batch_size, 64, 256, 512, chunk_size]
if chunk_size <= 1024: if chunk_size <= 1024:
return base_list return deduplicate_and_sort(base_list)
multiples = [i for i in range(1024, chunk_size + 1, 1024)] multiples = [i for i in range(1024, chunk_size + 1, 1024)]
......
ktransformers/server/balance_serve/inference/model_runner.py
View file @ b40f13ab
...@@ -45,7 +45,7 @@ def generate_cuda_graphs(chunk_size: int) -> list: ...@@ -45,7 +45,7 @@ def generate_cuda_graphs(chunk_size: int) -> list:
base_list = [1, 2, 3, Config().max_batch_size, 64, 256, 512, chunk_size] base_list = [1, 2, 3, Config().max_batch_size, 64, 256, 512, chunk_size]
if chunk_size <= 1024: if chunk_size <= 1024:
return base_list return deduplicate_and_sort(base_list)
multiples = [i for i in range(1024, chunk_size + 1, 1024)] multiples = [i for i in range(1024, chunk_size + 1, 1024)]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment