Skip to content

GitLab

"ssh:/git@developer.sourcefind.cn:2222/OpenDAS/vllm_cscc.git" did not exist on "682e0b6d2fbef012abde9b90c73da23f3065e2b8"
Unverified Commit 40d86ee4 authored by Chengji Yao's avatar Chengji Yao Committed by GitHub
Browse files

[TPU][Bugfix] fix OOM issue in CI test (#21550) 


Signed-off-by: default avatarChengji Yao <chengjiyao@google.com>
parent 85d051f0
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
tests/v1/tpu/test_basic.py
View file @ 40d86ee4
...@@ -59,7 +59,7 @@ def test_basic( ...@@ -59,7 +59,7 @@ def test_basic(
# actually test chunked prompt # actually test chunked prompt
max_num_batched_tokens=1024, max_num_batched_tokens=1024,
max_model_len=8192, max_model_len=8192,
gpu_memory_utilization=0.7, gpu_memory_utilization=0.95,
max_num_seqs=max_num_seqs, max_num_seqs=max_num_seqs,
tensor_parallel_size=tensor_parallel_size) as vllm_model: tensor_parallel_size=tensor_parallel_size) as vllm_model:
vllm_outputs = vllm_model.generate_greedy(example_prompts, vllm_outputs = vllm_model.generate_greedy(example_prompts,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment