Commit d4e72be3 authored by gaoqiong's avatar gaoqiong
Browse files

Merge branch 'v0.9.2-dev-main+mtp-zero' into 'v0.9.2-dev'

fix: chunk-prefill生成的token被错误返回

See merge request dcutoolkit/deeplearing/vllm!327
parents 072e3895 2c1de3fa
......@@ -98,6 +98,7 @@ class V1ZeroModelRunner(GPUModelRunner):
)
for req_idx, start_idx, end_idx in self.token_ids_cpu_fix_record:
if start_idx == -1:
self.fix_sampled_token_ids[req_idx].clear()
continue
num_accepted_tokens = len(self.fix_sampled_token_ids[req_idx])
req_id = self.fix_req_ids[req_idx]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment