Commit 19288a48 authored by yangql's avatar yangql
Browse files

Merge branch 'v0.9.2-dev-ds_auto_21' into 'v0.9.2-dev-ds_auto_12.29'

[feat]解决deepep auto模式卡住问题

See merge request dcutoolkit/deeplearing/vllm!324
parents de2a85b4 cb64c6bc
...@@ -108,6 +108,9 @@ class V1ZeroEagleProposer(EagleProposer): ...@@ -108,6 +108,9 @@ class V1ZeroEagleProposer(EagleProposer):
else: else:
num_input_tokens = num_tokens num_input_tokens = num_tokens
num_pad, num_tokens_across_dp = self.get_dp_padding(num_input_tokens)
num_input_tokens += num_pad
# copy inputs to buffer for cudagraph # copy inputs to buffer for cudagraph
self.positions[:num_tokens] = target_positions self.positions[:num_tokens] = target_positions
self.hidden_states[:num_tokens] = target_hidden_states self.hidden_states[:num_tokens] = target_hidden_states
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment