Commit 54ab90dd authored by zhuwenwen's avatar zhuwenwen
Browse files

remove num_speculative_tokens init val

parent b8b722eb
......@@ -55,7 +55,7 @@ class CommonAttentionMetadata:
"""Longest query in batch"""
num_rejected_tokens: list[int]
"""(batch_size,), record the rejected tokens number in cpu and gpu"""
num_speculative_tokens: int = 0
num_speculative_tokens: int
"""Number of speculative tokens"""
block_table_tensor: torch.Tensor
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment