Commit 54ab90dd authored by zhuwenwen's avatar zhuwenwen
Browse files

remove num_speculative_tokens init val

parent b8b722eb
...@@ -55,7 +55,7 @@ class CommonAttentionMetadata: ...@@ -55,7 +55,7 @@ class CommonAttentionMetadata:
"""Longest query in batch""" """Longest query in batch"""
num_rejected_tokens: list[int] num_rejected_tokens: list[int]
"""(batch_size,), record the rejected tokens number in cpu and gpu""" """(batch_size,), record the rejected tokens number in cpu and gpu"""
num_speculative_tokens: int = 0 num_speculative_tokens: int
"""Number of speculative tokens""" """Number of speculative tokens"""
block_table_tensor: torch.Tensor block_table_tensor: torch.Tensor
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment