Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
outputs.py 369 Bytes
Newer Older
lizhigong's avatar
add v1 engine zero overhead  
lizhigong committed
1
2
3
4
5
6
7
8
9
10 



from dataclasses import dataclass
from vllm.v1.outputs import ModelRunnerOutput

@dataclass
class ZeroV1ModelRunnerOutput(ModelRunnerOutput):
    # [num_reqs]
    fix_req_ids: list[str] = None
lizhigong's avatar
add v1 engine + deepseek r1 mtp + zero-overhead scheduler  
lizhigong committed
11
12
13
14 
    fix_sampled_token_ids:list[list[int]] = None
    fix_draft_req_ids:list[list[int]] = None
    fix_draft_tokens_ids:list[list[int]] = None
    is_output_valid:bool = True