Unverified Commit 41183c1f authored by Ekagra Ranjan's avatar Ekagra Ranjan Committed by GitHub
Browse files

[Spec Decode] Fix offline spec_decode.py (#24257)


Signed-off-by: default avatarEkagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: default avatarRoger Wang <hey@rogerw.io>
parent 43d9ad03
...@@ -1310,6 +1310,10 @@ def add_dataset_parser(parser: FlexibleArgumentParser): ...@@ -1310,6 +1310,10 @@ def add_dataset_parser(parser: FlexibleArgumentParser):
def get_samples(args, tokenizer) -> list[SampleRequest]: def get_samples(args, tokenizer) -> list[SampleRequest]:
if not hasattr(args, "request_id_prefix"):
args.request_id_prefix = ""
if args.dataset_name == "custom": if args.dataset_name == "custom":
dataset = CustomDataset(dataset_path=args.dataset_path) dataset = CustomDataset(dataset_path=args.dataset_path)
input_requests = dataset.sample( input_requests = dataset.sample(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment