Skip to content

GitLab

Unverified Commit a25866ac authored by zh Wang's avatar zh Wang Committed by GitHub
Browse files

[Bugfix] Fix profiling.py (#16202) 


Signed-off-by: default avatarzh Wang <rekind133@outlook.com>
parent 098900d7
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 3 deletions
examples/offline_inference/profiling.py
View file @ a25866ac
...@@ -234,9 +234,8 @@ def run_profile(context: ProfileContext, csv_output: Optional[str], ...@@ -234,9 +234,8 @@ def run_profile(context: ProfileContext, csv_output: Optional[str],
sampling_params.max_tokens = next(output_len_generator) sampling_params.max_tokens = next(output_len_generator)
assert isinstance(sampling_params.max_tokens, int) assert isinstance(sampling_params.max_tokens, int)
prompt_token_ids = torch.randint( prompt_token_ids = torch.randint(llm.get_tokenizer().vocab_size,
llm.llm_engine.model_config.get_vocab_size(), size=(prompt_len, )).tolist()
size=(prompt_len, )).tolist()
llm.llm_engine.add_request( llm.llm_engine.add_request(
request_id=f"seq{i}", request_id=f"seq{i}",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment