Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
c4774eb8
Unverified
Commit
c4774eb8
authored
Jul 11, 2024
by
Roger Wang
Committed by
GitHub
Jul 11, 2024
Browse files
[Bugfix] Fix snapshot download in serving benchmark (#6318)
parent
fc17110b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
9 deletions
+9
-9
benchmarks/backend_request_func.py
benchmarks/backend_request_func.py
+9
-9
No files found.
benchmarks/backend_request_func.py
View file @
c4774eb8
...
...
@@ -390,17 +390,17 @@ def remove_prefix(text: str, prefix: str) -> str:
return
text
def
get_model
(
pretrained_model_name_or_path
:
str
):
def
get_model
(
pretrained_model_name_or_path
:
str
)
->
str
:
if
os
.
getenv
(
'VLLM_USE_MODELSCOPE'
,
'False'
).
lower
()
==
'true'
:
from
modelscope
import
snapshot_download
else
:
from
huggingface_hub
import
snapshot_download
model_path
=
snapshot_download
(
model_id
=
pretrained_model_name_or_path
,
local_files_only
=
huggingface_hub
.
constants
.
HF_HUB_OFFLINE
,
ignore_file_pattern
=
[
".*.pt"
,
".*.safetensors"
,
".*.bin"
])
return
model
_path
model_path
=
snapshot_download
(
model_id
=
pretrained_model_name_or_path
,
local_files_only
=
huggingface_hub
.
constants
.
HF_HUB_OFFLINE
,
ignore_file_pattern
=
[
".*.pt"
,
".*.safetensors"
,
".*.bin"
])
return
model_path
return
pretrained_model_name_or
_path
def
get_tokenizer
(
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment