Skip to content

GitLab

"vllm/vscode:/vscode.git/clone" did not exist on "e16fa99a6ad5bae4aedfb76121d4e622d27f81c3"
Unverified Commit 6ad58f42 authored by fuchen.ljl's avatar fuchen.ljl Committed by GitHub
Browse files

fix_tokenizer_snapshot_download_bug (#4493)

parent dd1a50a8
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/transformers_utils/tokenizer.py
View file @ 6ad58f42
...@@ -79,7 +79,7 @@ def get_tokenizer( ...@@ -79,7 +79,7 @@ def get_tokenizer(
revision=revision, revision=revision,
local_files_only=huggingface_hub.constants.HF_HUB_OFFLINE, local_files_only=huggingface_hub.constants.HF_HUB_OFFLINE,
# Ignore weights - we only need the tokenizer. # Ignore weights - we only need the tokenizer.
ignore_file_pattern=["*.pt", "*.safetensors", "*.bin"]) ignore_file_pattern=[".*.pt", ".*.safetensors", ".*.bin"])
tokenizer_name = tokenizer_path tokenizer_name = tokenizer_path
if tokenizer_mode == "slow": if tokenizer_mode == "slow":
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment