Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
08500011
Unverified
Commit
08500011
authored
Jun 15, 2025
by
maobaolong
Committed by
GitHub
Jun 14, 2025
Browse files
[Fix] Convert kv_transfer_config from dict to KVTransferConfig (#19262)
parent
861a0a0a
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
18 additions
and
0 deletions
+18
-0
vllm/entrypoints/llm.py
vllm/entrypoints/llm.py
+18
-0
No files found.
vllm/entrypoints/llm.py
View file @
08500011
...
...
@@ -10,6 +10,7 @@ from typing import (TYPE_CHECKING, Any, Callable, ClassVar, Optional, Union,
import
cloudpickle
import
torch.nn
as
nn
from
pydantic
import
ValidationError
from
tqdm.auto
import
tqdm
from
typing_extensions
import
TypeVar
,
deprecated
...
...
@@ -195,6 +196,23 @@ class LLM:
if
isinstance
(
worker_cls
,
type
):
kwargs
[
"worker_cls"
]
=
cloudpickle
.
dumps
(
worker_cls
)
if
"kv_transfer_config"
in
kwargs
and
isinstance
(
kwargs
[
"kv_transfer_config"
],
dict
):
from
vllm.config
import
KVTransferConfig
raw_config_dict
=
kwargs
[
"kv_transfer_config"
]
try
:
kwargs
[
"kv_transfer_config"
]
=
KVTransferConfig
(
**
raw_config_dict
)
except
ValidationError
as
e
:
logger
.
error
(
"Failed to convert 'kv_transfer_config' dict to "
"KVTransferConfig object. Dict: %s. Error: %s"
,
raw_config_dict
,
e
)
# Consider re-raising a more specific vLLM error or ValueError
# to provide better context to the user.
raise
ValueError
(
f
"Invalid 'kv_transfer_config' provided:
{
e
}
"
)
from
e
if
hf_overrides
is
None
:
hf_overrides
=
{}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment