Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
f9dadfbe
Unverified
Commit
f9dadfbe
authored
Nov 11, 2024
by
Woosuk Kwon
Committed by
GitHub
Nov 11, 2024
Browse files
[V1] Fix detokenizer ports (#10224)
Signed-off-by:
Woosuk Kwon
<
woosuk.kwon@berkeley.edu
>
parent
25144cee
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
4 deletions
+6
-4
vllm/v1/tokenizer/detokenizer.py
vllm/v1/tokenizer/detokenizer.py
+6
-4
No files found.
vllm/v1/tokenizer/detokenizer.py
View file @
f9dadfbe
...
...
@@ -48,11 +48,13 @@ class Detokenizer:
# For example, it does not terminate properly. We need to improve this.
self
.
push_port
=
get_open_port
()
self
.
pull_port
=
get_open_port
()
# NOTE: The push port of the engine process should be the same as the
# pull port of the detokenizer process. Vice versa.
self
.
detokenizer
=
DetokenizerProc
(
tokenizer_name
=
tokenizer_name
,
tokenizer_mode
=
tokenizer_mode
,
trust_remote_code
=
trust_remote_code
,
push_port
=
self
.
pu
sh
_port
,
pull_port
=
self
.
pu
ll
_port
)
push_port
=
self
.
pu
ll
_port
,
pull_port
=
self
.
pu
sh
_port
)
self
.
detokenizer
.
start
()
self
.
zmq_context
=
zmq
.
Context
()
...
...
@@ -95,8 +97,8 @@ class DetokenizerProc(multiprocessing.Process):
self
.
tokenizer_name
=
tokenizer_name
self
.
tokenizer_mode
=
tokenizer_mode
self
.
trust_remote_code
=
trust_remote_code
# NOTE: The pull_port of the detokenizer should be the same as
the
# push_port of the engine. Vice versa.
# NOTE: The pull_port of the detokenizer
process
should be the same as
#
the
push_port of the engine
process
. Vice versa.
self
.
pull_port
=
pull_port
self
.
push_port
=
push_port
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment