Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
7f83f40d
Unverified
Commit
7f83f40d
authored
Jun 28, 2024
by
Woosuk Kwon
Committed by
GitHub
Jun 28, 2024
Browse files
[Bugfix][TPU] Fix pad slot id (#5977)
parent
54814fd8
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
vllm/worker/tpu_model_runner.py
vllm/worker/tpu_model_runner.py
+1
-1
No files found.
vllm/worker/tpu_model_runner.py
View file @
7f83f40d
...
...
@@ -19,7 +19,7 @@ from vllm.utils import make_tensor_with_pad
logger
=
init_logger
(
__name__
)
_PAD_SLOT_ID
=
0
#
FIXM
E(woosuk)
_PAD_SLOT_ID
=
-
1
#
NOT
E(woosuk)
: In PyTorch XLA, index -1 is ignored.
# FIXME(woosuk): Temporarily disabled top-p sampling since it's too slow.
_ENABLE_TOP_P
=
False
# FIXME(woosuk): A temporary hack to support `n > 1`.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment