Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
2b90ecd3
"vllm/vscode:/vscode.git/clone" did not exist on "c7d8724e7865ba4a54ffdba23ac77eb13d28234b"
Commit
2b90ecd3
authored
Oct 29, 2025
by
zhuwenwen
Browse files
remove fp8_e5m2 error
parent
90227352
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
3 deletions
+3
-3
vllm/attention/layer.py
vllm/attention/layer.py
+3
-3
No files found.
vllm/attention/layer.py
View file @
2b90ecd3
...
...
@@ -123,9 +123,9 @@ class Attention(nn.Module):
assert
isinstance
(
quant_method
,
BaseKVCacheMethod
)
# TODO (mgoin): kv cache dtype should be specified in the FP8
# checkpoint config and become the "auto" behavior
if
self
.
kv_cache_dtype
==
"fp8_e5m2"
:
raise
ValueError
(
"fp8_e5m2 kv-cache is not supported with "
"fp8 checkpoints."
)
#
if self.kv_cache_dtype == "fp8_e5m2":
#
raise ValueError("fp8_e5m2 kv-cache is not supported with "
#
"fp8 checkpoints.")
# If quantization is enabled, we make "k_scale" and "v_scale"
# parameters so that it can be loaded from the model checkpoint.
# The k/v_scale will then be converted back to native float32
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment