Skip to content

GitLab

"docs/source/git@developer.sourcefind.cn:norm/vllm.git" did not exist on "42c02f5892e984d308614f074f423a311aba8993"
Unverified Commit 220a4762 authored by Xiang Xu's avatar Xiang Xu Committed by GitHub
Browse files

Use head_dim in config if exists (#2622)

parent beb89f68
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/config.py
View file @ 220a4762
...@@ -213,6 +213,8 @@ class ModelConfig: ...@@ -213,6 +213,8 @@ class ModelConfig:
return self.hf_config.hidden_size return self.hf_config.hidden_size
def get_head_size(self) -> int: def get_head_size(self) -> int:
if hasattr(self.hf_config, "head_dim"):
return self.hf_config.head_dim
# FIXME(woosuk): This may not be true for all models. # FIXME(woosuk): This may not be true for all models.
return self.hf_config.hidden_size // self.hf_config.num_attention_heads return self.hf_config.hidden_size // self.hf_config.num_attention_heads
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment