Skip to content

GitLab

Unverified Commit 74692421 authored by Falko1's avatar Falko1 Committed by GitHub
Browse files

[Bugfix]: phi.py get rope_theta from config file (#9503) 


Co-authored-by: default avatarIsotr0py <2037008807@qq.com>
parent 29acd2c3
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 2 deletions
vllm/model_executor/models/phi.py
View file @ 74692421
...@@ -102,8 +102,9 @@ class PhiAttention(nn.Module): ...@@ -102,8 +102,9 @@ class PhiAttention(nn.Module):
# pylint: disable=C0301 # pylint: disable=C0301
# Refer to: # Refer to:
# https://huggingface.co/microsoft/phi-1_5/blob/d212a789620c380ff32ca1d1ee9943a777360987/modeling_phi.py#L518 # https://huggingface.co/microsoft/phi-1_5/blob/d212a789620c380ff32ca1d1ee9943a777360987/modeling_phi.py#L518
rope_theta = 10000 rope_theta = getattr(config, "rope_theta", 10000.0)
max_position_embeddings = getattr(config, "n_positions", 2048) max_position_embeddings = getattr(config, "max_position_embeddings",
2048)
self.rotary_emb = get_rope( self.rotary_emb = get_rope(
self.head_size, self.head_size,
rotary_dim=rotary_dim, rotary_dim=rotary_dim,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment