Skip to content

GitLab

Commit 799a193b authored by Nicolas Patry's avatar Nicolas Patry
Browse files

Fixing Phi3.

parent 08b3eac2
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
server/text_generation_server/models/custom_modeling/flash_llama_modeling.py
View file @ 799a193b
...@@ -52,7 +52,8 @@ if SYSTEM == "rocm": ...@@ -52,7 +52,8 @@ if SYSTEM == "rocm":
def load_attention(config, prefix, weights): def load_attention(config, prefix, weights):
bias = config.attention_bias # Only defined in granite.
bias = getattr(config, "attention_bias", False)
# if specific model type, load the correct attention # if specific model type, load the correct attention
if config.model_type == "phi3": if config.model_type == "phi3":
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment