Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
text-generation-inference
Commits
27ff1871
Unverified
Commit
27ff1871
authored
Oct 23, 2024
by
OlivierDehaene
Browse files
hotfix: fix flashllama
parent
03c9388b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
server/text_generation_server/models/custom_modeling/flash_llama_modeling.py
...ion_server/models/custom_modeling/flash_llama_modeling.py
+1
-1
No files found.
server/text_generation_server/models/custom_modeling/flash_llama_modeling.py
View file @
27ff1871
...
...
@@ -692,7 +692,7 @@ class FlashLlamaForCausalLM(torch.nn.Module):
logits
,
speculative_logits
=
self
.
lm_head
(
hidden_states
)
# Used in Granite
if
not
self
.
logits_scaled
:
if
self
.
logits_scaling
is
not
None
and
not
self
.
logits_scaled
:
logits
/=
self
.
logits_scaling
if
speculative_logits
is
not
None
:
speculative_logits
/=
self
.
logits_scaling
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment